FAQs

 Brittle
 Watch Me
 Needs Review
 Addressed
 Clear Flag

You may have noticed some new icons in the test result drop down menus. These new status options can be used to mark tests that may or may not require further action in the future. For example:

• The "Brittle" option can be used to mark a test that randomly fails as brittle. This is useful during regression because it means that the failures are not legitimate issues, and you can know to come back to it to fix it later when you have time.
• The "Watch me" status can be used to mark a test that may or may not be a legitimate failure that you simply want to keep an eye on over the next few runs (hence "Watch Me").
• The "Needs Review" status should be used to imply that either a test has become outdated and needs to be repaired - or that the failure is legitimate and a code change needs to be made.
• The "Addressed" status really comes in handy when there are multiple failures to sift through and you want an easy way to mark which tests you have already fixed - or at least attempted to fix.
• The "Clear Flag" option is used to simply clear any status that you may have applied to the result.

The end result of using the new status markers is that they can be seen when viewing the result history for a given test. If you've marked a test to be watched several times in the last few runs and see that it is consistently failing then you will know that you need to address the underlying issue as to why the test is failing - meaning something is probably broken in the code. Similarly, if you have repeatedly marked a test as being "Addressed" (fixed) and it is still failing then you know there is likely an issue with the environment or structuring of the test suite that is causing the failure. All of these are just suggestions based on the original intent of the status markers, but feel free to use them however they make sense for you! Also, you can use them to filter results when looking at test run results or in the result history screen.