This story was originally published on HackerNoon at:
https://hackernoon.com/why-did-the-test-fail-a-triage-rulebook-tested-on-two-llms.
Why did the test fail: the product, the test, the data or the environment? An evidence-based triage rulebook with rules and 21 labelled cases.
Check more stories related to programming at:
https://hackernoon.com/c/programming.
You can also check exclusive content about
#software-development,
#software-testing,
#qa,
#test-automation,
#llms,
#flaky-tests,
#ai-testing,
#hackernoon-top-story, and more.
This story was written by:
@yevhen-bozhenko. Learn more about this writer by checking
@yevhen-bozhenko's about page,
and for more stories, please visit
hackernoon.com.
A five-class taxonomy for deciding whether a failed test means the product, the test, the data or the environment is wrong, with the evidence rules that tell them apart — then checked by asking two LLMs to classify 21 labelled failures, with the rules and without.