{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"Programming Tech Brief By HackerNoon","title":"Why Did the Test Fail? A Triage Rulebook, Tested on Two LLMs","html":"<iframe width=\"100%\" height=\"180\" frameborder=\"no\" scrolling=\"no\" seamless src=\"https://share.transistor.fm/e/7083a5a6\"></iframe>","width":"100%","height":180,"duration":756,"description":"\n        This story was originally published on HackerNoon at: https://hackernoon.com/why-did-the-test-fail-a-triage-rulebook-tested-on-two-llms.\nWhy did the test fail: the product, the test, the data or the environment? An evidence-based triage rulebook with rules and 21 labelled cases.\nCheck more stories related to programming at: https://hackernoon.com/c/programming.\n            You can also check exclusive content about #software-development, #software-testing, #qa, #test-automation, #llms, #flaky-tests, #ai-testing, #hackernoon-top-story,  and more.\nThis story was written by: @yevhen-bozhenko. Learn more about this writer by checking @yevhen-bozhenko's about page,\n            and for more stories, please visit hackernoon.com.\nA five-class taxonomy for deciding whether a failed test means the product, the test, the data or the environment is wrong, with the evidence rules that tell them apart — then checked by asking two LLMs to classify 21 labelled failures, with the rules and without.","thumbnail_url":"https://img.transistorcdn.com/KhCapPSRkLGL2Xw8888yuChkNRWthaKapLYTvNdu4W4/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS9zaG93/LzQxMTY2LzE2ODM1/ODIzMzAtYXJ0d29y/ay5qcGc.webp","thumbnail_width":300,"thumbnail_height":300}