{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"Programming Tech Brief By HackerNoon","title":"Anatomy of a Silent 502: How Four HAProxy Characters Exposed a K8s Upload Failure","html":"<iframe width=\"100%\" height=\"180\" frameborder=\"no\" scrolling=\"no\" seamless src=\"https://share.transistor.fm/e/bd93e341\"></iframe>","width":"100%","height":180,"duration":616,"description":"\n        This story was originally published on HackerNoon at: https://hackernoon.com/anatomy-of-a-silent-502-how-four-haproxy-characters-exposed-a-k8s-upload-failure.\nA production file-upload incident returned HAProxy 502 errors while ingress-nginx, APISIX, and application logs stayed empty. H\nCheck more stories related to programming at: https://hackernoon.com/c/programming.\n            You can also check exclusive content about #kubernetes, #haproxy, #nginx-ingress, #http-client, #production-incident-debugging, #production-file-upload-failure, #502-bad-gateway, #silent-502,  and more.\nThis story was written by: @abhishekpareek. Learn more about this writer by checking @abhishekpareek's about page,\n            and for more stories, please visit hackernoon.com.\nA production file-upload failure returned `502 Bad Gateway` responses from HAProxy, while ingress-nginx, APISIX, and the application showed no matching requests. The key clues were HAProxy’s `SH--` termination state and two unrelated requests failing after exactly `60.132` seconds, pointing to a deterministic server-side timeout rather than random network loss.\n\nThe existing 20-minute `proxy_*` timeouts were misleading because they governed traffic from ingress-nginx to APISIX, not requests arriving from HAProxy. Increasing `client_header_timeout` from 60 to 300 seconds on the affected Ingress stopped the failures.\n\nThe fix was applied through a targeted `server-snippet` instead of changing the shared ingress controller globally. This reduced the blast radius but introduced a known security tradeoff because custom snippets can be dangerous in multi-tenant environments.\n\nThe main lesson: a 502 identifies where the error was reported, not necessarily where it originated. HAProxy termination flags, exact timing patterns, and connection direction often reveal more than the status code itself.","thumbnail_url":"https://img.transistorcdn.com/KhCapPSRkLGL2Xw8888yuChkNRWthaKapLYTvNdu4W4/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS9zaG93/LzQxMTY2LzE2ODM1/ODIzMzAtYXJ0d29y/ay5qcGc.webp","thumbnail_width":300,"thumbnail_height":300}