This story was originally published on HackerNoon at:
https://hackernoon.com/why-you-shouldnt-judge-by-pnl-alone.
PnL can lie. This hands-on guide shows traders how hypothesis testing separate luck from edge, with a Python example and tips on how not to fool yourself.
Check more stories related to data-science at:
https://hackernoon.com/c/data-science.
You can also check exclusive content about
#quantitative-research,
#trading,
#algorithmic-trading,
#pnl,
#udge-pnl,
#profit-and-loss,
#judge-profit-and-loss,
#hackernoon-top-story, and more.
This story was written by:
@ruslan4ezzz. Learn more about this writer by checking
@ruslan4ezzz's about page,
and for more stories, please visit
hackernoon.com.
I’ve spent years building and evaluating systematic strategies across highly adversarial markets. When you iterate on a trading system, PnL is the goal but a terrible day-to-day signal. It’s too noisy, too path-dependent, and too easy to cherry-pick. A simple framework—form a hypothesis, measure a test statistic, translate it into a probability under a “no-effect” world (the p-value)—helps you avoid false wins, iterate faster, and ship changes that actually stick. Below I’ll show a concrete example where two strategies look very different in cumulative PnL charts, yet standard tests say there’s no meaningful difference in their average per-trade outcome. I’ll also demystify the t-test in plain language: difference of means, scaled by uncertainty.