PINE LIBRARY
Updated EdgeStats
Library "EdgeStats"
A win rate on its own is not evidence. This library supplies the four things that turn one into a claim you can defend, none of which Pine ships: a base rate to subtract, a sample size corrected for overlapping forward windows, a confidence interval that behaves at small n, and a p-value that knows how many settings you tried before you picked this one.
The argument in three lines, all from the same 60 wins out of 100:
assess(60, 100, 0.5, horizon = 1) p = 0.046 significant
assess(60, 100, 0.5, horizon = 10) p = 0.527 not significant
assess(60, 100, 0.5, horizon = 10, trials = 30) p = 1.000 nothing at all
Nothing changed about the data. What changed is being honest that ten-bar forward returns sampled every bar are not a hundred independent observations, and that the best of thirty settings is not the same evidence as the only setting you tried.
WHAT THE DEMO SHOWS
Added to a chart directly, the library grades an ordinary signal: close above a 50 EMA, judged on whether price is higher ten bars later, over the last 500 bars. On BTCUSD 1h at the time of writing that is a hit rate of 39.9% against a base rate of 50.4%, an edge of -10.5 percentage points, and a two-sided p of 0.297.
Read that carefully, because it is the whole point. The signal looks bad. It is not reliably bad. Twenty-five independent observations cannot separate -10.5 points from noise, and the interval runs from 23.3% to 59.3%. A tool that says "I cannot tell" when it cannot tell is the only kind worth having.
THREE HONEST CAVEATS
n / horizon is a rough correction, not a theorem. It assumes overlap is the dominant source of dependence between observations. Where returns are autocorrelated beyond the window it is still optimistic. Treat it as a floor on your uncertainty rather than a ceiling.
zFor bisects normCdf, which is itself an approximation, so it inherits that error: zFor(0.95) lands about 1.2e-6 below the textbook 1.9599640. Irrelevant in practice, but it is an approximation of an approximation and you should hear that from me rather than discover it.
roll() uses ta.cum internally, so its call site must execute on every bar. Called inside "if barstate.islast" it has one bar of history and returns nonsense, and no max_bars_back setting repairs that. This is a property of Pine functions rather than of this library, and it is worth knowing generally.
VERIFICATION
Every fixed-input value is plotted to the Data Window and two are printed on the chart, so you can check the arithmetic rather than trust it. Against Python statistics.NormalDist:
normCdf(1.96) 0.9750022 true 0.9750021
normCdf(-1.0) 0.1586553
zFor(0.95) 1.9599628 true 1.9599640
zFor(0.99) 2.5758313 true 2.5758293
wilson(60, 100, 1.96) [0.5020026, 0.6905987]
selectionAdjusted(0.05, 30) 0.7853612
Corrections welcome, particularly to the effective sample size treatment, which is the part I would most like to be wrong about.
REFERENCE
normCdf(x)
Standard normal cumulative distribution. Abramowitz and Stegun 26.2.17, absolute error below 7.5e-8 across the whole real line.
Parameters:
x (float): Value to evaluate.
Returns: Probability that a standard normal variate is at most x.
zFor(conf)
Two-sided z multiplier for a confidence level. Bisects normCdf, so any level works rather than a lookup of the usual three.
Parameters:
conf (float): Confidence level in (0, 1). 0.95 returns 1.9599628.
Returns: The z for which the central interval of that width has the given coverage.
remark Converged to float precision against normCdf, which is itself an approximation, so the result inherits its error: zFor(0.95) lands about 1.2e-6 below the textbook 1.9599640. Irrelevant for anything you would do with it, but it is an approximation of an approximation and worth saying so.
nEff(n, horizon)
Effective independent sample size when observations use overlapping forward windows.
Parameters:
n (float): Raw observation count.
horizon (int): Length in bars of the forward window each observation measures.
Returns: n divided by the horizon, with the horizon floored at 1.
wilson(hits, n, z)
Wilson score interval for a proportion. Unlike the normal approximation it stays inside [0, 1] and stays sane when n is small or the rate sits near an edge.
Parameters:
hits (float): Successful observations.
n (float): Total observations. Pass an effective count here, not a raw bar count, when the windows overlap.
z (float): Multiplier from zFor().
Returns: A [lower, upper] tuple on the proportion, or [na, na] when there is no sample.
selectionAdjusted(p, trials)
Sidak correction. If you searched k settings and reported the best one, the p-value you found is not the p-value that best one deserves.
Parameters:
p (float): Uncorrected two-sided p-value.
trials (int): Settings, symbols or variants searched before this one was chosen. Pass 1 if you did not search.
Returns: Probability of seeing something at least this good in k independent tries.
roll(src, len)
Rolling window sum valid from the first bar, unlike math.sum which stays na until the window fills. Useful for counting events over a lookback.
Parameters:
src (float): Series to accumulate.
len (simple int): Window length in bars.
Returns: Sum of the last len values of src.
remark Uses ta.cum internally, so the CALL SITE must execute on every bar. Called inside `if barstate.islast` it has one bar of history and returns nonsense. That is a property of Pine functions rather than of this library, and no max_bars_back setting repairs it. len is `simple` so Pine can size the history buffer at compile time.
assess(hits, n, base, horizon, conf, trials)
The whole assessment in one call.
Parameters:
hits (float): Observations where the signal was right.
n (float): Total observations.
base (float): Rate at which the same outcome occurred unconditionally over the same horizon. This is the number that makes an edge an edge.
horizon (int): Bars in the forward window. Overlapping windows shrink the effective sample.
conf (float): Confidence level for the interval, default 0.95.
trials (int): Settings searched before choosing this one, default 1.
Returns: A Verdict.
describe(v)
One line of plain English for a Verdict, sized to drop straight into a table cell.
Parameters:
v (Verdict): The Verdict to describe.
Returns: A human-readable summary, or "no sample" when there is nothing to say.
Verdict
Everything needed to decide whether a measured hit rate means anything.
Fields:
rate (series float): Observed hit rate, 0 to 1.
base (series float): Base rate the signal is measured against, 0 to 1.
edge (series float): rate minus base, in percentage points.
n (series float): Raw observation count as supplied.
nEff (series float): Observation count after the overlapping-window correction.
lo (series float): Lower confidence bound on rate, computed on nEff.
hi (series float): Upper confidence bound on rate, computed on nEff.
z (series float): Test statistic of rate against base.
p (series float): Two-sided p-value, already Sidak-adjusted for the trials argument.
clears (series bool): True when the interval on the rate excludes the base rate.
A win rate on its own is not evidence. This library supplies the four things that turn one into a claim you can defend, none of which Pine ships: a base rate to subtract, a sample size corrected for overlapping forward windows, a confidence interval that behaves at small n, and a p-value that knows how many settings you tried before you picked this one.
The argument in three lines, all from the same 60 wins out of 100:
assess(60, 100, 0.5, horizon = 1) p = 0.046 significant
assess(60, 100, 0.5, horizon = 10) p = 0.527 not significant
assess(60, 100, 0.5, horizon = 10, trials = 30) p = 1.000 nothing at all
Nothing changed about the data. What changed is being honest that ten-bar forward returns sampled every bar are not a hundred independent observations, and that the best of thirty settings is not the same evidence as the only setting you tried.
WHAT THE DEMO SHOWS
Added to a chart directly, the library grades an ordinary signal: close above a 50 EMA, judged on whether price is higher ten bars later, over the last 500 bars. On BTCUSD 1h at the time of writing that is a hit rate of 39.9% against a base rate of 50.4%, an edge of -10.5 percentage points, and a two-sided p of 0.297.
Read that carefully, because it is the whole point. The signal looks bad. It is not reliably bad. Twenty-five independent observations cannot separate -10.5 points from noise, and the interval runs from 23.3% to 59.3%. A tool that says "I cannot tell" when it cannot tell is the only kind worth having.
THREE HONEST CAVEATS
n / horizon is a rough correction, not a theorem. It assumes overlap is the dominant source of dependence between observations. Where returns are autocorrelated beyond the window it is still optimistic. Treat it as a floor on your uncertainty rather than a ceiling.
zFor bisects normCdf, which is itself an approximation, so it inherits that error: zFor(0.95) lands about 1.2e-6 below the textbook 1.9599640. Irrelevant in practice, but it is an approximation of an approximation and you should hear that from me rather than discover it.
roll() uses ta.cum internally, so its call site must execute on every bar. Called inside "if barstate.islast" it has one bar of history and returns nonsense, and no max_bars_back setting repairs that. This is a property of Pine functions rather than of this library, and it is worth knowing generally.
VERIFICATION
Every fixed-input value is plotted to the Data Window and two are printed on the chart, so you can check the arithmetic rather than trust it. Against Python statistics.NormalDist:
normCdf(1.96) 0.9750022 true 0.9750021
normCdf(-1.0) 0.1586553
zFor(0.95) 1.9599628 true 1.9599640
zFor(0.99) 2.5758313 true 2.5758293
wilson(60, 100, 1.96) [0.5020026, 0.6905987]
selectionAdjusted(0.05, 30) 0.7853612
Corrections welcome, particularly to the effective sample size treatment, which is the part I would most like to be wrong about.
REFERENCE
normCdf(x)
Standard normal cumulative distribution. Abramowitz and Stegun 26.2.17, absolute error below 7.5e-8 across the whole real line.
Parameters:
x (float): Value to evaluate.
Returns: Probability that a standard normal variate is at most x.
zFor(conf)
Two-sided z multiplier for a confidence level. Bisects normCdf, so any level works rather than a lookup of the usual three.
Parameters:
conf (float): Confidence level in (0, 1). 0.95 returns 1.9599628.
Returns: The z for which the central interval of that width has the given coverage.
remark Converged to float precision against normCdf, which is itself an approximation, so the result inherits its error: zFor(0.95) lands about 1.2e-6 below the textbook 1.9599640. Irrelevant for anything you would do with it, but it is an approximation of an approximation and worth saying so.
nEff(n, horizon)
Effective independent sample size when observations use overlapping forward windows.
Parameters:
n (float): Raw observation count.
horizon (int): Length in bars of the forward window each observation measures.
Returns: n divided by the horizon, with the horizon floored at 1.
wilson(hits, n, z)
Wilson score interval for a proportion. Unlike the normal approximation it stays inside [0, 1] and stays sane when n is small or the rate sits near an edge.
Parameters:
hits (float): Successful observations.
n (float): Total observations. Pass an effective count here, not a raw bar count, when the windows overlap.
z (float): Multiplier from zFor().
Returns: A [lower, upper] tuple on the proportion, or [na, na] when there is no sample.
selectionAdjusted(p, trials)
Sidak correction. If you searched k settings and reported the best one, the p-value you found is not the p-value that best one deserves.
Parameters:
p (float): Uncorrected two-sided p-value.
trials (int): Settings, symbols or variants searched before this one was chosen. Pass 1 if you did not search.
Returns: Probability of seeing something at least this good in k independent tries.
roll(src, len)
Rolling window sum valid from the first bar, unlike math.sum which stays na until the window fills. Useful for counting events over a lookback.
Parameters:
src (float): Series to accumulate.
len (simple int): Window length in bars.
Returns: Sum of the last len values of src.
remark Uses ta.cum internally, so the CALL SITE must execute on every bar. Called inside `if barstate.islast` it has one bar of history and returns nonsense. That is a property of Pine functions rather than of this library, and no max_bars_back setting repairs it. len is `simple` so Pine can size the history buffer at compile time.
assess(hits, n, base, horizon, conf, trials)
The whole assessment in one call.
Parameters:
hits (float): Observations where the signal was right.
n (float): Total observations.
base (float): Rate at which the same outcome occurred unconditionally over the same horizon. This is the number that makes an edge an edge.
horizon (int): Bars in the forward window. Overlapping windows shrink the effective sample.
conf (float): Confidence level for the interval, default 0.95.
trials (int): Settings searched before choosing this one, default 1.
Returns: A Verdict.
describe(v)
One line of plain English for a Verdict, sized to drop straight into a table cell.
Parameters:
v (Verdict): The Verdict to describe.
Returns: A human-readable summary, or "no sample" when there is nothing to say.
Verdict
Everything needed to decide whether a measured hit rate means anything.
Fields:
rate (series float): Observed hit rate, 0 to 1.
base (series float): Base rate the signal is measured against, 0 to 1.
edge (series float): rate minus base, in percentage points.
n (series float): Raw observation count as supplied.
nEff (series float): Observation count after the overlapping-window correction.
lo (series float): Lower confidence bound on rate, computed on nEff.
hi (series float): Upper confidence bound on rate, computed on nEff.
z (series float): Test statistic of rate against base.
p (series float): Two-sided p-value, already Sidak-adjusted for the trials argument.
clears (series bool): True when the interval on the rate excludes the base rate.
Release Notes
v2 - correctness fixes found in an adversarial review after v1 went out. Nothing in the exported API changed shape: same functions, same type, same fields.WHAT WAS WRONG
1. clears and p could contradict each other. p was Sidak-adjusted for `trials`; the confidence interval was not, so `clears` came from an uncorrected interval. assess(60, 100, 0.5, 1, 0.95, 30) returned p = 0.753 and clears = true, and describe() printed both in the same line. The interval is now computed at a Sidak-adjusted level, conf^(1/trials), which makes `clears` exactly equivalent to p < 1 - conf. They can no longer disagree.
2. Invalid input produced confident answers instead of na.
base outside [0, 1] was accepted, and a 1e-10 floor on the variance turned it into z = -90000, p = 0, clears = true. assess(60, 100, 1.5) reported an impossible base rate as overwhelmingly significant. Now na. Where base is exactly 0 or 1 the variance is zero and no test exists, so z and p are na while the rate, interval and edge are still reported.
hits > n and hits < 0 were not checked. assess(150, 100, 0.5) reported edge = +100 pp with a plausible-looking interval, because wilson() clamps the proportion into [0, 1] and laundered the bad input. Now na.
horizon < 1 was clamped to 1, so assess(..., horizon = 0) returned the most optimistic possible sample size. For a library about not overstating your sample, that was the wrong direction to fail in. Now na.
3. wilson() with a negative z returned the bounds inverted, lower above upper. z is now taken as an absolute value.
4. describe() said "no sample" when the sample was fine and only the base rate was missing. It now distinguishes the two, and says "no test possible with these inputs" rather than "not distinguishable from chance" when no test was actually run.
5. roll()"s note claimed `simple` lets Pine size the history buffer at compile time. `simple` resolves at the start of the run; only `const` is compile time. A large len from input.int() can still exhaust the buffer, and the importer has to raise max_bars_back on their own indicator() call. Corrected.
6. The demo gradability filter was off by one, bar_index > H rather than >=, costing one observation in 500.
WHAT THE DESCRIPTION COULD NOT SAY
The description locks 15 minutes after publication and this did not make it in. assess() treats `base` as known exactly, which is the one-sample score test. If you estimate the base from the same bars that produced your events, as the demo does, two things follow and neither is in your favour. The base rate has its own sampling error and it is being discarded, so p is optimistic. And because the event bars sit inside the base window, the base is pulled toward the event rate, so the reported edge understates the signal-versus-everything-else difference by roughly the fraction of bars the signal covers. A signal firing on 60% of bars shows you about 40% of the gap. Both effects are small when events are rare and large when they are not. A two-proportion form is the honest fix and is not in this version. The source header now says all of this.
VERIFICATION
Four new fixed-input self tests are plotted to the Data Window, all reproduced against Python statistics.NormalDist:
assess(60,100,.5,h=1,k=30) p 0.7526703 CI [0.4441431, 0.7379410] clears false
assess(60,100,1.5).p na
assess(150,100,.5).rate na
assess(60,100,.5,h=0).nEff na
Every self-test value from v1 is unchanged, so the three-line argument in the description still holds exactly.
Release Notes
v3 - documentation only. No code change; every function is byte-identical to v2.Correcting the remark on roll(). Version 2 warned that a long len could exhaust the history buffer and told you to raise max_bars_back on the indicator doing the importing. That was a guess, and it does not reproduce.
Measured on a 22,683 bar chart with no max_bars_back set anywhere: roll(1.0, 500) returned 500.00, roll(1.0, 2000) returned 2000.00, roll(1.0, 4900) returned 4900.00 - exact in all three cases. The test was built so that failure would be loud: with src = 1.0 on every bar, ta.cum(src) is bar_index + 1, so a starved buffer makes nz(c[len]) fall back to 0 and roll returns 22,683 instead of len. It did not. And since Pine caps max_bars_back at 5000, there is no useful length at which the old advice would have applied anyway. Removed rather than left standing.
The two parts of that remark that are true stay in: len is simple rather than series, so the offset is fixed for the whole run - simple resolves at the start of the run, not at compile time, only const is compile time. And the call site must run on every bar, because a history read inside a user-defined function keeps per-call-site history and starves when the caller is gated on something like barstate.islast.
Pine library
In true PulseWire spirit, the author has published this Pine code as an open-source library so that other Pine programmers from our community can reuse it. Cheers to the author! You may use this library privately or in other open-source publications, but reuse of this code in publications is governed by House Rules.
I build Pine Script tooling. Quanticator turns plain-English descriptions into Pine Script v6 source - quanticator.app
Disclaimer
The information and publications are not meant to be, and do not constitute, financial, investment, trading, or other types of advice or recommendations supplied or endorsed by PulseWire. Read more in the Terms of Use.
Pine library
In true PulseWire spirit, the author has published this Pine code as an open-source library so that other Pine programmers from our community can reuse it. Cheers to the author! You may use this library privately or in other open-source publications, but reuse of this code in publications is governed by House Rules.
I build Pine Script tooling. Quanticator turns plain-English descriptions into Pine Script v6 source - quanticator.app
Disclaimer
The information and publications are not meant to be, and do not constitute, financial, investment, trading, or other types of advice or recommendations supplied or endorsed by PulseWire. Read more in the Terms of Use.