H1 The scorer puts forecasts of known quality in the right order.
In the plan. The scorer ranks forecasts of known quality correctly.
- Question
- Does the scoring module rank forecasts of known quality correctly, and is each benchmark computed as specified?
- Test
- the synthetic suite the protocol requires; a tracked directive covers the +5 bp judge, if she keeps it.
- Kill criterion
- Any pre-registered check fails. Every registration is blocked until it passes.
- Owner
eleonorabjornberg/repo-market-model.- Step
- 3 of 6: Development; status open; the manual’s own step 5
- Milestones
- Progress markers
- Checks passed out of the total: not measured
- Development variants
- not measured (the running count belongs to the progress page, which is not built yet)
Research log for H1
- 2026-10-12, step 3, log entry, tier development development, not a verdict
under the pooled count,risk_gbm+every_day+gate+settle+h41+iorbhas 3.46 (worst single lead 1.64) andngboost_laplace2.34 (1.73), tier 1 no where it was stored as yes.risk_gbm+settle+h411.44 / 1.48 (same-row / own-leg), tier 1 yes.
Against kill: the old judge's counting was wrong in a way that changed tier 1 verdicts; consistent with H1's concern (the plan's Why section). - 2026-10-13, step 1, log entry, tier development development, not a verdict
the first check passes: every stated order is recovered under grid CRPS and the upper-tail twCRPS. The second check fails: the one-sided 95% bound covers 0.802 (percentile) and 0.825 (studentized) of 1,500 replications on the real series, against 0.937 and 0.941 on the control; 0.831 at 10,000 draws. The real series' left tail drives it (0.929 with its one September 2019 day removed).
Against kill: the second check failed as first written, and passes with the calibrated value; per Eleonora's ruling no candidate is registered until a tracked directive's power study confirms coverage. - 2026-10-13, step 1, log entry, tier development development, not a verdict provisional: calendar not read at the target day · register
the verdict reads the grid comparison's values, 2.43 at 250 days and 3.42 at 125 (B3 − B1 on five levels: 5.45 and 9.50); null false detections on the full development history 0.85% and 0.25% by 250 and 125 days (B3 − B1: 0.0%). Futility values 1.93, 1.56 and 1.45 at 60, 125 and 250 days. Power at these values stays low (14% for a 3% skill by 250 days), and the confirmation design goes to a separate study.
Against kill: the second check passes at the grid comparison's value; no registration until the confirmation-design study is decided. - 2026-10-14, step 2, log entry, tier development development, not a verdict
at the critical value in force (the largest across four comparisons) no design has power: D0 to D2 under 1% at 3% and at 10% even on the post-2020 history, D3 2.6% and 3.9% there; the standardized D4 reaches 14.4% and 19.4% on the post-2020 history but fails size (8% to 13%). At reading (b) D0 reproduces PR a tracked directive's Q-E table on the post-2020 history (14.3% and 19.7% against 14.0% and 19.25%) with size 11.5%; every design fails size on that history. No design reaches 80% power at 3% or at 30% skill. The §7 rule's pick (D3) is the least bad of the admissible, not a useful design.
Against kill: not applicable (H1 is a validity check, not a gain claim); the page's reading is that the confirmation design cannot confirm the 3% MME in 500 days as the test stands. - 2026-10-14, step 3, log entry, tier development development, not a verdict
docs/research/literature-2026-10.md. Power at 60 to 125 days depends mainly on the dependence and tail weight of the CRPS differences, which no test choice changes; the sequential tests need a declared cap and are weakest at horizons above 1. No source sizes a confirmation period for a CRPS comparison, and none quantifies a sign test under dependence: gaps.
Against kill: not applicable (reading only; development). - 2026-10-14, step 3, log entry, tier development development, not a verdict
the ruled alignment holds (anchor = last public row; B1 adds h-row changes; B2 = same-type trailing-year climatology), and three recorded mutations fail it. Coverage of the central 90% is below 0.90 at every lead: B1 0.804 [0.802, 0.805] at lead 1 to 0.889 [0.887, 0.890] at lead 3; B2 0.863 to 0.870. B1's shortfall is the one-row gap between its anchor and h-row changes; B2's sits in the rare day types. Detail indocs/research/benchmark-simulation-762.md.
Against kill: no pre-registered check failed; coverage is reported, not a check. B1's definition is Eleonora's, so the departure is a marked question. - 2026-10-15, step 3, log entry, tier development development, not a verdict
on the declared process the forecaster beats B1 by +17.58% [+17.33%, +17.84%] pooled, every panel at or above the 3% MME, all of it from spike anchors (ordinary 0.00%; month-end +52.3%, tax date +32.6%, quarter-end +84.5%); with no spikes it has no gain (-1.27% [-1.54%, -1.01%]). Candidate evidence (a) for a §13 change; B1, B2, B3 and the MME unchanged. a tracked directive's candidate against B1 by anchor type on the development days: pooled +22.3% [+13.7%, +29.5%], ordinary anchors +11.2% [+6.9%, +15.9%], month-end +34.2%, tax date +67.5%, quarter-end +65.9%. Detail indocs/research/spike-blind-741.md.
Against kill: not applicable (H1's checks are pre-registered; none failed, none run). - 2026-10-15, groundwork G1, log entry, tier development development, not a verdict
the conflict block that reached main on 15 October would now fail the check; the tracked Markdown has none.
Against kill: not applicable (groundwork). - 2026-10-15, step 3, log entry, tier development development, not a verdict provisional: calendar not read at the target day · register
only B3 against B1 is a known-better pair (skill 21.8%); B2 scores worse than B1. At that gain the current test flags 66% to 73% of windows but flags 26% to 33% of sign-flipped B3 windows, so its size on a symmetric null is far above 5%; the fixed-b test flags 27% to 34% of the real windows and 4% to 10% of the sign-flipped. Synthetic gated power at a 3% gain, 250 days: 10% (current), 8% (matched null), 3% (fixed-b), 0% (e-process, whose bounded bets never grow on these scores). No family reaches 80% power at any gain up to 50% in 250 days; the current test does at 50% in 500 days. By normal-theory arithmetic the daily signal-to-noise of B3's 22% gain needs about 1,300 days. The rule's pick (the e-process on lead 1) holds the size limit with 0% power.
Against kill: not applicable (H1 is a validity check; development). - 2026-10-15, step 1, log entry, tier development development, not a verdict
a true 3% skill is detected on 0.8% (full history, circular), 7.8% (post-2020-04, circular), 1.8% (full, stationary) and 13.9% (post-2020-04, stationary, a tracked directive's own) of paths by 250 days; none reaches 80% at either look, so days needed and the smallest skill detected at target power are null in every reading. Size with no true skill by 250 days: 0.5%, 6.1%, 1.3% and 10.9%. Detail indocs/research/power-study-769-result.md.
Against kill: not applicable (groundwork; development data gives no verdict). - 2026-10-15, groundwork G1, log entry, tier development development, not a verdict
a merge to main no longer makes the page stale or conflicting in an open PR; the page's content is still checked byte for byte. No published figure moves.
Against kill: not applicable (groundwork). - 2026-10-17, step 4, log entry, tier development development, not a verdict provisional: calendar not read at the target day · register
efficacy 2.55 and 1.35 at 125 and 250 days (grid) against 3.42 and 2.43 in force; B3 five levels 1.43 and 1.01 against 9.50 and 5.45; futility 2.44, 2.29 and 2.30 against 1.93, 1.56 and 1.45. Power of a true 3% by 250 days: 8.7%, 24.3%, 11.8% and 30.8% in readings (a) to (d), against 0.8%, 7.8%, 1.8% and 13.9%; size by 250 days 5.9%, 20.2%, 9.4% and 26.6% against 0.5%, 6.1%, 1.2% and 10.9%. Replay of B3 against B1 at the grid values: 59.0% of windows flagged against 31.5% in force; sign-flipped windows 14.2% against 2.5%. No reading reaches the 80% target.
Against kill: not applicable (H1 is a validity check; development and simulated data only). - 2026-10-17, groundwork G1, log entry, tier development development, not a verdict
every chapter, view, figure and record of the page is still on it, folded or in the contents list; no figure, record or rule moves. The taglines are drafted from the plan's headings for Eleonora's review.
Against kill: not applicable (groundwork). - 2026-10-17, groundwork G1, log entry, tier development development, not a verdict
no false archive alarm on a weekend or holiday; a failed archive on a decision day can no longer finish green. No record, archive or published figure moves.
Against kill: not applicable (groundwork). - 2026-10-17, step 5, log entry, tier development development, not a verdict provisional: calendar not read at the target day · register
at the new values the size by 250 days is 5.0% on post-2020 stationary paths (26.6% at a tracked directive's values) and below 5% on the other three readings; power at 3% by 250 days falls to 6.1% (30.8%), at 10% to 9.6% (39.8%). a tracked directive's rule recommends D4 (250 days, standardized) where it recommended D3; at the full-history circular values no design is admissible. Development and simulated data, not a verdict.
Against kill: not applicable (a design and calibration measurement; development data gives no verdict). - 2026-10-19, step 4, log entry, tier development development, not a verdict provisional: calendar not read at the target day · register
B1 reproduces a tracked directive within the Monte Carlo error of two independent draws (full, 250 days: 24.6% amended, 4.8% earlier).gbm_funding, full history, 250 days: 9.6% amended, 0.9% earlier; post-2020-04: 9.2% and 0.8%. At 125 days: 0.6% and 0.1% (full). Development data, not a verdict. Detail indocs/research/b2-leg-size-922-result.md.
Against kill: not applicable (a sizing of a rule, not a model). - 2026-10-19, step 4, log entry, tier development development, not a verdict
leg against B1 2.438 (250 days) and 4.727 (125); leg against B2 1.701 and 2.682. At 250 days the B2 leg's size forgbm_fundingis 4.5% (9.2% at the amended value) and the B1 leg's 0.1% (6.4%); the stand-ins sit at the nominal 4.8%. a tracked directive's post-2020-04 values (2.427 / 4.808) come from a cut at 2020-04-01 and are reproduced by it. Development data, not a verdict. Detail indocs/research/per-leg-calibration-929-result.md.
Against kill: not applicable (a calibration of a rule, not a model). - 2026-10-19, step 1, log entry, tier development development, not a verdict
no figure; nothing is scored. The verdict is unchanged on every fixture whatever B3 shows.
Against kill: not applicable (the output of the scorer; no hypothesis result). - 2026-10-19, step 3, log entry, tier development development, not a verdict provisional: calendar not read at the target day · register
against B1 the lead-averaged differentials have rho(1) 0.16 and an automatic block of 4.8 days; against B2 rho(1) 0.87, rho(10) 0.47 and 73.2 days pooled (5.8 to 47.5 within regime). Pooled lower bounds at block 5, 10 and 20: +0.241, +0.236, +0.226 against B1; +0.485, +0.458, +0.417 against B2. Only the 2025-26 bound against B2 changes sign with the block (+0.121, +0.002, −0.165). The block stays 10. Detail indocs/research/block-length-949-result.md.
Against kill: not applicable (a check on the interval, not a model). - 2026-10-19, step 3, log entry, tier development development, not a verdict provisional: calendar not read at the target day · register
at 250 days the contiguous c is 7.19 for B2, 1.92 forspread_mr_ctxand 2.07 forgbm_funding, against 3.10, 1.49, 1.57 (stationary-10) and 2.44, 1.38, 1.45 (circular-10); the series hold 5.7 non-overlapping 250-day windows. By the declared reading rules the 250-day design survives on the candidates' own d and fails on B2's. Last 2 years, NW(10) t against fixed-b t: 4.13 / 3.53 (spread_mr_ctx), 3.66 / 3.21 (gbm_funding). Detail indocs/research/contiguous-calibration-963-result.md.
Against kill: not applicable (a check on the confirmation design, not a model). - 2026-10-20, step 3, log entry, tier development development, not a verdict
the hand and the module agree to 9 decimal places at all 25 levels on every sampled day, lead and target; the last public row is the day before the decision day for all three. No disagreement, so nothing is marked for Eleonora. One recorded mutation (B2 ignoring the day type) fails the test.
Against kill: not applicable (a check on the benchmarks, not a model). - 2026-10-20, step 2, log entry, tier development development, not a verdict provisional: calendar not read at the target day · register
the scale buys nothing under contiguous calibration. D4 (250 days, scaled) has size 0.0% and gated power 0.1% at 10% true skill (raw D0: 1.4%); no design on the grid reaches 80% power. The contiguous critical values are 4.3 at 250 days and 15.7 at the 125-day look, against 2.4 in force. The scaled score swaps the order of the two candidates in two regimes (2018-19 from 2019-07, and 2021-23, where the scaled means are a tie at +0.0316), so the misranking demonstration holds, thinly. Detail indocs/research/scaled-score-967-result.md.
Against kill: the declared rule "the scale buys nothing" fires (power at 10% below 20%); the rule "the misranking fails" does not. Development data, never a verdict. - 2026-10-20, step 3, log entry, tier development development, not a verdict provisional: calendar not read at the target day · register
at 250 days the contiguous c is 3.50 forspread_mr_ctxand 3.86 forgbm_funding(125 days: 6.61 and 6.46), against 1.92 and 2.07 on the leg against B1. By a tracked directive's reading rule the B1 leg survives and the B2 leg is between 3.0 and 4.0 (neither rule applies), so the joint reading is neither. Size at the value in force is 10.6% and 13.6% at 250 days, 18.1% forgbm_fundingat a tracked directive's draft value. Detail indocs/research/b2-leg-contiguous-1015-result.md.
Against kill: not applicable (a check on the confirmation design, not a model). - 2026-10-20, step 2, log entry, tier development development, not a verdict provisional: calendar not read at the target day · register
D4 (250 days, scaled) has size 1.1% and gated power 3.4% at 10% true skill (raw D0: 0.2%; a tracked directive: 0.1%); D5 (375 days) 6.7%. The contiguous critical values at the last two looks are 12.67 and 2.69 for D4 (a tracked directive: 15.66 and 4.32). No design on the grid reaches 80% power. The scaled score still swaps the order of the two candidates in 2018-19 and 2021-23. Detail indocs/research/scaled-score-1013-result.md.
Against kill: "the scale buys nothing" still fires (power at 10% below 20%); "the misranking fails" does not. Development data, never a verdict. - 2026-10-20, step 4, log entry, tier development development, not a verdict
calibrated here 4.359 (125 days) and 2.554 (250); the a tracked directive draft detects 22.9% and 35.0% of null paths by 125 and 250 days, the a tracked directive draft 11.4% and 21.9%, the value in force 0.0% and 0.1%. Detail indocs/research/b3-orientation-1014-result.md.
Against kill: unchanged. Development data, never a verdict. - 2026-10-21, step 1, log entry, tier development development, not a verdict
the set of classified functions is the same before and after (7 library entry points, 2 script entry points, 52 not-entry functions, 44 CRPS functions; every kind and reason compares equal). A new study declares itself in its own script and no longer edits the shared test. Of the last 50 merged pull requests 4 edited that file, and all 4 edits were one classification entry.
Against kill: not applicable (a guard, not a model). - 2026-10-21, step 1, log entry, tier development development, not a verdict
thepagesjob runs the same four checks; nothing is scored.
Against kill: not applicable; no score exists. - 2026-10-21, step 1, log entry, tier development development, not a verdict
the worker lives inrepo_model.reliability_regimes, so the pool works under fork and spawn and the cells are identical for 1 and 2 workers under both. No published figure moves.
Against kill: not applicable (infrastructure, not a model). - 2026-10-21, step 1, log entry, tier development development, not a verdict
the record and the code agree: grid 3.419 (125 days) and 2.426 (250 days), five-level 9.501 and 5.455, no per-leg value.
Against kill: not applicable; no score exists. - 2026-10-21, step 1, log entry, tier development development, not a verdict
a delegated-review merge can no longer add or change a decision record, and so cannot open the live-monitoring gate (live_monitor.require_decision_record), without her own GitHub action. No result, record or published figure is edited.
Against kill: not applicable (a guard, not a model). - 2026-10-22, step 3, log entry, tier development development, not a verdict provisional: calendar not read at the target day · register
a tracked directive reproduced to 1e-9 (4.8 against B1, 73.2 against B2, 1,868 days). Without 2019-09-18 the B1 leg's block moves 4.8 to 7.4 and the B2 leg's stays at 73. Post-2020-04 (1,429 days) the B1 leg's block is 10.2 and the B2 leg's 64.9 (89.1 widened); the proxy's 63.8 (97.9 widened) against Lens C's 108, with its ACF and long-run ratios reproduced closely. The block stays 10; the choice is a marked question for her (section 13). Detail indocs/research/block-length-1022-result.md.
Against kill: not applicable (a check on the interval, not a model). - 2026-10-22, step 1, log entry, tier development development, not a verdict
the workers live inrepo_model.confirmation_workersandrepo_model.scaled_score_workers, and the scripts import them. A fixed-seed run of the calibration, the design values and the path workers on a constructed design is byte-identical before and after. No published record or declaration changes.
Against kill: not applicable (infrastructure, not a model). - 2026-10-22, step 3, log entry, tier development development, not a verdict
the 2018-19 and 2025-26 cells equal a tracked directive's to the last digit (50 point values compared, none differs), so a tracked directive reproduces on the Mac; the 2020-24 cell of a tracked directive splits into 2020 (3 pressure days), 2021-23 (0) and 2024 (2), so the five-regime table has no scored cell there, and the 2020 cells carry no interval. Of 10 scored regime-by-lead cells per row (2018-19 and 2025-26), above zero in 5 for mean reversion, 5 for context-pooled residuals, 0 for two-part, 1 for gbm_funding; none below zero. The gain sits in 2018-19 and is small in 2025-26. Detail indocs/research/ordinary-day-p5-1023-result.md.
Against kill: not a kill test and no verdict (development data, scoring protocol v1). No pooled gain is claimed. The cells carry a tracked directive's percentile interval; on Eleonora's ruling of 22 October 2026 (comment 6340379028) each is also read on the protocol's studentized one-sided bound (scoring protocol section 5, over target days, Newey-West lag 10 on the AUROC difference's influence series). The bound column is computed by the re-score theneeds-mac:comment on a tracked directive asks for; until then the page says it is pending. - 2026-10-23, step 3, log entry, tier development development, not a verdict
each cell with an interval now carries "studentized lower bound x" beside a tracked directive's percentile interval. Cells whose one-sided studentized lower bound is above zero, out of 10 cells that have one: mean reversion 5, context-pooled residuals 3, two-part 0, gbm_funding 1. These count cells; they set no threshold. Read on the mean-d / studentized bound as ruled in a tracked directive. Detail indocs/research/ordinary-day-p5-1023-result.md.
Against kill: not a kill test and no verdict (development data, scoring protocol v1). No pooled gain is claimed. The page stays provisional under the register (row 10, a tracked directive, class b).