Science
Forecasting that shows its working.
Explore how HerPattern learns personal symptom rhythms, represents uncertainty, and measures performance against simple baselines.
See persistence, switching and uncertainty move together.
Adjust the three inputs. The simulation updates instantly so the model's behaviour is visible rather than hidden.
- Regime switches in 60 days
- 14
- Illustrative 80% band width
- ±1.76 points
- Simulated mean burden
- 4.87 / 10
Illustrative simulation only. It is a toy AR(1) process with two latent states, not the hierarchical Bayesian autoregression or switching state-space model used for real forecasts — those run in the external research service and are never reproduced in the browser.
Hierarchical Bayesian autoregression
Each participant's day-to-day symptom burden is modeled as an autoregressive process, but the per-participant parameters are partially pooled across the population using a hierarchical Bayesian model. This lets HerPattern personalize quickly: new participants borrow strength from population-level patterns, and the model shifts toward an individual's own history as more days are logged.
Switching state-space model
Symptom burden does not move smoothly all the time. People experience distinct latent states (e.g. calmer stretches vs. higher-burden stretches) with transitions between them. A switching state-space layer recovers these latent states and their transition probabilities, which are combined with the autoregressive component into the final probabilistic forecast.
Evaluation protocol
Models are evaluated using a leave-one-participant-out population fit combined with held-out early-history calibration and rolling-origin evaluation, so forecasts are always tested against data the model was not trained on, at multiple points across a participant's timeline, to avoid overstating performance.
Metrics are compared against documented baselines, including simple persistence, personal mean, and cycle-day heuristics, at 1, 2 and 3 day horizons. See the published metrics below for the current published metrics.
Uncertainty
Every forecast is reported with an 80% prediction interval and a plain-language confidence label, not just a point estimate. Uncertainty widens automatically when history is short, when recent logging has gaps, or when a participant's pattern is inherently more variable.
Limitations
- Observational associations are not causal effects.
- Cohorts evaluated to date are small and do not establish clinical generalizability.
- Synthetic smoke-test results are execution evidence only, not clinical performance. See the benchmark results below for current evaluation status.
- Forecasts can be, and sometimes will be, wrong. They are one input among many, not a substitute for clinical judgment.
Medical boundaries
HerPattern never uses diagnostic or treatment language. It does not diagnose endometriosis or any other condition, detect medical flares, predict ovulation, or recommend medications or treatments. Forecasts describe the likelihood of low, moderate, or high self-reported symptom burden only. Read the full medical disclaimer.
Open benchmark
A reproducible, public evaluation.
Synthetic execution results demonstrate that the benchmark pipeline runs. They do not constitute clinical evidence.
No clinical validation is claimed. Clinical performance requires authorized-access, real-data evaluation, which is still pending.
Evaluation methodology
The task is next-day forecasting of a participant’s overall symptom-burden score, together with a discrete symptom-pattern state and a prediction interval. Each forecast is produced from that participant’s history only up to the day before the target day, so no future information enters the prediction.
Evaluation is prospective and per participant: the model is fitted on the calibration window, then rolled forward one day at a time across the remaining history. A participant enters the evaluation only after meeting the minimum calibration requirement of 42 days; days with missing check-ins are skipped rather than imputed. A run is valid only when the same feature-contract and parameter-schema versions listed in the release summary were used end to end.
Every model is scored against the same reference baseline, persistence, on identical rows and horizons (1, 2, 3 day). Reported statistics are aggregates over all evaluated participants; individual-level records are never published. Statistical fitting runs entirely in the external modelling service — this page and the application only read published results.
Release summary
Benchmark version
0.1.0
Active model — public name
HerPattern Bayesian switching
Active model — machine name
herpattern_bayesian_switching
Model version
0.1.0
Feature-contract version
1.0
Parameter-schema version
1.0
Personalization mode
Partial pooling (hierarchical Bayesian) with switching state-space
Evaluation protocol
leave-one-participant-out population fit plus held-out early-history calibration and rolling-origin evaluation
Supported horizons
1, 2, 3 day(s)
Supported symptom roster
Per-participant chip roster (personalized) — see model card
Coverage requirements
≥ 42 calibration days
Interval method
80% bootstrap prediction interval
Calibration status
synthetic-smoke-test-only
Latest evaluation date
2026-07-19T11:47:16.882956+00:00
Dataset category
synthetic-smoke-test
Synthetic or real data
Synthetic
Reproducibility status
Not yet release-ready (6 open blocker(s))
Git commit
Pinned per release — see repository
Published aggregate metrics
All published rows are shown, including horizons and baselines where HerPattern does not lead. Nothing is filtered.
| Model | Horizon | MAE | MAE 95% CI | RMSE | Macro F1 | 80% coverage | vs persistence |
|---|---|---|---|---|---|---|---|
| HerPattern Bayesian switching | 1d | 0.047 | 0.037–0.058 | 0.065 | 0.856 | 90.0% | 7.2% |
| Persistence | 1d | 0.051 | 0.043–0.061 | 0.069 | 0.856 | 86.7% | 0.0% |
| Personal mean | 1d | 0.163 | 0.141–0.189 | 0.193 | 0.318 | 36.7% | -218.1% |
| Cycle-day baseline | 1d | 0.172 | 0.148–0.194 | 0.213 | 0.322 | 37.5% | -235.6% |
| HerPattern Bayesian switching | 2d | 0.085 | 0.075–0.094 | 0.115 | 0.758 | 79.2% | 3.9% |
| Persistence | 2d | 0.089 | 0.079–0.098 | 0.120 | 0.751 | 79.2% | 0.0% |
| Personal mean | 2d | 0.157 | 0.136–0.175 | 0.187 | 0.339 | 49.2% | -77.2% |
| Cycle-day baseline | 2d | 0.165 | 0.145–0.185 | 0.201 | 0.346 | 54.2% | -86.9% |
| HerPattern Bayesian switching | 3d | 0.124 | 0.110–0.141 | 0.164 | 0.652 | 71.7% | 1.8% |
| Persistence | 3d | 0.127 | 0.113–0.146 | 0.168 | 0.658 | 77.5% | 0.0% |
| Personal mean | 3d | 0.167 | 0.148–0.189 | 0.194 | 0.309 | 59.2% | -31.9% |
| Cycle-day baseline | 3d | 0.176 | 0.160–0.193 | 0.208 | 0.342 | 58.3% | -39.0% |
Metric definitions
Each metric below is computed over the published prediction rows for the stated horizon. Definitions are given so the table can be read without opening the repository.
- MAE(burden points, 0–10 scale)
- Mean absolute error between the predicted next-day symptom-burden score and the observed score. Lower is better.
- MAE 95% CI(burden points)
- Bootstrap 95% confidence interval around the reported MAE. Intervals that do not overlap indicate a difference the evidence supports.
- RMSE(burden points)
- Root mean squared error, which penalises large misses more heavily than MAE. Lower is better.
- Macro F1(0–1)
- Macro-averaged F1 over the discrete symptom-pattern states, weighting each state equally regardless of how often it occurs.
- 80% coverage(percentage of days)
- Share of days where the observed value fell inside the published 80% prediction interval. Well-calibrated uncertainty sits near 80%.
- vs persistence(percent change in MAE)
- Relative MAE change against the persistence baseline. Positive values mean lower error than the baseline.
Every model row is compared against the same reference baseline, persistence — tomorrow predicted as a repeat of today.
Cite this benchmark
Version and date below come from the release currently loaded on this page.
Plain citation
HerPattern. HerPattern open benchmark (version 0.1.0), 2026-07-19. https://herpattern.space/benchmark
BibTeX
@misc{herpattern_open_benchmark,
title = {HerPattern open benchmark},
author = {HerPattern},
year = {2026},
version = {0.1.0},
howpublished = {\url{https://herpattern.space/benchmark}},
note = {Benchmark repository: https://github.com/khrystynaklochko/herpattern-open-benchmark}
}Limitations
- Observational associations are not causal effects.
- The cohort is small and does not establish clinical generalizability.
- Synthetic smoke-test results are execution evidence only, not clinical performance.
Open release blockers
- mcphases_aggregate_results_present
- ki_endolist_aggregate_results_present
- nhanes_coverage_audit_present
- live_deployment_evidence_present
- demo_video_evidence_present
- edge_parameters_are_not_real-data-validated