Great Seneca Creek Record

How the expected-score model works

Context Trained on 7,332 surveys at 1,162 sites, it explains 39% of the variation at sites it never saw.

0.39held-out R², sensitive share
0.02season and form alone
How each survey is judged against what its setting predictsSeasonSurvey formWatershed land coverSoil, elevation, climateRecent rain and heatWater temperature and flowTeam sizeModeltrained on 7,332 surveysat 1,162 other sites(never sees the year)Expected range for a surveyexpectedfoundgap = found − expectedas expected2005–092010–142015–192020–242025–26Site 1's median gap, by five-year periodHeld-out accuracy: R² 0.39, against 0.02 for season and form alone
How the model is used. The line is Site 1's median gap in each five-year period.

For every usable biological survey, the model predicts what it should find given its season, survey form, watershed (paved and urban share, forest, farmland, soils, elevation, climate), rain and air temperature in the week before, water temperature, flow and team. It never sees the year, so change over time stays in the gap between observed and expected.

Every expected value comes from a version of the model that never saw any survey from that site (five folds grouped by site), which is how a site is judged by what the rest of the country predicts for it.

PredictsSurveysSitesHeld-out R²Season and form onlyTypical error
Share of groups found that are pollution-sensitive7,3321,1620.390.028 pts
Score percentile within its survey form7,2941,1470.380.0018 percentile pts
Drop in held-out accuracy (R²) when each group of inputs is scrambled; larger means the model leans on it more.

Rainfall in the week before each survey is included. Volunteer data is noisy, so an R² near 0.4 at unseen sites is useful: enough to set an expected range for each survey and to tell a change in the stream from a change in the season or team.

Related

How this was worked out

scikit-learn HistGradientBoostingRegressor (500 iterations, learning rate 0.04, 15 leaves, at least 40 surveys per leaf); importance by scrambling each input group on held-out folds.