diff --git a/AICONTROL/docs/results-file.md b/AICONTROL/docs/results-file.md new file mode 100644 index 0000000..0883813 --- /dev/null +++ b/AICONTROL/docs/results-file.md @@ -0,0 +1,131 @@ +# The AI → Business results file — draft v0.1 + +| | | +|---|---| +| **What this is** | The third handover in the Project Manual: "one file with a fixed set of columns: one row per controller, per scenario, per run. Business builds the euro comparison straight from it. Agree the columns before the first benchmark is run" | +| **Owner** | RL Evaluation seat (person 3). Reviewed by the lead 11 Sep 2026 from person 3's spreadsheet draft | +| **Next** | Encode as `aicontrol/evaluate/results_schema.py` (week 4), show to Business's Commercial Model Analyst, agree the open questions at the bottom, then freeze | +| **Format** | Parquet is the version of record; a CSV is written beside it so Business can double-click it. Rows are appended, never edited | + +## Rules that make the file usable by a script + +- Column names are lowercase `snake_case`, no spaces, no hyphens, and carry the unit where one + exists (`_kwh`, `_kg`, `_kw`). Fractions are stored as 0–1, never as percentages. +- One row is one run: one controller, on one scenario, with one seed. A controller is a *value* + in the `controller` column, not a row label, so the file grows to hundreds of rows. +- Controller names are fixed strings: `rule_based`, `mpc`, `perfect_knowledge`, `agent`, and + the Q1 stand-ins `random` and `do_nothing`. Business's S0 baseline (the hospital without the + hydrogen system) is not a controller but a plant configuration, so it is a *scenario* from + Simulations' library, appearing here as `scenario = no_hydrogen_` with + `controller = rule_based`. +- A column may be empty for a run (for example, `wandb_run` for a rule-based run). A column + that is empty for every run at the freeze is dropped. + +## Columns + +### Identity — which run this row is + +| column | type | meaning | +|---|---|---| +| `controller` | string | one of the fixed names above | +| `controller_version` | string | git commit for rule-based / MPC; model checkpoint tag for the agent | +| `scenario` | string | name of the scenario YAML file, e.g. `baseline_2018`, `blackout_72h`, `dark_december_week` | +| `data_year` | int | which year of data the run used | +| `held_out` | bool | true if the agent never saw this year in training; the manual's proof of generalisation | +| `run` | int | repetition index within a setup (1, 2, 3, …) | +| `seed` | int | the random seed of that repetition | +| `steps` | int | number of steps simulated | +| `step_minutes` | int | 5 | +| `twin_version` | string | version of the Simulations package that produced the physics | +| `parameter_book_version` | string | version of Energy Management's Parameter Book | +| `dataset_version` | string | version of the clean hospital dataset | +| `run_at` | timestamp | when the run finished (UTC) | +| `wandb_run` | string | Weights & Biases run id, if any | + +The three `*_version` columns are the manual's rule that "every result records which versions +produced it". + +### KPIs — the manual's four, plus money and carbon + +Names are fixed now; the exact equations come from Energy Management's Dispatch & Grid +Engineer and are applied by Simulations' KPI calculator. We copy the numbers; we do not +recompute them. + +| column | type | meaning | +|---|---|---| +| `self_sufficiency_rate` | fraction | share of load served without the grid | +| `grid_dependency_ratio` | fraction | share of load that came from the grid | +| `critical_load_uptime` | fraction | share of steps with Tier 1 fully served; must be 1.0 | +| `soc_violation_rate` | fraction | share of steps with the battery outside its window | +| `energy_cost` | float | net cost of grid electricity over the run (import cost minus export revenue) | +| `currency` | string | `USD` or `EUR`; see open questions | +| `co2_kg` | float | grid electricity used × carbon intensity, summed | + +### Energy totals — the evidence behind the KPIs, and what Business's register needs + +All summed over the run. + +| column | type | meaning | +|---|---|---| +| `load_kwh` | float | total hospital demand | +| `load_served_kwh` | float | demand actually supplied | +| `tier1_unserved_kwh` | float | **must be 0** in every run that counts | +| `tier2_shed_kwh` | float | essential load cut | +| `tier3_shed_kwh` | float | non-critical load cut | +| `pv_kwh` | float | solar available | +| `pv_curtailed_kwh` | float | solar thrown away | +| `grid_import_kwh` | float | | +| `grid_export_kwh` | float | | +| `grid_import_peak_kwh` | float | import during tariff peak hours | +| `grid_import_offpeak_kwh` | float | import during off-peak hours | +| `peak_import_kw` | float | highest grid import in any step; drives demand charges | +| `battery_charge_kwh` | float | | +| `battery_discharge_kwh` | float | | +| `battery_full_cycles` | float | throughput ÷ capacity; the wear proxy | +| `electrolyser_kwh` | float | electricity into the electrolyser | +| `electrolyser_offpeak_kwh` | float | the part of it in off-peak hours | +| `h2_produced_kg` | float | | +| `h2_consumed_kg` | float | | +| `fuel_cell_kwh` | float | electricity out of the fuel cell | +| `fuel_cell_heat_kwh` | float | recovered heat, if the twin models it; else empty | +| `chp_kwh` | float | electricity from the CHP plant | +| `electrolyser_starts` | int | on/off switches | +| `fuel_cell_starts` | int | on/off switches | +| `limit_violations` | int | steps where the twin clipped an action against a limit | +| `notes` | string | free text | + +Why the peak / off-peak and hydrogen columns: Business's cost register has seven rows marked +"ModelOutput / Missing" — off-peak share of hydrogen production, MWh shifted from peak to +off-peak, peak shaving, effective cost of electrolysis, cost per kWh from hydrogen, gas +displaced by fuel-cell heat, and the CO₂ change — and each is a ratio or difference of the +columns above between the no-hydrogen scenario, the `rule_based` rows and the `agent` rows. +The peak / off-peak definition is the tariff's, which Business and Energy Management own. + +## The companion file + +Business's register also asks for "simulation of net grid imports" as a series. That does not +fit one row per run. Each run therefore also writes its per-step table (the same quantities, +one row per 5-minute step) as Parquet under `results/runs/.parquet`. The summary file +above is the one they open; the per-step file is there when they need a chart. + +## Changes from person 3's spreadsheet draft + +- Added the `controller` header (the draft's column A had none) and made controller a column + value rather than a row label. +- Fixed `KPI_Grid_Dependecy_Ratio` → `grid_dependency_ratio`; dropped the `KPI_` prefix, + hyphens and capitals from every name so the file loads without renaming. +- Replaced `cost_eur` with `energy_cost` + `currency`, because the data source is ISO New + England, which prices in dollars, while the business case is in euros. +- Added identity columns (`held_out`, `data_year`, the three versions, `run_at`), the energy + totals and the switching counts. +- Kept `scenario`, `run`, `seed`, the four KPIs and `co2_kg` as drafted. + +## Open questions for Business (person 3 brings these) + +1. Currency: do they want dollars as recorded, or euros converted at a fixed rate they choose? +2. Peak / off-peak: their tariff hours, so the split columns match their model. +3. Are the seven register quantities computed by us (extra columns) or by them from these + columns? Either works; theirs keeps one owner per number. +4. What exactly their S0 baseline contains (no hydrogen only, or no battery either), so we can + ask Simulations for that scenario file. +5. Anything they need per run that is not here.