# SHIFT Documentation

Every current document in the project, in markdown, in one place.

Documents that arrived as Word, PDF, Excel, or MATLAB Live Script have a **markdown report**
here — readable in the browser, diffable in git, searchable with `grep`. The originals are kept
alongside (root-level ones in [`_originals/`](_originals/), the rest in place under
`Shift Matlab Drive/`); each report links back to its source and records the file's MD5.

Outdated material lives in [`../superseded/`](../superseded/README.md), with the reason for
each retirement.

---

## Start here

| If you want to… | Read |
|---|---|
| Understand what SHIFT is building | [Project Explanation](01-project/project-explanation.md) |
| Know what the simulator must accept and return | [Simulator I/O Interface](02-specifications/simulator-io-interface.md) |
| Know what the RL agent sees, does, and is scored on | [State](02-specifications/rl-state-variables.md) · [Action](02-specifications/rl-action-variables.md) · [Reward](02-specifications/rl-reward-function.md) |
| Find which Simulink model is the live one | [Simulink Model Inventory](04-simulations/simulink-model-inventory.md) |
| Know what data exists and where it came from | [Dataset Inventory](06-data/dataset-inventory.md) |
| See what is unresolved | [Open questions across the project](#open-questions-across-the-project) |

---

## 01 · Project

| Document | What it covers |
|---|---|
| [Project Explanation](01-project/project-explanation.md) | Vision, mission, values, the five departments, benchmarking against Tibo/Zympler and the H2B2–PowiDian Rijnstate pilot, business case and routes to market |
| [Work Packages](01-project/work-packages.md) | WP1–WP7: literature review, forecasting, simulation, RL control, evaluation, extensions, reporting |
| [AI Efficiency Improvements](01-project/ai-efficiency-improvements.md) | Where the losses are — ~30–40% electrolyser, ~40% fuel cell — and the scheduling changes that attack them |
| [Reference Links](01-project/reference-links.md) | Four external sources: two IEEE/ScienceDirect papers, a review, and the H2B2–PowiDian announcement |

> [!NOTE]
> [Work Packages](01-project/work-packages.md) WP6 describes *"a basic crop growth/CO₂
> enrichment module"*, *"greenhouse-to-grid interactions"*, and *"scalability tests for multiple
> greenhouses"*. SHIFT builds a **hospital** microgrid. WP6 is a leftover from an earlier
> greenhouse concept and does not describe current scope. It is kept rather than retired
> because WP1–WP5 and WP7 are current and still the project's work breakdown — but WP6 needs
> rewriting, not reading.

## 02 · Specifications

The contract layer. These define what gets built.

| Document | Source | Status |
|---|---|---|
| [Simulator I/O Interface](02-specifications/simulator-io-interface.md) | PDF, 10 pp | **Draft v0.1** — 8 open questions |
| [RL State Variables](02-specifications/rl-state-variables.md) | Word | 32 variables, no units |
| [RL Action Variables](02-specifications/rl-action-variables.md) | Word | 16 actions; 9 have no simulator counterpart |
| [RL Reward Function](02-specifications/rl-reward-function.md) | Word | 26 terms, **no weights** |
| [Forecasting Requirements](02-specifications/forecasting-requirements.md) | PDF, 10 pp | Complete; PV, demand, and price forecasters |
| [Rule-Based Controller](02-specifications/rule-based-controller.md) | markdown | The baseline policy RL must beat |

> [!WARNING]
> **The specification set is not internally consistent.** Two naming conventions are in use —
> the Simulator I/O sheet says `SoC(t)`, `P_PV`, `u_ele`; the RL documents say `Battery_SOC`,
> `PV_Generation`, `Electrolyzer_PowerSetpoint` — and neither references the other. The action
> spaces disagree in size (5 vs 16), and five reward terms need quantities the simulator does
> not return. Each report documents its own mismatches; resolving them is a decision, not an
> edit.

## 03 · Energy Management

Case-study data and equipment for the two reference sites: **BIDMC** (Beth Israel Deaconess
Medical Center, Boston) and **UCSD** (UC San Diego).

| Document | What it covers |
|---|---|
| [BIDMC/UCSD Energy Flow and Balances](03-energy-management/bidmc-ucsd-energy-flow-and-balances.md) | Four-level dispatch priority, three-tier load model, energy balance equations, four KPI formulae |
| [UCSD Equipment Manual](03-energy-management/ucsd-equipment-manual.md) | DFC3000 fuel cell, BYD battery, 30 MW CHP, planned PEM electrolyser — each parameter tagged State / Action / Reward |
| [BIDMC Datasets Manual](03-energy-management/bidmc-datasets-manual.md) | Full column reference for the three MIMIC-IV demo clinical files and the synthetic building-energy CSV |
| [BIDMC ComStock Load Profile](03-energy-management/bidmc-parquet-load-data-manual.md) | The hospital load proxy: 8,026 MWh/yr electricity, 15-min resolution, full end-use breakdown |
| [BIDMC Digital Twin Readiness Checklist](03-energy-management/bidmc-digital-twin-readiness-checklist.md) | 8-section readiness gate. **Sections 1–3 scored; 4–8 blank** |
| [PV Hourly Dataset Manual](03-energy-management/pv-hourly-dataset-manual.md) | The recommended PV dataset |
| [Satellite PV Dataset Manual](03-energy-management/satellite-pv-dataset-manual.md) | The 167.9 MB NSRDB meteorological dataset — detailed, but not recommended for PV rates |

## 04 · Simulations

| Document | What it covers |
|---|---|
| [Simulink Model Inventory](04-simulations/simulink-model-inventory.md) | Every `.slx`, with **real lineage** from Simulink's internal revision metadata. Identifies the current model |
| [Rule-Based Controller Implementation](04-simulations/rule-based-controller-implementation.md) | `controller-claude.m` — the dispatch cascade in MATLAB, and why it does not yet run |
| [PEM Electrolyser and Fuel Cell Models](04-simulations/pem-electrolyser-and-fuel-cell-models.md) | The MathWorks Simscape packages and their full parameter sets |
| [PV + Battery Simulink Model](04-simulations/pv-battery-simulink.md) | The Excel→Simulink vertical slice, and its known time-dependency bug |
| [MATLAB Live Scripts](04-simulations/matlab-live-scripts.md) | All five `.mlx` files transcribed |
| [Generation Profiles Workbook](04-simulations/generation-profiles-workbook.md) | Both sheets of the profiles spreadsheet, plus the PowerWorld results analysis |
| [PowerWorld Microgrid Model](04-simulations/powerworld-microgrid.md) | The campus-scale network study |
| [Shift Input Data](04-simulations/shift-input-data.md) | The per-component variable list the simulator contract grew from |
| [Existing EST Model](04-simulations/existing-est-model.md) | The vendored MIT-licensed TU/e model, and SHIFT's gravity-storage experiments on top |

## 05 · Business & Economics

| Document | What it covers |
|---|---|
| [Cost Analysis](05-business-economics/cost-analysis.md) | A 48-row parameter register — value, source, why it matters, and a **Confirmed / Estimated / Derived / Missing** status per row. The most rigorous single artefact in the project |
| [Social Cost-Benefit Analysis](05-business-economics/social-cost-benefit-analysis.md) | SCBA methodology: the questions a robust analysis must answer, discounting, NPV in Excel |

> The SCBA document is **methodology, not a SHIFT analysis** — it is course material on how to
> run an SCBA, worked through a generic wind-farm example. The SHIFT-specific analysis has not
> been written yet. The parameter register in [Cost Analysis](05-business-economics/cost-analysis.md)
> is what would feed it.

## 06 · Data

| Document | What it covers |
|---|---|
| [Dataset Inventory](06-data/dataset-inventory.md) | Every data file: size, rows, provenance, licence obligations, and the redundancies |

## 07 · Team & Operations

| Document | What it covers |
|---|---|
| [Interview Questions](07-team-and-operations/interview-questions.md) | The recruitment question template — general, technical, business, PR |
| [Team Design](07-team-and-operations/team-design.md) | Polo/sweatshirt and business card design, logo concept |

---

## Open questions across the project

Collected from the individual reports. Each links to where it is documented in full.

### Blocking

| Question | Where |
|---|---|
| **Timestep**: hourly or sub-hourly? Affects RL compute cost, battery fast-response fidelity, and every energy/power conversion | [Simulator I/O §0](02-specifications/simulator-io-interface.md#timestep) |
| **Reward weights**: 26 terms, no relative magnitudes beyond "critical load dominates" | [RL Reward](02-specifications/rl-reward-function.md#open-issues-with-this-reward-design) |
| **Naming convention**: two incompatible variable vocabularies across the spec set | [RL State](02-specifications/rl-state-variables.md#naming-mismatch-with-the-simulator-io-contract) |
| **Critical / non-critical load split**: can the hospital profile actually be split, or is a proxy fraction needed? | [Simulator I/O §5](02-specifications/simulator-io-interface.md#5-open-questions) |
| **PEM models are still at vendor defaults** — the "10× bigger" sizing TODO was never done, and the fuel cell still runs an automotive drive cycle | [PEM Models](04-simulations/pem-electrolyser-and-fuel-cell-models.md#the-unfinished-sizing-work) |
| **The dispatch controller does not compile** — undefined variables, outputs unassigned on several paths | [Controller Implementation](04-simulations/rule-based-controller-implementation.md#blocking-defects) |
| **Readiness checklist §4 (Safety) is unscored** — the document's own rule makes any FAIL here a deployment blocker | [Readiness Checklist](03-energy-management/bidmc-digital-twin-readiness-checklist.md#4-operational-constraints--safety--the-boundary) |

### Data

| Question | Where |
|---|---|
| **No measured hospital load** — every figure is simulated or synthetic | [Dataset Inventory](06-data/dataset-inventory.md#gaps) |
| **No price or carbon-intensity series** — both are required simulator inputs | [Dataset Inventory](06-data/dataset-inventory.md#gaps) |
| **PV array configuration is inconsistent** — 4 kW / 100 kWp / 460 kW across three documents | [Dataset Inventory](06-data/dataset-inventory.md#pvwatts_hourlycsv--058-mb) |
| **Boston or Netherlands?** — all datasets are Boston; the price forecaster targets the Dutch market | [Dataset Inventory](06-data/dataset-inventory.md#gaps) |
| **Licence and citation files missing** for MIMIC-IV demo (ODbL), ComStock, NSRDB, PVWatts | [Dataset Inventory](06-data/dataset-inventory.md#clinical-data--mimic-iv-demo) |

### Modelling

| Question | Where |
|---|---|
| Tank model — isothermal + ideal gas, or full pressure/temperature dynamics? | [Simulator I/O §5](02-specifications/simulator-io-interface.md#5-open-questions) |
| Battery degradation — modelled or ignored? | [Simulator I/O §5](02-specifications/simulator-io-interface.md#5-open-questions) |
| Should simultaneous electrolyser + fuel cell operation be structurally forbidden, or merely penalised? | [Simulator I/O §5](02-specifications/simulator-io-interface.md#5-open-questions) |
| Two-tier or three-tier load model? The specs and the energy-flow doc disagree | [Energy Flow](03-energy-management/bidmc-ucsd-energy-flow-and-balances.md#notes-on-scope) |
| Is the clinical/patient layer in scope at all? | [BIDMC Datasets](03-energy-management/bidmc-datasets-manual.md#how-this-fits-the-wider-project) |
| Was `chal`'s `_CLEAR_CONNECTIONS` cleanup (rev 2.6) ever merged into rev 2.7? | [Simulink Inventory](04-simulations/simulink-model-inventory.md#findings) |

---

## Conventions used in these reports

Every report generated from a non-markdown source opens with a provenance block:

| Field | Meaning |
|---|---|
| **Source** | Repo-relative path to the original file |
| **Format** | File type and size |
| **MD5** | Checksum at the time of writing — lets you tell whether the original has changed since |
| **Status** | Current, superseded, or draft |
| **Report generated** | The date this markdown was produced |

Where machine extraction was lossy — PDF tables that interleave columns, Word equations, images
— the report says so explicitly and points back to the original. Reconstructed tables are
labelled as reconstructed. Author asides and first-person notes from the sources are preserved
as quotes rather than smoothed away, because they usually mark exactly the parts that are
uncertain.

Observations that go beyond the source — inconsistencies between documents, unit errors,
scale mismatches — are marked with callouts and attributed to this audit, not to the original
author.