ALLSHIFT/docs
pepe 72dd781dbc Organize documentation into docs/ and superseded/
Audit every document in the repository, convert the non-markdown ones into
markdown reports, and split current documentation from outdated material.

docs/ — 31 markdown documents in seven numbered sections. Twenty are new
reports generated from .docx / .pdf / .xlsx / .mlx / .m sources that were
previously unreadable in the browser and undiffable in git. Each report
carries a provenance block (source path, format, MD5) and links back to its
original; all 13 recorded checksums verify against the files on disk.
Machine-extraction losses (PDF table column interleaving, Word OMML
equations, embedded figures) are called out explicitly rather than silently
smoothed over.

superseded/ — outdated material with a documented reason per entry:
two byte-identical ClickUp re-exports, an older revision of the BIDMC/UCSD
energy-flow doc (the retained copy adds the SoC Violation Rate KPI), a
duplicate of Shift input data.docx, the May 2026 simulation plan, the
root PV+Battery.md now covered by a fuller report, GitHub's stock
demo-repository template, and a zero-byte placeholder. Its README also
records what was deliberately NOT retired and why — the "Old Frameworks"
and "Old Simulations" folders hold unique Simulink revisions, and
"Big Ugly Folder" holds the only copy of framework revision 1.3.

Findings worth flagging, all documented in the reports:
- Simulink lineage recovered from each .slx's internal coreProperties.xml
  revision counter. The current model is
  Current Framework/Bobert0206_Initial_Simulation_Framework.slx (rev 2.7);
  the top-level copy is rev 1.3, five revisions behind.
- Simulations/Constants.m is a truncated byte-prefix of the Current
  Framework copy, silently missing H2_leak, H2_cap and E_H2_vol_h.
- The PEM electrolyser and fuel cell are unmodified MathWorks Simscape
  examples still at vendor defaults; the "10x bigger" sizing TODO recorded
  in Constants.m was never carried out.
- controller-claude.m does not compile — undefined P_Electro_max, outputs
  unassigned on several paths.
- The specification set uses two incompatible variable naming conventions
  and disagrees on action-space size (5 vs 16).
- MA_hourly_load.csv (13.7 MB) is the same 35,040 rows as 89993-0.parquet
  (2.4 MB).
- Clinical data is the MIMIC-IV *demo* (ODbL, 100 patients), not full
  MIMIC-IV — redistributable, but the licence and citation are unrecorded.

Housekeeping: untrack 21 Simulink build artefacts (slprj/, *.slxc) and add
ignore rules for them. Root README rewritten around the new layout.

Recruitment notes naming individual candidates are excluded from version
control via .gitignore rather than committed; the generic question template
is kept in docs/07-team-and-operations/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-25 21:20:33 -07:00
..
01-project Organize documentation into docs/ and superseded/ 2026-07-25 21:20:33 -07:00
02-specifications Organize documentation into docs/ and superseded/ 2026-07-25 21:20:33 -07:00
03-energy-management Organize documentation into docs/ and superseded/ 2026-07-25 21:20:33 -07:00
04-simulations Organize documentation into docs/ and superseded/ 2026-07-25 21:20:33 -07:00
05-business-economics Organize documentation into docs/ and superseded/ 2026-07-25 21:20:33 -07:00
06-data Organize documentation into docs/ and superseded/ 2026-07-25 21:20:33 -07:00
07-team-and-operations Organize documentation into docs/ and superseded/ 2026-07-25 21:20:33 -07:00
_originals Organize documentation into docs/ and superseded/ 2026-07-25 21:20:33 -07:00
README.md Organize documentation into docs/ and superseded/ 2026-07-25 21:20:33 -07:00

SHIFT Documentation

Every current document in the project, in markdown, in one place.

Documents that arrived as Word, PDF, Excel, or MATLAB Live Script have a markdown report here — readable in the browser, diffable in git, searchable with grep. The originals are kept alongside (root-level ones in _originals/, the rest in place under Shift Matlab Drive/); each report links back to its source and records the file's MD5.

Outdated material lives in ../superseded/, with the reason for each retirement.


Start here

If you want to… Read
Understand what SHIFT is building Project Explanation
Know what the simulator must accept and return Simulator I/O Interface
Know what the RL agent sees, does, and is scored on State · Action · Reward
Find which Simulink model is the live one Simulink Model Inventory
Know what data exists and where it came from Dataset Inventory
See what is unresolved Open questions across the project

01 · Project

Document What it covers
Project Explanation Vision, mission, values, the five departments, benchmarking against Tibo/Zympler and the H2B2PowiDian Rijnstate pilot, business case and routes to market
Work Packages WP1WP7: literature review, forecasting, simulation, RL control, evaluation, extensions, reporting
AI Efficiency Improvements Where the losses are — ~3040% electrolyser, ~40% fuel cell — and the scheduling changes that attack them
Reference Links Four external sources: two IEEE/ScienceDirect papers, a review, and the H2B2PowiDian announcement

Note

Work Packages WP6 describes "a basic crop growth/CO₂ enrichment module", "greenhouse-to-grid interactions", and "scalability tests for multiple greenhouses". SHIFT builds a hospital microgrid. WP6 is a leftover from an earlier greenhouse concept and does not describe current scope. It is kept rather than retired because WP1WP5 and WP7 are current and still the project's work breakdown — but WP6 needs rewriting, not reading.

02 · Specifications

The contract layer. These define what gets built.

Document Source Status
Simulator I/O Interface PDF, 10 pp Draft v0.1 — 8 open questions
RL State Variables Word 32 variables, no units
RL Action Variables Word 16 actions; 9 have no simulator counterpart
RL Reward Function Word 26 terms, no weights
Forecasting Requirements PDF, 10 pp Complete; PV, demand, and price forecasters
Rule-Based Controller markdown The baseline policy RL must beat

Warning

The specification set is not internally consistent. Two naming conventions are in use — the Simulator I/O sheet says SoC(t), P_PV, u_ele; the RL documents say Battery_SOC, PV_Generation, Electrolyzer_PowerSetpoint — and neither references the other. The action spaces disagree in size (5 vs 16), and five reward terms need quantities the simulator does not return. Each report documents its own mismatches; resolving them is a decision, not an edit.

03 · Energy Management

Case-study data and equipment for the two reference sites: BIDMC (Beth Israel Deaconess Medical Center, Boston) and UCSD (UC San Diego).

Document What it covers
BIDMC/UCSD Energy Flow and Balances Four-level dispatch priority, three-tier load model, energy balance equations, four KPI formulae
UCSD Equipment Manual DFC3000 fuel cell, BYD battery, 30 MW CHP, planned PEM electrolyser — each parameter tagged State / Action / Reward
BIDMC Datasets Manual Full column reference for the three MIMIC-IV demo clinical files and the synthetic building-energy CSV
BIDMC ComStock Load Profile The hospital load proxy: 8,026 MWh/yr electricity, 15-min resolution, full end-use breakdown
BIDMC Digital Twin Readiness Checklist 8-section readiness gate. Sections 13 scored; 48 blank
PV Hourly Dataset Manual The recommended PV dataset
Satellite PV Dataset Manual The 167.9 MB NSRDB meteorological dataset — detailed, but not recommended for PV rates

04 · Simulations

Document What it covers
Simulink Model Inventory Every .slx, with real lineage from Simulink's internal revision metadata. Identifies the current model
Rule-Based Controller Implementation controller-claude.m — the dispatch cascade in MATLAB, and why it does not yet run
PEM Electrolyser and Fuel Cell Models The MathWorks Simscape packages and their full parameter sets
PV + Battery Simulink Model The Excel→Simulink vertical slice, and its known time-dependency bug
MATLAB Live Scripts All five .mlx files transcribed
Generation Profiles Workbook Both sheets of the profiles spreadsheet, plus the PowerWorld results analysis
PowerWorld Microgrid Model The campus-scale network study
Shift Input Data The per-component variable list the simulator contract grew from
Existing EST Model The vendored MIT-licensed TU/e model, and SHIFT's gravity-storage experiments on top

05 · Business & Economics

Document What it covers
Cost Analysis A 48-row parameter register — value, source, why it matters, and a Confirmed / Estimated / Derived / Missing status per row. The most rigorous single artefact in the project
Social Cost-Benefit Analysis SCBA methodology: the questions a robust analysis must answer, discounting, NPV in Excel

The SCBA document is methodology, not a SHIFT analysis — it is course material on how to run an SCBA, worked through a generic wind-farm example. The SHIFT-specific analysis has not been written yet. The parameter register in Cost Analysis is what would feed it.

06 · Data

Document What it covers
Dataset Inventory Every data file: size, rows, provenance, licence obligations, and the redundancies

07 · Team & Operations

Document What it covers
Interview Questions The recruitment question template — general, technical, business, PR
Team Design Polo/sweatshirt and business card design, logo concept

Open questions across the project

Collected from the individual reports. Each links to where it is documented in full.

Blocking

Question Where
Timestep: hourly or sub-hourly? Affects RL compute cost, battery fast-response fidelity, and every energy/power conversion Simulator I/O §0
Reward weights: 26 terms, no relative magnitudes beyond "critical load dominates" RL Reward
Naming convention: two incompatible variable vocabularies across the spec set RL State
Critical / non-critical load split: can the hospital profile actually be split, or is a proxy fraction needed? Simulator I/O §5
PEM models are still at vendor defaults — the "10× bigger" sizing TODO was never done, and the fuel cell still runs an automotive drive cycle PEM Models
The dispatch controller does not compile — undefined variables, outputs unassigned on several paths Controller Implementation
Readiness checklist §4 (Safety) is unscored — the document's own rule makes any FAIL here a deployment blocker Readiness Checklist

Data

Question Where
No measured hospital load — every figure is simulated or synthetic Dataset Inventory
No price or carbon-intensity series — both are required simulator inputs Dataset Inventory
PV array configuration is inconsistent — 4 kW / 100 kWp / 460 kW across three documents Dataset Inventory
Boston or Netherlands? — all datasets are Boston; the price forecaster targets the Dutch market Dataset Inventory
Licence and citation files missing for MIMIC-IV demo (ODbL), ComStock, NSRDB, PVWatts Dataset Inventory

Modelling

Question Where
Tank model — isothermal + ideal gas, or full pressure/temperature dynamics? Simulator I/O §5
Battery degradation — modelled or ignored? Simulator I/O §5
Should simultaneous electrolyser + fuel cell operation be structurally forbidden, or merely penalised? Simulator I/O §5
Two-tier or three-tier load model? The specs and the energy-flow doc disagree Energy Flow
Is the clinical/patient layer in scope at all? BIDMC Datasets
Was chal's _CLEAR_CONNECTIONS cleanup (rev 2.6) ever merged into rev 2.7? Simulink Inventory

Conventions used in these reports

Every report generated from a non-markdown source opens with a provenance block:

Field Meaning
Source Repo-relative path to the original file
Format File type and size
MD5 Checksum at the time of writing — lets you tell whether the original has changed since
Status Current, superseded, or draft
Report generated The date this markdown was produced

Where machine extraction was lossy — PDF tables that interleave columns, Word equations, images — the report says so explicitly and points back to the original. Reconstructed tables are labelled as reconstructed. Author asides and first-person notes from the sources are preserved as quotes rather than smoothed away, because they usually mark exactly the parts that are uncertain.

Observations that go beyond the source — inconsistencies between documents, unit errors, scale mismatches — are marked with callouts and attributed to this audit, not to the original author.