pepe
72dd781dbc
Audit every document in the repository, convert the non-markdown ones into markdown reports, and split current documentation from outdated material. docs/ — 31 markdown documents in seven numbered sections. Twenty are new reports generated from .docx / .pdf / .xlsx / .mlx / .m sources that were previously unreadable in the browser and undiffable in git. Each report carries a provenance block (source path, format, MD5) and links back to its original; all 13 recorded checksums verify against the files on disk. Machine-extraction losses (PDF table column interleaving, Word OMML equations, embedded figures) are called out explicitly rather than silently smoothed over. superseded/ — outdated material with a documented reason per entry: two byte-identical ClickUp re-exports, an older revision of the BIDMC/UCSD energy-flow doc (the retained copy adds the SoC Violation Rate KPI), a duplicate of Shift input data.docx, the May 2026 simulation plan, the root PV+Battery.md now covered by a fuller report, GitHub's stock demo-repository template, and a zero-byte placeholder. Its README also records what was deliberately NOT retired and why — the "Old Frameworks" and "Old Simulations" folders hold unique Simulink revisions, and "Big Ugly Folder" holds the only copy of framework revision 1.3. Findings worth flagging, all documented in the reports: - Simulink lineage recovered from each .slx's internal coreProperties.xml revision counter. The current model is Current Framework/Bobert0206_Initial_Simulation_Framework.slx (rev 2.7); the top-level copy is rev 1.3, five revisions behind. - Simulations/Constants.m is a truncated byte-prefix of the Current Framework copy, silently missing H2_leak, H2_cap and E_H2_vol_h. - The PEM electrolyser and fuel cell are unmodified MathWorks Simscape examples still at vendor defaults; the "10x bigger" sizing TODO recorded in Constants.m was never carried out. - controller-claude.m does not compile — undefined P_Electro_max, outputs unassigned on several paths. - The specification set uses two incompatible variable naming conventions and disagrees on action-space size (5 vs 16). - MA_hourly_load.csv (13.7 MB) is the same 35,040 rows as 89993-0.parquet (2.4 MB). - Clinical data is the MIMIC-IV *demo* (ODbL, 100 patients), not full MIMIC-IV — redistributable, but the licence and citation are unrecorded. Housekeeping: untrack 21 Simulink build artefacts (slprj/, *.slxc) and add ignore rules for them. Root README rewritten around the new layout. Recruitment notes naming individual candidates are excluded from version control via .gitignore rather than committed; the generic question template is kept in docs/07-team-and-operations/. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
111 lines
6.3 KiB
Markdown
111 lines
6.3 KiB
Markdown
# RL Action Variables
|
|
|
|
> **Markdown report of a non-markdown source document.**
|
|
>
|
|
> | | |
|
|
> |---|---|
|
|
> | **Source** | [`docs/_originals/ACTION VARIABLES AND DESCRIPTIONS.docx`](../_originals/ACTION%20VARIABLES%20AND%20DESCRIPTIONS.docx) |
|
|
> | **Format** | Microsoft Word (.docx), 16.2 kB |
|
|
> | **MD5** | `9939b9c2f76fbb5bf54fbfeff25a31f4` |
|
|
> | **Owner** | AI & Control Systems cluster |
|
|
> | **Status** | Current — but broader than the simulator currently accepts |
|
|
> | **Report generated** | 2026-07-25 |
|
|
|
|
## What this document is
|
|
|
|
The action space for the reinforcement-learning controller: every lever the agent is allowed to
|
|
pull at each timestep. Sixteen actions, spanning direct power setpoints, discrete mode
|
|
selections, and demand-side commands.
|
|
|
|
## Action variables
|
|
|
|
### Generation and storage setpoints
|
|
|
|
| Action | Description |
|
|
|---|---|
|
|
| `PV_Setpoint` | Command that limits or caps the PV inverter output to avoid overgeneration, match load, or reduce battery stress. |
|
|
| `Battery_ChargePower` | Amount of power (kW) the controller instructs the battery to absorb; used when excess solar or cheap grid power is available. |
|
|
| `Battery_DischargePower` | Amount of power (kW) the controller instructs the battery to release; used to support critical loads, reduce grid cost, or stabilize voltage. |
|
|
| `Battery_Mode` | Direct mode selection (charge, discharge, idle) that constrains what the battery is allowed to do in the next control interval. |
|
|
|
|
### Hydrogen chain
|
|
|
|
| Action | Description |
|
|
|---|---|
|
|
| `Electrolyzer_PowerSetpoint` | Electrical power sent to the electrolyzer to control hydrogen production; higher power increases H₂ output but consumes more energy and heats the stack. |
|
|
| `Electrolyzer_Mode` | Operating mode such as standby, active production, cooldown, or shutdown; controls whether the electrolyzer is allowed to start producing hydrogen. |
|
|
| `FuelCell_PowerSetpoint` | Power command that determines how much hydrogen is converted back into electricity; used to supply critical loads during low-PV or grid failure conditions. |
|
|
| `FuelCell_Mode` | Operational mode (idle, ramp-up, nominal, emergency) that governs stack usage and prevents rapid degradation. |
|
|
| `HydrogenTank_ValveCommand` | Opening/closing command controlling the flow of hydrogen between the tank and fuel cell or between tank and buffer systems. |
|
|
|
|
### Grid exchange
|
|
|
|
| Action | Description |
|
|
|---|---|
|
|
| `Grid_ImportPower` | Amount of electrical power drawn from the grid; controlled to minimize cost, follow load, or maintain battery SOC. |
|
|
| `Grid_ExportPower` | Amount of power sent back to the grid; used when PV or fuel cell production exceeds local demand and export is economically or operationally beneficial. |
|
|
|
|
### Demand side
|
|
|
|
| Action | Description |
|
|
|---|---|
|
|
| `Load_SheddingCommand` | Decision to temporarily turn off certain non-critical loads to preserve battery SOC, maintain resilience, or avoid overloading. |
|
|
| `Load_ShiftingCommand` | Reschedules flexible loads (e.g., HVAC pre-cooling, non-critical equipment) to times of high solar availability or low grid pricing. |
|
|
| `ThermalManagement_Command` | Controls cooling or heating for PV inverters, electrolyzers, battery packs, and fuel cells to maintain safe temperatures. |
|
|
|
|
### Supervisory
|
|
|
|
| Action | Description |
|
|
|---|---|
|
|
| `Microgrid_Mode` | High-level system mode (normal operation, island mode, emergency mode, cost-optimization mode, resilience mode) determining overall behavior and constraints. |
|
|
| `Forecast_UpdateTrigger` | Forces the agent to update internal predictions (PV forecast, load forecast) earlier than scheduled when sudden weather/load changes are detected. |
|
|
|
|
## Scope gap against the simulator contract
|
|
|
|
The [Simulator I/O Interface](simulator-io-interface.md) defines a much narrower control input
|
|
set — five signals, of which two are optional:
|
|
|
|
| Simulator control input | Corresponding action here |
|
|
|---|---|
|
|
| `u_ele(t)` — electrolyser power setpoint, W | `Electrolyzer_PowerSetpoint` |
|
|
| `u_fc(t)` — fuel cell power setpoint, W | `FuelCell_PowerSetpoint` |
|
|
| `u_batt(t)` — signed battery setpoint, W | `Battery_ChargePower` + `Battery_DischargePower` collapsed into one signed value |
|
|
| `sw_ele(t)` — electrolyser ON/OFF (optional) | `Electrolyzer_Mode` (coarser) |
|
|
| `sw_fc(t)` — fuel cell ON/OFF (optional) | `FuelCell_Mode` (coarser) |
|
|
|
|
**Nine of the sixteen actions listed here have no simulator counterpart:** `PV_Setpoint`,
|
|
`HydrogenTank_ValveCommand`, `Grid_ImportPower`, `Grid_ExportPower`, `Load_SheddingCommand`,
|
|
`Load_ShiftingCommand`, `ThermalManagement_Command`, `Microgrid_Mode`, and
|
|
`Forecast_UpdateTrigger`.
|
|
|
|
Two of those gaps are deliberate design decisions recorded in the simulator sheet:
|
|
|
|
- **Grid import/export is not an action.** The simulator treats the grid as the *slack bus* —
|
|
it absorbs whatever residual remains after every other component acts, bounded by
|
|
`P_grid_max` and `grid_on(t)`. Exposing `Grid_ImportPower`/`Grid_ExportPower` as independent
|
|
actions would over-determine the power balance.
|
|
- **Load shedding is an outcome, not an action.** The simulator reports `unmet_crit(t)` and
|
|
`unmet_noncrit(t)` as consequences of the setpoints, and overrides the policy outright if
|
|
critical load would go unserved.
|
|
|
|
The rest — PV curtailment control, thermal management, mode supervision, forecast triggering —
|
|
are genuine capability gaps between the two documents that need a decision.
|
|
|
|
## Design questions still open
|
|
|
|
The simulator sheet raises two action-space questions this document does not answer:
|
|
|
|
1. **Continuous or discrete?** Continuous setpoints (`u_ele ∈ [0, P_ele_max]`) versus discrete
|
|
buckets. This document implies continuous power setpoints alongside discrete `*_Mode`
|
|
selections — i.e. a hybrid action space — but never says so explicitly.
|
|
2. **Should simultaneous electrolyser and fuel cell operation be structurally forbidden**
|
|
(via a mode-switch action space) or merely penalised in the reward? This document's
|
|
`Electrolyzer_Mode` and `FuelCell_Mode` are independent, which permits the physically
|
|
wasteful both-on state.
|
|
|
|
## Related
|
|
|
|
- [RL State Variables](rl-state-variables.md) — the observation space
|
|
- [RL Reward Function](rl-reward-function.md) — the reward terms
|
|
- [Simulator I/O Interface](simulator-io-interface.md) — the typed simulator contract
|
|
- [Rule-Based Controller](rule-based-controller.md) — the baseline policy
|