# RL Action Variables

> **Markdown report of a non-markdown source document.**
>
> | | |
> |---|---|
> | **Source** | [`docs/_originals/ACTION VARIABLES AND DESCRIPTIONS.docx`](../_originals/ACTION%20VARIABLES%20AND%20DESCRIPTIONS.docx) |
> | **Format** | Microsoft Word (.docx), 16.2 kB |
> | **MD5** | `9939b9c2f76fbb5bf54fbfeff25a31f4` |
> | **Owner** | AI & Control Systems cluster |
> | **Status** | Current — but broader than the simulator currently accepts |
> | **Report generated** | 2026-07-25 |

## What this document is

The action space for the reinforcement-learning controller: every lever the agent is allowed to
pull at each timestep. Sixteen actions, spanning direct power setpoints, discrete mode
selections, and demand-side commands.

## Action variables

### Generation and storage setpoints

| Action | Description |
|---|---|
| `PV_Setpoint` | Command that limits or caps the PV inverter output to avoid overgeneration, match load, or reduce battery stress. |
| `Battery_ChargePower` | Amount of power (kW) the controller instructs the battery to absorb; used when excess solar or cheap grid power is available. |
| `Battery_DischargePower` | Amount of power (kW) the controller instructs the battery to release; used to support critical loads, reduce grid cost, or stabilize voltage. |
| `Battery_Mode` | Direct mode selection (charge, discharge, idle) that constrains what the battery is allowed to do in the next control interval. |

### Hydrogen chain

| Action | Description |
|---|---|
| `Electrolyzer_PowerSetpoint` | Electrical power sent to the electrolyzer to control hydrogen production; higher power increases H₂ output but consumes more energy and heats the stack. |
| `Electrolyzer_Mode` | Operating mode such as standby, active production, cooldown, or shutdown; controls whether the electrolyzer is allowed to start producing hydrogen. |
| `FuelCell_PowerSetpoint` | Power command that determines how much hydrogen is converted back into electricity; used to supply critical loads during low-PV or grid failure conditions. |
| `FuelCell_Mode` | Operational mode (idle, ramp-up, nominal, emergency) that governs stack usage and prevents rapid degradation. |
| `HydrogenTank_ValveCommand` | Opening/closing command controlling the flow of hydrogen between the tank and fuel cell or between tank and buffer systems. |

### Grid exchange

| Action | Description |
|---|---|
| `Grid_ImportPower` | Amount of electrical power drawn from the grid; controlled to minimize cost, follow load, or maintain battery SOC. |
| `Grid_ExportPower` | Amount of power sent back to the grid; used when PV or fuel cell production exceeds local demand and export is economically or operationally beneficial. |

### Demand side

| Action | Description |
|---|---|
| `Load_SheddingCommand` | Decision to temporarily turn off certain non-critical loads to preserve battery SOC, maintain resilience, or avoid overloading. |
| `Load_ShiftingCommand` | Reschedules flexible loads (e.g., HVAC pre-cooling, non-critical equipment) to times of high solar availability or low grid pricing. |
| `ThermalManagement_Command` | Controls cooling or heating for PV inverters, electrolyzers, battery packs, and fuel cells to maintain safe temperatures. |

### Supervisory

| Action | Description |
|---|---|
| `Microgrid_Mode` | High-level system mode (normal operation, island mode, emergency mode, cost-optimization mode, resilience mode) determining overall behavior and constraints. |
| `Forecast_UpdateTrigger` | Forces the agent to update internal predictions (PV forecast, load forecast) earlier than scheduled when sudden weather/load changes are detected. |

## Scope gap against the simulator contract

The [Simulator I/O Interface](simulator-io-interface.md) defines a much narrower control input
set — five signals, of which two are optional:

| Simulator control input | Corresponding action here |
|---|---|
| `u_ele(t)` — electrolyser power setpoint, W | `Electrolyzer_PowerSetpoint` |
| `u_fc(t)` — fuel cell power setpoint, W | `FuelCell_PowerSetpoint` |
| `u_batt(t)` — signed battery setpoint, W | `Battery_ChargePower` + `Battery_DischargePower` collapsed into one signed value |
| `sw_ele(t)` — electrolyser ON/OFF (optional) | `Electrolyzer_Mode` (coarser) |
| `sw_fc(t)` — fuel cell ON/OFF (optional) | `FuelCell_Mode` (coarser) |

**Nine of the sixteen actions listed here have no simulator counterpart:** `PV_Setpoint`,
`HydrogenTank_ValveCommand`, `Grid_ImportPower`, `Grid_ExportPower`, `Load_SheddingCommand`,
`Load_ShiftingCommand`, `ThermalManagement_Command`, `Microgrid_Mode`, and
`Forecast_UpdateTrigger`.

Two of those gaps are deliberate design decisions recorded in the simulator sheet:

- **Grid import/export is not an action.** The simulator treats the grid as the *slack bus* —
  it absorbs whatever residual remains after every other component acts, bounded by
  `P_grid_max` and `grid_on(t)`. Exposing `Grid_ImportPower`/`Grid_ExportPower` as independent
  actions would over-determine the power balance.
- **Load shedding is an outcome, not an action.** The simulator reports `unmet_crit(t)` and
  `unmet_noncrit(t)` as consequences of the setpoints, and overrides the policy outright if
  critical load would go unserved.

The rest — PV curtailment control, thermal management, mode supervision, forecast triggering —
are genuine capability gaps between the two documents that need a decision.

## Design questions still open

The simulator sheet raises two action-space questions this document does not answer:

1. **Continuous or discrete?** Continuous setpoints (`u_ele ∈ [0, P_ele_max]`) versus discrete
   buckets. This document implies continuous power setpoints alongside discrete `*_Mode`
   selections — i.e. a hybrid action space — but never says so explicitly.
2. **Should simultaneous electrolyser and fuel cell operation be structurally forbidden**
   (via a mode-switch action space) or merely penalised in the reward? This document's
   `Electrolyzer_Mode` and `FuelCell_Mode` are independent, which permits the physically
   wasteful both-on state.

## Related

- [RL State Variables](rl-state-variables.md) — the observation space
- [RL Reward Function](rl-reward-function.md) — the reward terms
- [Simulator I/O Interface](simulator-io-interface.md) — the typed simulator contract
- [Rule-Based Controller](rule-based-controller.md) — the baseline policy