ALLSHIFT/docs/03-energy-management/bidmc-datasets-manual.md

BIDMC Datasets Manual (v2)

Markdown report of a non-markdown source document.

Source	Shift Matlab Drive/Shift Matlab Drive/Energy_Managment/Beth Israel Deaconess Medical Center/BIDMC_Datasets_Manual_v2.docx
Format	Microsoft Word (.docx), 29.3 kB
MD5	78426a0944272a1549be13772a83e9c1
Owner	Energy Management cluster
Status	Current (v2) — no v1 exists in this repository
Report generated	2026-07-25

What this document is

Manual for the data related to the BIDMC building structure. It documents four datasets — three clinical and one synthetic building-energy file — and explains how each maps onto the reinforcement-learning formulation (state space, action space, reward).

Important

Provenance of the clinical files. The column names, itemid codes, and de-identification scheme are those of MIMIC-IV. The Digital Twin Readiness Checklist in the same folder links to physionet.org/content/mimic-iv-demo/2.2/icu/inputevents.csv.gz, and this manual records 100 patients — both consistent with the MIMIC-IV Clinical Database Demo, which is openly published under the Open Data Commons ODbL v1.0 and does not require credentialed access.

That makes redistribution here defensible, but the licence and citation are not recorded anywhere in the repository. ODbL requires attribution and share-alike. Adding a LICENSE/SOURCE note next to these files is a small fix worth making — see Dataset Inventory.

These files are not BIDMC's own clinical records. Despite the folder name, no real BIDMC patient data is present.

How the datasets link together

All three clinical files share the join key subject_id. It connects patients (patients.csv) to ICU monitoring (chartevents.csv) and treatments (inputevents.csv). The energy file links via Patient ID (format P0001).

Within the clinical files:

• hadm_id = hospital admission
• stay_id = specific ICU stay within that admission
• One patient can have multiple admissions and multiple ICU stays

All dates in the clinical files are shifted forward by around 100 years for de-identification. Only the length of stay can be understood from the dates.

---

1. patients.csv

• Unique patient data giving an understanding of patient behaviours and needs. 100 rows, 6 columns, no repeated rows per patient.
• Static dataset, recorded once.
• Can be used as a "virtual patient avatar" — by modelling different ages the system can predict energy used per person.
• For RL, this data is in the state space.

Column	Type	Example	Description
subject_id	Integer	10014729	Unique patient identifier. Primary join key across all clinical files. Filter chartevents and inputevents by this to get one patient's records
gender	String	F or M	Biological sex at ICU admission
anchor_age	Integer (years)	21–91, mean 62	Patient age at their first ICU admission
anchor_year	Integer (shifted)	2110–2201	De-identified year of first ICU admission. Cannot be used as a real calendar date
anchor_year_group	String	2011 – 2013	The real calendar period of the patient's admission. Useful for understanding what medical equipment was in use
dod	Date or NULL	2137-09-02 or empty	De-identified date of death. NULL for 69 of 100 patients (alive or unknown)

---

2. chartevents.csv

(5.29 MB gzipped)

• Acts as a chart, tracking how the patient's body is performing while in ICU: heart rate, blood pressure, oxygen levels, body temperature, ventilator settings.
• Units: Heart Rate bpm | Oxygen/SpO2 % | Respiratory Rate breaths/min | Temperature °C or °F (check valueuom per row) | Blood Pressure mmHg | Weight kg
• Data intervals are irregular — monitored whenever a nurse takes measurements. Always use charttime (not storetime) for time-series work.
• Gives a real-time trajectory of a patient's health in the digital twin.
• Represents the dynamic state space. The AI can judge whether a patient is getting better or worse and thus predict whether more equipment usage will be required.
• Can also feed the reward: when the AI adjusts environment flows and an unstable heart rate returns to a safe range, it can receive a major positive reward.

Column	Type	Example	Description
subject_id / hadm_id / stay_id	Integer	10005817 / 20626031 / 32604416	subject_id links to patients.csv. hadm_id = hospital admission. stay_id = specific ICU stay. One patient can have multiple stay_ids. Always filter by stay_id when analysing one ICU episode
caregiver_id	Float or NULL	6770.0	ID of the nurse or clinician who recorded this reading. NULL means an automated monitor recording. Can be used to model nurse visit frequency, which correlates with lighting and device power usage per room
charttime	Datetime (shifted)	2132-12-16 00:00:00	When the measurement was taken. Use this for all time-series work. Dates are de-identified so use relative time differences only
storetime	Datetime (shifted)	2132-12-15 23:45:00	When the value was entered into the system. Can be ignored; charttime is better
itemid	Integer (code)	220045, 220050, 220210	Numeric code identifying what was measured. Key codes: 220045 = Heart Rate (bpm), 220050 = Arterial BP Systolic (mmHg), 220051 = BP Diastolic, 220052 = BP Mean, 220210 = Respiratory Rate, 220277 = SpO2 (%), 220048 = Heart Rhythm. 483 unique codes in this dataset
value	String	"100" or "On"	Raw recorded value, always stored as text. For numeric readings also use valuenum. For categorical readings (e.g. ventilator On/Off) valuenum is NULL and you must read value
valuenum	Float or NULL	100.0	Numeric version of value. Used for all calculations. NULL for categorical readings
valueuom	String (unit)	%, mmHg, bpm, /min, °C	Unit of measurement for this row. Temperature appears in both °C and °F depending on the monitor
warning	Integer (0/1)	0 or 1	Clinical alarm flag. 1 = value was outside normal thresholds at the time of recording. A flagged vital after an AI action should trigger a negative reward

---

3. inputevents.csv

(0.75 MB gzipped)

• Tracks everything actively pumped or injected into the patient's body: IV fluids, active medications, nutrition.
• Records the type of medication or fluid (ordercategoryname), the exact amount, how fast it was given (rate), and exact start and end times.
• Units: mL or mg for dosage (amountuom); mL/hour or mg/hour for infusion rate (rateuom); weight in kg (patientweight).
• Logs continuous periods with exact start and end times, showing how long an infusion lasted.
• If a patient is actively receiving high-risk continuous IV medications, their body is under high stress. The digital twin can use this timeline to flag that the room environment must remain perfectly steady — the AI manager reads this status and knows it cannot alter the HVAC settings in that room right now.

Column	Type	Example	Description
starttime / endtime	Datetime (shifted)	2132-12-16 19:50:00	Start and end of the infusion. Any chartevents.charttime within this window occurred during active treatment
ordercategoryname	String	01-Drips, 08-Antibiotics (IV)	Clinical type of what is being given. 01-Drips = continuous vasopressor or sedative infusions (highly unstable patient — do not alter room). Full list in this dataset: 01-Drips, 02-Fluids (Crystalloids), 05-Med Bolus, 06-Insulin, 07-Blood Products, 08-Antibiotics (IV), 10-Prophylaxis (IV), 13-Enteral Nutrition, 14-Oral/Gastric Intake
amount / amountuom	Float / String	1.0 / "dose" or 500 / "ml"	Quantity given and its unit. Units seen: ml, mg, mcg, grams, units, mEq, dose. Always read both together. "dose" means a standard clinical dose with no specific volume recorded
rate / rateuom	Float or NULL / String	5.0 / "mL/hour"	Speed of infusion. NULL = given all at once. Non-NULL = continuous drip, patient is actively on a pump. Active pumps consume measurable medical equipment power
statusdescription	String	FinishedRunning, Stopped, ChangeDose/Rate, Paused, Bolus	Outcome of this infusion record. FinishedRunning = completed normally. Stopped = discontinued early (possible adverse event). ChangeDose/Rate = dose adjusted mid-infusion (a new row follows)
patientweight	Float (kg)	91.0	Patient body weight at the time of this event. Weight-based dosing is common in ICU. Also feeds the virtual avatar: heavier patients have higher metabolic output, affecting room temperature modelling
ordercategorydescription	String	Drug Push, Continuous Med	How the order was administered. Drug Push = fast IV push. Continuous Med = ongoing infusion. Useful for classifying clinical intensity
isopenbag / continueinnextdept	Integer (0/1)	0 or 1	isopenbag = 1 if the bag was already open when this event started. continueinnextdept = 1 if the infusion continued after the patient moved to another unit. Both useful for tracking infusion continuity across ICU transfers

---

4. hospital_communication_energy_system.csv

(3.16 MB)

• Merges real-time patient environment data with the hospital's power grid, climate systems, and hardware performance. 10,000 rows, 25 columns. 5-minute intervals from 2025-01-01 to 2025-02-04. No missing values.
• Contains room and outdoor temperatures/humidity, grid and renewable energy sources, energy consumed by lighting/HVAC/medical gear, and system health checks.
• Units: Power & Energy Consumption kWh | Renewable Energy Usage % | Vitals bpm, mmHg, % | Temperatures °C | Humidity %
• Synchronised and recorded at exactly 5-minute intervals.
• Creates the digital twin infrastructure of the building, linking patients to environment. Can be used to predict how a heatwave or sudden grid blackout affects indoor room temperatures and the power stability of life-support gear.
• Contains the exact switches of AI controls: Energy Saving Mode on/off, HVAC Mode (Heating/Cooling), power distribution adjustments.
• At every 5-minute interval the AI can look at Total Power Usage and Renewable Energy Usage. If it reduces grid reliance or lowers kWh usage while keeping all patient rooms comfortable, it can receive a positive reward.
• By comparing Outdoor Temperature with Internal Room Temperature, the AI can predict upcoming thermal loads (e.g. afternoon heat spikes) and pre-cool the building using cheap, clean renewable energy before peak pricing hits the main grid.

Warning

This is synthetic data. The manual states the timestamps are "synthetic operational data that is not shifted", energy sources are distributed "~33% each" across Grid/Solar/Wind, and HVAC Mode is a "~50/50 split" — signatures of generated rather than measured data. It is useful as a schema and an RL-environment mock-up, not as evidence about BIDMC's real operation. Note also that each row is a single patient room snapshot, not a whole-hospital total.

Column	Type	Example	Description
Timestamp	Datetime	2025-01-01 00:00:00	The RL time-step index. Each row = one decision point (every 5 min). Sort by this before any time-series work
Patient ID	String	P0001	Room/patient identifier. Links to clinical files via subject_id mapping. Each row is a different patient's room snapshot
Temperature (°C)	Float	35.96	Patient body temperature. Normal ICU range 36.1–37.2 °C. Above 38 °C = fever, below 36 °C = hypothermia. AI uses this to decide if HVAC needs to compensate
Humidity (%)	Float	53.1	Relative humidity near the patient. ICU target 30–60%. High humidity raises infection risk; low humidity causes respiratory discomfort for ventilated patients. Use as an HVAC reward constraint
Oxygen Level (%)	Float	98.9	Blood oxygen saturation (SpO2). Normal 95–100%. Below 90% = clinical emergency
Heart Rate (bpm)	Integer	76	Beats per minute. Normal ICU resting range 60–100 bpm
Blood Pressure (mmHg)	String	(111, 84)	Stored as a tuple string (systolic, diastolic). Normal ICU systolic 90–140 mmHg. Requires parsing
Energy Consumption (kWh)	Float	16.75	Total energy consumed in this 5-min window. Primary cost signal for the RL agent. Distinct from Total Power Usage — this is measured consumption; that is the sum of subsystem loads
Energy Source	String	Grid, Solar, Wind	Dominant power source at this timestamp (~33% each). RL agent should prefer Solar and Wind over Grid
Renewable Energy Usage (%)	Float	52.8	Percentage of total power drawn from renewable sources. Direct reward component: higher = better
HVAC Power Usage (kWh)	Float	5.12	Energy used by HVAC in this interval. Controllable by the AI action space. Reducing this while maintaining Room Temperature = positive reward
Lighting Power Usage (kWh)	Float	1.87	Energy used by lighting. Also controllable. Can be dimmed during low-activity periods without affecting patient safety
Medical Equipment Power Usage (kWh)	Float	2.30	Energy used by medical devices. Treated as a fixed load
Total Power Usage (kWh)	Float	9.29	= HVAC + Lighting + Medical Equipment. Minimising this while keeping Room Temperature and Humidity in safe ranges is an AI reward
Room Temperature (°C)	Float	21.7	Ambient temperature inside the patient's room — what the HVAC controls. AI target: 21–23 °C for ICU comfort
Room Humidity (%)	Float	33.7	Ambient humidity inside the patient room. AI target 30–60%
HVAC Mode	String	Heating or Cooling	Current HVAC operating mode (~50/50 split). Both an observable state and the AI's last action. Heating = 1, Cooling = 0
HVAC Efficiency (%)	Float	92.9	How efficiently HVAC converts energy into useful heating/cooling. Low efficiency = system working hard achieving little. Reward penalty: high power + low efficiency = bad
Outdoor Temperature (°C)	Float	18.9	External ambient temperature. Gap between outdoor and room temperature = the thermal load HVAC must overcome. Used for predictive pre-conditioning
Outdoor Humidity (%)	Float	58.3	External ambient humidity. Used alongside outdoor temperature to predict HVAC load and pre-plan energy sourcing
Day of the Week	Integer (0–6)	2	0 = Monday, 6 = Sunday. Used for time-of-week patterns in energy demand
Season	String	Fall, Winter, Spring, Summer	Calendar season. Affects expected outdoor temperature range and baseline HVAC load. Context variable in the RL state space
Energy Saving Mode	Integer (0/1)	0 or 1	AI action flag. 1 = energy saving mode active (reduced HVAC and lighting intensity). A direct action the RL agent can toggle each step
System Health Check	Integer (0/1)	0 or 1	Hardware status. 0 = all systems healthy, 1 = fault or maintenance alert active. When 1, the AI should avoid aggressive energy optimisation and prioritise stability
AI Predicted Health Status	String	Healthy, Unhealthy	AI's prediction of overall patient/system health at this timestep. When Unhealthy, the RL agent should weight patient safety constraints more heavily

---

How this fits the wider project

This manual describes a building-and-patient digital twin — HVAC control, room comfort, patient vitals as reward signals. That is a materially different system from the one the Simulator I/O Interface specifies, which is a power-dispatch environment: PV, battery, electrolyser, fuel cell, H₂ tank, grid.

The two are complementary rather than contradictory — HVAC is the largest shiftable load in the tier model — but nothing yet connects them. In particular:

• No hydrogen, electrolyser, fuel cell, or battery SoC appears in any of these datasets.
• Season covering all four values within a 2025-01-01 → 2025-02-04 window is internally inconsistent, reinforcing that the file is generated.
• Clinical reward signals (heart rate returning to safe range) are not present in the RL reward specification, which is purely energy/resilience-oriented.

Deciding whether the clinical layer is in scope is an open project question, not a documentation gap.

Related

• BIDMC ComStock Load Profile — the whole-building load proxy that complements these files
• BIDMC Digital Twin Readiness Checklist — the readiness assessment over this data
• BIDMC/UCSD Energy Flow and Balances — where the tier power figures come from
• Dataset Inventory — every data file in the repo