pepe/ALLSHIFT
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ALLSHIFT/docs/03-energy-management/bidmc-parquet-load-data-manual.md
pepe 72dd781dbc Organize documentation into docs/ and superseded/ 
Audit every document in the repository, convert the non-markdown ones into
markdown reports, and split current documentation from outdated material.

docs/ — 31 markdown documents in seven numbered sections. Twenty are new
reports generated from .docx / .pdf / .xlsx / .mlx / .m sources that were
previously unreadable in the browser and undiffable in git. Each report
carries a provenance block (source path, format, MD5) and links back to its
original; all 13 recorded checksums verify against the files on disk.
Machine-extraction losses (PDF table column interleaving, Word OMML
equations, embedded figures) are called out explicitly rather than silently
smoothed over.

superseded/ — outdated material with a documented reason per entry:
two byte-identical ClickUp re-exports, an older revision of the BIDMC/UCSD
energy-flow doc (the retained copy adds the SoC Violation Rate KPI), a
duplicate of Shift input data.docx, the May 2026 simulation plan, the
root PV+Battery.md now covered by a fuller report, GitHub's stock
demo-repository template, and a zero-byte placeholder. Its README also
records what was deliberately NOT retired and why — the "Old Frameworks"
and "Old Simulations" folders hold unique Simulink revisions, and
"Big Ugly Folder" holds the only copy of framework revision 1.3.

Findings worth flagging, all documented in the reports:
- Simulink lineage recovered from each .slx's internal coreProperties.xml
  revision counter. The current model is
  Current Framework/Bobert0206_Initial_Simulation_Framework.slx (rev 2.7);
  the top-level copy is rev 1.3, five revisions behind.
- Simulations/Constants.m is a truncated byte-prefix of the Current
  Framework copy, silently missing H2_leak, H2_cap and E_H2_vol_h.
- The PEM electrolyser and fuel cell are unmodified MathWorks Simscape
  examples still at vendor defaults; the "10x bigger" sizing TODO recorded
  in Constants.m was never carried out.
- controller-claude.m does not compile — undefined P_Electro_max, outputs
  unassigned on several paths.
- The specification set uses two incompatible variable naming conventions
  and disagrees on action-space size (5 vs 16).
- MA_hourly_load.csv (13.7 MB) is the same 35,040 rows as 89993-0.parquet
  (2.4 MB).
- Clinical data is the MIMIC-IV *demo* (ODbL, 100 patients), not full
  MIMIC-IV — redistributable, but the licence and citation are unrecorded.

Housekeeping: untrack 21 Simulink build artefacts (slprj/, *.slxc) and add
ignore rules for them. Root README rewritten around the new layout.

Recruitment notes naming individual candidates are excluded from version
control via .gitignore rather than committed; the generic question template
is kept in docs/07-team-and-operations/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-25 21:20:33 -07:00

164 lines
7.9 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# BIDMC Load Profile — NREL ComStock Parquet Manual
> **Markdown report of a non-markdown source document.**
>
> | | |
> |---|---|
> | **Source** | `Shift Matlab Drive/Shift Matlab Drive/Energy_Managment/Beth Israel Deaconess Medical Center/BIDMC_Parquet_Load_Data_Manual.docx` |
> | **Format** | Microsoft Word (.docx) |
> | **MD5** | `af1178d355dda935363046f04a64e34d` |
> | **Describes** | `89993-0.parquet` (2.37 MB) in the same folder |
> | **Owner** | Energy Management cluster |
> | **Status** | Current |
> | **Report generated** | 2026-07-25 |
## What the file contains
`89993-0.parquet` holds **a full year of simulated energy consumption** for a 500,000 sq ft
hospital building in Massachusetts. It was generated by NREL's **ComStock** simulation platform
using EnergyPlus and actual 2018 meteorological weather data (AMY2018) for the Boston climate.
Building **ID 89993** was selected as the best available public proxy for Beth Israel Deaconess
Medical Center based on:
- similar floor area (~500,000 sq ft),
- building date (late-1950s construction),
- HVAC system type (VAV chiller + gas boiler),
- climate zone.
This is the **baseline** scenario (`upgrade00`).
> [!IMPORTANT]
> This is **simulated stock-model data, not measured BIDMC data**. It is a proxy. The
> [Forecasting Requirements](../02-specifications/forecasting-requirements.md#21-inputs) flags
> real historical hospital load as the project's biggest data gap — this file is the current
> stand-in for it, not a resolution of it.
## File overview
| Property | Value |
|---|---|
| File name | `89993-0.parquet` |
| Building ID | 89993 |
| Source | NREL End-Use Load Profiles for US Building Stock (ComStock 2025) |
| Building type | Hospital |
| State | Massachusetts (Climate Zone 5A — Boston) |
| Floor area | 500,000 sq ft (46,450 m²) |
| Vintage / year built | 1946–1959 (built 1959) |
| Number of floors | 4 |
| File version | Upgrade 00 — Baseline |
| Time resolution | **15 minutes** |
| Total rows | 35,040 (full year 2018) |
| Time period | 2018-01-01 00:15 to 2019-01-01 00:00 |
| HVAC system | VAV chiller with gas boiler reheat |
| Heating fuel | Natural gas |
## Column reference
Each energy end use appears in **two forms** across the 64 columns: absolute consumption (kWh
per 15-minute interval) and intensity (kWh per sq ft per 15-minute interval). The intensity
columns are useful for scaling the data to BIDMC's exact floor area.
| Column name | Unit | Annual total | Peak (15-min) |
|---|---|---|---|
| `bldg_id` | — | 89993 (constant) | — |
| `timestamp` | date | 2018-01-01 to 2019-01-01 | — |
| `out.electricity.cooling.energy_consumption` | kWh | 1,025,061 | 210.14 |
| `out.electricity.fans.energy_consumption` | kWh | 1,042,572 | 131.76 |
| `out.electricity.interior_equipment.energy_consumption` | kWh | 3,247,485 | 170.15 |
| `out.electricity.interior_lighting.energy_consumption` | kWh | 1,597,696 | 118.15 |
| `out.electricity.pumps.energy_consumption` | kWh | 544,941 | 25.12 |
| `out.electricity.heat_recovery.energy_consumption` | kWh | 186,740 | 7.69 |
| `out.electricity.heat_rejection.energy_consumption` | kWh | 81,596 | 22.49 |
| `out.electricity.exterior_lighting.energy_consumption` | kWh | 107,751 | 6.69 |
| `out.electricity.water_systems.energy_consumption` | kWh | 153,266 | 4.51 |
| `out.electricity.refrigeration.energy_consumption` | kWh | 39,259 | 2.79 |
| `out.electricity.purchased.energy_consumption` | kWh | 8,026,366 | — |
| `out.electricity.total.energy_consumption` | kWh | 8,026,366 | — |
| `out.natural_gas.heating.energy_consumption` | kWh | 6,893,812 | — |
| `out.natural_gas.water_systems.energy_consumption` | kWh | 1,195,680 | — |
| `out.natural_gas.interior_equipment.energy_consumption` | kWh | 861,204 | — |
| `out.site_energy.total.energy_consumption` | kWh | 16,977,060 | — |
**All-zero columns:** `out.electricity.heating`, `out.electricity.pv`,
`out.district_cooling.*`, `out.district_heating.*`, `out.fuel_oil.*`, and `out.propane.*`. This
building uses natural gas for heating and has no on-site PV or district energy connections in
the baseline case.
> The absence of an `out.electricity.pv` signal is convenient rather than limiting: SHIFT
> supplies its own PV generation from the
> [PV hourly dataset](pv-hourly-dataset-manual.md), so the ComStock profile provides pure
> *demand* with no embedded generation to net out.
## Filling gaps in the existing datasets
These values are complementary to those in `hospital_communication_energy_system.csv` (see the
[BIDMC Datasets Manual](bidmc-datasets-manual.md)). Only the rows marked *New column* do not
exist in that CSV, but can be added.
| Parquet data | Maps to | How it's used |
|---|---|---|
| `electricity.interior_equipment` | Medical Equipment Power Usage (kWh) | Equipment load correlated with MIMIC clinical events |
| `electricity.cooling + fans + pumps` | HVAC Power Usage (kWh) | RL action target for demand shifting |
| `electricity.interior_lighting` | Lighting Power Usage (kWh) | Partially controllable — follows occupancy/shift schedule |
| `electricity.purchased` (hourly sum) | Total Power Usage (kWh) | RL environment state: total demand the agent must satisfy |
| `natural_gas.heating` | **New column** — Thermal Load (kWh) | Drives HVAC heating decisions; links to UCSD thermal storage data |
| All intensity columns (`*_intensity`) | **New column** — Load per sq ft | Normalization factor for scaling BIDMC to actual floor area |
## Overall energy statistics
### Annual summary
| Metric | Value |
|---|---|
| Total electricity purchased from grid | **8,026,366 kWh/year** |
| Total natural gas consumed | **8,950,696 kWh** equivalent/year |
| Total site energy (all fuels) | **16,977,060 kWh/year** |
| Energy use intensity (EUI) | **33.9 kWh/sq ft/year** |
At 8,026 MWh/year the building averages **≈ 916 kW** of continuous electrical demand.
### Electricity end-use breakdown
| End use | Annual (kWh) | Share of electricity |
|---|---:|---:|
| Interior equipment (medical devices, computers, plug loads) | 3,247,485 | **40.5%** |
| Interior lighting | 1,597,696 | 19.9% |
| HVAC fans | 1,042,572 | 13.0% |
| Cooling (chiller) | 1,025,061 | 12.8% |
| Pumps (HVAC circulation) | 544,941 | 6.8% |
| Heat recovery | 186,740 | 2.3% |
| Water systems | 153,266 | 1.9% |
| Exterior lighting | 107,751 | 1.3% |
| Cooling tower (heat rejection) | 81,596 | 1.0% |
| Refrigeration | 39,259 | 0.5% |
### Natural gas breakdown
| End use | Annual (kWh) | Note |
|---|---:|---|
| Space heating | 6,893,812 | Mostly in Boston winters |
| Service water heating | 1,195,680 | Hot water for clinical/kitchen use |
| Interior equipment (gas-fired) | 861,204 | Kitchen equipment, sterilizers |
## Practical notes
- **Resolution matches the forecasting spec, not the simulator default.** 15-minute resolution
lines up with the `t−96`/`t−672` lags in the
[PV forecasting inputs](../02-specifications/forecasting-requirements.md#11-inputs), but the
[Simulator I/O Interface](../02-specifications/simulator-io-interface.md#timestep) defaults to
hourly steps. Aggregating 4→1 is trivial; the reverse is not.
- **No critical/non-critical split.** ComStock reports by *end use*, not by clinical criticality.
Deriving `L_crit` / `L_noncrit` requires the tier mapping in
[Energy Flow and Balances](bidmc-ucsd-energy-flow-and-balances.md) or a proxy fraction — an
open question the simulator sheet explicitly flags.
- **Natural gas is larger than electricity.** 8,951 MWh gas vs 8,026 MWh electricity. A
hydrogen microgrid that only displaces *electricity* addresses under half of this building's
site energy.
## Related
- [BIDMC Datasets Manual](bidmc-datasets-manual.md) — the clinical and synthetic-energy datasets this complements
- [BIDMC/UCSD Energy Flow and Balances](bidmc-ucsd-energy-flow-and-balances.md) — the load-tier model
- [Forecasting Requirements](../02-specifications/forecasting-requirements.md) — §2, hospital demand forecasting
- [Dataset Inventory](../06-data/dataset-inventory.md) — every data file in the repo
Reference in a new issue View git blame Copy permalink