pepe/ALLSHIFT
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ALLSHIFT/docs/03-energy-management/pv-hourly-dataset-manual.md
pepe 72dd781dbc Organize documentation into docs/ and superseded/ 
Audit every document in the repository, convert the non-markdown ones into
markdown reports, and split current documentation from outdated material.

docs/ — 31 markdown documents in seven numbered sections. Twenty are new
reports generated from .docx / .pdf / .xlsx / .mlx / .m sources that were
previously unreadable in the browser and undiffable in git. Each report
carries a provenance block (source path, format, MD5) and links back to its
original; all 13 recorded checksums verify against the files on disk.
Machine-extraction losses (PDF table column interleaving, Word OMML
equations, embedded figures) are called out explicitly rather than silently
smoothed over.

superseded/ — outdated material with a documented reason per entry:
two byte-identical ClickUp re-exports, an older revision of the BIDMC/UCSD
energy-flow doc (the retained copy adds the SoC Violation Rate KPI), a
duplicate of Shift input data.docx, the May 2026 simulation plan, the
root PV+Battery.md now covered by a fuller report, GitHub's stock
demo-repository template, and a zero-byte placeholder. Its README also
records what was deliberately NOT retired and why — the "Old Frameworks"
and "Old Simulations" folders hold unique Simulink revisions, and
"Big Ugly Folder" holds the only copy of framework revision 1.3.

Findings worth flagging, all documented in the reports:
- Simulink lineage recovered from each .slx's internal coreProperties.xml
  revision counter. The current model is
  Current Framework/Bobert0206_Initial_Simulation_Framework.slx (rev 2.7);
  the top-level copy is rev 1.3, five revisions behind.
- Simulations/Constants.m is a truncated byte-prefix of the Current
  Framework copy, silently missing H2_leak, H2_cap and E_H2_vol_h.
- The PEM electrolyser and fuel cell are unmodified MathWorks Simscape
  examples still at vendor defaults; the "10x bigger" sizing TODO recorded
  in Constants.m was never carried out.
- controller-claude.m does not compile — undefined P_Electro_max, outputs
  unassigned on several paths.
- The specification set uses two incompatible variable naming conventions
  and disagrees on action-space size (5 vs 16).
- MA_hourly_load.csv (13.7 MB) is the same 35,040 rows as 89993-0.parquet
  (2.4 MB).
- Clinical data is the MIMIC-IV *demo* (ODbL, 100 patients), not full
  MIMIC-IV — redistributable, but the licence and citation are unrecorded.

Housekeeping: untrack 21 Simulink build artefacts (slprj/, *.slxc) and add
ignore rules for them. Root README rewritten around the new layout.

Recruitment notes naming individual candidates are excluded from version
control via .gitignore rather than committed; the generic question template
is kept in docs/07-team-and-operations/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-25 21:20:33 -07:00

4 KiB
Raw Permalink Blame History

PV Hourly Dataset Manual

Markdown report of a non-markdown source document.
Source Shift Matlab Drive/Shift Matlab Drive/Energy_Managment/PV hourly data/Readme- Manual.docx
Format Microsoft Word (.docx)
MD5 420336b9a65140e7390bdf3ab21349bb
Describes pvwatts_hourly.csv (0.57 MB) in the same folder
Owner Energy Management cluster
Status Current — this is the recommended PV dataset
Report generated 2026-07-25

Summary

This dataset contains all the PV hourly performance data of last year for BIDMC's location. A DC array output and an AC system output are available in watts, along with a few meteorological variables such as albedo and wind speed.

"Very easy to understand, would be recommended over the complex second dataset."

The "complex second dataset" is the satellite meteorological dataset — 167 MB of NSRDB-style data that the same author explicitly does not recommend for extracting PV rates.

Generated by NREL PVWatts (per the filename pvwatts_hourly.csv).

Column reference

Physical quantity Definition Units
Beam Irradiance The amount of solar radiation received per unit area by a surface held perpendicular (normal) to the rays that come in a straight line from the direction of the sun W/m²
Diffuse Irradiance The solar radiation that reaches the Earth's surface after being scattered out of the direct beam by molecules, aerosols, dust, and clouds in the atmosphere W/m²
Ambient Temperature Temperature of the surrounding air in the immediate environment °C
Wind Speed — m/s
Albedo A measure of the reflectivity of a surface —
Plane of Array Irradiance — W/m²
Cell Temperature The actual operating temperature of the individual solar cells inside the solar module °C
DC Array Output — W
AC Array Output — W

Remarks

  • Cell Temperature can take negative values, which is when it is most efficient.
  • Ambient Temperature can also take negative values.
  • The AC output includes an overall inverter efficiency.

Note

Because the AC figure already has inverter losses baked in, do not apply a second inverter efficiency when feeding this into the simulator. The Simulator I/O Interface treats P_PV_max as an optional inverter/rated cap that clips output — using AC Array Output as G(t) is the consistent choice; using DC would double-count the conversion stage.

Fit for the forecasting spec

The PV forecasting inputs call for GHI, DNI, DHI, 2 m temperature, cloud fraction, wind speed, relative humidity, plus solar zenith/azimuth and a clear-sky index.

Required feature Present here?
DNI (beam irradiance) ✅
DHI (diffuse irradiance) ✅
GHI ⚠️ Not listed directly; derivable from beam + diffuse and solar geometry
Ambient temperature ✅
Wind speed ✅
Cloud fraction ❌
Relative humidity ❌
Solar zenith / azimuth ❌ — compute via pvlib as the spec advises
Clear-sky index ❌ — needs a clear-sky GHI reference
Historical PV output (target) ✅ DC and AC array output

For the missing meteorological variables the satellite dataset is the complement — it has clear-sky irradiance, humidity, dew point, precipitable water, and solar angles. Use this dataset for the PV target and core drivers, and the satellite dataset for extra meteorological features.

Related