Key IO Data Structures

Tip

  1. Need help? Please let us know in the SUEWS Community.

  2. Please report issues with the manual on GitHub Issues (or use Report Issue for This Page for page-specific feedback).

  3. Please cite SUEWS with proper information from our Zenodo page.

6.8. Key IO Data Structures#

6.8.1. Introduction#

The Python API uses pandas DataFrame structures for analysis data and a typed checkpoint object for restart workflows:

from supy import SUEWSSimulation

# Load sample data and run simulation
sim = SUEWSSimulation.from_sample_data()
output = sim.run()

# Access the data structures
df_state_init = sim.df_state_init   # Input: initial states
df_forcing = sim.df_forcing         # Input: forcing data
df_output = sim.results             # Output: simulation results
df_state_final = sim.df_state_final # Legacy/developer final states
checkpoint = output.checkpoint      # Preferred restart artefact

Input DataFrames:

  • df_state_init: Model initial states and configuration parameters

  • df_forcing: Meteorological forcing data time series

Output and restart objects:

  • df_output (or sim.results): Model output results for scientific analysis

  • df_state_final: Legacy/developer final states for compatibility and inspection

  • SUEWSCheckpoint: Typed runtime state for continuation runs

6.8.2. Input#

6.8.2.1. df_state_init: model initial states#

df_state_init is organised with grids in rows and state variables in columns.

>>> df_state_init.head()
var     ah_min       ah_slope_cooling      ...  z z0m_in zdm_in
ind_dim   (0,)  (1,)             (0,) (1,) ...  0      0      0
grid
98        15.0  15.0              2.7  2.7 ... 49.6   1.9   14.2

[1 rows x 1200 columns]

The details of all state variables can be found in df_state variables.

Dimensionality encoding:

Properties are stored as flattened values to fit into the tabular DataFrame format, though they may actually be of higher dimension. The ind_dim level in columns indicates variable dimensionality:

  • 0 for scalars

  • (ind_dim1, ind_dim2, ...) for arrays (vectors are 1D arrays)

For example, ohm_coef has dimension {8, 4, 3} according to its description. The flattened representation:

>>> df_state_init.loc[:, "ohm_coef"]
ind_dim  (0, 0, 0)  (0, 0, 1)  (0, 0, 2)  ...  (7, 3, 1)  (7, 3, 2)
grid
98           0.719      0.194      -36.6  ...        0.6      -30.0

[1 rows x 96 columns]

Users should follow the dimensionality requirements when preparing or modifying df_state_init.

6.8.2.2. df_forcing: forcing data#

df_forcing is organised with temporal records in rows and forcing variables in columns.

>>> df_forcing.head()
                       iy  id  it  imin  ...   U     RH    Tair    pres  rain  kdown
2012-01-01 00:05:00  2012   1   0     5  ...  4.5  85.46  11.77  1001.5   0.0   0.15
2012-01-01 00:10:00  2012   1   0    10  ...  4.5  85.46  11.77  1001.5   0.0   0.15
...

The details of all forcing variables can be found in df_forcing variables.

Missing values can be specified with -999, which is the default NaN value accepted by SUEWS.

Note

The index of df_forcing must be a DatetimeIndex. The model time-step size is determined by the forcing data’s temporal resolution:

>>> df_forcing.index.freq
<300 * Seconds>  # 5-minute time step

6.8.3. Output#

6.8.3.1. df_output: model output results#

df_output is organised with temporal records of grids in rows (MultiIndex: grid, datetime) and output variables by group in columns (MultiIndex: group, variable).

>>> df_output.head()
group                        SUEWS                              ...
var                          Kdown       Kup    Ldown      Lup  ...
grid datetime
98   2012-01-01 00:05:00  0.153333  0.018279  344.31  371.99   ...
     2012-01-01 00:10:00  0.153333  0.018279  344.31  371.99   ...
...

[5 rows x 218 columns]

The details of all output variables can be found in df_output variables.

Output is recorded at the same temporal resolution as the forcing data:

>>> df_output.index.levels[1].freq == df_forcing.index.freq
True

6.8.3.2. df_state_final: model final states#

df_state_final has the identical data structure as df_state_init except for an extra datetime level in the index, which stores temporal information associated with model states.

>>> df_state_final.head()
var                      ah_min       ah_slope_cooling  ...  z z0m_in zdm_in
ind_dim                    (0,)  (1,)             (0,)  ...  0      0      0
datetime            grid
2012-01-01 00:05:00 98     15.0  15.0              2.7  ... 49.6   1.9   14.2
2013-01-01 00:05:00 98     15.0  15.0              2.7  ... 49.6   1.9   14.2

[2 rows x 1200 columns]

This structure is retained for compatibility and state inspection:

  • Runtime diagnostics: Save intermediate states with save_state=True in run_supy

  • Developer inspection: Examine flattened state variables in DataFrame form

The meanings of state variables in df_state_final are the same as in df_state_init, documented in df_state variables.

For new object-oriented continuation workflows, use SUEWSCheckpoint rather than df_state_final as the restart artefact.

6.8.3.3. SUEWSCheckpoint: typed restart state#

SUEWSCheckpoint carries typed backend runtime state keyed by grid ID. It is the preferred restart artefact for new object-oriented workflows.

The checkpoint is intentionally only the typed runtime state. It does not contain the full YAML configuration or forcing data. To continue a run, load the same YAML configuration, attach the next forcing period, and run from SUEWSSimulation.from_checkpoint(...):

from supy import SUEWSCheckpoint, SUEWSSimulation

checkpoint = SUEWSCheckpoint.from_file("{site}_SUEWS_checkpoint.json")

sim_next = SUEWSSimulation.from_checkpoint(
    "config.yml",
    checkpoint,
)
sim_next.update_forcing("forcing_next_period.txt")
output_next = sim_next.run()

Use checkpoint.to_file(path) to write the checkpoint explicitly, or sim.save(path) to save it alongside the configured output files. Checkpoints are keyed by grid ID, so checkpoint/configuration grid mismatches are rejected by the runtime.

class supy.SUEWSCheckpoint(grid_states: dict[int, str], supy_version: str = '2026.6.5.dev3', state_schema_version: int | None = None, created_at: str = <factory>, last_timestamp: str | None = None)[source]#

Typed restart artifact carrying Rust SUEWS state JSON by grid ID.

classmethod from_file(path: str | Path) → SUEWSCheckpoint[source]#

Read a checkpoint JSON file.

to_file(path: str | Path) → Path[source]#

Write the checkpoint JSON file and return its path.