Building Custom Scenarios#

Developing custom scenarios for contributing to the pystorms library or your personal use is really easy.

Steps for building a custom scenario

Anonymize the stormwater network, if you are considering contributing to the pystorms repository.
Identify the event drivers (e.g. stormevents, tides).
Choose an objective and performance metric for evaluating the objective.
Decide on the state and action space.
Build the class and populate the yaml file.

Anonymize the stormwater network#

To ensure that the stormwater network is anonymized, we apply randomized transformations on the network that transform the coordinates of the network, while maintaining the integrity of the network topology.

In a SWMM input file the geometry of the model elements (length, elevation, cross section, etc.,) and their coordinate locations (i.e. the location to be displayed in the visualization of the model in a program) are defined in separate sections. This separation allows the transformation of the coordinate data without altering the geometry of the model elements. This means that the coordinates of the model elements can be rotated, scaled, and translated without effecting any of the geometry values relevant to the execution of the model.

Anonymization procedure#

Rotate all coordinates by a common (random) angle between 0 and 360 degrees. This randomly generated rotation angle is not exposed.

Scale all coordinates such that they are within a 100,000 x 100,000 unit box.

Translate coordinates to be within the bounding box defined by its two corners of (0,0) and (100000, 100000).

Output the anonymized coordinates to a new SWMM input file. By default comments are scrubbed from the input file, therefore no identifying metadata is kept in the anonymized version.

Inspect anonymized SWMM input file for any identifiable information remaining.

More information on the anonymization routine and the code to do so can be found here.

Once the network is anonymized, the names of the nodes, links, or any components that can be traced back to the original model have to be updated. Currently, we do not have a proper nomenclature style for renaming the network components, so we leave this up to the better judgement of the users.

Event driver#

The event driver can be a rainfall event, time series data of flows, or some initial volumes in nodes. Basically, anything that is supported by the stormwater simulation engine.

Control objective and performance metric#

The performance measure quantifies the ability of the control algorithm in achieving its objective. For example, if your objective was to minimize the CSO volume, a performance measure could be the CSO volume that occurs with the implementation of control.

Choose the state and action space#

After choosing an appropriate performance measure for quantifying the control objective, pick the states (i.e., the network elements where certain states, such as flow, water level, or pollutant concentrations, are measured) and the action space (i.e., controllable network assets).

Building the `pystorms` scenario class#

Every scenario in the library follows the same shape, shown below. The network goes in pystorms/networks/<name>.inp and the state space, action space and performance targets in pystorms/config/<name>.yaml (see Contributing for the yaml layout). The class reads both, hands them to the environment that drives SWMM, and computes the performance measure on every step.

from pystorms.environment import environment, validate_level
from pystorms.networks import load_network
from pystorms.config import load_config
from pystorms.scenarios import scenario
from pystorms.scenarios.scenario import validate_version
import yaml


class myawesomescenario(scenario):
    """
    One paragraph describing the network, the event and the objective.

    Parameters
    ----------
    version : str
        "1" is the scenario as published. Describe what "2" changes, or
        state that only "1" is defined.
    level : str
        difficulty level of the instrumentation

    Notes
    -----
    Anything else worth knowing about the scenario.
    """

    def __init__(self, version="1", level="1"):
        # Validate the keywords first, before any simulation is opened.
        # Name the versions this scenario defines; ("1",) if there is no second one.
        self.version = validate_version(version, ("1", "2"), "myawesomescenario")
        self.level = validate_level(level)

        # Scenario meta data is defined in the yaml file
        with open(load_config("myawesomescenario"), "r") as fh:
            self.config = yaml.load(fh, yaml.FullLoader)

        # Load the network input file
        self.config["swmm_input"] = load_network(self.config["name"])

        # Anything version 2 changes goes here: a tighter threshold, a
        # different action space, or a rewritten copy of the network saved
        # under pystorms.networks.derived_network_path(...)
        self.threshold = 0.5
        if self.version == "2":
            self.threshold = 0.25

        # Create the simulation environment
        self.env = environment(
            self.config, ctrl=True, version=self.version, level=self.level
        )

        # Create an object for logging data. Include everything used for the
        # performance measure and anything useful for debugging. Keep the
        # "performance_measure" and "simulation_time" keys.
        self.data_log = {
            "performance_measure": [],
            "simulation_time": [],
            "flow": {},
            "flooding": {},
        }

        # Populate the data_log with the elements whose measurements are recorded
        for ID, attribute in self.config["performance_targets"]:
            self.data_log[attribute][ID] = []

    def step(self, actions=None, log=True, level=None, version=None):
        # Implement the actions and step ahead. The environment applies the
        # level the scenario was built with unless one is passed explicitly.
        done = self.env.step(actions, level=level)

        # Log the states, actions, performance targets, and anything else useful
        if log:
            self._logger()

        # Compute the performance measure for this step
        __performance = 0.0
        for ID, attribute in self.config["performance_targets"]:
            value = self.env.methods[attribute](ID)
            # Implement your performance evaluation here, for example
            if attribute == "flooding" and value > 0.0:
                __performance += 10 ** 6
            if attribute == "flow" and value > self.threshold:
                __performance += (value - self.threshold) * 10.0

        self.data_log["performance_measure"].append(__performance)

        # Close the simulation when the event ends
        if done:
            self.env.terminate()

        return done

The base class provides state(), performance(), save(), terminate() and context manager support, so the new scenario behaves like the shipped ones. The difficulty levels come for free from the environment: the fault schedule is drawn from the state and action space in the yaml file, and applied to every reading and every command.