Skip to content

CosimGym v1.0.0

Python Docs Status

Cosim-GYM logo

CosimGym is a Flexible, Modular and Customizable framework that use a fully declarative approach to easily set-up complex co-simulation scenarios embedding different models. It is meant to be a gym for dimulating real world cyberphisical systems composed by multiple independent simulators enabling the usage of these scenarios in Control and Optimization application such as beign a test-bed for ReinforcementLearning-based applications. It bridges the following gaps:

  • Declarative co-simulation set up and custom model integration via standardized approach
  • Bridging complex co-simulation environments and REinforcement Learning applications
  • Digital Twin enabler with generalized interfaces for realtime MQTT-based communication (sensor and adapter linking and realtime execution visualization)

The Challenge: Modern engineering systems—such as power grids, building energy systems, and robotic swarms—often rely on multiple interacting subsystems that are best modeled with specialized tools. Co-simulation (powered by the HELICS middleware) enables these heterogeneous models to run simultaneously and exchange data at every time step, providing a unified system-level perspective.

The Solution: By natively integrating Gymnasium, CosimGym translates complex publish/subscribe data exchanges into the standard reset() and step() paradigm. This allows RL agents to directly interact with, learn from, and control realistic, physics-based simulations without the heavy burden of networking and synchronization boilerplate.

Overview

With this framework you can setup co-simulation scenario by plugging your own models or reuse the community shared ones. The main orchestration mechanism relies on HELICS middleware and by wrapping those low level functionalities enables a standard and reproducible approach to plug and co-simulate models. In addition this framework is intended to be a Reinforcement Learning gym for training and testing your agents on composable realistic phisical environments.

Cosim-GYM logo

✨ Main Features

  • 📜 Declarative YAML Scenarios: Define complete simulation architectures, timing windows, multi-federation setups, and RL hyperparameters purely through YAML configuration files—no hardcoded orchestration required.
  • ⚙️ Automated Orchestration: The ScenarioManager handles the heavy lifting: starting HELICS brokers, spawning individual federate processes, distributing configurations via Redis, managing time synchronization, and ensuring graceful shutdowns.
  • 🌐 Distributed Execution: The framework can execute and distribute simualtors across different machines enabling the horizontal scaling of simulated systems and reduing possible computational bottlenecks
  • 🧩 Plug-and-Play Models: Rely on a standardized BaseModel interface and a centralized Model Catalog to easily drop in custom physics models, data readers, FMUs, or arbitrary RL algorithms.
  • 🤖 Gymnasium Compatibility: The built-in HelicsGymEnv wrapper automatically binds HELICS pub/sub variables to standard Observation and Action spaces, making it out-of-the-box compatible with popular RL libraries (e.g. stablebaseline3, RLlib).
  • 🔄 Flexible Workflows: Run standard physics-only co-simulations, conduct live online RL training (with built-in agents like DQN or SAC), or evaluate pre-trained policies within a single unified framework.
  • 📊 Built-in Dashboard: Monitor simulation metrics and analyze RL performance through an interactive, ready-to-use Streamlit dashboard, including a live view for data as it streams during a run.
  • 🔌 Digital-Twin Interfaces: Mirror any federate's data to MQTT for live observability, or swap a simulated federate for an interface federate bridging to real sensors/actuators — a one-line, config-only sim-to-real change (see Digital-Twin Interfaces & Live Streaming).

🚀 Quick Start: Choose Your Way

In order to start using this repository there are different options listed and explained here Installation

Examples

Simple example of runnable test cases (see full list for scenario names and how to run them): - Simple spring mass damper system (single & multi-federation) --> simple_test (Case 0) - Building + Heatpump + Weather + PID controller --> bui_hp_test_base (Case 1) - Building + Heatpump + Weather + RL agent (DQN/SAC) --> bui_hp_DQN / bui_hp_SAC (Cases 2 & 3) - PV + Battery + Load + Weather + RB controller --> pv_batt_test_base (Case 4) - PV + Battery + Load + Weather + RL controller (DQN/SAC)--> pv_batt_DQN / pv_batt_SAC (Cases 5 & 6)

Published Applications: Incoming

⚠️ Disclaimer:

This repository is in an early prototype stage. Testing is still ongoing, and the codebase is under active development. Expect changes, refactoring, and new features in the near future!

Contributing

Contributions, issues, and feature requests are welcome. Feel free to open an issue or submit a pull request. A discussion session will be open so in this first stage start brainstorming there. This repo is meant to be a collective place for multidisciplinary exchanges.

Cite This

incoming

Acknowledgements

This project has been supported by the following institutions and organizations:

Politecnico di Torino
Politecnico di Torino 		Energy Center Lab EC-lab
Energy Center Lab EC-lab