Skip to content

TargetGym

Reach a setpoint. Hold it forever. Against disturbances.
24 JAX environments for target MDPs, the control problems industry actually has.


One example task from each family, under PID control.

24 environments: 10 aircraft, 6 process control, 6 industrial / energy, 2 renewable energy. Every one of them is in the gallery with its own page and clip, and its recorded baseline numbers are in Baseline results.

pip install target-gym

Until 0.7.0 is on PyPI

TargetGym is in active development: PyPI releases are the stable line, and main can change quickly between them. This site documents main, whose API (starting with tg.make) is not on PyPI until 0.7.0. Until then, install from GitHub to use it: pip install "git+https://github.com/YannBerthelot/TargetGym".

Or try it in the browser: Colab quickstart.

import jax
import numpy as np
import target_gym as tg

env, params = tg.make("plane")          # the scored configuration
pid = tg.make_expert("plane").host()    # the shipped expert; keeps its own memory
obs, state = env.reset(jax.random.PRNGKey(0), params)

for t in range(200):
    action = np.atleast_1d(pid(np.asarray(obs)))
    obs, state, reward, terminated, truncated, info = env.step(
        jax.random.PRNGKey(t), state, action, params
    )
    if terminated or truncated:
        break

Browse the twenty-four environments → Getting started →

Why these environments

Holding a setpoint forever breaks differently than reaching a goal once, and these are the failure modes that come with it:
Irrecoverable states A boiler drum that carries water into the turbine, a reactor past runaway, a kiln that has gone cold
Partial observability The furnace hides 6 of 9 states, the reactor 7 of 11, the kiln 64 behind 8 measurements
Non-minimum phase Drum level rises as mass leaves; the four-tank's obvious loop pairing is unstable
Transport delay Half the kiln's response to a fuel change takes a full 25-minute residence time
Multi-timescale Millisecond neutronics against hour-long xenon; sub-second flame gas against 30 h glass residence
Finite budgets A battery whose charge limits trip the pack, though exact dispatch tracking stays inside them for a 30-minute episode, so tracking costs wear rather than later tracking
Open-loop instability A stirred-tank reactor held on its unstable middle steady state, where an uncontrolled error grows e-fold every 19 to 43 s (derived, scripts/unstable_cstr_numbers.py --section targets)
A trip the reward does not track A compressor trips when its flow margin to surge runs out. No reward term reads that margin, and the recycle valve that protects it has no tracked output of its own

Every environment ships a tuned PID and an MPC (on patrol_bearing_only the MPC reads the true state, a full-state bound), so a learned policy has something real to beat. And where a baseline is weak, the docs say how weak.

Documentation