Baseline results¶
Every environment ships two reference controllers, scored on its test
configuration (spec.make_test_params()):
- The expert (the
"pid"slot): reads the observation only, as your agent does. It is the score to beat. - The oracle (the
"mpc"slot): reads the true state, including what the observation hides. It shows what information and planning are worth.
Costs on this page are per step, lower is better, in each environment's own units. To score your own policy, see Evaluating against it. How the controllers are built and measured is in How the baselines work.
At a glance¶
How many times cheaper the oracle holds each target than the expert, and where the ceiling sits: tracking every quantity within its instrument's resolution. Switch the anchor to the ceiling to see how far each controller is from as good as a real plant can verify. Hover a row for the numbers, including the whole episode.
Room to grow¶
Hold is the tracking cost per step once a controller has settled on
each target, after a settle fixed per environment. The ceiling is
tracking every quantity within its instrument's resolution: as good as a
real plant can verify. Room is the number of decades from the expert's
hold down to the ceiling; 0 means the expert is already there. The
oracle score places the oracle between the expert (0) and the ceiling
(1). Score a policy the same way with target_gym.scoring.score_policy.
3 seeds each.
| environment | expert hold | ceiling | room (decades) | oracle hold | oracle score |
|---|---|---|---|---|---|
| Aircraft | |||||
plane |
4.36 | 1 | 0.64 | 0.519 | 1.00 |
plane_energy |
2.77 | 0.694 | 0.60 | 1.4 | 0.49 |
plane_sine |
3.19 | 0.63 | 0.70 | 0.319 | 1.00 |
plane3d_heading |
3.47 | 2 | 0.24 | 0.612 | 1.00 |
plane3d_circle |
3.46 | 2 | 0.24 | 3.15 | 0.17 |
plane3d_racetrack |
12.7 | 2 | 0.80 | 1.07 | 1.00 |
plane3d_figure8 |
324 | 1 | 2.51 | 0.519 | 1.00 |
patrol |
0.847 | 2 | 0.00 | 0.0532 | — |
patrol_bearing_only |
1.65 | 2 | 0.00 | 0.0532 | — |
formation |
4.52 | 6 | 0.00 | 1.01 | — |
| Process Control | |||||
cstr |
4.89e-7 | 1 | 0.00 | 8.85e-8 | — |
first_order |
3.02e-12 | 1 | 0.00 | 0 | — |
four_tank |
1.38 | 2 | 0.00 | 2.5e-8 | — |
ph_neutralization |
0.753 | 1 | 0.00 | 0.0892 | — |
distillation |
0.477 | 2 | 0.00 | 0.0228 | — |
unstable_cstr |
6.35 | 1 | 0.80 | 0.00731 | 1.00 |
| Industrial / Energy | |||||
glass_furnace |
0.668 | 1 | 0.00 | 0.102 | — |
reactor |
4.71 | 0.00278 | 3.23 | 1.48 | 0.16 |
hvac |
0.0238 | 7.5e-5 | 2.50 | 0.0142 | 0.09 |
cement_kiln |
14.3 | 1 | 1.16 | 0.0251 | 1.00 |
boiler_drum |
120 | 1.14 | 2.02 | 5.53 | 0.66 |
compressor_surge |
3.6 | 1 | 0.56 | 0.0093 | 1.00 |
| Renewable Energy | |||||
wind_turbine |
9.81e-5 | 6.94e-6 | 1.15 | 1.18e-5 | 0.80 |
battery |
6.8e-4 | 1.39e-4 | 0.69 | 2.29e-4 | 0.68 |
Whole episode¶
The gain is the mean cost per step after the plant's burn-in, moves between targets included: what the plant pays. Oracle vs expert is the expert's gain divided by the oracle's, so 10 means the oracle pays a tenth. The transient is the summed cost around each target change.
| environment | expert gain | oracle gain | oracle vs expert | expert transient | oracle transient |
|---|---|---|---|---|---|
| Aircraft | |||||
plane |
4.36 | 0.677 | 6.44 | 1.64e6 | 3.2e5 |
plane_energy |
14,437 | 354 | 40.8 | 2.49e6 | 86,448 |
plane_sine |
3.19 | 0.529 | 6.03 | 2.84e6 | 2.27e5 |
plane3d_heading |
3.47 | 0.612 | 5.66 | 2.9e6 | 9e5 |
plane3d_circle |
3.46 | 3.15 | 1.1 | 1.63e6 | 3.42e5 |
plane3d_racetrack |
12.7 | 1.07 | 11.9 | 1.64e6 | 3.26e5 |
plane3d_figure8 |
324 | 0.519 | 624 | 5.5e6 | 6,068 |
patrol |
0.847 | 0.0532 | 15.9 | 57,985 | 3,168 |
patrol_bearing_only |
1.65 | 0.0532 | 31 | 73,792 | 3,168 |
formation |
4.52 | 1.01 | 4.46 | 17,428 | 5,121 |
| Process Control | |||||
cstr |
13.3 | 7.04e-8 | 1.88e8 | 3.51e5 | 3.09e5 |
first_order |
3.32e-5 | 0 | — | 54,064 | 53,873 |
four_tank |
18.7 | 4.45e-8 | 4.19e8 | 5.94e5 | 5e5 |
ph_neutralization |
0.757 | 0.0925 | 8.18 | 3,017 | 2,557 |
distillation |
0.477 | 0.0228 | 20.9 | 50,808 | 38,594 |
unstable_cstr |
60,668 | 3,800 | 16 | 5.84e5 | 2.48e5 |
| Industrial / Energy | |||||
glass_furnace |
0.967 | 0.234 | 4.13 | 363 | 100 |
reactor |
4.71 | 1.48 | 3.19 | 3,794 | 639 |
hvac |
0.025 | 0.0137 | 1.82 | 1.11 | 0.519 |
cement_kiln |
14.4 | 0.0721 | 200 | 3,650 | 154 |
boiler_drum |
120 | 5.62 | 21.4 | 21,136 | 6,919 |
compressor_surge |
527 | 23 | 22.9 | 6.32e5 | 31,036 |
| Renewable Energy | |||||
wind_turbine |
1.04e-4 | 1.26e-5 | 8.25 | 0.0316 | 0.00324 |
battery |
0.00333 | 9.25e-4 | 3.6 | 0.0585 | 9.08e-4 |
Episode returns¶
The cost per step over the whole test episode (minus the return divided by the episode length): reaching the targets and holding them, together. 10 seeds, paired: oracle wins counts the seeds where the oracle beat the expert.
| environment | steps | expert cost/step | oracle cost/step | oracle wins |
|---|---|---|---|---|
| Aircraft | ||||
plane |
280 | 3,778 | 1,121 | 9/10 |
plane_energy |
1600 | 13,746 | 494 | 10/10 |
plane_sine |
480 | 2,241 | 387 | 10/10 |
plane3d_heading |
300 | 5,573 | 2,298 | 9/10 |
plane3d_circle |
300 | 3,535 | 1,123 | 8/10 |
plane3d_racetrack |
650 | 1,637 | 495 | 10/10 |
plane3d_figure8 |
400 | 9,291 | 13.9 | 10/10 |
patrol |
200 | 142 | 19.5 | 10/10 |
patrol_bearing_only |
200 | 182 | 19.5 | 10/10 |
formation |
200 | 292 | 26.6 | 10/10 |
| Process Control | ||||
cstr |
150 | 4,176 | 3,992 | 10/10 |
first_order |
150 | 276 | 274 | 10/10 |
four_tank |
1200 | 222 | 155 | 10/10 |
ph_neutralization |
300 | 149 | 5.31 | 10/10 |
distillation |
200 | 415 | 123 | 10/10 |
unstable_cstr |
1200 | 60,930 | 3,950 | 10/10 |
| Industrial / Energy | ||||
glass_furnace |
1600 | 1.17 | 0.312 | 10/10 |
reactor |
864 | 6.64 | 1.4 | 10/10 |
hvac |
720 | 0.0226 | 0.0113 | 10/10 |
cement_kiln |
700 | 18.2 | 0.158 | 10/10 |
boiler_drum |
400 | 170 | 22.1 | 10/10 |
compressor_surge |
1200 | 1,840 | 102 | 10/10 |
| Renewable Energy | ||||
wind_turbine |
400 | 1.4e-4 | 1.48e-5 | 10/10 |
battery |
360 | 0.00354 | 9.48e-4 | 10/10 |
Caveats¶
patrol_bearing_only: The PID is a lead-state estimator feeding the same formation hold the full-observation variant uses. Range with azimuth and elevation is a complete relative-position measurement, so the only unobservable quantity is the lead's heading, which the slot needs because it is expressed in the lead's frame; the estimator takes the lead's velocity from the follower's own and the change in the relative position, and its heading from that track less the crab the follower flies in the same air. The MPC slot holds patrol's oracle reading the true state, so it is a full-state bound, not a bearing-only controller. On a task defined by what the observation withholds, its score measures information and control together: what the hidden lead state is worth plus what a policy leaves on the table. It never reads the observation, so its per-step costs equal patrol's seed for seed.formation: The PID flies the lead seat as a formation lead: the 3D heading task's course hold and pitch on height, aimed at the heading the route commands, at a set power, so its wingmen have a steady reference. Each wingman seat flies the patrol's formation hold. The MPC slot is the patrol oracle's design applied to the whole formation: the patrol's lead autopilot in the lead seat and its twin-autopilot law in each wingman seat, with one residual planner over every seat on the team cost.