Patrol - wingman, bearing-only¶

The patrol task flown from a passive sensor: the wingman sees range, azimuth and elevation to the lead instead of the lead's relative state. The lead's heading, which places the slot, is not measured and has to be inferred from how the bearing changes. The tracked value is the measured range, against the distance at which the slot sits from the lead.

import target_gym as tg

env, params = tg.make("patrol_bearing_only")


Action space	`Box((3,))`, all actions in [-1, 1]
Observation space	`Box((21,))`
Tracked variable(s)	measured range (m)
Episode length	200 steps of 1 s (3.33 min)
Cite as	`patrol_bearing_only-v3`
Physics	contract

Action space¶

Actions are normalised to [-1, 1] and mapped onto the plant's real actuator range inside the environment.

#	meaning	min	max
0	power	-1	1
1	stick	-1	1
2	aileron	-1	1

Observation space¶

21 values in the plant's own units. The space declares no bounds (it spans the float32 range), so scale them before training. The reward scores obs[19] against obs[20].

#	meaning
0-9	the wingman's own state, as in `plane3d_heading`
10	range to the lead (m)
11	azimuth of the lead in the wingman's body frame (rad)
12	elevation of the lead (rad)
13-15	the slot in the lead's frame: back, right, up (m)
16	throttle, 0 to 1
17	elevator deflection (rad)
18	aileron deflection (rad)
19	range to the lead (m), the tracked value
20	distance from the lead to the slot (m), its target

Rewards¶

See the environment's compute_reward.

Every environment in this suite scores on one contract. The reward is minus the sum of three non-negative costs, -(tracking_cost + running_cost + failure_cost). The running cost charges consumption and the failure cost charges trips out of the operating envelope. The reward is never positive, and its scale differs by orders of magnitude between plants. Some plants are priced in dollars or euros and the rest are dimensionless; the per-plant summary says which. See Reward shaping for how each cost is built.

Starting state¶

reset samples the initial condition and the target; state has 12 fields.

Episode end¶

Termination. None by default. Leaving the operating envelope trips the plant: the step that leaves it is charged the trip cost and sets info["tripped"] and info["trip_cost"], and the plant restarts as reset_env would while the episode clock keeps running. Episodic use is optional: with continuing = False the plant is lost and terminated is raised on that step instead. See Reward shaping.

Truncation. After 200 steps.

Baselines¶

Recorded over 10 seeds of the 200-step episode (see Baseline results). The reward is a cost, so a return closer to zero is better.

controller	mean return	cost per step
PID	-3.644e+04	182.2
MPC	-3907	19.54

The MPC beats the PID on 10 of 10 seeds and does not trip the plant on any of them.

The PID is a lead-state estimator feeding the same formation hold the full-observation variant uses. Range with azimuth and elevation is a complete relative-position measurement, so the only unobservable quantity is the lead's heading, which the slot needs because it is expressed in the lead's frame; the estimator takes the lead's velocity from the follower's own and the change in the relative position, and its heading from that track less the crab the follower flies in the same air. The MPC slot holds patrol's oracle reading the true state, so it is a full-state bound, not a bearing-only controller. On a task defined by what the observation withholds, its score measures information and control together: what the hidden lead state is worth plus what a policy leaves on the table. It never reads the observation, so its per-step costs equal patrol's seed for seed.

Arguments¶

The scored configuration, which tg.make returns. A keyword argument changes one field, as in tg.make("patrol_bearing_only", max_steps_in_episode=...).

parameter	value	unit
`delta_t`	1	s
`max_steps_in_episode`	200	steps
`gravity`	9.81	m/s^2
`initial_mass`	73500	kg
`thrust_output_at_sea_level`	240000	N
`air_density_at_sea_level`	1.225	kg/m^3
`frontal_surface`	12.6	m^2
`wings_surface`	122.6	m^2
`C_x0`	0.095	-
`C_z0`	0.9	-
`initial_fuel_quantity`	19088	kg
`specific_fuel_consumption`	0.0175	kg/(kN.s)
`power_response_rate`	0.05	1/s
`stick_response_rate`	0.9	1/s
`aileron_response_rate`	0.9	1/s
`aspect_ratio`	9.48	-
`aileron_effectiveness`	0.4	-
`cl_alpha`	0.08786	1/deg
`cl0`	0.2	-
`cd0`	0.02	-
`k`	0.045	-
`aoa_stall`	15	deg
`CL_max`	1.5	-
`CL_min`	-1	-
`aoa_stall_width`	3	deg
`M_crit`	0.8	-
`k_shock_stall`	4	-
`shock_stall_floor`	0.25	-
`k_drag`	5	-
`I`	9000000	kg.m^2
`I_x`	2500000	kg.m^2
`moment_arm_stabilizer`	15	m
`moment_arm_wings`	1.5	m
`stabilizer_surface`	27	m^2
`elevator_surface`	10	m^2
`wingspan`	35.8	m
`aileron_surface`	6	m^2
`moment_arm_aileron`	14	m
`C_lp`	-0.4	-
`min_alt`	0	m
`max_alt`	12191.4	m
`precision_floor`	1	m
`heading_precision_floor`	0.0087	rad
`position_precision_floor`	3	m
`reward_version`	2	-
`e_floor_altitude`	1.44	m
`e_tol_altitude`	0	m
`e_floor_heading`	0.0087	rad
`e_floor_path`	3	m
`tracking_exponent`	2	-
`failure_cost`	760790	reward
`restart_steps`	3600	steps
`rho_floor_tracking`	0.0363037	reward
`rho_floor`	0.0363037	reward
`figure8_altitude_amplitude`	200	m
`initial_z_dot`	0	m/s
`initial_x_dot`	200	m/s
`initial_y_dot`	0	m/s
`initial_theta_dot`	0	deg/s
`initial_theta`	0	deg
`initial_phi`	0	deg
`initial_phi_dot`	0	deg/s
`initial_heading`	0	rad
`initial_power`	1	-
`initial_stick`	0	deg
`initial_aileron`	0	deg
`wind_x`	0	m/s
`wind_y`	0	m/s
`wind_z`	0	m/s
`turbulence_sigma`	1.2	m/s
`turbulence_theta`	0.2	1/s
`wind_shear_x`	0	1/s
`wind_shear_y`	0	1/s
`shear_ref_alt`	0	m
`slot_tolerance`	60	m
`heading_tolerance`	0.5236	rad
`min_separation`	25	m
`max_slot_error`	1500	m
`slot_precision_floor`	3	m
`e_floor_slot`	3	m
`e_tol_slot`	0	m
`lead_leg_seconds`	60	s
`follower_spawn_noise`	40	m