Compatibility

Which module runs on which agent, as the agent is built. Generated by tests/agents/test_compatibility.py; do not edit by hand.

yes: supported. no: the algorithm lacks what the module needs (listed below). not yet: the agent has it, and its code does not pass it on yet.

Module	SAC	TD3	REDQ	ASAC	AVG	CrossQ	TQC	IQL	APG	TDMPC2	PPO	PPO/discrete	APO	APO/discrete	UDRL	UDRL/discrete	DAgger	DAgger/discrete	DreamerV3	DreamerV3/discrete	DQN	PQN	TDMPC2MultiTask
AdversarialReward	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet	yes	no	not yet	no	not yet	no	not yet	no	not yet	no	no	no	not yet
BellmanPretrain	yes	not yet	not yet	not yet	no	not yet	not yet	not yet	no	not yet	no	no	no	no	no	no	no	no	not yet	not yet	not yet	no	not yet
BiasVoreDecomposition	yes	yes	yes	yes	yes	yes	yes	yes	no	yes	yes	yes	yes	yes	no	no	no	no	yes	yes	yes	yes	yes
BiasVorePenalty	yes	yes	yes	yes	yes	yes	no	yes	no	not yet	yes	yes	yes	yes	no	no	no	no	not yet	not yet	yes	yes	not yet
CalQL	yes	no	yes	no	no	yes	yes	yes	no	not yet	no	no	no	no	no	no	no	no	no	no	no	no	not yet
CloningPretrain	yes	yes	yes	yes	yes	yes	yes	yes	not yet	not yet	not yet	not yet	yes	not yet	not yet	not yet	yes	not yet	not yet	not yet	no	no	not yet
ConditioningMetrics	yes	yes	yes	yes	yes	yes	yes	yes	not yet	yes	yes	yes	yes	yes	not yet	not yet	not yet	not yet	yes	yes	yes	yes	yes
ConservativeQ	yes	no	yes	yes	yes	yes	yes	yes	no	not yet	no	no	no	no	no	no	no	no	no	no	no	no	not yet
CriticBlend	yes	not yet	not yet	no	not yet	not yet	no	not yet	no	not yet	no	no	no	no	no	no	no	no	no	no	not yet	not yet	not yet
EDGEExploration	yes	not yet	not yet	not yet	not yet	not yet	not yet	not yet	no	not yet	no	no	no	no	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet
ExpertValueMetrics	yes	yes	yes	yes	no	yes	yes	not yet	no	not yet	no	no	no	no	no	no	no	no	no	no	not yet	no	not yet
IBRL	yes	yes	yes	no	no	no	yes	yes	no	not yet	no	no	no	no	no	no	no	no	no	no	no	no	not yet
ImitationLoss	yes	yes	yes	yes	yes	yes	yes	yes	yes	not yet	yes	not yet	yes	not yet	yes	not yet	yes	not yet	not yet	not yet	no	no	not yet
JSRLCurriculum	yes	not yet	not yet	not yet	not yet	not yet	not yet	not yet	no	not yet	no	no	no	no	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet
KernelRepulsion	yes	yes	yes	yes	no	yes	yes	yes	no	not yet	no	no	no	no	no	no	no	no	no	no	no	no	not yet
LCBGatedBootstrap	yes	yes	yes	no	no	no	yes	yes	no	not yet	no	no	no	no	no	no	no	no	no	no	no	no	not yet
MCPretrain	yes	not yet	not yet	no	not yet	not yet	not yet	not yet	no	not yet	not yet	not yet	no	no	no	no	no	no	not yet	not yet	not yet	not yet	not yet
MCVarianceCorrection	yes	not yet	not yet	no	no	not yet	no	not yet	no	not yet	no	no	no	no	no	no	no	no	no	no	no	no	not yet
OnlineBC	yes	not yet	not yet	no	not yet	not yet	not yet	not yet	no	not yet	no	no	no	no	no	no	no	no	no	no	no	no	not yet
PhiRefresh	yes	not yet	not yet	no	no	not yet	not yet	not yet	no	not yet	no	no	no	no	no	no	no	no	no	no	not yet	no	not yet
PriorData	yes	yes	yes	yes	no	yes	yes	yes	no	not yet	no	no	no	no	not yet	not yet	not yet	not yet	not yet	not yet	yes	no	not yet
ResidualPolicy	yes	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet	no	not yet	no	not yet	no	not yet	no	not yet	no	no	no	not yet
TargetBC	not yet	yes	not yet	no	not yet	not yet	not yet	not yet	no	not yet	no	no	no	no	no	no	no	no	no	no	no	no	not yet
ValueBox	yes	not yet	not yet	no	not yet	not yet	not yet	not yet	no	not yet	no	no	no	no	no	no	no	no	no	no	not yet	not yet	not yet
ImitationLoss(q_filter=True)	yes	yes	yes	yes	yes	yes	yes	yes	no	not yet	no	no	no	no	no	no	no	no	no	no	no	no	not yet
ImitationLoss(target='replay', alpha=2.5)	yes	yes	yes	yes	no	yes	yes	yes	no	not yet	no	no	no	no	no	no	no	no	no	no	no	no	not yet
memory	yes	yes	yes	yes	not yet	not yet	yes	not yet	yes	no	yes	yes	not yet	not yet	not yet	not yet	not yet	not yet	no	no	not yet	not yet	no
cnn	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet	yes	yes	not yet	not yet	not yet	not yet	not yet	not yet	not yet	not yet	yes	yes	not yet

Refused: what the algorithm lacks

• AdversarialReward needs continuous_actions: not on PPO/discrete, APO/discrete, UDRL/discrete, DAgger/discrete, DreamerV3/discrete, DQN, PQN
• BellmanPretrain needs critic: not on UDRL, UDRL/discrete, DAgger, DAgger/discrete
• BellmanPretrain needs critic, replay: not on APG
• BellmanPretrain needs replay: not on AVG, PPO, PPO/discrete, APO, APO/discrete, PQN
• BiasVoreDecomposition needs critic: not on APG, UDRL, UDRL/discrete, DAgger, DAgger/discrete
• BiasVorePenalty needs critic, scalar_target: not on APG, UDRL, UDRL/discrete, DAgger, DAgger/discrete
• BiasVorePenalty needs scalar_target: not on TQC
• CalQL needs actor, continuous_actions, policy_density: not on DQN
• CalQL needs actor, continuous_actions, policy_density, replay: not on PQN
• CalQL needs continuous_actions, critic, discounted, policy_density, q_critic: not on DAgger/discrete
• CalQL needs continuous_actions, critic, discounted, q_critic: not on UDRL/discrete
• CalQL needs continuous_actions, discounted, q_critic, replay: not on APO/discrete
• CalQL needs continuous_actions, q_critic: not on DreamerV3/discrete
• CalQL needs continuous_actions, q_critic, replay: not on PPO/discrete
• CalQL needs critic, discounted, policy_density, q_critic: not on DAgger
• CalQL needs critic, discounted, policy_density, q_critic, replay: not on APG
• CalQL needs critic, discounted, q_critic: not on UDRL
• CalQL needs discounted: not on ASAC
• CalQL needs discounted, q_critic, replay: not on APO
• CalQL needs policy_density: not on TD3
• CalQL needs q_critic: not on DreamerV3
• CalQL needs q_critic, replay: not on PPO
• CalQL needs replay: not on AVG
• CloningPretrain needs actor, continuous_actions: not on DQN, PQN
• ConservativeQ needs actor, continuous_actions, policy_density: not on DQN, PQN
• ConservativeQ needs continuous_actions, critic, policy_density, q_critic: not on DAgger/discrete
• ConservativeQ needs continuous_actions, critic, q_critic: not on UDRL/discrete
• ConservativeQ needs continuous_actions, q_critic: not on PPO/discrete, APO/discrete, DreamerV3/discrete
• ConservativeQ needs critic, policy_density, q_critic: not on APG, DAgger
• ConservativeQ needs critic, q_critic: not on UDRL
• ConservativeQ needs policy_density: not on TD3
• ConservativeQ needs q_critic: not on PPO, APO, DreamerV3
• CriticBlend needs critic, discounted, q_critic, scalar_target: not on APG, UDRL, UDRL/discrete, DAgger, DAgger/discrete
• CriticBlend needs discounted: not on ASAC
• CriticBlend needs discounted, q_critic: not on APO, APO/discrete
• CriticBlend needs q_critic: not on PPO, PPO/discrete, DreamerV3, DreamerV3/discrete
• CriticBlend needs scalar_target: not on TQC
• EDGEExploration needs off_policy: not on APG, PPO, PPO/discrete, APO, APO/discrete
• ExpertValueMetrics needs q_critic: not on UDRL, UDRL/discrete, DAgger, DAgger/discrete, DreamerV3, DreamerV3/discrete
• ExpertValueMetrics needs q_critic, replay: not on APG, PPO, PPO/discrete, APO, APO/discrete
• ExpertValueMetrics needs replay: not on AVG, PQN
• IBRL needs actor: not on DQN
• IBRL needs actor, target_critic: not on PQN
• IBRL needs critic, discounted, q_critic, target_critic: not on APG, UDRL, UDRL/discrete, DAgger, DAgger/discrete
• IBRL needs discounted: not on ASAC
• IBRL needs discounted, q_critic, target_critic: not on APO, APO/discrete
• IBRL needs q_critic, target_critic: not on PPO, PPO/discrete, DreamerV3, DreamerV3/discrete
• IBRL needs target_critic: not on AVG, CrossQ
• ImitationLoss needs actor, continuous_actions: not on DQN, PQN
• ImitationLoss(q_filter=True) needs actor, continuous_actions: not on DQN, PQN
• ImitationLoss(q_filter=True) needs continuous_actions, q_critic: not on PPO/discrete, APO/discrete, UDRL/discrete, DAgger/discrete, DreamerV3/discrete
• ImitationLoss(q_filter=True) needs q_critic: not on APG, PPO, APO, UDRL, DAgger, DreamerV3
• ImitationLoss(target='replay', alpha=2.5) needs actor, continuous_actions: not on DQN
• ImitationLoss(target='replay', alpha=2.5) needs actor, continuous_actions, replay: not on PQN
• ImitationLoss(target='replay', alpha=2.5) needs continuous_actions, q_critic: not on UDRL/discrete, DAgger/discrete, DreamerV3/discrete
• ImitationLoss(target='replay', alpha=2.5) needs continuous_actions, q_critic, replay: not on PPO/discrete, APO/discrete
• ImitationLoss(target='replay', alpha=2.5) needs q_critic: not on UDRL, DAgger, DreamerV3
• ImitationLoss(target='replay', alpha=2.5) needs q_critic, replay: not on APG, PPO, APO
• ImitationLoss(target='replay', alpha=2.5) needs replay: not on AVG
• JSRLCurriculum needs off_policy: not on APG, PPO, PPO/discrete, APO, APO/discrete
• KernelRepulsion needs critic, critic_ensemble, q_critic: not on APG, UDRL, UDRL/discrete, DAgger, DAgger/discrete
• KernelRepulsion needs critic_ensemble: not on AVG, DQN, PQN
• KernelRepulsion needs critic_ensemble, q_critic: not on PPO, PPO/discrete, APO, APO/discrete, DreamerV3, DreamerV3/discrete
• LCBGatedBootstrap needs actor, critic_ensemble: not on DQN
• LCBGatedBootstrap needs actor, critic_ensemble, target_critic: not on PQN
• LCBGatedBootstrap needs critic, critic_ensemble, discounted, q_critic, target_critic: not on APG, UDRL, UDRL/discrete, DAgger, DAgger/discrete
• LCBGatedBootstrap needs critic_ensemble, discounted, q_critic, target_critic: not on APO, APO/discrete
• LCBGatedBootstrap needs critic_ensemble, q_critic, target_critic: not on PPO, PPO/discrete, DreamerV3, DreamerV3/discrete
• LCBGatedBootstrap needs critic_ensemble, target_critic: not on AVG
• LCBGatedBootstrap needs discounted: not on ASAC
• LCBGatedBootstrap needs target_critic: not on CrossQ
• MCPretrain needs critic, discounted: not on APG, UDRL, UDRL/discrete, DAgger, DAgger/discrete
• MCPretrain needs discounted: not on ASAC, APO, APO/discrete
• MCVarianceCorrection needs critic, critic_ensemble, discounted, q_critic, scalar_target: not on APG, UDRL, UDRL/discrete, DAgger, DAgger/discrete
• MCVarianceCorrection needs critic_ensemble: not on AVG, DQN, PQN
• MCVarianceCorrection needs critic_ensemble, discounted, q_critic: not on APO, APO/discrete
• MCVarianceCorrection needs critic_ensemble, q_critic: not on PPO, PPO/discrete, DreamerV3, DreamerV3/discrete
• MCVarianceCorrection needs discounted: not on ASAC
• MCVarianceCorrection needs scalar_target: not on TQC
• OnlineBC needs actor, continuous_actions: not on DQN, PQN
• OnlineBC needs continuous_actions, critic, discounted, q_critic: not on UDRL/discrete, DAgger/discrete
• OnlineBC needs continuous_actions, discounted, q_critic: not on APO/discrete
• OnlineBC needs continuous_actions, q_critic: not on PPO/discrete, DreamerV3/discrete
• OnlineBC needs critic, discounted, q_critic: not on APG, UDRL, DAgger
• OnlineBC needs discounted: not on ASAC
• OnlineBC needs discounted, q_critic: not on APO
• OnlineBC needs q_critic: not on PPO, DreamerV3
• PhiRefresh needs critic, discounted, q_critic: not on UDRL, UDRL/discrete, DAgger, DAgger/discrete
• PhiRefresh needs critic, discounted, q_critic, replay: not on APG
• PhiRefresh needs discounted: not on ASAC
• PhiRefresh needs discounted, q_critic, replay: not on APO, APO/discrete
• PhiRefresh needs q_critic: not on DreamerV3, DreamerV3/discrete
• PhiRefresh needs q_critic, replay: not on PPO, PPO/discrete
• PhiRefresh needs replay: not on AVG, PQN
• PriorData needs off_policy, replay: not on APG, PPO, PPO/discrete, APO, APO/discrete
• PriorData needs replay: not on AVG, PQN
• ResidualPolicy needs continuous_actions: not on PPO/discrete, APO/discrete, UDRL/discrete, DAgger/discrete, DreamerV3/discrete, DQN, PQN
• TargetBC needs actor, continuous_actions: not on DQN, PQN
• TargetBC needs continuous_actions, critic, discounted, q_critic: not on UDRL/discrete, DAgger/discrete
• TargetBC needs continuous_actions, discounted, q_critic: not on APO/discrete
• TargetBC needs continuous_actions, q_critic: not on PPO/discrete, DreamerV3/discrete
• TargetBC needs critic, discounted, q_critic: not on APG, UDRL, DAgger
• TargetBC needs discounted: not on ASAC
• TargetBC needs discounted, q_critic: not on APO
• TargetBC needs q_critic: not on PPO, DreamerV3
• ValueBox needs critic, discounted, off_policy, q_critic: not on APG
• ValueBox needs critic, discounted, q_critic: not on UDRL, UDRL/discrete, DAgger, DAgger/discrete
• ValueBox needs discounted: not on ASAC
• ValueBox needs discounted, off_policy, q_critic: not on APO, APO/discrete
• ValueBox needs off_policy, q_critic: not on PPO, PPO/discrete
• ValueBox needs q_critic: not on DreamerV3, DreamerV3/discrete
• memory needs obs_networks: not on TDMPC2, DreamerV3, DreamerV3/discrete, TDMPC2MultiTask

Not yet wired

• only SAC folds action and transform_action (batch module-reach): EDGEExploration on TD3 REDQ TQC CrossQ ASAC AVG UDRL UDRL/discrete DAgger DAgger/discrete DQN PQN IQL; JSRLCurriculum on TD3 REDQ TQC CrossQ ASAC AVG UDRL UDRL/discrete DAgger DAgger/discrete DQN PQN IQL; ResidualPolicy on TD3 REDQ TQC CrossQ ASAC AVG APG PPO APO UDRL DAgger IQL; ValueBox on TD3 REDQ TQC CrossQ AVG DQN PQN IQL
• MCPretrain binds only on SAC (batch module-reach): CriticBlend on TD3 REDQ CrossQ AVG DQN PQN IQL; MCPretrain on TD3 REDQ TQC CrossQ AVG PPO PPO/discrete DQN PQN IQL; MCVarianceCorrection on TD3 REDQ CrossQ IQL; OnlineBC on TD3 REDQ TQC CrossQ AVG IQL; PhiRefresh on TD3 REDQ TQC CrossQ DQN IQL
• the agent binds no context for it yet: BellmanPretrain on TD3 REDQ TQC CrossQ ASAC; CloningPretrain on APG PPO UDRL
• the agent takes no expert_policy yet (expert reach): BellmanPretrain on DQN IQL; ExpertValueMetrics on DQN IQL
• no form for this agent yet (cross-entropy for discrete actors; a policy's encoder): CloningPretrain on PPO/discrete APO/discrete UDRL/discrete DAgger/discrete DreamerV3/discrete; ImitationLoss on PPO/discrete APO/discrete UDRL/discrete DAgger/discrete DreamerV3/discrete; ConditioningMetrics on APG UDRL UDRL/discrete DAgger DAgger/discrete
• world models fold no loss phase for extensions and build their own encoders: CriticBlend on TDMPC2 TDMPC2MultiTask; IBRL on TDMPC2 TDMPC2MultiTask; KernelRepulsion on TDMPC2 TDMPC2MultiTask; LCBGatedBootstrap on TDMPC2 TDMPC2MultiTask; MCVarianceCorrection on TDMPC2 TDMPC2MultiTask; OnlineBC on TDMPC2 TDMPC2MultiTask; PhiRefresh on TDMPC2 TDMPC2MultiTask; ValueBox on TDMPC2 TDMPC2MultiTask; ExpertValueMetrics on TDMPC2 TDMPC2MultiTask; ImitationLoss(q_filter=True) on TDMPC2 TDMPC2MultiTask; ImitationLoss(target='replay', alpha=2.5) on TDMPC2 TDMPC2MultiTask; ConservativeQ on TDMPC2 TDMPC2MultiTask; CalQL on TDMPC2 TDMPC2MultiTask; TargetBC on TDMPC2 TDMPC2MultiTask; EDGEExploration on TDMPC2 DreamerV3 DreamerV3/discrete TDMPC2MultiTask; JSRLCurriculum on TDMPC2 DreamerV3 DreamerV3/discrete TDMPC2MultiTask; MCPretrain on TDMPC2 DreamerV3 DreamerV3/discrete TDMPC2MultiTask; BellmanPretrain on TDMPC2 DreamerV3 DreamerV3/discrete TDMPC2MultiTask; BiasVorePenalty on TDMPC2 DreamerV3 DreamerV3/discrete TDMPC2MultiTask; cnn on TDMPC2 DreamerV3 DreamerV3/discrete TDMPC2MultiTask; CloningPretrain on TDMPC2 DreamerV3 TDMPC2MultiTask; ImitationLoss on TDMPC2 DreamerV3 TDMPC2MultiTask; ResidualPolicy on TDMPC2 DreamerV3 TDMPC2MultiTask
• the cnn options are not on this agent yet (batch network-reach): cnn on SAC TD3 REDQ TQC CrossQ ASAC AVG APG APO APO/discrete UDRL UDRL/discrete DAgger DAgger/discrete IQL
• memory is not wired on this agent yet (batch network-reach): memory on APO APO/discrete DQN PQN DAgger DAgger/discrete IQL
• memory on this agent has no published form (deferred): memory on AVG CrossQ UDRL UDRL/discrete
• only TD3 hands on_target the next actions its target scored (ReBRAC's TargetBC): TargetBC on SAC REDQ TQC CrossQ AVG IQL
• only PPO folds the reward phase (GAIL's AdversarialReward): AdversarialReward on SAC TD3 REDQ TQC CrossQ ASAC AVG APG APO UDRL DAgger IQL DreamerV3 TDMPC2 TDMPC2MultiTask
• the agent's own replay is not handed to its pretraining yet: PriorData on UDRL UDRL/discrete DAgger DAgger/discrete TDMPC2 DreamerV3 DreamerV3/discrete TDMPC2MultiTask