openrl.modules.networks package

Subpackages

• openrl.modules.networks.utils package
  • Subpackages
    • openrl.modules.networks.utils.nlp package
      • Submodules
      • openrl.modules.networks.utils.nlp.base_policy module
      • openrl.modules.networks.utils.nlp.causal_policy module
      • Module contents
  • Submodules
  • openrl.modules.networks.utils.act module
  • openrl.modules.networks.utils.attention module
  • openrl.modules.networks.utils.cnn module
  • openrl.modules.networks.utils.distributed_utils module
  • openrl.modules.networks.utils.distributions module
  • openrl.modules.networks.utils.mix module
  • openrl.modules.networks.utils.mlp module
  • openrl.modules.networks.utils.popart module
  • openrl.modules.networks.utils.rnn module
  • openrl.modules.networks.utils.running_mean_std module
  • openrl.modules.networks.utils.transformer_act module
  • openrl.modules.networks.utils.util module
  • openrl.modules.networks.utils.vdn module
  • Module contents

Submodules

openrl.modules.networks.MAT_network module

class openrl.modules.networks.MAT_network.DecodeBlock(n_embd, n_head, n_agent)

Bases: torch.nn.modules.module.Module

an unassuming Transformer block

forward(x, rep_enc)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class openrl.modules.networks.MAT_network.Decoder(obs_dim, action_dim, n_block, n_embd, n_head, n_agent, action_type='Discrete', dec_actor=False, share_actor=False)

Bases: torch.nn.modules.module.Module

forward(action, obs_rep, obs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

zero_std(device)

class openrl.modules.networks.MAT_network.EncodeBlock(n_embd, n_head, n_agent)

Bases: torch.nn.modules.module.Module

an unassuming Transformer block

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class openrl.modules.networks.MAT_network.Encoder(state_dim, obs_dim, n_block, n_embd, n_head, n_agent, encode_state)

Bases: torch.nn.modules.module.Module

forward(state, obs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class openrl.modules.networks.MAT_network.MultiAgentTransformer(cfg, input_space, action_space, device=device(type='cpu'), use_half=False, extra_args=None)

Bases: openrl.modules.networks.base_value_policy_network.BaseValuePolicyNetwork

eval_actions(obs, rnn_states, action, masks, action_masks=None, active_masks=None)

get_actions(obs, rnn_states_actor=None, masks=None, action_masks=None, deterministic=False)

get_actor_para()

get_critic_para()

get_values(critic_obs, rnn_states_critic=None, masks=None)

zero_std()

class openrl.modules.networks.MAT_network.SelfAttention(n_embd, n_head, n_agent, masked=False)

Bases: torch.nn.modules.module.Module

forward(key, value, query)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

openrl.modules.networks.MAT_network.init_(m, gain=0.01, activate=False)

openrl.modules.networks.base_policy_network module

class openrl.modules.networks.base_policy_network.BasePolicyNetwork(cfg, device)

Bases: torch.nn.modules.module.Module

openrl.modules.networks.base_value_network module

class openrl.modules.networks.base_value_network.BaseValueNetwork(cfg, device)

Bases: abc.ABC, torch.nn.modules.module.Module

abstract forward()

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

openrl.modules.networks.base_value_policy_network module

class openrl.modules.networks.base_value_policy_network.BaseValuePolicyNetwork(cfg, device)

Bases: abc.ABC, torch.nn.modules.module.Module

abstract eval_actions(*args, **kwargs)

forward(forward_type, *args, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

abstract get_actions(*args, **kwargs)

get_actor_para()

get_critic_para()

abstract get_values(*args, **kwargs)

openrl.modules.networks.ddpg_network module

class openrl.modules.networks.ddpg_network.ActorNetwork(cfg, input_space, action_space, device=device(type='cpu'), use_half=False, extra_args=None)

Bases: openrl.modules.networks.base_policy_network.BasePolicyNetwork

forward(obs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class openrl.modules.networks.ddpg_network.CriticNetwork(cfg, input_space, action_space, device=device(type='cpu'), use_half=False, extra_args=None)

Bases: openrl.modules.networks.base_value_network.BaseValueNetwork

forward(state, action, rnn_states, masks)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class openrl.modules.networks.ddpg_network.CriticNetwork_v0(cfg, input_space, action_space, device=device(type='cpu'), use_half=False)

Bases: openrl.modules.networks.base_value_network.BaseValueNetwork

forward(state, action, rnn_states, masks)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

openrl.modules.networks.gail_discriminator module

class openrl.modules.networks.gail_discriminator.Discriminator(cfg, input_space, action_space, device, use_half, extra_args=None)

Bases: torch.nn.modules.module.Module

compute_grad_pen(expert_state, expert_action, policy_state, policy_action, lambda_=10)

predict_reward(state, action, gamma, masks, update_rms=True)

update(expert_loader, buffer, obsfilt=None)

openrl.modules.networks.policy_network module

class openrl.modules.networks.policy_network.PolicyNetwork(cfg, input_space, action_space, device=device(type='cpu'), use_half=False, extra_args=None)

Bases: openrl.modules.networks.base_policy_network.BasePolicyNetwork

eval_actions(obs, rnn_states, action, masks, action_masks=None, active_masks=None)

forward(forward_type, *args, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

forward_original(raw_obs, rnn_states, masks, action_masks=None, deterministic=False)

get_policy_values(obs, rnn_states, masks)

openrl.modules.networks.policy_network_gpt module

openrl.modules.networks.policy_value_network module

class openrl.modules.networks.policy_value_network.PolicyValueNetwork(cfg, input_space, action_space, device=device(type='cpu'), use_half=False, extra_args=None)

Bases: openrl.modules.networks.base_value_policy_network.BaseValuePolicyNetwork

eval_actions(obs, rnn_states, action, masks, action_masks, active_masks=None)

get_actions(obs, rnn_states, masks, action_masks=None, deterministic=False)

get_values(critic_obs, rnn_states, masks)

openrl.modules.networks.policy_value_network_gpt module

openrl.modules.networks.policy_value_network_sb3 module

openrl.modules.networks.q_network module

class openrl.modules.networks.q_network.QNetwork(cfg, input_space, action_space, device=device(type='cpu'), use_half=False, extra_args=None)

Bases: openrl.modules.networks.base_value_network.BaseValueNetwork

forward(obs, rnn_states, masks, action_masks=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

openrl.modules.networks.sac_network module

class openrl.modules.networks.sac_network.SACActorNetwork(cfg, input_space, action_space, device=device(type='cpu'), use_half=False, extra_args=None, log_std_min=- 20, log_std_max=2)

Bases: openrl.modules.networks.ddpg_network.ActorNetwork

evaluate(obs, deterministic=True)

forward(obs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

openrl.modules.networks.value_network module

class openrl.modules.networks.value_network.ValueNetwork(cfg, input_space, action_space=None, use_half=False, device=device(type='cpu'), extra_args=None)

Bases: openrl.modules.networks.base_value_network.BaseValueNetwork

forward(critic_obs, rnn_states, masks)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

openrl.modules.networks.value_network_gpt module

openrl.modules.networks.vdn_network module

class openrl.modules.networks.vdn_network.VDNNetwork(cfg, input_space, action_space, device=device(type='cpu'), use_half=False, extra_args=None)

Bases: openrl.modules.networks.base_value_policy_network.BaseValuePolicyNetwork

eval_actions(obs, rnn_states, action, masks, action_masks, active_masks=None)

eval_actions_target(obs, rnn_states, action, masks, action_masks, active_masks=None)

eval_values(obs, rnn_states, masks, action_masks=None)

forward(forward_type, *args, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

get_actions(*args, **kwargs)

get_values(obs, rnn_states, masks, action_masks=None)

Module contents