openrl.selfplay.callbacks package

Submodules

openrl.selfplay.callbacks.base_callback module

class openrl.selfplay.callbacks.base_callback.BaseSelfplayCallback(verbose: int = 0)

Bases: openrl.utils.callbacks.callbacks.BaseCallback

openrl.selfplay.callbacks.selfplay_api module

openrl.selfplay.callbacks.selfplay_callback module

class openrl.selfplay.callbacks.selfplay_callback.SelfplayCallback(save_freq: int, opponent_pool_path: Union[str, pathlib.Path], api_address: str, name_prefix: str = 'opponent', save_replay_buffer: bool = False, opponent_template: Optional[str] = None, clear_past_opponents: bool = False, copy_script_file: bool = False, verbose: int = 0)

Bases: openrl.selfplay.callbacks.base_callback.BaseSelfplayCallback

Callback for saving a model every save_freq calls to env.step(). By default, it only saves model checkpoints, you need to pass save_replay_buffer=True to save replay buffer checkpoints.

Warning

When using multiple environments, each call to env.step() will effectively correspond to n_envs steps. To account for that, you can use save_freq = max(save_freq // n_envs, 1)

Parameters

• save_freq – Save checkpoints every save_freq call of the callback.

• opponent_pool_path – Path to the folder where the model will be saved.

• name_prefix – Common prefix to the saved models

• save_replay_buffer – Save the model replay buffer

• verbose – Verbosity level: 0 for no output, 2 for indicating when saving model checkpoint

get_opponent_path(checkpoint_type: str = '', extension: str = '') → pathlib.Path

Helper to get checkpoint path for each type of checkpoint.

Parameters

• checkpoint_type – empty for the model, “replay_buffer_” for the other checkpoints.

• extension – Checkpoint file extension (zip for model, pkl for others)

Returns

Path to the checkpoint

save_opponent()

Module contents