OpenAI Gym

Integrate W&B with OpenAI Gym to track reinforcement learning experiments and record episode performance videos.

“The team that has been maintaining Gym since 2021 has moved all future development to Gymnasium, a drop in replacement for Gym (import gymnasium as gym), and Gym will not be receiving any future updates.” (Source)

Since Gym is no longer a maintained project, try out the integration with Gymnasium.

This page describes how to use W&B with OpenAI Gym to automatically capture videos of your reinforcement learning environments. You can then review agent behavior alongside your experiment metrics in W&B.

If you’re using OpenAI Gym, W&B automatically logs videos of your environment generated by gym.wrappers.Monitor. To enable this, set the monitor_gym keyword argument to wandb.init() to True, or call wandb.gym.monitor().

The gym integration is lightweight. It looks at the name of the video file being logged from gym and names it after that, or falls back to "videos" if it doesn’t find a match. If you want more control, you can manually log a video.

The OpenRL Benchmark by CleanRL uses this integration for its OpenAI Gym examples. You can find source code (including the code used for specific runs) that demonstrates how to use gym with