Description
PlaNet is a purely model-based reinforcement learning algorithm designed to solve control tasks directly from pixel inputs. It achieves this by learning a compact sequence of hidden states that model the world's dynamics. For planning, PlaNet first encodes the history of past images into the current state. From this state, it efficiently predicts future rewards for various action sequences within its learned latent space.
The algorithm operates by executing the first action of the most promising sequence identified through planning. After observing the next image, it replans. This iterative process allows PlaNet to make decisions based on a predictive model of the environment, rather than relying solely on trial and error. A key advantage of PlaNet is its efficiency, requiring substantially less interaction with the environment compared to many model-free reinforcement learning methods, while still achieving competitive performance.
This project offers the open-source implementation of the PlaNet agent, enabling researchers and developers to utilize and build upon its capabilities. The implementation includes components for learning latent transition models, encoder and decoder networks, and scripts for training agents on various tasks. Instructions are provided for training an agent, including dependency installation and command-line arguments for task selection and parameter configuration.
The PlaNet repository is archived and read-only as of May 28, 2024. However, the code remains available for inspection and use. Modifications to the code can be made by exploring directories like `scripts/configs.py` for parameter adjustments, `scripts/tasks.py` for environment modifications, and `models` and `networks` for altering latent transition models and neural network architectures, respectively. Tips for development include using a debug configuration for faster iteration and exploring different environment isolation strategies.
The project was tested under Ubuntu 18 and requires specific package versions, including `tensorflow-gpu==1.13.1`, `tensorflow_probability==0.6.0`, `dm_control`, `gym`, `scikit-image`, `scipy`, `ruamel.yaml`, and `matplotlib`. It is important to note that this is not an official Google product. The project website and a PDF of the associated paper provide further details on the method and its applications.
PlaNet Highlights
Model-based reinforcement learning
Planning from pixels
Learned latent dynamics
Efficient prediction of future rewards
Model-free method comparison
Reduced environmental interaction
Open-source implementation
Latent transition models
Encoder and decoder networks
Reinforcement learning agent training
Control tasks from images
Getting Started with PlaNet
Access model: Clone the GitHub repository.
Set up environment: Install Python dependencies including TensorFlow and dm_control.
Integrate via API: Run training scripts with specified parameters and log directories.
Optimise: Modify configuration files for tasks, parameters, and network architectures.
PlaNet's Use Cases
- Robotics control
- Autonomous navigation
- Game playing
- Simulated environments
- Visual reinforcement learning
