The Beginner's RL Playground

An interactive web-based demonstration of fundamental tabular Reinforcement Learning (RL) algorithms in a simple grid world environment. Built with vanilla HTML, CSS, and JavaScript. * Interactive Grid World: Represents a simple Markov Decision Process (MDP). * Configurable grid size (2x2 to 10x10). * Agent (🤖), Gems (💎), Hazards (☠️), and Walls (🚧) representing states, rewards, and transitions. * Click to cycle cell states (💎 -> ☠️ -> 🚧 -> empty). * Shift+Click to set the agent's starting position (🏠). * RL Algorithms: Demonstrating various approaches to learning optimal behavior. * Q-Learning: (Off-Policy Temporal Difference Learning) * SARSA: (On-Policy Temporal Difference Learning) * Expected SARSA: (On-Policy Temporal Difference Learning) * Monte Carlo (Control): (Learning from complete episode trajectories) * Actor-Critic (Advantage Actor-Critic): (On-Policy, combining policy and value-based learning) * Successor Representation (SR): (Learning a predictive state representation, decoupling states and rewards) * Exploration Strategies: Illustrating the Exploration vs. Exploitation trade-off. * ε-Greedy (Used for Q-Learning, SARSA, Expected SARSA, Successor Representation) * Softmax (Boltzmann) (Used for Q-Learning, SARSA, Expected SARSA, Actor-Critic, Successor Representation) * Configurable Parameters: Allowing experimentation with key RL hyperparameters. * Learning Rate (α): Controls update step size, affecting convergence speed and stability. * Discount Factor (γ): Determines the importance of future rewards (Discounting). * Exploration Rate (ε): (Used for ε-Greedy) * Softmax Temperature (β): (Used for Softmax exploration & Actor-Critic policy) * Step Penalty: Influences the agent's path preference. * Max Steps per Episode: Defines episode termination conditions. * Simulation Speed: Controls visualization pace. * Shareable Configurations: Save and share complete algorithm and environment settings via URL parameters. * Visualizations: Providing insights into the learning process and learned knowledge. * Real-time agent movement and learning. * Animated floating text for rewards received. * Cell display modes: * State Value `V(s)` (Color / Color + Text): Visualizing the learned Value Function. Computed via Q-max, V-table, or SR (M, w). * Greedy Policy `π(s)` (Arrows colored by probability): Visualizing the learned Policy. * Current state's Action Values `Q(s,a)` display: Showing the expected return for each action (derived from Q-table or SR). Demonstrates the core of value-based methods. * Current state's Action Probabilities `π(a|s)` display: Showing the agent's current action selection distribution (Policy). * Learning progress plot (Episodic Reward - Raw & Smoothed): Tracking overall performance over time using Chart.js. * Educational Content: * In-app explanations for selected algorithms and exploration strategies. * Mathematical formulas (like components of the Bellman Equation) rendered using KaTeX. * Controls: * Start / Stop Learning. * Reset Algorithm (clears learned values like Q-tables, V-tables, policies, SR matrices). * Reset Environment (resets grid layout). * Responsive Design: Adapts to different screen sizes. 2. Open the index.html file in your web browser. A simple way is often to use a live server extension in your code editor (like VS Code's Live Server) or run a basic HTTP server: The Beginner's RL Playground is the open-source project awjuliani/web-rl-playground (https://github.com/awjuliani/web-rl-playground) by awjuliani, released under MIT. Ion packages it unmodified from commit 4855420b6977, adding only a manifest, an icon and a storage compatibility script.

Written by awjuliani, published to Oikou by Ben Grande.

• ★ 105 stars on GitHub
• Licence: MIT
• Upstream version v1.2.0
• Built from commit 4855420b6977

View the repository on GitHub

Not rated. Nobody has said who this app is suitable for, so Personal Clouds and the Oikou app treat it as 18+.

Internet

It asks to reach these hosts, and no others:

• cdn.jsdelivr.net
• upload.wikimedia.org

Personal Cloud tools

• Keep a database of its own on your Personal Cloud ion.db

Notifications

It does not send push notifications.

Storage

Up to 16 MB on your Personal Cloud.

Sign in to add this web app to a Personal Cloud of yours. It runs there, on your own computer, and you open it from there; nothing is installed on your phone.

Sign in to add