- Advanced
- 4 to 5 hours
- 13 min read
- You marked this lesson done
- Updated
After this lesson you can
- Explain imitation learning in plain words and say what a demonstration dataset must contain
- Open a public LeRobotDataset, read its metadata and describe each field
- Train a small policy on demonstrations and watch it run on fixed starting positions
- Say why a policy with a low training error can still fail the task, and name two modern methods that reduce the problem
Read these first
Supports: the Robot learning track and the Master Developer Product Package, where a robot product needs evidence that its policy works. Time: about four to five hours.
1. Why this matters
Writing rules for a robot arm is slow. “Move above the cup, lower by 4 centimetres, close the gripper” works until the cup sits a little differently. A person can show the arm what to do in a minute. If the robot could learn from that, an hour of examples could replace days of rules.
That is imitation learning: training a policy to copy actions that people demonstrated. A policy is a model that reads what the robot senses and chooses its next action. It is the most direct way to get a learned policy, and it is behind much recent progress in robot manipulation.
It moves your effort. You now own a dataset. Who demonstrated? How many times? Did anyone show how to recover from a slip? A policy can only copy what it was shown. Choosing that data is a product decision, and it stays with you.
What the PDM decides here
| You decide | You may delegate to an agent | You verify | Other disciplines involved |
|---|---|---|---|
| Which task to teach, who demonstrates, how many demonstrations, which data you may use and what “worked” means | Writing loading, training and plotting scripts; running training; producing videos | The dataset’s licence and contents, a held-out check, and what the policy really does on screen | Safety (a policy is not a safety guard), legal (dataset licence), operations (who records data and how often) |
2. Key ideas
Observation, action, policy
At each time step the robot has an observation: camera images and joint positions, for example. It sends an action: the next target position for its joints or gripper. A policy maps one to the other.
Behaviour cloning is supervised learning
Behaviour cloning is the simplest imitation learning. Each recorded step is an example: “given this observation, the person did this action”. You train a model to predict the action. The model holds numbers called weights. Training nudges them to shrink a loss: one number for how wrong the predictions are. The nudge direction is the gradient. The library does that calculus for you.
Teleoperation and episodes
A person records demonstrations by teleoperation: driving the robot with a leader arm, a gamepad or a mouse. The software saves what the robot senses and does at a fixed rate, such as 10 times a second. One attempt, from start to finish, is an episode. One time step in it is a frame.
The LeRobotDataset format
Hugging Face makes LeRobot, an open toolkit for robot learning under the Apache 2.0 licence. It defines the LeRobotDataset format. The current version is 3.0 and needs lerobot 0.4.0 or later. From the official documentation:
| Part | What it holds |
|---|---|
data/ Parquet files |
Numbers per frame: state, action, timestamps, episode index |
videos/ MP4 files |
Camera recordings, one stream per camera |
meta/info.json |
Field names and shapes, frames per second, version |
meta/stats.json |
Mean, spread and range of each field, for scaling |
meta/tasks.parquet |
Task sentences, such as “Push the T-shaped block onto the T-shaped target” |
meta/episodes/ |
Where each episode starts and ends |
Parquet is a compact table format. Datasets load from the Hugging Face Hub, a public site for sharing models and data.
Compounding errors
The policy trains on states the demonstrator visited, and a demonstrator rarely errs. When the policy makes a small mistake, it lands in a state that is not in the data. With nothing to copy, it makes a bigger mistake. Errors pile up. The ACT paper names this in its abstract: errors in the policy can compound over time.
Two modern fixes
- Action chunking. Predict the next several actions at once and run them before looking again. That means fewer decisions and fewer chances to drift. ACT (Action Chunking with Transformers) does this. Its paper reports 80 to 90 per cent success on six hard real-world tasks, each learned from about 10 minutes of demonstrations. Those are the authors’ results on their own robot, not a promise for yours.
- Diffusion policy. Generate each action by starting from noise and removing it step by step. The paper reports an average gain of 46.9 per cent over earlier methods across 12 tasks from four benchmarks. It copes when several actions are equally good, such as pushing a block left or right of an obstacle. Averaging two good actions can give a bad one.
LeRobot ships both, as act and diffusion. Its hardware guide gives rough GPU memory needs at batch size 8: about 2 to 6 GB for ACT and 8 to 14 GB for diffusion. It says a CPU-only computer is not meant for training them.
What imitation learning cannot do
- It cannot beat its demonstrators. Slow or sloppy demonstrations give a slow or sloppy policy.
- It cannot recover from a situation it never saw.
- A low loss is not success. Only running the policy tells you.
3. Step by step
The code is small so that it trains on a laptop CPU. It is a teaching baseline, not a good policy. You reuse it in the capstone.
-
Set up. Make a folder
lesson-01/in yourrobot-learningrepository and work inside it: every command and path below is relative to that folder. Create a Python 3.12 virtual environment and install LeRobot with the PushT simulator extra. Follow the official install page for the current command and forffmpeg, which decodes the videos.python3.12 -m venv .venv source .venv/bin/activate pip install "lerobot[pusht]" imageio imageio-ffmpeg -
Look at the data. PushT is a public benchmark: a round pusher moves a T-shaped block onto a T-shaped target. The dataset
lerobot/pushtholds human demonstrations of it. Run this and read every line of the output.from lerobot.datasets.lerobot_dataset import LeRobotDataset dataset = LeRobotDataset("lerobot/pusht") print(dataset.num_episodes, dataset.num_frames, dataset.fps) print(list(dataset.meta.features)) sample = dataset[0] print(sample["observation.image"].shape, sample["observation.state"], sample["action"]) print(sample["task"])On the day this lesson was checked it printed 206 episodes, 25,650 frames and 10 frames a second. The image is 96 by 96 pixels, the state is the pusher’s position and the action is where it moves next. The dataset card on the Hub names the licence: MIT. Copy that into your notes.
-
Decode the data once. Reading video at every training step is slow, so this script decodes it once into one NumPy file. Save it as
src/pusht_bc/prepare_data.py, with an emptysrc/pusht_bc/__init__.pybeside it."""Decode the public lerobot/pusht dataset once and save it as one NumPy file.""" from pathlib import Path import numpy as np import torch from lerobot.datasets.lerobot_dataset import LeRobotDataset OUT = Path("data/pusht.npz") dataset = LeRobotDataset("lerobot/pusht") loader = torch.utils.data.DataLoader(dataset, batch_size=256, num_workers=4) images, states, actions, episodes = [], [], [], [] for batch in loader: images.append((batch["observation.image"] * 255).round().to(torch.uint8).numpy()) states.append(batch["observation.state"].numpy()) actions.append(batch["action"].numpy()) episodes.append(batch["episode_index"].numpy()) OUT.parent.mkdir(exist_ok=True) np.savez_compressed(OUT, image=np.concatenate(images), state=np.concatenate(states), action=np.concatenate(actions), episode=np.concatenate(episodes)) print(f"saved {OUT}: {dataset.num_frames} frames from {dataset.num_episodes} episodes")PYTHONPATH=src python -m pusht_bc.prepare_data -
Write the policy. A small convolutional network reads the picture, and the pusher position joins near the end. The output is a small step from the current position, which is easier to learn than an absolute one. Save as
src/pusht_bc/policy.py."""A small policy: a picture of the table and the pusher position in, the next pusher target out.""" import torch import torch.nn.functional as F from torch import nn WORLD = 512.0 # PushT positions run from 0 to 512 in both axes def prepare(image_u8: torch.Tensor, position: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]: """Turn a (B, 3, 96, 96) uint8 image into (B, 3, 48, 48) floats, and scale positions to 0..1.""" image = F.avg_pool2d(image_u8.float() / 255.0, kernel_size=2) return image, position.float() / WORLD class SmallPolicy(nn.Module): def __init__(self) -> None: super().__init__() self.conv = nn.Sequential( nn.Conv2d(3, 16, 5, stride=2), nn.ReLU(), nn.Conv2d(16, 32, 3, stride=2), nn.ReLU(), nn.Conv2d(32, 64, 3, stride=2), nn.ReLU(), nn.AdaptiveAvgPool2d(2), nn.Flatten(), ) self.head = nn.Sequential(nn.Linear(64 * 4 + 2, 128), nn.ReLU(), nn.Linear(128, 2)) def forward(self, image: torch.Tensor, position: torch.Tensor) -> torch.Tensor: # a small step from the current position return position + 0.1 * self.head(torch.cat([self.conv(image), position], dim=1)) -
Train, with episodes held out. The model never trains on the last 20 episodes, so their loss shows if it copies unseen demonstrations. Save as
src/pusht_bc/train.py."""Train the small policy on the demonstrations in data/pusht.npz.""" import argparse import time from pathlib import Path import numpy as np import torch from torch import nn from pusht_bc.policy import SmallPolicy, WORLD, prepare def main() -> None: parser = argparse.ArgumentParser() parser.add_argument("--epochs", type=int, default=10) parser.add_argument("--seed", type=int, default=0) parser.add_argument("--out", type=Path, default=Path("runs/bc_policy.pt")) args = parser.parse_args() torch.manual_seed(args.seed) data = np.load(Path("data/pusht.npz")) image = torch.from_numpy(data["image"]) # uint8, (N, 3, 96, 96) position = torch.from_numpy(data["state"]) # (N, 2), pixels action = torch.from_numpy(data["action"]).float() / WORLD episode = data["episode"] held_out = torch.from_numpy(episode > episode.max() - 20) # the last 20 episodes are never trained on train_idx = torch.nonzero(~held_out).squeeze(1) val_idx = torch.nonzero(held_out).squeeze(1) model = SmallPolicy() optimiser = torch.optim.AdamW(model.parameters(), lr=1e-3) loss_fn = nn.MSELoss() started = time.time() for epoch in range(1, args.epochs + 1): model.train() order = train_idx[torch.randperm(len(train_idx))] for start in range(0, len(order), 128): batch = order[start : start + 128] img, pos = prepare(image[batch], position[batch]) loss = loss_fn(model(img, pos), action[batch]) optimiser.zero_grad() loss.backward() optimiser.step() model.eval() with torch.no_grad(): img, pos = prepare(image[val_idx], position[val_idx]) val_loss = loss_fn(model(img, pos), action[val_idx]).item() print(f"epoch {epoch} val_loss {val_loss:.5f} elapsed {time.time() - started:.0f}s", flush=True) args.out.parent.mkdir(parents=True, exist_ok=True) torch.save(model.state_dict(), args.out) if __name__ == "__main__": main()PYTHONPATH=src python -m pusht_bc.train --epochs 10 | tee train-log.txtOn a 28-core desktop CPU, 10 epochs took under a minute; a laptop may need several. The loss is the mean squared error: the average squared gap between predicted and demonstrated positions.
-
Watch it play. Run the policy on three fixed seeds. A seed fixes the random starting positions, so anyone can repeat the episodes. Save as
src/pusht_bc/rollout.py."""Watch the policy play on fixed seeds and save one video per seed.""" import sys from pathlib import Path import gym_pusht # noqa: F401 registers gym_pusht/PushT-v0 import gymnasium as gym import imageio.v2 as imageio import numpy as np import torch from pusht_bc.policy import SmallPolicy, WORLD, prepare SEEDS = [10_000, 10_001, 10_002] weights = Path(sys.argv[1] if len(sys.argv) > 1 else "runs/bc_policy.pt") policy = SmallPolicy() policy.load_state_dict(torch.load(weights, map_location="cpu", weights_only=True)) policy.eval() env = gym.make("gym_pusht/PushT-v0", obs_type="pixels_agent_pos", render_mode="rgb_array") Path("videos").mkdir(exist_ok=True) for seed in SEEDS: obs, _ = env.reset(seed=seed) frames, best = [], 0.0 done = False while not done: image = torch.from_numpy(obs["pixels"]).permute(2, 0, 1).unsqueeze(0) position = torch.from_numpy(obs["agent_pos"]).unsqueeze(0) with torch.no_grad(): action = policy(*prepare(image, position)).squeeze(0).numpy() * WORLD obs, _, terminated, truncated, info = env.step(np.clip(action, 0.0, WORLD)) frames.append(env.render()) best = max(best, float(info["coverage"])) done = terminated or truncated imageio.mimsave(f"videos/seed{seed}.mp4", frames, fps=10, macro_block_size=1) print(f"seed {seed}: success={info['is_success']} best_coverage={best:.2f} steps={len(frames)}") env.close()PYTHONPATH=src python -m pusht_bc.rolloutThe coverage is how much of the target the block covers. The environment counts success at 95 per cent or more.
-
Read the result honestly. Open the videos. Expect a weak policy, with success at or near zero. That is not a bug in your set-up: a tiny network trained for minutes on 206 demonstrations cannot match published methods. The lesson is what you see. Where does the pusher drift, freeze or push the block the wrong way? Write it down. Lesson 5 turns those notes into a failure taxonomy.
4. Worked example: TrailMug
TrailMug is fictional. The arm, the rack and every number below are invented for teaching.
The TrailMug team imagines a café robot that takes a returned cup from a tray and places it in a wash rack. They have a simulated arm and no real one. A team member drives the arm with a gamepad and records 50 demonstrations. Before training, they fill in a dataset note:
| Field | Their note (example) |
|---|---|
| Source | One person, in the team’s own simulator |
| Licence | Internal use only, not published |
| Size | 50 episodes, about 10 frames per second |
| Inputs | One top camera and the arm’s joint positions |
| Variation | The tray never moves; the cup is always upright |
| Missing | No dropped cup, no tilted cup, no retry after a miss |
They train a small policy. The held-out loss is low. On screen the arm reaches the cup, closes its gripper a little late, and the cup slips. The arm then stops, because it never saw a slipped cup. One small error became an unseen state.
The team blames the data, not the model. They record 30 more episodes in which the demonstrator misses on purpose and then recovers, and they move the tray a few centimetres between episodes. They retrain and test the same fixed starting positions as before. That is the Product Development Manager’s work: reading a failure as a data problem, choosing what to record next, and comparing before and after on the same test.
5. Working with AI agents on this
What the PDM decides. The task, who records the demonstrations, what the dataset may contain and when the policy is good enough. You also decide what you will not claim: a policy that copies demonstrations in a simulator says nothing about the real world.
What you delegate, and the exact brief. You can hand an agent the job of writing the loading, training and rollout scripts. This is the delegate rung: a written brief with acceptance criteria. Adapt this one:
Task: in the folder lesson-01, write a PushT behaviour-cloning baseline for a laptop CPU.
Use the public dataset lerobot/pusht through LeRobotDataset (current lerobot release).
Files under src/pusht_bc/: prepare_data.py, policy.py, train.py, rollout.py.
Rules:
1. Hold out the last 20 episodes. Never train on them. Print their loss each epoch.
2. Take the torch seed from a --seed argument.
3. Run the policy in gym_pusht/PushT-v0 on fixed seeds 10000, 10001 and 10002.
4. Save one MP4 per seed. Print success and best coverage per seed.
5. Nothing may need a GPU. Do not download any other dataset.
6. If unsure of a function name, read the installed package. Do not guess.
Output: the files, the exact commands to run them, and a list of assumptions.
How you verify. Run it yourself. Check that the licence and episode count in your notes match the dataset page. Read the line that splits the data: an agent can write a loop that looks right but trains on the test episodes. Watch all three videos. Do not ask the same agent whether its result is good.
Which disciplines are involved. Safety asks what the robot does when the policy is wrong; a learned policy gives no guarantee. Legal asks whether you may use and share the dataset. Operations plans who records new demonstrations.
Where this sits on the ladder. This is delegate. It becomes orchestrate when several agents train variants at once, and you need one results table. A rule that no run is reported until its held-out check and videos are filed, enforced before a merge, is governing.
Disclose. Add to your README: “Scripts were drafted by an AI agent from my brief. I read every file, ran every command and watched every video.” See Disclose AI use and Reviewing and verifying AI output.
6. Common mistakes
- Trusting the loss. A low held-out loss means the model copies well, not that the robot succeeds. Run the simulator.
- Testing on training data. Hold out whole episodes. Frames from one episode look alike, so a random split of frames leaks.
- Never looking at the videos. Numbers hide how a policy fails. Watch at least three episodes.
- Skipping the licence. Write the dataset’s licence in your notes.
- Claiming too much. A weak baseline on a toy task proves nothing about a real robot.
7. Practice: a dataset note and a baseline
Deliverable: a folder lesson-01/ in your own git repository.
dataset-note.md: forlerobot/pusht, its source, licence, episode and frame counts, frames per second, each field in the features list, and three things the data does not show.- The four scripts from steps 3 to 6, and a
README.mdwith the exact commands, the versions oflerobot,torchandgymnasium(frompip list) and the date. train-log.txtand the three videos.observations.md: one row per video (seed, what the pusher did, how it failed), and a paragraph on what data you would record next and why.- One line in the README on how an AI agent helped.
Check: ask a classmate or Sensei to run your commands in a fresh environment.
8. Checklist
- I can explain a policy, an episode and behaviour cloning in my own words.
- I wrote down the dataset’s licence, size and fields.
- I held out whole episodes and checked the code excludes them.
- I ran the policy on fixed seeds and watched every video.
- I wrote what failed, not only a number.
- I wrote what the data does not show.
- I recorded the library versions and my AI use.
9. Further learning
The LeRobot documentation is the main reference, and the dataset page explains the v3.0 layout. The Hugging Face robotics course teaches the same ideas as a course. The ACT and Diffusion Policy papers describe the two methods above; read the abstracts first, and note that the numbers come from the authors’ own tasks. The hardware guide shows what a GPU does for training before you rent one. Next, Reinforcement learning in MuJoCo removes the teacher: the robot learns by trial and error.
Finished this lesson?
This is only for you. It is saved in this browser, on this device, and nothing is sent anywhere. It is not a grade and not a certificate.
Go deeper
Further learning
Free or low-cost material from other publishers. We opened every link; check the cost label before you start.
Hugging Face
LeRobot (opens the publisher's site in a new tab)
- Free
- Docs
- Intermediate
The official documentation for LeRobot, Hugging Face's open toolkit for robot learning, released under the Apache 2.0 licence. It covers installation, recording and loading datasets, training policies and testing them in simulation. Use it as the reference for the commands in this track, and follow the install steps it shows on the day you work.
Hugging Face
LeRobotDataset v3.0 (opens the publisher's site in a new tab)
- Free
- Docs
- Intermediate
The format page for LeRobot datasets: Parquet files for numbers, MP4 files for video and a meta folder that describes them. It shows how to record a dataset, load it for training and stream it from the Hugging Face Hub. Read it before you collect or share demonstrations.
Hugging Face
Welcome to the Robotics Course (opens the publisher's site in a new tab)
- Free
- Course
- Beginner
Explore classical robotics and learning-based approaches using the LeRobot ecosystem; some later units are still being developed. Ask an assistant to plan tests for a simulated robot and compare the plan to the course material. Practice can start with datasets or simulation; hardware and cloud compute can cost money, and physical use requires independent safety review.
arXiv
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware (opens the publisher's site in a new tab)
- Free
- Article
- Advanced
The paper that introduced Action Chunking with Transformers (ACT). It names the problem of errors that compound over time in imitation learning, and reports 80 to 90 per cent success on six real-world tasks from about 10 minutes of demonstrations each. Those results are the authors' own, on their own hardware.
arXiv
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (opens the publisher's site in a new tab)
- Free
- Article
- Advanced
The paper that represents a robot policy as a denoising diffusion process. It reports results on 12 tasks from four robot manipulation benchmarks and explains why the method handles actions that have several good answers. Read the abstract first, then the evaluation section.
Hugging Face
Compute HW Guide for LeRobot Training (opens the publisher's site in a new tab)
- Free
- Docs
- Intermediate
Rough GPU memory needs and training times for LeRobot policies, from small action-chunking models to large vision-language-action models. The guide says its numbers are indicative, not exact. Read it before you rent or buy compute.



