LLMs Learn to Avoid Pedestrians in Driving Simulations
dscape · x · 2026-07-15
This repost highlights a presentation from Hugging Face at the Lisbon AI stage by TRL maintainer Sergio Paniego.
The talk focuses on two main areas:
- He is developing OpenEnv, an open-source toolkit designed for "training models in environments with real consequences."
- He also demonstrates a CARLA driving simulation experiment: by placing an LLM in a physics-based simulator, a 0.6B model learns to brake and steer when pedestrians appear after just 50 training steps.
The overall theme is how to train language models within embodied or simulated environments rather than solely on text tasks.
More from Embodied
- Tesla expands Robotaxi rides to seven areas, including new Orlando and Tampa zones — elonmusk · 2026-07-22
- Hands-on robotics workshop on Saturday may be the last in-person session before August — StewartalsopIII · 2026-07-22
- NVIDIA pitches World Foundation Models as a way to scale physical AI data generation — MonaJalal_ · 2026-07-22
- RoboMME Podcast Preview: Benchmarking Memory for Robotic Policies — chris_j_paxton · 2026-07-21
- Gritt says an 8-person crew now installs 3,000 to 4,000 solar panels a day — HaktanSuren · 2026-07-21
- A helium-powered flying robot whale aims to be a quiet companion pet — chris_j_paxton · 2026-07-21