1. [Topics](/topics) 2. [Drive](/drive) 3. Model Development # Autonomous Vehicle Model Development: Open Models & Tools Discover open source models, datasets, and training frameworks for building and deploying reasoning-based AV models. Explore toolkits, SDKs, and more. [Download Center](/drive/downloads) Key Topics - [Data Curation](https://developer.nvidia.com) - [Scenario Retrieval](https://developer.nvidia.com) - [Automated Labeling](https://developer.nvidia.com) - [VLA Model Training](https://developer.nvidia.com) ## Understanding Model Development Model development for autonomous vehicles starts with data collected from fleet vehicles, curated and annotated with [NVIDIA Cosmos™ Curator](https://github.com/NVIDIA/cosmos-curator), and augmented with synthetic scenarios from [NVIDIA Cosmos](/cosmos) world foundation models. [NVIDIA Cosmos Dataset Search](https://github.com/NVIDIA-Omniverse-blueprints/cosmos-dataset-search) retrieves rare and long-tail scenarios instantly, accelerating post-training data pipelines. [NVIDIA Alpamayo VLA reasoning model](https://huggingface.co/nvidia/Alpamayo-1.5-10B) is then fine-tuned on those datasets, with [NVIDIA AlpaGym](https://github.com/NVlabs/alpagym) running closed-loop reinforcement learning post-training. ![Reference diagram highlighting the model development portion of the pipeline.](https://developer.download.nvidia.com/images/drive/autonomous-vehicle-model-pipeline-ari-1920x1080.jpg) * * * ## Tools for Data Processing and Searching AV Training Data ### Cosmos 3 NVIDIA Cosmos 3 simulation model is designed to expand training and testing for autonomous driving through synchronized multi camera and lidar generation. Planned multimodal generation responds to driving actions and configurable sensor layouts, targeting up to 11 camera views, including fisheye, for closed-loop testing and synthetic data generation. [Get Started with Cosmos 3 for Simulation](https://huggingface.co/nvidia/Cosmos3-Nano-Transfer-Auto) ### Cosmos Data Factory for AV Dataset Generation The NVIDIA Cosmos Data Factory’s Autonomous Vehicle Dataset Generation agent skill transforms driving video and vehicle signals into curated, searchable, augmented, evaluated datasets. The workflow reduces manual orchestration and expands training coverage without additional road collection. [Get Started With Cosmos Data Factory for AV](https://github.com/NVIDIA/cosmos-data-factory) ### Cosmos Curator [NVIDIA Cosmos Curator](https://github.com/NVIDIA/cosmos-curator/tree/main/docs) is a framework for quickly filtering, annotating, and deduplicating large volumes of sensor data for physical AI development. It uses NVIDIA Cosmos Reason VLM to annotate and critique data, shortening processing pipelines from months to days. [Get Started With Cosmos Curator](https://github.com/NVIDIA/cosmos-curator/tree/main/docs) ### Cosmos Dataset Search The [NVIDIA Cosmos Dataset Search](https://build.nvidia.com/nvidia/cosmos-dataset-search) vector search workflow retrieves relevant driving scenarios from massive datasets instantly. Powered by NVIDIA Cosmos Embed NIM™ for semantic search across billions of clips, it cuts rare scenario discovery from weeks to seconds. [Get Started with Cosmos Dataset Search](https://build.nvidia.com/nvidia/cosmos-dataset-search) ### CoC Autolabeling Pipeline The [NVIDIA CoC Auto-Labeling Pipeline](https://github.com/NVlabs/alpamayo-coc-autolabeler) automatically generates Chain of Causation (CoC) reasoning labels for driving clips. It identifies decision-making keyframes, labels driving decisions from a closed action set, and organizes cause-and-effect relationships into structured CoC traces. [Get Started With CoC Autolabeler](https://github.com/NVlabs/alpamayo-coc-autolabeler) ### Physical AI Dataset for Autonomous Vehicles The [NVIDIA Physical AI Dataset for Autonomous Vehicles](https://huggingface.co/datasets/nvidia/PhysicalAI-Autonomous-Vehicles) is an open, large-scale multimodal driving dataset spanning over a thousand hours across dozens of countries. It includes CoC reasoning labels that provide causally grounded decision traces for training reasoning-based AV models. [Get Started with Physical AI Dataset for AV](https://huggingface.co/datasets/nvidia/PhysicalAI-Autonomous-Vehicles) * * * ## Generate Photorealistic Synthetic Training Data ### Cosmos 3 [NVIDIA Cosmos™ 3](https://github.com/nvidia/cosmos#cosmos-3) is the frontier open world foundation model for physical AI and the first full omnimodel to understand and generate text, image, video, ambient sound, and action. It unifies vision reasoning, world and action generation to accelerate vision analytics, synthetic data, simulation, and policy improvement. [Get Started with Cosmos 3](https://github.com/nvidia/Cosmos) ### Omniverse NuRec [NVIDIA Omniverse™ NuRec](https://developer.nvidia.com/omniverse/nurec) is a set of Gaussian-based libraries that reconstruct real-world driving scenes into interactive 3D environments. This enables new sensor views and scenario variations for synthetic training data generation. [Get Started With Omniverse NuRec](https://developer.nvidia.com/omniverse/nurec) ### Cosmos Evaluator [NVIDIA Cosmos Evaluator](https://github.com/NVIDIA/cosmos-evaluator) is an open framework that automatically scores and validates synthetic video generated by NVIDIA Cosmos, measuring physical accuracy and object fidelity to improve data quality for AV model training. [Get Started With Cosmos Evaluator](https://github.com/nvidia-cosmos/cosmos-evaluator) * * * ## Model Training and Optimization ### Alpamayo 2 Edge NVIDIA Alpamayo 2 Edge is a 2B-parameter in-vehicle reasoning vision-language-action model using adaptive reasoning to generate trajectories and causal explanations for fine-tuning and efficient deployment. [Get Started With Alpamayo 2 Edge](https://nvidiahttps://huggingface.co/nvidia/Alpamayo2-Edgenews.nvidia.com/news/european-mobility-leaders-build-safe-vehicles-and-robotaxi-fleets-on-nvidia-hyperion) ### Alpamayo 2 Super [NVIDIA Alpamayo 2 Super](https://huggingface.co/blog/drmapavone/nvidia-alpamayo-2) is an open 34-billion-parameter reasoning VLA model that reasons, plans, and acts across the full driving stack for safer, scalable Level 4 development. [Get Started With Alpamayo 2 Super](https://huggingface.co/nvidia/Alpamayo2-Super) [Get Started With Alpamayo 2 Super NIM](https://catalog.ngc.nvidia.com/orgs/nim/nvidia/containers/alpamayo2-super/latest) ### Alpamayo 1.5 Nano NVIDIA Alpamayo 1.5 Nano is a 10B-parameter reasoning vision-language-action model for autonomous driving research, generating trajectories and causal explanations to evaluate and adapt driving policies. [Get Started With Alpamayo 1.5 Nano](https://huggingface.co/nvidia/Alpamayo-1.5-10B) [Get Started With Alpamayo 1.5 NIM](https://catalog.ngc.nvidia.com/orgs/nim/nvidia/models/alpamayo-1-5-10b/-) ### AlpaGym NVIDIA AlpaGym is an open-source modular closed-loop reinforcement learning framework for training AV policy models at GPU scale. It helps expose compounding errors that open-loop, log-replay training leaves uncorrected. [Get Started with AlpaGym](https://github.com/NVlabs/alpagym) ### AlpaJudge NVIDIA AlpaJudge is a human-calibrated evaluation and reward model that scores causal driving explanations to assess reasoning quality, filter weak annotations and reduce manual review of training data. [Get Started with AlpaJudge](https://huggingface.co/nvidia/alpajudge) ### TensorRT The NVIDIA TensorRT™ ecosystem of tools for high-performance deep learning inference includes compilers, runtimes, and model optimizations that deliver low latency and high throughput for production applications. [Get Started with TensorRT](https://docs.nvidia.com/deeplearning/tensorrt/latest/#) ### TensorRT-LLM The NVIDIA TensorRT-LLM open-source library optimizes large language and vision-language model inference on NVIDIA GPUs, with support for FP8 and NVFP4 quantization, in-flight batching, and paged attention. It’s the inference backend for Alpamayo 1.5. [Get Started with TensorRT LLM](https://docs.nvidia.com/tensorrt-llm/index.html) ### Dynamo-Triton NVIDIA Dynamo-Triton™ is an open-source model serving framework that deploys AI models across NVIDIA TensorRT, PyTorch, ONNX, and other major frameworks. It uses dynamic batching, concurrent model execution, and ensemble pipelines to run multiple AI models, including preprocessing and postprocessing on GPU. [Get Started With Dynamo-Triton](/dynamo-triton) ### Learn About NVIDIA Alpamayo NVIDIA Alpamayo is an open family of vision-language-action models, simulation frameworks, and physical AI datasets for reasoning-based autonomous vehicle development. - Reason through complex driving scenarios with [Alpamayo 2 Super](https://github.com/NVlabs/alpamayo-recipes). - Auto-generate causally grounded reasoning labels with [CoC Auto-Labeling Pipeline](https://github.com/NVlabs/alpamayo-coc-autolabeler). - Simulate and evaluate AV policies in closed-loop with [AlpaSim](https://github.com/NVlabs/alpasim). - Post-train AV policies with GPU-scale reinforcement learning with [AlpaGym](https://github.com/NVlabs/alpagym). - Access the [Physical AI Open Dataset](https://huggingface.co/collections/nvidia/physical-ai-67c643edbb024053dcbcd6d8), an open-source large, diverse multimodal driving dataset. [Get started with Alpamayo](https://github.com/NVlabs/alpamayo-recipes) ## Model Development Learning Resources The rows below are the raw result records for this component — one row per record. Column names are the source index fields; the full data is also downloadable in the links that follow. | title | featured | x_formats | document_url | technologies | document_date | short_summary | document_title | learning_level | x_content_types | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Accelerating AV Simulation with Neural Reconstruction and World Foundation Models | false | blog | https://developer.nvidia.com/blog/accelerating-av-simulation-with-neural-reconstruction-and-world-foundation-models/ | Cosmos, Omniverse NuRec | 2025-11-13T00:00:00.000Z | Implement NuRec and Cosmos WFMs in CARLA to build photorealistic AV simulation pipelines. | Accelerating AV Simulation with Neural Reconstruction and World Foundation Models | Technical - Intermediate | Explainer | | How to Build In-Vehicle AI Agents: From Cloud to Car | true | blog | https://developer.nvidia.com/blog/how-to-build-in-vehicle-ai-agents-with-nvidia-from-cloud-to-car/ | DRIVE AGX, Nemotron | 2026-05-14T00:00:00.000Z | Design and deploy agentic AI cockpit systems using DRIVE AGX Thor and Nemotron VLMs. | How to Build In-Vehicle AI Agents: From Cloud to Car | Technical - Intermediate | How-to | | Inside the Alpamayo Ecosystem: Partners, Open Research, and What's Next | false | video | https://www.nvidia.com/en-us/on-demand/session/alpamayo26-alpa06/ | Alpamayo | 2026-06-03T00:00:00.000Z | Survey the Alpamayo open ecosystem, active partners, and upcoming research directions. | Inside the Alpamayo Ecosystem: Partners, Open Research, and What's Next | Technical - Beginner | Overview | | Safety in the Loop: Scaling AV Safety Through Industry Standards | false | video | https://www.nvidia.com/en-us/on-demand/session/alpamayo26-s1/ | DRIVE AGX, DRIVE AV | 2026-06-03T00:00:00.000Z | Evaluate how DriveOS and Hyperion meet safety standards for scalable L4 AV deployment. | Safety in the Loop: Scaling AV Safety Through Industry Standards | Technical - Intermediate | Explainer | | From Research to Production: How Alpamayo Accelerates AV Development | false | video | https://www.nvidia.com/en-us/on-demand/session/gtc26-s81779/ | Alpamayo | 2026-03-17T00:00:00.000Z | Discover how Alpamayo advances end-to-end AV stacks toward scalable Level 4 deployment. | From Research to Production: How Alpamayo Accelerates AV Development | Technical - Intermediate | Overview | | How Open World Models Are Powering Breakthroughs in Physical AI | false | video | https://www.nvidia.com/en-us/on-demand/session/gtc26-s81667/ | Cosmos | 2026-03-17T00:00:00.000Z | Examine how open world foundation models generate photorealistic data for physical AI reasoning. | How Open World Models Are Powering Breakthroughs in Physical AI | Technical - Intermediate | Overview | | Advancing Autonomous Vehicles With World Models | false | video | https://www.nvidia.com/en-us/on-demand/session/gtc26-s82446/ | Alpamayo, Cosmos | 2026-03-17T00:00:00.000Z | Evaluate how world foundation models accelerate AV policy simulation and validation. | Advancing Autonomous Vehicles With World Models | Technical - Intermediate | Explainer | | An Introduction to NVIDIA Cosmos for Physical AI | false | hands-on | https://www.nvidia.com/en-us/on-demand/session/gtcdc25-dct51187/ | Cosmos | 2025-10-29T00:00:00.000Z | Explore Cosmos world foundation models, tokenizers, and data pipelines in a hands-on introductory course. | An Introduction to NVIDIA Cosmos for Physical AI | Technical - Beginner | Tutorial | | Safety in the Loop: Open Datasets and Models for Safe AVs | false | video | https://www.youtube.com/watch?v=-5jfeuv2Lkw | Alpamayo | 2026-01-05T00:00:00.000Z | Discover how open datasets and models support safe AV development and validation. | Safety in the Loop: Open Datasets and Models for Safe AVs | Technical - Intermediate | Explainer | | Autonomous Driving With Reasoning Models | false | video | https://www.nvidia.com/en-us/on-demand/session/alpamayo26-alpa03/ | Alpamayo | 2026-06-03T00:00:00.000Z | Apply reasoning model concepts to end-to-end autonomous vehicle policy development. | Autonomous Driving With Reasoning Models | Technical - Intermediate | Explainer | | Inside Alpamayo 1: Making Autonomous Vehicles Reason | false | video | https://www.youtube.com/watch?v=V9E4GX5vaC8 | Alpamayo | 2026-02-11T00:00:00.000Z | Examine the architecture and capabilities of the Alpamayo 1 reasoning model. | Inside Alpamayo 1: Making Autonomous Vehicles Reason | Technical - Beginner | Explainer | | Simplify End-to-End AV Development with Cosmos World Foundation Models | false | blog | https://developer.nvidia.com/blog/simplify-end-to-end-autonomous-vehicle-development-with-new-nvidia-cosmos-world-foundation-models/ | Cosmos | 2025-06-11T00:00:00.000Z | Integrate Cosmos Predict, Transfer, and Reason WFMs to accelerate AV synthetic data pipelines. | Simplify End-to-End AV Development with Cosmos World Foundation Models | Technical - Intermediate | Explainer | | Build Physical AI with Edge-First LLMs for Autonomous Vehicles | false | blog | https://developer.nvidia.com/blog/build-next-gen-physical-ai-with-edge%e2%80%91first-llms-for-autonomous-vehicles-and-robotics/ | Alpamayo, DRIVE AGX | 2026-03-12T00:00:00.000Z | Deploy Alpamayo with TensorRT Edge-LLM for on-vehicle inference in production AV stacks. | Build Physical AI with Edge-First LLMs for Autonomous Vehicles | Technical - Advanced | Tutorial | | How to Enhance 3D Gaussian Reconstruction Quality for Simulation | false | blog | https://developer.nvidia.com/blog/how-to-enhance-3d-gaussian-reconstruction-quality-for-simulation/ | Cosmos, Omniverse NuRec | 2025-12-11T00:00:00.000Z | Apply NuRec Fixer to remove artifacts and restore fidelity in 3DGS simulation environments. | How to Enhance 3D Gaussian Reconstruction Quality for Simulation | Technical - Intermediate | How-to | | How to Post-Train AV Models in Closed-Loop with Alpamayo | true | blog | https://developer.nvidia.com/blog/how-to-post-train-autonomous-vehicle-models-in-closed-loop-with-nvidia-alpamayo/ | Alpamayo | 2026-06-03T00:00:00.000Z | Apply AlpaGym closed-loop RL to post-train AV driving policies with compounding-error awareness. | How to Post-Train AV Models in Closed-Loop with Alpamayo | Technical - Advanced | Tutorial | | Generate Synthetic Data for Physical AI with Cosmos World Foundation Models | false | hands-on | https://www.nvidia.com/en-us/on-demand/session/gtc26-dlit81644/ | Cosmos | 2026-03-17T00:00:00.000Z | Build synthetic data generation pipelines for physical AI using Cosmos WFMs in a hands-on lab. | Generate Synthetic Data for Physical AI with Cosmos World Foundation Models | Technical - Intermediate | Tutorial | | Closing the Development Loop: Simulation and Training With AlpaSim and AlpaGym | false | video | https://www.nvidia.com/en-us/on-demand/session/alpamayo26-alpa04/ | Alpamayo | 2026-06-03T00:00:00.000Z | Apply AlpaSim and AlpaGym to build closed-loop reinforcement learning pipelines for AVs. | Closing the Development Loop: Simulation and Training With AlpaSim and AlpaGym | Technical - Advanced | Tutorial | | Building Autonomous Vehicles That Reason with Alpamayo | false | blog | https://developer.nvidia.com/blog/building-autonomous-vehicles-that-reason-with-nvidia-alpamayo/ | Alpamayo | 2026-05-01T00:00:00.000Z | Build and evaluate a reasoning-based AV pipeline using Alpamayo 1 and AlpaSim. | Building Autonomous Vehicles That Reason with Alpamayo | Technical - Beginner | Tutorial | | Advancing Level 4 Autonomy: Scalable, Safe AVs and Robotaxis | false | video | https://www.youtube.com/watch?v=vUBytUceIjM | DRIVE AV | 2026-03-16T00:00:00.000Z | Analyze the technical path to scalable, safe Level 4 autonomous vehicle deployment. | Advancing Level 4 Autonomy: Scalable, Safe AVs and Robotaxis | Technical - Intermediate | Overview | | How AI Helps Autonomous Vehicles See Outside the Box | false | video | https://www.youtube.com/watch?v=HS1wV9NMLr8 | DRIVE AV | 2019-10-23T00:00:00.000Z | Explore DRIVE Labs perception techniques that extend AV sensor coverage and understanding. | How AI Helps Autonomous Vehicles See Outside the Box | Technical - Beginner | Explainer | | Alpamayo 2 Super: The Open Reasoning Model for Robotaxis | true | video | https://www.youtube.com/watch?v=kJRVwaYwvt0 | Alpamayo | 2026-06-16T00:00:00.000Z | Explore Alpamayo 2 Super, a 32B reasoning VLA model for Level 4 robotaxi development. | Alpamayo 2 Super: The Open Reasoning Model for Robotaxis | Technical - Beginner | Overview | | Accelerate AV Development with the DRIVE AGX Thor Developer Kit | false | blog | https://developer.nvidia.com/blog/accelerate-autonomous-vehicle-development-with-the-nvidia-drive-agx-thor-developer-kit/ | DRIVE AGX | 2026-01-07T00:00:00.000Z | Set up and deploy the DRIVE AGX Thor Developer Kit for production-level AV development. | Accelerate AV Development with the DRIVE AGX Thor Developer Kit | Technical - Intermediate | How-to | | The ChatGPT Moment for Autonomous Driving | false | video | https://www.nvidia.com/en-us/on-demand/session/alpamayo26-alpa01/ | Alpamayo | 2026-06-03T00:00:00.000Z | Understand how reasoning-based VLA models are transforming autonomous vehicle development. | The ChatGPT Moment for Autonomous Driving | Technical - Beginner | Overview | | Train and Test End-to-End Autonomous Vehicles with Alpamayo | true | video | https://www.nvidia.com/en-us/on-demand/session/gtc26-dlit82311/ | Alpamayo | 2026-03-17T00:00:00.000Z | Learn to train and evaluate end-to-end AV models using Alpamayo and AlpaSim. | Train and Test End-to-End Autonomous Vehicles with Alpamayo | Technical - Intermediate | Tutorial | | How Simulation Enables Safer Autonomous Vehicles | false | video | https://www.nvidia.com/en-us/on-demand/session/ces25-foretellix/ | DRIVE AV | 2026-01-06T00:00:00.000Z | Learn how the Foretellix toolchain uses simulation to test and validate AV safety at scale. | How Simulation Enables Safer Autonomous Vehicles | Technical - Beginner | Explainer | | Training at Scale: The Physical AI Dataset | false | video | https://www.nvidia.com/en-us/on-demand/session/alpamayo26-alpa02/ | Alpamayo | 2026-06-03T00:00:00.000Z | Discover how the Physical AI dataset enables large-scale AV model training across 25 countries. | Training at Scale: The Physical AI Dataset | Technical - Intermediate | Explainer | | The Generative AI In-Vehicle Experience Powered by NVIDIA DRIVE | false | video | https://www.youtube.com/watch?v=t-UPlPlrYgQ | DRIVE AGX, Nemotron | 2024-03-18T00:00:00.000Z | Examine generative AI capabilities for personalized in-cabin experiences on DRIVE hardware. | The Generative AI In-Vehicle Experience Powered by NVIDIA DRIVE | Technical - Beginner | Demo | [Download the raw results data (JSON)](https://developer.nvidia.com/search-data/automotive.json) * * * ## Other DRIVE Resources ### Simulation and Validation Test and validate AV models in high-fidelity closed-loop simulation. [Learn More](/drive/simulation) ### DRIVE AGX Get developer kits and SDKs for in-vehicle compute development. [Learn More](/drive/agx) ### Downloads Center Download SDKs, open models, and simulation tools. [Explore](/drive/downloads) ### Support Ask questions and get answers in our developer forum. [Explore](https://forums.developer.nvidia.com/c/autonomous-vehicles/517)