Importance Sampling and PCA for Finding Failures in Commercial Autonomous Vehicles

2026-07-20 · 16:07 UTC ·Hailey Warner, Duncan Eddy, Shreya Parjan... ·1 min read

Brief

The paper tackles rare-failure discovery in a commercial autonomous trucking stack by applying AST (RL-based) and DiFS (denoising diffusion) to find collisions in merge and cut-in scenarios that Monte Carlo missed. It adds a PCA-based analysis to cluster failure modes, pinpoint influential timesteps, and invert components to synthesize generalized noise trajectories that reproduce failures, enabling systematic perception diagnosis. (Only the abstract was available.)

Why it matters

Applied adaptive stress testing (AST, reinforcement-learning-based) and diffusion-based failure sampling (DiFS, denoising diffusion model) to a commercial autonomous trucking stack and discovered simulated collisions during merge and cut-in maneuvers that traditional Monte Carlo simulation did not find.

Key details

Developed a PCA-based statistical workflow that clusters failure modes, identifies timesteps most influencing outcomes, and inverts principal components to recover generalized noise trajectories which reproduce failures in identical and similar scenarios, enabling perception-level diagnosis.

Preprint published on arXiv 2026-07-20 (https://arxiv.org/abs/2607.18106v1); submitted to IEEE ICVES 2026.

Abstract

Methods for discovering rare failures in autonomous systems have so far been demonstrated almost exclusively in simulations with simple, academic driving stacks, leaving open whether they generalize to the more robust planners used in commercial systems. We address this gap by applying two rare-event discovery algorithms to a commercial autonomous trucking stack. Adaptive stress testing (AST) uses reinforcement learning to search for the most likely noise trajectories leading to a simulated collision, while diffusion-based failure sampling (DiFS) trains a denoising diffusion model to sample a diverse set of failures. We show that both algorithms find simulated collisions during merge and cut-in maneuvers where traditional Monte Carlo simulation does not. To make these failures actionable, we introduce a statistical analysis based on principal component analysis (PCA) that classifies failures into common modes and identifies the timesteps that most influence the outcome. We cluster the principal components and invert the PCA transform to recover generalized noise trajectories, and show that these trajectories reproduce failures in identical and similar scenarios. This provides a path from failure discovery to systematic diagnosis of perception-level flaws.

Comment: IEEE ICVES 2026 (Submitted)