CARLA Observability Toolkit v1.0.0

A complete workflow for capturing CARLA simulation telemetry, validating run artifacts, and comparing experiments through a Flask API and React dashboard.

PythonCARLAFlaskReactViteRechartspytest

Overview

CARLA simulations produce vehicle behavior that is difficult to evaluate from a live driving session alone. I built the CARLA Observability Toolkit to turn those sessions into saved datasets that can be inspected and compared after capture. Released on May 8, 2026 as my Spring 2026 capstone submission, v1.0.0 connects Python telemetry collection, artifact validation, a Flask API, and a React dashboard in one workflow. Its scope is a local demonstration and research proof of concept.

Highlights

Implemented start, stop, abort, and manual tagging controls for simulation runs, with guards against invalid lifecycle actions.
Captured speed, acceleration, steering, throttle, brake, position, and heading alongside lifecycle and collision events.
Built Run Explorer and Compare Runs views with summary statistics, event timelines, metric toggles, and side-by-side speed and control charts.
Added validation for individual runs and batches, plus experiment reports derived from saved telemetry.

Architecture

CARLA adapters and modular collectors publish telemetry and events through a metric bus, separating data collection from persistence.
Each run has its own metadata.json, metrics.csv, and events.json files. These artifacts are shared inputs for validation, summaries, and dashboard analysis.
Flask exposes run-list and run-detail endpoints, loading artifacts and computing statistics such as average speed, duration, and collision count.
React, Vite, and Recharts provide exploration and comparison views. In demo mode, Flask also serves the built frontend from a single local server.

Key Learnings

Separating capture from analysis lets saved runs be revisited without restarting CARLA; it does not require a live dashboard connection during collection.
Normalizing each run's simulation time to t=0 makes comparison charts meaningful even when captures begin at different CARLA world times.
Using the same artifacts for API responses, validation, and reports keeps raw data and derived results traceable.
Validation evidence needs to distinguish executed checks, code-inspected behavior, and scenarios blocked by missing dependencies or an inactive simulator.

Outcomes