X-Sense ECCV 2026 X-Sense Tutorial

Malmö, Sweden | ECCV 2026

X-Sense: Cooperative Perception and Distributed Foundation Models for Smart Mobility

A full-day tutorial on cooperative sensing, multi-agent perception, distributed VLA agents, and scenario-based safety validation.

View Modules Speaker List
4 modules
V2X vehicles, infrastructure, aerial platforms
VLA distributed reasoning, communication, action

Tutorial Thesis

From isolated ego-centric perception to distributed visual intelligence

X-Sense frames cooperation as a fundamental shift in computer vision: from isolated ego-centric perception toward distributed, foundation-model-driven visual intelligence operating across agents and infrastructure.

The tutorial connects cooperative sensing, multi-view learning, distributed foundation models, VLA agents, communication-aware inference, and scenario-based safety validation into one smart mobility research agenda.

Core Questions

  • Why is cooperation necessary beyond onboard ego-centric sensing?
  • What data infrastructure enables scalable cross-view cooperation?
  • How do we build distributed foundation models and VLA agents?
  • How do we validate and deploy safely under real constraints?

Overall Structure

Four modules for cooperative autonomy

01

Motivation and Fundamental Challenges

Why cooperation is necessary, and what new constraints it introduces.

  • Safety, efficiency, scalability, cost, privacy
  • Ego-only limits: occlusion, limited FoV, long-tail events
  • Cooperation constraints: synchronization, calibration, latency, bandwidth, privacy, security
02

Datasets and Data Infrastructure

What ego, infrastructure, V2X, aerial, multi-view, and simulation data can teach us.

  • Time synchronization and calibration requirements
  • Annotation scope, geographic coverage, rare events
  • Bias, inconsistent protocols, missing long-tail scenarios
03

Cooperative Perception and Distributed Foundation Models

How cooperative systems perceive, align, reason, communicate, and plan.

  • Cooperative perception and spatio-temporal modeling
  • Foundation models and multimodal reasoning
  • Communication-efficient learning, uncertainty-aware fusion, VLA agents
04

Scenario-Based Safety Validation and Deployment

How we trust cooperative systems before deployment.

  • Accuracy plus latency, stability, compute, and fallback metrics
  • Splits, OOD suites, stress tests, reproducibility
  • Drift monitoring, fail-safe behavior, go/no-go criteria

Organizers + Speakers

X-Sense Organizing Team

Primary organizer: Yi-Ting Chen, National Yang Ming Chiao Tung University, Taiwan.

Yi-Ting Chen

Yi-Ting Chen

National Yang Ming Chiao Tung University

Visual risk perception, heterogeneous traffic modeling, scenario-based safety validation
Zhenzhen Liu

Zhenzhen Liu

Cornell University

V2X, infrastructure sensing, cooperative perception, foundation models
Minkyoung Cho

Minkyoung Cho

University of Michigan

Robustness, uncertainty-aware fusion, real-world V2X deployment
Walter Zimmer

Walter Zimmer

University of California, Los Angeles

Roadside infrastructure and V2X datasets
Ross Greer

Ross Greer

University of California, Merced

Long-tail scenarios, data curation, language-based scene datasets
Yu-Hsiang Chen

Yu-Hsiang Chen

National Yang Ming Chiao Tung University

Drone datasets and heterogeneous traffic understanding
Rui Song

Rui Song

University of California, Los Angeles

Synthesized scene reconstruction and scalable data infrastructure
Zewei Zhou

Zewei Zhou

University of California, Los Angeles

Multi-agent spatio-temporal scene understanding
Seth Zhao

Seth Zhao

University of California, Los Angeles

Efficient V2X systems, quantization, cooperative perception
Hsu-kuang Chiu

Hsu-kuang Chiu

Carnegie Mellon University

Multimodal LLM-based cooperative autonomous driving
Jinsu Yoo

Jinsu Yoo

The Ohio State University

Cross-view learning and collaboration as supervision
Min-Hung Chen

Min-Hung Chen

NVIDIA Research

Foundation models, multimodal perception and reasoning, 4D understanding
Qi Chen

Qi Chen

Toyota Motor North America

Closed-loop evaluation and safety validation
Jiawei Yong

Jiawei Yong

Toyota Motor Corporation

Simulation, benchmarks, evaluation protocols
Yongkang Liu

Yongkang Liu

Toyota Motor North America

Scenario-based safety validation
Deyuan Qu

Deyuan Qu

Toyota Motor North America

Closed-loop evaluation and deployment criteria
Ruiyang Zhu

Ruiyang Zhu

University of Michigan

Collaborative map perception and robust V2X systems

Program Flow

Full-day teaching plan

09:00

Opening Remarks and Tutorial Overview

Lead: Yi-Ting Chen

Introduce X-Sense, the organizing thesis, and the four guiding questions for the day.

09:10

Module 1: Motivation and Fundamental Challenges

Core speaker: Minkyoung Cho. Contributors: Minkyoung Cho, Zhenzhen Liu.

Limitations of ego-centric perception and real-world constraints in cooperation.

  • Occlusion
  • Limited FoV
  • Long-tail events
  • Synchronization
  • Latency
  • Privacy and security
10:50

Module 2: Datasets and Data Infrastructure

Core speaker: Walter Zimmer. Contributors: Ross Greer, Yu-Hsiang Chen, Rui Song.

Ego, infrastructure, V2X, aerial, multi-view, and simulation datasets for cooperative perception.

  • Temporal alignment
  • Calibration
  • Cross-view annotation
  • Dataset bias
  • Domain gaps
  • Reproducibility
13:30

Module 3: Algorithms for Cooperative Perception and Distributed Foundation Models

Core speaker: Min-Hung Chen. Contributors: Zewei Zhou, Seth Zhao, Hsu-kuang Chiu, Jinsu Yoo.

Communication-aware algorithms, robust fusion, distributed foundation models, and VLA agents.

  • Cooperative 3D perception
  • Cross-view representation
  • Spatio-temporal modeling
  • V2V-LLM
  • VLA agents
  • Uncertainty-aware fusion
15:20

Module 4: Scenario-Based Safety Validation and Deployment

Core speaker: Yi-Ting Chen. Contributors: Qi Chen, Minkyoung Cho, Jiawei Yong, Yongkang Liu, Deyuan Qu.

Closed-loop evaluation, deployment readiness, system integration, and open problems.

  • Stress testing
  • System metrics
  • OOD validation
  • Drift monitoring
  • Fail-safe design
  • Go/no-go criteria

Deployment Reality

Verification needs behavior, system health, and auditability

Format and logistics

  • Format: full-day, in-person ECCV 2026 tutorial
  • Audience: computer vision, machine learning, autonomous driving, embodied AI, and multi-agent systems
  • Expected attendance: 100-200 participants
  • Outcome: a structured design space for real-world distributed perception systems

Deployment validation focus

  • Closed-loop evaluation and scenario-based stress testing
  • System-level metrics for latency, reliability, and resource headroom
  • OOD and long-tail validation with reproducible protocols
  • Drift monitoring and interpretable multi-agent state estimation
  • Fail-safe design and explicit deployment readiness criteria