---

The question that decides everything inside a fire: Is there a living person behind this wall — in the next room — where exactly, and is there still time — answered before the firefighter commits to entering.

Vision-X is a four-device AI system that helps firefighters and rescue teams locate living people through walls, across rooms, and smoke — and confirm they are alive — using impulse ultra-wideband (IR-UWB) radar, distributed intelligence, and hands-free spoken guidance. It does not replace the thermal camera; it adds the see-through-walls, is-it-alive sense the camera lacks.

Full design document — view it live (GitHub Pages) (print-friendly HTML, with all figures) · markdown version · raw HTML source

---

Contents

• The 30-second version
• The problem
• The solution — one principle
• System architecture
• The four devices
• How it works
• AI / ML pipeline & toolchain
• Tech stack
• Repository structure
• Demonstration plan
• Datasets
• Feasibility & challenges
• Novelty & prior art
• Roadmap
• Getting started
• Team
• References
• License

---

The 30-second version

What	A see-through-walls, across-rooms "is-it-alive" sense for firefighters, distributed across four Snapdragon-class devices.
How	IR-UWB radar passes through walls and doors into the next room — and through smoke; on-device AI confirms a living person from the millimetre motion of breathing.
Why multi-device	One capability is split so each device does only what that device can — and every tier keeps working if the one above it loses connectivity.
The demo	A hidden person in the next room → the firefighter, on the other side of a solid wall, pauses and detects "living person, ~2 m, behind this wall, breathing" before entering — and it still works with the room full of smoke.
Why it's low-risk	Every ingredient (through-wall UWB vitals, RF localization, on-device AI) is independently proven. The integration is the contribution.

---

The problem

Inside a burning building, a firefighter's best perception tool is the thermal imaging camera — but it has two blind spots that kill:

• It is line-of-sight. It cannot see through a closed door, a wall, or a floor.
• It reads heat, not life. A hot appliance or a just-vacated spot looks like a person; a person behind a barrier looks like nothing.

So the decisive question — is there a living person behind this wall, in the next room? — goes unanswered at the exact moment it matters most. (Optical cameras and LiDAR don't even get that far: smoke defeats them entirely.)

Radio frequency answers what heat and light cannot. UWB penetrates interior walls, doors, and floors — so a firefighter can sense a living person in the next room before entering — it passes straight through smoke, and radar can detect the millimetre chest motion of breathing. The same RF that finds trapped civilians also finds a downed colleague — and loss of situational awareness inside structures is tied to a large share of the dozens of firefighter line-of-duty deaths each year.

Most Indian state Fire and Emergency Services operate without thermal imaging at all, so the perception gap here is even wider. Vision-X targets India's state fire services and the National Disaster Response Force (NDRF), running on commodity IR-UWB radar and Snapdragon edge hardware at a fraction of a thermal camera's cost.

---

The solution — one principle

Push every decision to the lowest device that can make it, and ensure each tier keeps working if the one above it disappears.

That single design rule produces the entire architecture and its strongest argument.

• Data shrinks as it climbs the stack — a raw radar echo becomes a detection, then a position, then a building-wide picture — while intelligence grows.
• The result is graceful degradation:
  • the node finds a victim with no network,
  • the phone guides one firefighter with no command post,
  • the PC coordinates a building with no internet,
  • the cloud links sites only when a backhaul exists.

Disaster zones force exactly this property — and it happens to be the precise distributed edge-to-cloud AI story this hardware is built for. Every device is non-substitutable because of where the intelligence has to live, not because it was added for effect.

---

System architecture

Four tiers, from the physical world up to the cloud. Each arrow carries a more abstract product than the one below it; two refined products flow back down.

Read the diagram: solid arrows are always-available local links; dashed arrows are opportunistic. Each tier runs even if the tier above it is unreachable. Every figure also lives in docs/images/ and the full design document.

Graceful degradation — the multi-device payoff

---

The four devices

Tier	Device	Job — and why nothing else can do it
Edge node	Arduino UNO Q (Dragonwing QRB2210 + STM32U585)	Carries the IR-UWB radar. The STM32 captures radar frames on a hard real-time clock; the Dragonwing MPU runs on-device AI (clutter removal + lightweight ML classifier) and emits a compact detection. Raw RF never leaves the board. An autonomous detector that works in a burning structure with no network.
Mobile	Firefighter's phone (worn on SCBA/chest)	Fuses radar sweeps with its own IMU to localize the victim, tracks the firefighter's own path by dead reckoning, and runs a small on-device LLM that speaks guidance into the comms (a gloved, masked firefighter can't read a screen). Also shows every teammate's live position.
AI PC	Copilot+ PC (X2 Elite) (Hexagon NPU)	The incident-command brain at the scene — private, internet-free. Fuses every responder and every detection into one live building map, pushes the team picture back to every phone, raises the downed-firefighter alarm, and writes the commander's situation report. Many-to-one fusion is what a single moving phone structurally cannot do.
Cloud	Qualcomm Cloud AI 100	The heavy, building-scale work: trains/improves the edge models (OTA), and from a building blueprint runs structural-collapse + fire-spread simulation to predict a heatmap of where people are likely trapped. Non-blocking — with no uplink the system uses the last precomputed heatmap and loses nothing critical.

Full role write-ups: node/ · mobile/ · pc/ · cloud/

---

How it works

1. Seeing through walls, into the next room

The radar emits ultra-wideband pulses and reads the reflections. Radio at these frequencies passes through common non-metallic walls, doors, and floors — and through smoke — and the human body reflects it, so a firefighter can sense who is in the next room without entering. Echo delay gives distance (range bins); a person is found by the motion in those bins.

2. Detecting a person (presence + range)

The radar emits ultra-wideband pulses and reads the reflections through walls, doors, and floors. Humans reflect RF energy differently than static clutter, so Vision-X identifies a likely person return by combining clutter removal with range-domain motion/returns across sweeps. Echo delay gives distance (range bins), turning “a return is present” into range estimates for the next-room cue.

3. The motion problem → detect robustly

On a moving firefighter, the sensor’s own motion can be larger than the person-related signatures. Vision-X handles this with detect-robustly rather than vitals-confirmation:

• While advancing, motion detection helps filter dynamic clutter and isolates candidate person returns (“contact, that way”).
• The firefighter sweeps and stabilizes briefly at the wall to improve the signal-to-noise ratio for detection and range estimation.

More advanced IMU motion compensation and motion-trained models that perform reliably during continuous movement are roadmap items.

4. Localizing the victim

One reading gives range, not a position. As the firefighter sweeps from slightly different spots, the phone fuses successive ranges with its IMU to triangulate: "person behind this wall, ~2 m, low to the floor."

5. Tracking the firefighter across the blueprint

Indoors GPS fails, so position comes from IMU pedestrian dead reckoning (gyro heading + accelerometer footstep detection, integrated from the entry door = map origin; barometer for floor). Drift is anchored by zero-velocity updates at each footfall, map-matching to corridors/doors, and optional UWB anchors at the entry. This yields a live "where is everyone" view, greys out swept rooms, and gives — for free — a downed firefighter's last-known location.

Honest accuracy: a few metres of absolute position over a mission (enough to place a contact in the right room); the relative victim cue is sub-metre.

6. Data-flow recap

raw RF echo (node, real-time)
  → confirmed detection (node MPU, offline)
    → relative position + firefighter track (phone, offline)
      → fused building map + situation report (PC NPU, local)
        → occupancy prediction + multi-site picture (cloud, opportunistic)

Two flows run back down: the cloud's occupancy heatmap → the PC, and the PC's fused team positions → every phone. Awareness that needs the whole picture is computed once, centrally, and distributed.

7. From collapse to search priority — the cloud's contribution

When a backhaul exists, the cloud turns the building blueprint and live detections into a search plan: it reconstructs the collapsed structure from radar + IMU data, builds a 3D occupancy heatmap of where people are most likely present, and ranks rooms by search priority so teams hit the highest-probability zones first. It stays non-blocking — if the uplink drops, the last computed heatmap is used.

---

AI / ML pipeline & toolchain

Where	What runs	Stack
Node	clutter removal → range-FFT → micro-Doppler / breathing analysis → small ML classifier	Edge Impulse (UNO Q's native TinyML path) or TensorFlow Lite on the Dragonwing MPU; STM32 does real-time capture
Phone	small on-device LLM (e.g. Llama 3.2 3B) for offline spoken guidance, speech in/out	Qualcomm AI Hub → QNN / Genie on the NPU (ONNX is the NPU path on Snapdragon, not GGUF)
PC	sensor-fusion + occupancy prediction, larger on-device LLM with retrieval grounded in the floor plan	AnythingLLM on the Hexagon NPU (~45 TOPS), all private / offline-capable
Cloud	heavier occupancy-likelihood model; (future) retraining the edge detector from aggregated data	Qualcomm Cloud AI 100

Toolchain: Qualcomm AI Hub + qai_hub_models (supports Snapdragon X Elite and X2 Elite), QNN / QAIRT, the Genie LLM runtime, ONNX Runtime, Edge Impulse, TensorFlow Lite, and Arduino App Lab.

---

Tech stack

Hardware — Arduino UNO Q (Qualcomm Dragonwing QRB2210 + STM32U585) · a team-supplied IR-UWB radar module (BYOD) · 6/9-axis IMU · CO + temperature sensor · Snapdragon Copilot+ PC (X2 Elite) · mobile device · Qualcomm Cloud AI 100.

Software / AI — Qualcomm AI Hub · QNN / QAIRT · Genie · ONNX Runtime · Edge Impulse · TensorFlow Lite · Arduino App Lab · AnythingLLM.

Connectivity — BLE / Wi-Fi Direct (node ↔ phone) · scene Wi-Fi / mesh (units ↔ PC, two-way) · opportunistic cellular / satellite (PC ↔ cloud).

---

Repository structure

Vision-X-Through-Wall/
├── README.md                      ← you are here
├── LICENSE
│
├── docs/                          ← design documents & figures
│   ├── DESIGN.md                    full design document (markdown)
│   ├── ARCHITECTURE.md              deep architecture & data-flow dive
│   ├── DEMO.md                      demonstration runbook
│   ├── DATASETS.md                  public datasets + how we use them
│   ├── DEPLOYMENT_GUIDE.md          exporting & running models on-device
│   ├── RESULTS.md                   benchmark results
│   ├── ROADMAP.md                   milestones & future scope
│   ├── PROPOSAL.md                  condensed 500-word overview
│   ├── PLANNING.md                  Fireground Command AI build spec
│   ├── BUILD_BRIEF_PHASE0-1.md      command-post laptop track, phases 0–1
│   ├── images/                      all diagrams & figures (SVG + PNG)
│   └── Vision-X_design_document.html   print-friendly design doc
│
├── firmware/                      ← sensing hardware (see firmware/README.md)
│   ├── esp32/                       ESP32 WiFi-CSI firmware (main.cc, sdkconfig)
│   └── unoq-radar/                  Arduino UNO Q app — STM32 sketch/ + Linux python/
│
├── fallback/                      ← laptop-mode CSI radar (ESP on USB → WebSocket → dashboard)
│   └── 01_csi_presence/             ruview_presence.py, dashboard, run scripts
│
├── training/                      ← optional ML kits (the live radar is pure DSP, no ML)
│   ├── tinyml/                      HB100 doppler: collect / train / infer apps
│   └── csi/                         CSI feature-MLP: collect / train / infer
│
├── mobile/                        ← firefighter phone — hands-free mobile brain
│   ├── app/                         React Native / Expo field app (+ Whisper-NPU module)
│   └── MOBILE_BRAIN.md              on-device compute & guidance design
│
├── visionx-laptop/                ← command post, phase 1: MQTT ingest → fusion → dashboard
│   ├── backend/                     FastAPI + aiomqtt + WebSocket fan-out
│   └── frontend/                    built React dashboard (served statically)
│
├── fireground-command-ai/         ← offline incident-command decision-support co-pilot
│   ├── schemas/                     Pydantic v2 data contracts (source of truth)
│   ├── backend/                     agent roster, event bus, incident state, I/O backends
│   └── dashboard/                   React + Vite command dashboard
│
├── src/visionx/                   ← the Python package: data, training, evaluation, export
├── tests/                         ← pytest suite for the package
├── configs/                       ← layered YAML experiment configs
├── examples/                      ← standalone snippets (edge_infer.py)
├── hall-map/                      ← standalone floor-plan viewer (single HTML page)
│
├── node/  ·  pc/  ·  cloud/       ← per-tier architecture notes (see "The four devices")
└── assets/                        ← logos, banners, media

Where to start: firmware/README.md for the hardware (wiring, gotchas, and the exact data each sensor publishes), fallback/01_csi_presence/RUN_FALLBACK.md to bring the radar up on a laptop, and docs/DESIGN.md for the full system design.

Note on hardware config: WiFi credentials and transmitter MACs in the firmware and CSI tooling are placeholders (YOUR_WIFI_SSID, AA:BB:CC:DD:EE:FF). Set your own before flashing — via firmware/esp32/sdkconfig, and --tx-mac / CSI_TX_MAC for the pipeline.

---

Demonstration plan

The demo build delivers one vertical slice: a single UNO Q with one IR-UWB module detecting presence, distance, and breathing through a solid wall; the phone showing the live cue; the PC showing a fused map with one LLM situation line. The cloud runs as a thin sync stub or a second site on a slide.

The demo moment: a person hidden in the next room, behind a solid wall. The "firefighter" — in the adjoining room, visor obscured — pauses at the wall, and before entering gets the hands-free cue "living person detected, ~2 m, behind this wall, breathing," while the command-post screen lights up the contact on its map. Fill the room with smoke and nothing changes — RF doesn't care.

The detect-then-confirm flow (walk up, pause, sweep) is the real intended workflow, and genuinely sensing a living person through a solid wall — no line of sight, no trick — makes the demo authentic rather than simulated. Full runbook → docs/DEMO.md.

The illustration at the top of this README shows this exact moment, end to end.

---

Datasets

The detector is pretrained on public IR-UWB data and fine-tuned on the actual wall on-site. Two halves are covered: detect-through-the-barrier and confirm-alive.

• Through-wall detection — a dataset built specifically for victim detection behind walls/obstacles with a UWB radar sensor.
• Vitals under motion — nesl/MobiVital (IR-UWB chest signals + synchronized IMU + ground-truth respiration) and RadarDataforCSBHRD (breathing + heart rate during activity). These are close-range, line-of-sight, cooperative — pair them with the through-wall set.
• Optional gesture control — UWB-gestures (9,600 labelled samples).
• Index — awesome-radar-perception, a curated hub of radar datasets and detection / domain-adaptation papers.

Details and licensing → docs/DATASETS.md.

---

Feasibility & challenges

Challenge	How Vision-X addresses it
Respiration from a moving worn sensor	The demo confirms breathing during a stationary dwell — a clean FFT spectral peak, a proven stationary-sensor technique. Confirming breathing while moving (IMU motion compensation + a motion-trained ML model) is roadmap, not the demo build.
Justifying the cloud	scoped to opportunistic occupancy-likelihood prediction and made non-blocking, so it reinforces graceful degradation instead of undermining the offline thesis.
The phone as more than a screen	it runs localization, firefighter tracking, and the guidance SLM on its NPU; the display is the output of real compute.
Firefighter self-localization	IMU dead reckoning anchored by zero-velocity updates, map-matching, and optional UWB fixes; absolute accuracy claimed at a few metres, the relative cue at sub-metre.
Downed firefighter	rides on the same sensors: motion-stop + vitals for detection, last-known trail point for location, UWB homing to reach them through smoke.

---

Novelty & prior art

The physics is proven (through-wall UWB vital-sign detection), the AI is proven (RF-based pose and localization), and individual pieces of the firefighter stack already exist — SmokeNav (mmWave + IMU navigation), C-THRU (a thermal see-through-smoke HUD), and POINTER (responder tracking). But none of them fuse, as one distributed multi-device system on Snapdragon silicon:

1. commodity RF that confirms a living victim through walls and across rooms (and through smoke),
2. the detection AI running on the sensor node itself,
3. a hands-free responder unit that localizes the victim, and
4. a command-level coordination + occupancy-prediction layer.

That integration is the contribution — and it is low-risk precisely because every ingredient is independently validated.

---

Roadmap

• Breathing confirmation under motion — IMU motion compensation and a motion-trained ML model, removing the need for a still dwell.
• Federated, privacy-preserving training across fire services — shared models improve without centralizing sensitive incident data.
• 3D pose & skeletons (the RF-Pose direction) — tell a slumped victim from a moving one.
• Multi-victim simultaneous tracking and antenna-array imaging for richer through-wall scenes.
• Heart-rate during stationary dwells as signal processing matures.
• Integration with building systems — fire-safety panels, pre-plans, dispatch / CAD.
• Productization — heat-hardened enclosure, ruggedization, and field certification; target unit cost well below a thermal imaging camera, procured through state fire services and the NDRF.
• Adjacent applications — earthquake search-and-rescue, privacy-preserving elder-care fall detection, secure presence sensing.

Full milestone plan → docs/ROADMAP.md.

---

Getting started

Run the radar on a laptop — the quickest path to something live. An ESP32 on USB, no UNO Q needed:

cd fallback/01_csi_presence
pip install -r requirements.txt
./run_ruview.sh /dev/ttyUSB0 <YOUR_ROUTER_MAC>     # Windows: run_ruview.bat COM3 <YOUR_ROUTER_MAC>

Then open dashboard/index.html. Full walkthrough: RUN_FALLBACK.md.

Everything else:

What	Where
Hardware wiring, gotchas, sensor data contract	firmware/README.md
Flash the ESP32 CSI firmware	firmware/esp32/
Build & deploy the UNO Q app	firmware/unoq-radar/REPRODUCE.md
Field app (Android APK)	mobile/app/ · mobile/README.md
Command-post dashboard (phase 1)	visionx-laptop/README.md
Incident-command AI	fireground-command-ai/README.md
Train the ML models (visionx package)	docs/DEPLOYMENT_GUIDE.md
Optional on-device TinyML kits	training/

Per-tier architecture notes: node/ · mobile/ · pc/ · cloud/

---

Team

Team HoneyBadger.

Role	Member
Team Lead · AI/LLM & software	Omkar (@HoneyBadger-010)
Signal processing, radar & embedded firmware	Ashutosh Chaudhari (@Ashutosh-Chaudhari)
DevOps / cloud	Atul
Software · mobile & PC	Vedant

---

References

Selected prior art grounding the feasibility and novelty claims — full list in docs/DESIGN.md.

• Zhao et al., "Through-Wall Human Pose Estimation Using Radio Signals" (RF-Pose), CVPR 2018, MIT CSAIL.
• Adib et al., "WiTrack: 3D Tracking via Body Radio Reflections," USENIX NSDI 2014.
• Adib & Katabi, "See Through Walls with WiFi," ACM SIGCOMM 2013.
• "Ultra-Wideband Impulse Radar Through-Wall Detection of Vital Signs," Scientific Reports 2018.
• Chen et al., "SmokeNav," Advanced Intelligent Systems 2024.
• Qwake / DHS S&T C-THRU; NASA JPL / DHS S&T POINTER.

---

License

Released under the MIT License.