Computer Vision
Real-time object detection on the edge — from the Basler camera's raw capture to TensorRT YOLO and pixel-to-ground geolocation.
Overview
The Computer Vision subsystem finds the competition objects and works out where they are on the ground. It runs on the same 15 W Jetson Orin Nano as the mission logic, taking 5320×3040 frames from a global-shutter Basler camera and producing georeferenced detections several times a second.
The core engineering is in the constraints: a fixed power budget, a camera with no in-sensor color, and a chip with no hardware video encoder.
Engineering Objectives
- Detect and classify the competition objects reliably at 150 ft altitude.
- Keep the capture path fast enough that the model always sees the newest frame.
- Turn a detection pixel into a latitude/longitude on the ground.
- Stream what the model sees to the safety pilot in real time.
Major Components
Basler a2A5320 camera
A 16 MP global-shutter camera delivering raw Bayer frames over USB 3.
Two-thread pipeline
A grab thread that only copies raw bytes and a consumer thread that demosaics, resizes, and runs the model.
TensorRT YOLO26
FP16 detection engines exported on the Jetson itself; a large accurate model by default, a faster small model as an option.
Georeferencing
Casts the detection pixel as a ray, rotates it by the aircraft pose at the frame timestamp, and intersects it with the ground.
RTSP overlay stream
A software x264 encode served to the Herelink so the pilot sees the model's detections live.
Debounced confirmation
A 3-of-5 gate that separates a real target from a single spurious hit.
Integration with ASCENT-1
Computer Vision pairs each detection with an aircraft pose from AI & Communication's time-indexed pose buffer — both stamped on the same monotonic clock — to georeference it. Confirmed, localized targets are handed to the mission state machine in Autonomous Flight to commit to.
The subsystem shares the Jetson's power envelope from Power Systems, placing exactly one tenant on the GPU and absorbing everything else on the CPU cores.
Subsystem Architecture
Engineering Gallery
Technical Highlights
Move bytes, not pixels
The capture callback only copies raw Bayer; all pixel work happens on the consumer thread.
Newest frame wins
A size-1 latest-frame slot beats a queue for a live-perception loop.
One clock end to end
Frames and poses share a monotonic clock, so georeferencing has no timing bias.
Debug in the air
The RTSP overlay shows the model's opinion while the drone is still flying.
- Expand the training set with more flight-realistic aerial imagery.
- Add multi-object tracking across frames to strengthen confirmation.
- Evaluate a hardware-encoded video path on future compute.