flightreacts
a safety-first drone mission-control platform built at red team hacks in calgary - a very selective hackathon with under 10% acceptance. an operator speaks plain english, an llm parses the intent, and deterministic safety checks decide what the drone is actually allowed to do.
the problem
operators under pressure want to command a drone quickly in plain language - but a language model can misread a command, invent a location, or produce something unsafe. the challenge was getting natural-language speed without ever letting the model bypass mission rules.
how it works
an operator says "alpha take off to 10 meters" or "fly to the northwest watch tower." whisper transcribes it, an llm command brain parses the intent into a structured action, natural-language locations resolve to canonical waypoints, hard safety checks run, and the result comes back with confidence and risk context in a live mission view.
design choices
- the llm parses, it doesn't decide. every parsed command goes through deterministic checks before execution - unknown waypoints, no-go-zone incursions, altitude violations at constrained waypoints and unsafe targeting are blocked, and invalid weaponized intent is hard-rejected. no phrasing can talk its way past them.
- confirm / cancel for risky commands. ambiguous or high-risk commands wait for explicit operator confirmation instead of guessing.
- waypoint aliasing. phrases like "northwest watch tower" resolve to canonical ids like TOWER_NW, so the system works from known mission waypoints instead of free-form coordinates.
- session memory for follow-ups ("do that again", "same target") and multi-waypoint routes in a single command.
architecture
- frontend/ - operator ui: command log, mission status, interaction panels.
- whisper-backend/ - fastapi stt + command-execution bridge.
- groq-integration/ - llm parser, validator, and mission command brain.
- challenge/ + mavsdk sim stack - waypoints, no-go zones, sitl/gazebo context.
challenges
- keeping the parser flexible while the safety layer stayed deterministic.
- integrating branches as the ui and command-brain variants evolved in parallel.
- path and runtime mismatches between backend launch contexts.
- handling verification both with and without live telemetry.
- keeping confirm / cancel robust to all the ways people actually say yes and no.
what i learned
if a model's output ends up moving something physical, the model shouldn't be the last line of defence. putting a validation layer between the llm and the actuator is the interesting engineering - it's what makes natural language usable around a drone.
what's next
- full live mission execution against sitl controls, not just validation.
- a richer planner for multi-drone concurrent tasking.
- automatic recovery and replanning on verification failure.
- timeline replay, mission audit reports, and role-based command permissions.
team
built with harsit baral, dip khadka, om upadhyay and shreay patel.
gallery
the backend simulation - gazebo sim running the compound_ops world, with walls, gate posts, a command building and the iris drone in the entity tree.