Flight Delay Prediction

Context

This project implements an end-to-end system for predicting US commercial flight delays using only pre-departure metadata from the US DOT Flight Delays dataset. The model works with information that is available before departure:

• month and day of week
• scheduled departure and arrival hours
• origin and destination airport codes
• airline code
• distance in miles

The goal is to estimate both the probability that a flight will arrive more than 15 minutes late and, if delayed, the expected delay in minutes. On top of the model sits a conversational assistant — the Flight Deck Console — so a user can describe a flight in plain English and get a grounded prediction back with the reasoning behind it.

System architecture

The system is a small pipeline of independent layers, each doing one job:

• Next.js chat UI (frontend/) — the Flight Deck Console, sending messages to the backend and rendering the conversation.
• FastAPI service (app/) — hosts /predict, /agent/chat, /health and /model/performance.
• Agent layer (assistant/) — a ReAct loop with tool-calling, driven by Google Gemini 2.5 Flash.
• FAISS retrieval — in-memory semantic lookup for airports and airlines.
• LightGBM two-stage model (ml_pipeline/) — the classifier + regressor that actually produces the numbers.

The whole thing is packaged with Docker / docker-compose so the API and agent come up with a single command.

ML pipeline

The ML pipeline (ml_pipeline/) loads the Kaggle CSVs, cleans and filters the data, and builds features using only pre-flight information:

• removal of cancelled and diverted flights
• explicit dropping of leakage columns (fields only known after departure/arrival)
• time-based features and time slots
• calendar flags (weekend, summer, holiday season)
• cyclic encodings for month, day-of-week and hours
• route and distance categories, hub indicators
• historical delay statistics for routes, airlines and time bands

A two-stage model based on LightGBM is trained:

• a classifier (LGBMClassifier) that estimates P(delay > 15)
• a regressor (LGBMRegressor) that predicts delay minutes on delayed flights

Feature engineering runs identically at training and inference time — models, feature statistics and encoding metadata are serialized so there is no train/serve skew.

Agent, RAG and guardrails

The model is exposed through the FastAPI backend and an LLM-powered assistant that turns free-form conversation into a valid prediction request.

• ReAct loop — the agent reasons, calls a tool, observes the result and repeats, bounded to 8 iterations. It has six tools: airport search, airline search, time resolution, distance calculation, slot updates and the model prediction itself.
• FAISS RAG — in-memory indexes over airports and airlines, queried with multilingual embeddings for fuzzy entity resolution ("JFK", "New York", "Kennedy" all resolve to the same code). This grounds every airport/airline code in real data instead of letting the LLM invent one.
• Deterministic slot-filling — the agent fills the 8 required inputs from tool results and enforces a complete slot state before it is allowed to predict.

The loop is deliberately deterministic where it can be: the LLM handles conversation and slot-filling, while distance calculation and the prediction itself are plain tool calls — so the numbers a user sees always come from the model pipeline, never from the LLM's imagination.

Guardrails keep the assistant honest:

• a hard prediction gate — all 8 slots must be filled before any prediction is returned
• no guessing — the LLM is forbidden from inventing airport codes, distances or values
• an 8-iteration bound on the agent loop
• graceful fallbacks when a tool call fails or an entity can't be resolved

Results

The project uses a temporal split (months 1–10 for training, 11–12 for testing), which is honest about the fact that you predict the future from the past.

By design, the system relies only on schedule and route metadata. High-signal real-time factors such as weather, airport congestion and gate status are not present in the dataset, which caps the achievable performance; this is partially mitigated by historical aggregates on routes, airlines and time bands.

What I'd do next

• More robust temporal cross-validation and probability calibration.
• Higher-signal features (weather, airport congestion, real-time gate status) when available.
• Persistent session state (Redis) instead of in-memory, and a persisted FAISS index instead of rebuilding at startup.
• Streaming agent responses and a richer chat frontend.
• Structured logging and conversation tracing.
• Exposing feature importance in the API response and surfacing the key drivers in the LLM's answer.