Revolutionizing Short-term Traffic Congestion Prediction with Machine Learning

In this success story, we developed an AI-powered traffic management system that predicts short-term congestion using machine learning and computer vision. Trained on over 100,000 traffic camera images from cities across Europe and North America, our EfficientNet-based model achieved high accuracy in real-time forecasting, helping reduce travel delays, optimize routes, and make urban mobility smarter and more sustainable.

The Problem

The Challenges of Traffic Congestion in Urban Areas

Traffic congestion is one of the biggest challenges facing modern cities. It slows down vehicle movement, increases travel times, and creates long queues, especially in busy metropolitan areas. For policymakers and city planners, it remains a major obstacle to improving urban life.

The effects of congestion go far beyond delays:

• Economic losses: Every hour spent in traffic wastes time, fuel, and productivity — costing individuals, businesses, and governments billions each year.
• Environmental damage: Idling vehicles emit harmful gases, worsening air quality and accelerating climate change.
• Health impacts: Exposure to air pollution from traffic increases the risk of respiratory and cardiovascular diseases.
• Reduced quality of life: Long commutes cause stress, frustration, and less time for family and recreation.
• Emergency response delays: Congested roads slow down ambulances and emergency vehicles, putting lives at risk.

To tackle these complex challenges, cities must move beyond traditional approaches. Data-driven, AI-powered systems that combine machine learning and computer vision can help predict and reduce congestion — improving safety, sustainability, and overall livability.

To address this complex challenge, data-driven solutions that leverage advanced technologies like machine learning and computer vision must be developed. These solutions can help predict and mitigate congestion, ultimately improving the health, prosperity, and livability of cities.

The Background

What is Short-term Traffic Congestion Prediction?

Short-term traffic congestion prediction focuses on forecasting how crowded a specific road or network will be in the immediate future, typically within a few minutes to an hour. Unlike long-term predictions used for urban planning, short-term forecasting provides real-time insights that can directly support traffic management decisions.

The goal is to deliver timely information that enables city managers to take proactive action, such as adjusting traffic signals, deploying control measures, or providing live updates to drivers. Accurate predictions allow authorities to prevent bottlenecks before they form, helping keep traffic flowing smoothly and efficiently.

The Untapped Potential of Traffic Cameras in Congestion Prediction

Traffic cameras are now a common feature in cities across the globe, providing continuous visual data on road conditions. These cameras are primarily used for surveillance and incident monitoring, helping operators detect accidents or blockages. However, their potential for congestion forecasting remains largely underutilized.

Each camera captures rich visual information about vehicle density, movement, and road occupancy. When analyzed correctly, this data can reveal traffic patterns in real time. By combining traffic camera footage with advanced machine learning and computer vision models, it becomes possible to forecast short-term congestion with far greater precision.

This approach can transform how cities manage road traffic. Instead of reacting to gridlocks after they occur, authorities can act in advance to prevent or reduce their impact, creating smarter and more adaptive traffic systems.

The Goal

The Traffic Congestion Prediction Project aimed to unlock the hidden potential of traffic camera data through artificial intelligence. By combining machine learning and computer vision, the goal was to create a reliable system capable of predicting short-term traffic congestion in real time.

This system was designed to provide city managers and transportation authorities with accurate, actionable insights that could help reduce congestion, improve travel times, and enhance urban mobility.

Beyond the main objective, the project also set out to achieve several supporting goals:

By achieving these goals, the project demonstrated how AI can play a transformative role in creating smarter, cleaner, and more efficient transportation networks for the future.

Our Approach

Step 1. Comprehensive Data Collection

Tapping into a Wealth of Visual Data: Accessing Traffic Camera APIs Worldwide

To broaden the project’s geographic reach, we explored multiple public APIs that provide access to live traffic camera images from cities in Europe and North America. Using these sources, we collected images at intervals of 5 to 30 minutes over a 24-hour period from cameras located in Ottawa, New York, Illinois, and London. This effort resulted in a dataset of more than 100,000 raw images.

Some APIs had request limitations, allowing only one image retrieval every 60 seconds. To overcome this, we designed a flexible data collection process that ensured balanced coverage across various times of day and lighting conditions. This approach enabled us to capture both daytime and nighttime traffic scenarios, providing a more comprehensive dataset for model training.

It is worth noting that the data collection occurred during the festive season, which likely influenced traffic volume and flow patterns. However, this diversity in data strengthened our models by exposing them to a wide range of traffic conditions, from light to heavy congestion.

The resulting dataset served as a strong foundation for testing and improving our machine learning models, helping ensure they could generalize across different cities, lighting environments, and traffic conditions.

Step 2. Collaborative Data Annotation

Harnessing the Power of Crowdsourcing: Labeling Images with Labelbox

To train our machine learning models, we needed a labeled dataset where each image was annotated with its corresponding congestion level. We focused on the 5,183 images collected from Ottawa, as they provided a good balance of highway and city scenes.

Using the Labelbox platform, our team collaborated to manually label each image as “High,” “Medium,” or “Low” congestion, as well as “Blurry” or “Camera-off” for images that were unclear or unavailable. To ensure consistency, we established criteria for each label and held group discussions to review edge cases.

Screenshot of the Labelbox interface

To streamline the process, we uploaded the predictions from our base model as pre-labels, allowing annotators to quickly approve correct labels while focusing their efforts on refining inaccurate ones. This iterative approach enabled us to efficiently create a high-quality labeled dataset, which we later split into training and testing subsets to evaluate our models’ performance.

Examples of traffic light detection using the Yolov7 object detection-based filter

Step 3. Intelligent Data Preprocessing

Filtering Out the Noise: Developing Blurry Image and Traffic Light Detectors

To address the challenge of environmental noise that can confound traffic congestion predictions, we developed a suite of preprocessing techniques to filter out problematic images before feeding them to our models.

First, we tackled the issue of blurry images by creating two complementary solutions: a YOLOv5-based binary classifier and an anomaly detection model. Both approaches aimed to identify and remove images that were too blurry to yield accurate predictions, thereby improving the overall quality of our dataset.

Sample predictions by the anomaly detection-based filter

Next, we recognized that the presence of traffic lights could also mislead our models, as they often indicate localized congestion unrelated to overall traffic flow. To mitigate this, we developed a YOLOv7-based traffic light detector, which could flag images containing traffic lights and exclude them from further analysis.

Zeroing In on What Matters: Creating Lane Masks to Isolate Key Features

Beyond filtering out noise, we hypothesized that focusing our models’ attention on the most relevant parts of the image – namely, the road lanes themselves – could significantly improve prediction accuracy. To test this, we created binary masks that isolated the lanes from the background, using a combination of manual annotation and automatic image clustering techniques.

Mask creation and Inference results on masked images

By applying these masks to our images before feeding them to the models, we effectively removed distracting elements like buildings, trees, and sidewalks, allowing the algorithms to concentrate on the key features that directly influence traffic congestion levels. This targeted approach aimed to boost the signal-to-noise ratio and enable more precise predictions.

Step 4. Cutting-Edge Modeling

Pushing the Boundaries of Accuracy: Training State-of-the-Art Neural Networks

With a clean, high-quality dataset prepared, the next step was to build powerful machine learning models capable of accurately predicting short-term traffic congestion. Our modeling phase focused on leveraging state-of-the-art neural network architectures and advanced training strategies to achieve top-tier performance.

Training EfficientNet Models

We chose the EfficientNet architecture as the foundation of our system due to its strong performance in computer vision tasks and computational efficiency. By fine-tuning pre-trained EfficientNet models on our labeled traffic dataset, we effectively transferred existing visual knowledge to our domain-specific problem.

Through iterative experimentation and hyperparameter tuning, we optimized learning rates, batch sizes, and augmentation strategies. These refinements led to significant performance improvements, enabling the models to classify congestion levels with high precision and generalize well across cities and lighting conditions.

Plot of training accuracy, validation accuracy, and test accuracy

Enhancing Model Robustness

To further boost accuracy, we introduced several techniques:

• Data augmentation to simulate varied real-world conditions.

• Class balancing to handle uneven representation of congestion levels.

• Ensemble methods to combine multiple model predictions for improved stability and reliability.

This combination produced consistent, accurate predictions even when dealing with diverse traffic scenes and environmental variations.

Beyond Classification: Measuring Traffic Density

While congestion classification provided actionable insights, we aimed to go deeper by quantifying traffic flow. To achieve this, we integrated YOLOv7-based object detection to identify and count vehicles within each frame.

By overlaying these detections with our lane segmentation masks, we calculated vehicle density per lane and analyzed directional flow patterns using a StrongSORT tracking algorithm.

Vehicle and Vehicle Direction Detection

This approach offered two major advantages:

1. It provided interpretable, data-driven metrics directly tied to vehicle counts and movements.

2. It enabled fine-grained congestion monitoring, identifying bottlenecks or imbalances across multiple lanes.

StrongSORT tracker for traffic flow rate measurement

Combining the Best of Both Worlds

By merging the strengths of classification-based congestion prediction with vehicle density estimation, we built a more comprehensive traffic intelligence system. This hybrid method not only predicts congestion levels but also explains why and where it occurs—empowering city planners and transportation managers with actionable insights.

Step 5. Seamless Deployment

Building an End-to-End Solution

Once the models were trained and validated, our goal was to integrate every component into a single, cohesive system capable of making real-time congestion predictions. This required combining data ingestion, preprocessing filters, and predictive models within a unified application.

We developed a Flask-based web application that connected directly to live traffic camera APIs in Ottawa, demonstrating how real-time data could flow through our entire pipeline. The interface displayed predicted congestion levels for each active camera feed, giving users a clear view of current traffic conditions across the city.

Smart-Traffic user interface

Integrating Filters for Data Quality

To maintain accuracy, we embedded the preprocessing filters into the live workflow.

• The blurry image detection filter automatically screened out unclear frames before prediction, ensuring only high-quality inputs reached the model.

• The YOLOv7-based traffic light detector added valuable context by identifying areas where congestion might be temporary due to red lights.

Optimizing Performance

Efficiency was critical for real-time analysis. We optimized model inference using techniques such as model tracing to reduce processing time. As a result, the system achieved 2–3 seconds per image on standard hardware, making it suitable for real-world deployment in urban traffic centers.

We continuously monitored performance, identifying and addressing bottlenecks in both the data pipeline and API communication layers. This ensured smooth operation and scalability across multiple camera feeds.

From Research to Real-World Readiness

The final deployed system combined machine learning, computer vision, and live data streaming into a fully functional traffic intelligence platform. It showcased how real-time AI solutions could empower city managers to anticipate congestion, adjust signal timings, and optimize road usage before gridlock occurs.

This deployment marked the transition from research to impact—demonstrating how advanced AI systems can directly improve urban mobility and quality of life.

The Outcome

Benefits and Applications

Transforming Transportation: The Far-Reaching Impact of Intelligent Traffic Management

The methodologies and technologies developed in this project have the potential to revolutionize traffic management and urban planning, offering a wide range of benefits and applications beyond congestion prediction.