Revolutionizing Short-term Traffic Congestion Prediction with Machine Learning

In this success story, we explore a smart traffic management system. By harnessing the power of machine learning and computer vision, we’ve developed a system designed to predict short-term traffic congestion on urban roads, while combating the challenges posed by environmental noise and limited data accessibility.

Here’s how we did it!

The Problem

The Challenges of Traffic Congestion in Urban Areas

Traffic congestion is a significant problem in many urban areas around the world. It is characterized by slow vehicle speeds, longer travel times, and long queues, particularly in metropolitan areas with a high number of road users. This issue presents various challenges for policymakers and city managers in their efforts to improve urban quality of life.

Traffic congestion can lead to several negative consequences:

To address this complex challenge, data-driven solutions that leverage advanced technologies like machine learning and computer vision must be developed. These solutions can help predict and mitigate congestion, ultimately improving the health, prosperity, and livability of cities.

The Background

What is Short-term Traffic Congestion Prediction?

Short-term traffic congestion prediction involves forecasting the level of congestion on a given road segment or network in the near future, typically within the next few minutes to an hour. This is distinct from long-term predictions, which focus on larger time horizons and are often used for strategic planning purposes.

The goal of short-term prediction is to provide actionable insights that can be used for real-time traffic management, such as adjusting signal timings, deploying traffic control measures, or providing information to drivers to help them make informed route choices. By accurately anticipating congestion levels, city managers can proactively mitigate the impacts of gridlock and keep traffic flowing smoothly.

The Untapped Potential of Traffic Cameras in Congestion Prediction

Traffic cameras have become ubiquitous in cities around the world, providing real-time visual information about road conditions. These cameras are primarily used for monitoring and surveillance purposes, helping traffic management centers identify incidents, accidents, and other disruptions to the flow of vehicles.

However, despite their prevalence, traffic cameras have been largely underutilized for the purpose of predicting and mitigating congestion. The rich visual data captured by these cameras contains valuable insights about traffic patterns, vehicle density, and road occupancy, which could be harnessed to develop more accurate and responsive congestion prediction models.

By applying advanced machine learning and computer vision techniques to analyze traffic camera footage, we can extract real-time information about traffic conditions and use this data to forecast short-term congestion levels. This approach has the potential to revolutionize traffic management, enabling cities to proactively address congestion before it becomes a major problem, rather than merely reacting to it after the fact.

The Goal

The main objective of the Traffic Congestion Prediction Project was to harness the untapped potential of traffic camera data and advanced AI techniques to revolutionize short-term traffic congestion prediction. By developing a comprehensive, end-to-end solution that leveraged machine learning and computer vision to analyze real-time traffic camera footage, we aimed to provide city managers with accurate, actionable insights for proactive traffic management.

Beyond our main goal, we set out to achieve the following as well:

By achieving these goals, we aimed to revolutionize the way cities manage and mitigate traffic congestion, harnessing the power of AI for smarter, more sustainable urban mobility.

Our Approach

Step 1. Comprehensive Data Collection

Tapping into a Wealth of Visual Data: Accessing Traffic Camera APIs Worldwide

To expand the geographic scope of our project, we identified multiple public APIs providing access to traffic camera images from cities across Europe and North America. We collected images at 5-30 minute intervals over a period of 24 hours from cameras in Ottawa, New York, Illinois, and London, amassing a dataset of over 100,000 raw images.

While some API providers imposed rate limits, such as allowing only one request per 60 seconds, we adapted our collection process to ensure adequate coverage throughout the day and night. By capturing images at regular intervals, we aimed to build a representative dataset encompassing a wide range of traffic conditions and lighting scenarios.

It’s worth noting that our data collection took place during the festive season, which may have influenced the observed traffic patterns. Nevertheless, this diverse dataset provided a solid foundation for training and testing our machine learning models, enabling us to assess their performance across different geographies and time periods.

Step 2. Collaborative Data Annotation

Harnessing the Power of Crowdsourcing: Labeling Images with Labelbox

To train our machine learning models, we needed a labeled dataset where each image was annotated with its corresponding congestion level. We focused on the 5,183 images collected from Ottawa, as they provided a good balance of highway and city scenes.

Using the Labelbox platform, our team collaborated to manually label each image as “High,” “Medium,” or “Low” congestion, as well as “Blurry” or “Camera-off” for images that were unclear or unavailable. To ensure consistency, we established criteria for each label and held group discussions to review edge cases.

To streamline the process, we uploaded the predictions from our base model as pre-labels, allowing annotators to quickly approve correct labels while focusing their efforts on refining inaccurate ones. This iterative approach enabled us to efficiently create a high-quality labeled dataset, which we later split into training and testing subsets to evaluate our models’ performance.

Step 3. Intelligent Data Preprocessing

Filtering Out the Noise: Developing Blurry Image and Traffic Light Detectors

To address the challenge of environmental noise that can confound traffic congestion predictions, we developed a suite of preprocessing techniques to filter out problematic images before feeding them to our models.

First, we tackled the issue of blurry images by creating two complementary solutions: a YOLOv5-based binary classifier and an anomaly detection model. Both approaches aimed to identify and remove images that were too blurry to yield accurate predictions, thereby improving the overall quality of our dataset.

Next, we recognized that the presence of traffic lights could also mislead our models, as they often indicate localized congestion unrelated to overall traffic flow. To mitigate this, we developed a YOLOv7-based traffic light detector, which could flag images containing traffic lights and exclude them from further analysis.

Zeroing In on What Matters: Creating Lane Masks to Isolate Key Features

Beyond filtering out noise, we hypothesized that focusing our models’ attention on the most relevant parts of the image – namely, the road lanes themselves – could significantly improve prediction accuracy. To test this, we created binary masks that isolated the lanes from the background, using a combination of manual annotation and automatic image clustering techniques.

By applying these masks to our images before feeding them to the models, we effectively removed distracting elements like buildings, trees, and sidewalks, allowing the algorithms to concentrate on the key features that directly influence traffic congestion levels. This targeted approach aimed to boost the signal-to-noise ratio and enable more precise predictions.

Step 4. Cutting-Edge Modeling

Pushing the Boundaries of Accuracy: Training State-of-the-Art Neural Networks

Building on the solid foundation of our preprocessed and annotated dataset, we embarked on a comprehensive modeling effort to develop highly accurate traffic congestion prediction algorithms. Our approach centered on training state-of-the-art neural networks, with a particular focus on the EfficientNet architecture, which has achieved remarkable results in computer vision tasks.

We began by fine-tuning pre-trained EfficientNet models on our labeled dataset, leveraging transfer learning to adapt these powerful networks to the specific characteristics of traffic camera images. Through a process of iterative experimentation and hyperparameter tuning, we gradually improved the performance of our models, achieving significant gains over the baseline.

To further enhance accuracy, we explored a range of techniques, including data augmentation, class balancing, and ensemble methods. By carefully crafting training pipelines that incorporated these strategies, we were able to create models that consistently outperformed benchmarks and demonstrated robust generalization across different geographies and lighting conditions.

Beyond Classification: Estimating Traffic Density through Vehicle Detection

While our primary focus was on predicting congestion levels through image classification, we recognized that a more granular understanding of traffic density could provide valuable insights for city planners and traffic managers. To this end, we developed a complementary approach based on vehicle detection and counting using the YOLOv7 object detection model.

By training YOLOv7 on a large dataset of annotated vehicle images, we created a model capable of accurately localizing and classifying vehicles in traffic camera footage. We then integrated this model with our lane segmentation masks, enabling us to count the number of vehicles in each lane and estimate the overall traffic density.

This density estimation approach offered several advantages over pure classification. First, it provided a more interpretable and actionable metric for assessing congestion levels, as the number of vehicles directly corresponds to the observed traffic conditions. Second, it allowed for more fine-grained analysis of traffic patterns, such as identifying bottlenecks or asymmetric flows between lanes.

By combining the strengths of our classification and density estimation models, we aimed to create a comprehensive system for predicting and understanding short-term traffic congestion, empowering cities to make data-driven decisions and optimize their transportation networks.

Step 5. Seamless Deployment

Integrating APIs, Filters, and Models: Building an End-to-End Solution

To bring our traffic congestion prediction system to life, we focused on creating a seamless, end-to-end pipeline that integrated the various components developed throughout the project. This included the Ottawa traffic camera API, the YOLOv5-based blurry image filter, the traffic light detection filter, and the base congestion prediction model.

We began by developing a Flask application that could pull live images from the Ottawa highway and city camera APIs, demonstrating the system’s ability to process real-time data from multiple sources. The application’s user interface provided an intuitive way to visualize the predicted congestion levels for each camera feed.

Next, we incorporated the blurry image filter into the pipeline, applying it to each image before passing it to the base model for congestion prediction. This ensured that only high-quality images were used for analysis, improving the overall accuracy of the system. When tested with live data from the Ottawa API, the filter performed well in most cases, although it struggled with some edge cases like foggy weather conditions. This highlighted the need to further diversify the training data to improve the filter’s robustness.

We also integrated the YOLOv7-based traffic light detection filter, which could flag images containing traffic lights and provide additional context for the congestion predictions. When a traffic light was detected with a confidence score of 30% or higher, the system would display a message indicating that users might experience temporary wait times. This feature aimed to provide a more nuanced understanding of the factors contributing to congestion at a given location.

Throughout the deployment process, we focused on optimizing the system’s performance and efficiency. For example, we traced the traffic light detection model at the start of the application to reduce inference times, achieving speeds of 2-3 seconds per image. We also carefully monitored the processing times for each component, identifying bottlenecks and implementing optimizations where possible.

By integrating these various elements into a cohesive, end-to-end solution, we demonstrated the feasibility and potential impact of our approach to traffic congestion prediction. The deployed system showcased the power of combining advanced machine learning techniques, computer vision algorithms, and real-time data streams to provide actionable insights for traffic management and urban planning.

The Outcome

Benefits and Applications

Transforming Transportation: The Far-Reaching Impact of Intelligent Traffic Management

The methodologies and technologies developed in this project have the potential to revolutionize traffic management and urban planning, offering a wide range of benefits and applications beyond congestion prediction.