Towards Safer Highway Work Zones: Insights from Deep Learning Analysis of Thermal Footage
Project Timeline: Mar 2022-May 2024
Links
Z. Bhuyan, Y. Xie, R. Liu, Y. Cao and B. Liu, "Towards Safer Highway Work Zones: Insights from Deep Learning Analysis of Thermal Footage," 17th IFAC Symposium on Control in Transportation Systems (2024)
This research project aims to develop methods to extract vehicle trajectories, use the trajectories to analyze driver behavior, particularly lane-changing behavior under different conditions, and identify safety hazards and opportunities to improve work zone safety and operations.
Project Overview
Highway work zones present unique safety challenges for both workers and drivers. This project leverages thermal imaging and deep learning to analyze traffic patterns and driver behavior in work zones, providing insights for improved safety measures.
 |
 |
| Medford, MA: Segmentation of a highway traffic vehicles. | Medford, MA: Segmentation of a lane closure channelizer drums. |
Equipment
Data collection was carried out using FLIR TrafiSense AI-632 thermal imaging cameras, with the recorded footage stored locally. The cameras were installed on trailers positioned by the highway and a boom lift for optimal coverage.
Data collection was conducted with FLIR TrafiSense AI-632 thermal imaging cameras, and the footage was stored on local drives. These cameras were strategically mounted on trailers alongside roads and on a boom lift to ensure extensive coverage.
Thermal Vehicle Segmentation with Deep Learning
Deep learning excels in object detection due to its effective feature recognition capabilities, independent of color information. When integrated with thermal imaging, which detects heat emissions rather than relying on visible light, this approach ensures superior performance under conditions where optical cameras are ineffective.The success of deep learning applications heavily relies on a high-quality, diverse training dataset. Therefore, a substantial effort went into developing a detailed thermal vehicle dataset, designed for the specific conditions of highway work zones.
This dataset, critical for training our models to perform effectively in real-world scenarios, was created from thermal footage collected in Medford and Danvers. The dataset has three vehicle categories: small, medium, and large. Small vehicles consist of sedans, motorbikes, SUVs, and pickups. Medium vehicles cover trucks like garbage and concrete mixers, construction vehicles, and buses. Large vehicles include tractor-trailers.
Video Processing Framework Highlights
- Model Deployment: The trained model and tracking module were deployed on two separate computing environments with NVIDIA A100 and RTX 4090 GPUs, enabling parallel video processing of Medford and Danvers locations.
- Thermal Video Inference: Conducted frame-by-frame vehicle detection and tracking on thermal videos, leveraging powerful hardware for near real-time processing.
- Output Postprocessing: Aggregated detection and tracking data per video, compiling results into a structured JSON format for easy analysis and integration with analytical tools.
- Video Compression: Compress processed videos for efficient storage and sharing. Achieved a compression ratio of ~95%, reducing 1-hour videos from 3.5-4 GB to 60-110 MB.
Occluded vehicles
 |
 |
| Occluding vehicle. | Occluded vehicle. |
Detecting vehicles travelling close to work zone
 |
 |
| Danvers, MA. | Medford, MA. |
Sample analytics visualization
Funding
MassDOT Research Program with funding from FHWA SPR funds