Zubin Bhuyan, Ph.D.
← Back to Portfolio

Project Timeline: 2024-2027 (Ongoing)

This project proposes an integrated, low-cost system to detect and deter railroad trespassing incidents, with a focus on improving safety and operational reliability.

Project Overview

This project integrates thermal imaging, machine learning algorithms, and edge computing to develop a cost-effective detection and alerting solution. Core system elements include adaptive warning signage, on-site real-time video analytics, and automated identification of unauthorized access. Powered by solar energy with battery backup, the system enhances operational reliability, enables accurate detection, supports local data storage, minimizes communication expenses, and generates comprehensive insights into trespassing events. Implementation of this approach would allow MassDOT to deploy targeted mitigation strategies, thereby improving rail safety and strengthening overall service reliability.

Sample thermal frames captured at railroad sites in Greenfield, MA.

Key Features

• Thermal camera-based detection for all-weather operation
• Real-time machine learning processing at the edge
• Dynamic warning signs for immediate alerts
• Solar-powered, off-grid operation
• Automated cloud data transmission
• Cost-effective deployment solution

Technical Implementation

The system employs thermal cameras that can operate effectively in various weather conditions and lighting scenarios. Edge computing devices process video feeds in real-time using trained machine learning models to detect human presence on tracks. When trespassing is detected, the system activates dynamic warning signs and logs the incident.

Thermal camera imagery captured at railroad deployment sites in Taunton, Massachusetts, with an overlaid map indicating the approximate camera field of view.

System Architecture

Comparison of cameras with different FoV, illustrating variations in scene coverage.

• Thermal Imaging: High-resolution thermal cameras for reliable detection
• Edge Processing: Local ML inference for real-time response
• Warning System: Dynamic signage and alert mechanisms
• Power Management: Solar panels with battery backup
• Communication: Intermittent cloud connectivity to reduce costs

Edge computing plays a central role in the system architecture by enabling all video analytics and decision-making processes to occur locally at the deployment site. By executing machine learning inference directly on the edge device, the system achieves low-latency detection while eliminating dependence on continuous network connectivity. This localized processing approach enhances reliability, reduces bandwidth requirements, and ensures continued operation even in environments with limited or intermittent communication infrastructure.

Example detection results obtained using varying numbers of negative samples in the training dataset. Results indicate that adding an excessive number of negative samples degrades performance in previously unseen scenes, likely due to the limited informational content of thermal imagery, where distant individuals may be misclassified as artifacts by the model.

Hardware enclosure for the LED warning sign, including the microcontroller, battery, and PV charge controller. The unit communicates wirelessly via Wi-Fi or LoRaWAN with an edge computing cabinet containing the deep learning module and thermal–RGB camera system.

The system is designed for fully off-grid operation through the use of solar power combined with battery energy storage. This configuration allows for continuous functionality independent of existing electrical infrastructure, making the solution suitable for deployment in remote or hard-to-access rail corridors. The inclusion of battery backup improves system uptime and resilience, ensuring reliable operation during periods of low solar availability or adverse weather conditions.

Communication within the system is handled through a hybrid wireless architecture. The edge computing module transmits detection events wirelessly to dynamic warning signs, enabling immediate on-site alerts. For remote monitoring and data aggregation, the system utilizes a 4G cellular connection to communicate with cloud-based services. To minimize data usage and communication costs, all video processing is performed locally, with only essential metadata, such as object trajectories and selected key frames, uploaded for further analysis and record-keeping.

Impact

This system provides an affordable and effective solution for railroad safety, potentially preventing accidents and saving lives while reducing operational costs through its off-grid, low-maintenance design.