Enhancing Real-World Finnish Traffic Sign Detection and Classification Using Synthetic Data Augmentation and Transfer Learning Techniques

Abstract

Traffic sign identification and classification are two of the most important modules in road safety and autonomous vehicle technology development. This dissertation focuses on developing a real-time Finnish traffic sign detection and classification system using synthetic data to alleviate the shortage of available real-world Finnish traffic sign datasets.

Initially, there was a dataset of synthetic cropped images representing 38 Finnish traffic sign classes, created by a researcher who worked on this study earlier at JAMK University of Applied Sciences. Training a custom Convolutional Neural Network (CNN) on this dataset resulted in 99% accuracy on synthetic data but only 36% accuracy on real-world data. To address these deficiencies, a new, improved dataset of cropped traffic sign images was created using Blender and computer vision techniques. Synthetic traffic sign images with transparent backgrounds and different angles were generated from Scalable Vector Graphics (SVG) files in the Finnish Traffic Agency GitHub repository. These were further enhanced through an OpenCV-based module that overlaid traffic signs onto real-world backgrounds with augmentations including night lighting, blur, and pixelation.

The augmented data, together with an EfficientNetB0 model initialized with ImageNet weights, reached an accuracy of 99% on the synthetic data and 95% on the small real-world data, showing a massive improvement compared to the original dataset and model. Subsequently, the dataset was further extended to include 17 additional traffic sign classes, bringing the total number to 55 classes. Transfer learning was applied to fine-tune the EfficientNetB0 by freezing most of its layers and adding a new output layer. This updated model reached 99% accuracy on the synthetic dataset, 100% for the original 38 classes on the small real-world dataset, and 96% for all 55 classes, proving that the enhanced dataset and model are robust.

For real-time applications, a You Only Look Once version 11 (YOLOv11)-based dataset with 640x640 images was prepared, and a YOLOv11 model was trained to detect traffic sign bounding boxes in video feeds. While the YOLO model detected most of the traffic signs in the videos from Finnish roads effectively, class mismatches constrained its accuracy. To soften this issue, the classifier EfficientNetB0 was added by processing cropped regions of bounding boxes from YOLO, which considerably improved the class prediction accuracy. Yet, the dataset, being over 500 GB in size and the corresponding computational demands, posed serious hurdles to further optimization.

This work proves the effectiveness of synthetic datasets in training deep learning models and, therefore, bridges the gap between synthetic and real-world data to enable robust performance in real-world applications. The corresponding system can provide a low-cost method for traffic sign detection and classification to improve intelligent transportation systems and speed up the development of autonomous vehicle technologies.