Lessons learned from labeling images for AI development

Our experience can help you avoid costly mistakes

Before becoming an industry leader in providing real-time and historical imagery and data from roadside and on-vehicle cameras, Vizzion got its start as an imaging consulting firm. With our background in machine vision, we’ve developed internal tools and products that rely on AI to automate quality processes and detect certain conditions in the imagery. Our experience developing these algorithms has taught us that quality labeled data is the foundation for any good model. We’ve taken this knowledge to the market and are now providing some of our partners with sets of images labeled in-house. Labeled data is a selection of data that has been tagged based on shared defining characteristics, which can be used to train a computer to classify images based on these characteristics.

Images labeled left to right as high, medium, and low congestion.

Combat low-prevalence conditions with billions of images

To take advantage of the 100,000+ cameras available in our live image service, our partners are training algorithms to recognize road conditions of interest. To provide them with images for training, we first had to determine the most efficient way to use our own service to source images. The solution is to use our archive of over one billion images from roadside cameras located in the top 30 North American markets and pre-selected locations for on-vehicle image collection. New images are saved every five minutes and are kept for a year, allowing us to start labeling images from all seasons immediately.

Train algorithms for different seasons with images collected year-round

Throughout the collection process, we aimed to minimize bias and provide a comprehensive set of contextual conditions. Selecting imagery from throughout the year allows our partners to control for changes caused by seasonal effects. Seasonal changes in the background of the image can have confounding effects on how the algorithm processes them. For example, foliage and vegetation change colour and size throughout the year. A large branch that extends to cover the camera’s view of the road could be construed as a vehicle, thereby impacting congestion metrics. The ground adjacent to the road changes colour as precipitation levels vary throughout the year, potentially impacting how the algorithm determines if a road is wet or dry. Similarly, light angles and shadows shift throughout the year.

Images showing the effects of seasonality.

Avoid bias by selecting images at random

A model will be most effective if the data it’s trained on matches real-world conditions. Creating an image set that matches the conditions encountered in production settings requires images pulled from our archive to be extracted in a random manner. Bias created by selecting images from certain seasons, times of day, and cameras needs to be eliminated. A mix of zoom levels, image resolutions, and a variety of city and highway images are required to match the conditions seen in the live service.

Use a significant amount of images to account for variations in conditions

From our experience, an image set should contain at least two thousand images to train, validate, and evaluate a model, and more is always better. 80% of the images should be used to train the model, with the remaining 20% split evenly into validation and testing sets. The slight differences in the object or condition being detected and the surrounding context within the image necessitate a set of images large enough to cover the vast majority of possible conditions. For example, a road can be wet if it’s currently raining, rained recently, or from snow melt. All three conditions look different, but cause the same effect on the road surface. Acquiring a sufficient number of images can be time-consuming. By using our image sets, our partners can focus their time and efforts on developing an algorithm and applying it to their application.

Wet roads can look different but cause the same effect to vehicle traction

We have the experience to produce accurately labeled image sets...

Vizzion released our first traffic camera API in 2005. Since then, we’ve learned a lot about traffic, weather, and other road conditions and how they appear in the cameras in our network. For example, our experienced team of content engineers can tell the difference between an image with low visibility caused by heavy fog and an image that appears blurry due to a low-resolution camera. They can tell when a road is wet, compared to when the pavement is dark in color. They can determine with contextual clues whether a road is truly congested, or if traffic has simply accumulated at an intersection.

Cameras providing a blurry image and cameras showing low-visibility conditions may appear similar.

…and use definitions that reflect real-world conditions

As with any scientific endeavor, the successful development of an algorithm depends on a defined end point. We worked with our partners to create precise definitions of road conditions. To use road congestion as an example, most images show multiple travel lanes. Vehicle density by lane can vary significantly in situations where commuters are entering a city during the morning rush hour, large trucks are clogging the slow lane, or a broken-down vehicle is blocking a lane. From an internal study we conducted to measure traffic congestion, we found that the best results are reached by labeling the lane in the image with the greatest density of traffic. This creates a consistent definition that works despite the disparity in traffic levels and also presents results that provide actionable insights to the operator working with the traffic data. Snow conditions are another example. The roadway is not affected by snow off the shoulder, so we only classified images as showing snow if it was present where vehicles travel.

We can provide assistance beyond collecting images

We can also provide insight into the algorithm development process. Our success in building image analysis tools that detect out-of-service cameras, direction of current view, and low-visibility conditions, for example, makes Vizzion a knowledgeable partner. Not only can we provide images, but we can also supply EXIF data alongside them. One of our partners requested this, as they required the camera's latitude and longitude coordinates to compare their algorithm’s output against other data sources.

Interested in seeing how labeled images can help you with your algorithm development? Connect with us to receive a set of sample images.

About Vizzion

Vizzion is the leading provider of road imagery for traffic, weather, road condition, and safety operations and applications. Through partnerships with over 200 different transport agencies and on-vehicle camera providers, Vizzion offers live feeds from over 100,000 cameras in 40 countries across North America, Europe, Asia, Australasia, and key markets in South America and Africa. Both on-vehicle and roadside traffic camera services are available through Vizzion’s flexible API and turnkey Video Wall application. Vizzion’s content is trusted by major apps, map providers, broadcasters, fleets, and automotive organizations. Contact [email protected] for more information.