Dirt and Pollution Forecast on Vehicles Using a Computer Vision Approach

This case study details how we helped a chemical group to investigate a computer vision solution for forecasting the level of dirt and pollution on vehicles. From collecting the requirements to creating a database of labeled images from scratch and developing the deep learning models, we proved our client that the solution was feasible, and provided them with the keys to bring it on the market.

• Data Science
• Computer Vision
• Deep Learning
• Classification
• Image Analytics
• Artificial Intelligence
• Actionable Roadmap

Key Challenges

Our client is one of the leading chemical manufacturers in Germany that invests heavily in research and development and process optimization for itself and its customers. The company wants to be among the pioneers in the field of Artificial Intelligence and industrial innovation.

The group produces, among others, chemicals for car washing machines. In an automatic car washing station, no matter how dirty a car is, the same amount of chemicals will be used to clean it. Our client therefore wanted to investigate the possibility of having a personalized solution based on a camera that would evaluate the degree of dirtiness of a car. Depending on this evaluation, the person being about to clean a car could adapt the amount of chemicals needed.

The objective was obviously to reduce the costs associated with excessive product consumption by implementing a “pay as you go” model. But also, they wanted to limit the environmental impact that the consumption of chemicals and water may unnecessarily cause.

It is in this context that the German manufacturer contacted us. They were looking to:

• Find out if it was possible to create a computer vision solution for forecasting the type and degree of dirtiness of vehicles
• See the solution in action
• Have a clear plan of how to implement such a solution for their customers

Our Approach

In order to meet the challenge that our client assigned to us, we:

1. Organized a kick-off session in the form of workshops with our client’s scientists, business and technical departments. The objective was to collect all the needs and expectations, and to evaluate the process in place.
2. Created a database of labeled images. It is important to note that our client did not have such a database, and we needed both dirty and clean car images to move forward. Our teams went to a car washing facility to capture the first images. We quickly realized that the type of pollution on the cars actually depends a lot on the current weather and season (e.g. pollen in spring, and salt on the roads in winter). We thus defined the different dirt categories for the dataset and created high-quality annotations for the images to get started with our computer vision activities. We took pictures irregularly and individually, over one year, analyzed them and created an annotated and labeled image dataset.
3. Developed and trained deep learning models. Initially, we chose an approach consisting of analyzing small parts of each image individually (pixel wise classification) to get the most accurate results. But this approach turned out to be far too complex. We then switched to an alternative approach, using the entire image, and we quickly observed very promising results. Our models were able to predict how dirty a car was and what type of dirt was visible. On the technical side, we used pre-trained models, open-source tools to build the pipeline and utilized an Azure GPU to train the models.
4. Established a set of recommendations on how to optimize the models and the image capturing process to improve the predictions and give our client the ability to scale. We successfully presented the solution to the teams and provided guidelines for operationalizing the models. One of our main recommendations was to start a pilot phase with a selection of car washing stations to test the solution at a larger scale. Furthermore, we enabled the client to extend the dataset with the help of fine-grained guidelines for image captioning and labelling.

Benefits

• The computer vision solution was able to detect and forecast the type and degree of dirt with high accuracy.
• The image data set and model can be extended at will to include new types of dirt classes.
• The image recognition methods for dirt detection can be operationalized and we provided our client with the guidelines to do so.
• Our client got an initial estimate in terms of resources and an actionable plan to expand the solution to all car washing facilities.
• We handed over the keys to our client so that their teams could carry out the expansion of the project, and the operationalization and optimization of the models on their own.

Team involved

One Project Manager, one Data Scientist and two Deep Learning Engineer collaborated with our client on this project for 12 months.

Technologies

Share

Thanks to our
Author

Michael A.