The vision: Identify and classify roadway deficiencies with greater efficiency through power of AI.
Southwestern Ontario is no stranger to harsh Canadian winters and its side effects. Road deficiencies, like potholes, where pavement weakens and cracks after continually freezing and thawing, are especially common — alongside cracks in general and spalling.
One specific county had put a manual system in place to comply with provincial regulatory requirements (and its own internal standards) to fix identified deficiencies within a certain number of days after first getting reported. To streamline the process they decided to go automated, with help from Insight Canada.
All automated systems go.
Having previously worked with Insight, the county knew they had a partner on whom they could rely. In coordination with the community’s IT department, the Solutions Integrator, which has previously supplied it with hardware, is in the process of rolling out the automated road-deficiency detection maintenance system in question: the Municipal Operations Digital Integration Platform.
Instead of having county workers manually seek out deficiencies, the county of more than 60,000 residents is mounting cameras on patroller trucks that are set to automatically identify and log potholes as workers travel their routes. Currently in the testing stage, the system also brings together technology from a slew of Insight partners.
The in-vehicle AI processing is driven by a Lenovo SE70 edge computer in each truck, which captures a live video feed and processes an object-detection model developed using Microsoft Azure Cognitive Services. Mobility networking technology handles cellular connectivity, also providing local Wi-Fi and Global Positioning System (GPS) services across the fleet of around 20 trucks — 20, at least for the moment.
One solution fits all.
Phase 1 is to equip the maintenance/patroller trucks in the county with cameras — and the drivers with the custom Insight app and mobile access via tablets. The ultimate goal is to do the same for an expanded portfolio of the county’s government vehicles, including buses for example.
The county envisions a standardized solution such that the cameras can seamlessly be mounted on any vehicle, the system itself easily leveraged by non-patrollers. The hands-off AI would freely identify and log road deficiencies without the obstacle of a driver learning curve with respect to the app, which revolves around a largely manual process right now.
Automation through integration
The system’s current incarnation leverages three application sets: Geographic Information System (GIS) software that tracks the locations of assets, a scheduling module, through which workers get their assignments, and the patrol application, in which they make their observations. If a road deficiency is identified and approved, a work order then gets created.
From an integration perspective, Insight automated it all. The heart of the solution is a custom web application hosted on Azure, powered by Intel Xeon Scalable processors, which deliver built-in accelerators and advanced security technologies, to provide integration, automation and a seamless user experience for patrollers to validate deficiencies automatically detected by the solution or to manually enter their own observations. However, the goal is to prevent two workers from having to patrol in a single vehicle, one to drive and one to report. Now, the second worker will be able to drive a separate vehicle or get dispatched elsewhere. All the while, the first relies on the system to find deficiencies on its own. The system then creates a work order when necessary before assigning it.
To ensure transparency and the app’s intuitiveness, Insight conducted workshops with the drivers, consulting them on its design early on in the process. Of note, Insight also conducted workshops to gather information to properly respond to the county’s Request for Proposal (RFP) before the project even began. However, the project truly originated through Insight’s longstanding relationship with the county and a PoC from a few years prior that resulted in a smaller-scale model to cache data locally in a single vehicle.
Looking down the road
Over the roll-out of the PoC, provincial funding became available, leading to the project’s scope evolving, the RFP getting issued and Insight’s eventual selection. The scope may also evolve further, as the county has expressed interest in enhancing the system’s sensing capabilities to measure potholes down to the centimetre.
The future could realistically see additional uses for the platform. Thermal cameras could measure the temperature of roads, determine where for example brine, which is less expensive than salt, can be used, further adding to the return on investment. The county has also considered sharing imagery taken by the cameras to alert citizens to adverse road conditions, the importance of public safety consistently staying front and centre.