Argis Solutions’ CEO Brady Hustad presented on computer vision and augmented reality and how to use the Open Computer Vision Library with ArcGIS at Esri’s 2019 DevSummit.
By Alyssa Grant
Argis Solutions’ CEO Brady Hustad had the honor of sharing how to integrate computer vision and Esri, explaining how to use the Open Computer Vision Library (Open CV) with ArcGIS. The presentation covered the technical setup of Open CV, some interesting tips on how to successfully connect it to ArcGIS, and coding tips that will help Open CV function properly for developers interested in using computer vision in their next project. Reflecting on his talk, Hustad remarked, “It was exciting to see how people are getting creative and how computer vision and machine learning are impacting the way they are doing business. People are seeing ways to go beyond doing maps and creating systems software.”
Computer vision enables a computer to see something and make a decision that a human no longer has to make. For example, if hundreds of miles of roadways need to be analyzed for damage, computer vision could allow the computer to look through thousands of images and determine which images contain road damage. In robotics, computer vision has been one of the hardest problems for programmers to solve. Now with access to libraries such as OpenCV and some some finesse to connect it to ArcGIS, programmers are able to attempt to integrate this complex technology in ways that will shape the future of business.
Big cities can be GPS black holes—their large volume of concrete and metal throws off GPS service and blocks signals. It can be difficult to get accurate geographic data. Using the rough satellite location of the mobile device, the computer can compare and compute location using two known points, giving improved accuracy in the city. Computer vision could enable the mobile device to detect an asset such as a manhole cover, storm drain, or hydrant. It just needs to be trained to see these known above-ground facilities, enabling the ability to generate accuracy where before none existed.
Together, AR and computer vision could be used to document missing GIS assets. The computer could be programmed to be passively viewing in the background with the ability to notice an asset that is not documented in the ArcGIS data. The computer would then generate a basic record with spatial location. Its final step would be to ask the end user for further details, for instance: “This asset is not found in your data. Is this correct?” This would be an efficient way to quality check data.
Computer vision could also be used to train a computer to look at an asset when a field worker is looking at it and then automatically pull up the right manual to work on that asset, streamlining field work. As AR visualizer improves with computer vision and image recognition this will all be possible. Computer vision is a game changer in improving accuracy and processes.
How do we get there? Spatial referencing is required. Once you know where something is in space by way of dual cameras, you can extrapolate 3D space around it. Most mobile devices now come standard with dual cameras, paving the way for programming these functionalities. Argis Solutions has also made the code for Brady Hustad’s talk available on GITHub for developers interested in building projects using the OpenCV library and ArcGIS.
What is needed to program computer vision? A knowledge of a modern programming language like Swift, C++, Java, Python, Kotlin, etc. A developer will also need to be savvy with mobile software like ARCORE, ARKit, OpenCV, Esri, etc. Lastly, a project using Open CV and ArcGIS will need decent software and high availability GIS data. If you would like tips and further information on programming, please visit GITHub, where our example is built in Android using Java and Esri’s ArcGIS Runtime SDK for Android.