augmented reality

Addressing Aging Infrastructure: Make AR Part of Your Solution

GIS data shown through augmented reality technology can help communicate discrepancies during the remediation process for upgrading underground infrastructure. 

By Alyssa Grant 

The Flint water crisis highlights the consequences of aging underground infrastructure and the deeply negative impact it can have on a community. Remediation can be a daunting, complex process. The remediation efforts to replace Flint’s lead and galvanized steel pipes with copper piping will be winding down by the end of the year. As of September 2018, 15,031 pipes have been excavated and 7,233 pipes have been identified as requiring replacement, underlining the scope of this headline-making water utility remediation project. 

Flint is not alone—aging underground infrastructure is a nationwide issue. The 2017 Infrastructure Report Card gives America’s drinking water a “D,” communicating that communities across the country are in dire need of water pipeline upgrades. 

Outdated natural gas pipelines are another concern. Old pipelines can leak, damaging the environment, even resulting in explosions causing property damage and fatalities, like the Merrimack Valley gas explosions on September 13, 2018 that resulted in 80 fires and one death. 

Updating aging utilities can be a complicated task.  Gregory Korte’s investigation on the state of our nation’s natural gas pipelines for USA Today revealed that the aging gas pipelines in Merrimack Valley were acknowledged to be a challenge to remediate:

“. . . Columbia Gas warned state regulators that replacing pipes in places like Lawrence would be difficult. The pipelines were in densely populated areas dominated by paved surfaces. They're intertwined with other utilities in crowded rights of way. " 

Unfortunately, the explosions occurred before improvements could be fully addressed. The costs related to the resulting damage could reach $1 billion

The complexities of underground infrastructure and the delicate excavation that can be involved in remediation underline the need for a sophisticated way to view and share mapping data. Some companies and municipalities are finding that augmented reality technology meets that need. 

What is augmented reality (AR)? This technology interposes computer-generated images in a real-world setting. Imagine being able to see GIS data through a cell phone camera—a scene you might see through the camera’s eye with AR could be a sewer line below a sidewalk intersected by a natural gas pipeline that crosses underneath an adjacent roadway. 

With the Argis Lens, a mobile AR application, that imagined scene is reality. The Lens quickly visually communicates what lies beneath the ground because a picture is worth a thousand words, especially when the complex GIS data of water or natural gas pipelines and sewer or stormwater systems are involved. The Argis Lens dynamically translates GIS data into AR imagery on mobile iOS devices.  

Using AR, underground infrastructure stakeholders can project their GIS data on new job sites to show foremen and construction crews where underground assets are located in real time. In addition, with the Lens, they can confirm that all assets are marked appropriately before excavation begins. Leveraging this technology, these companies are seeing a new level of collaboration between their asset protection teams and contractors because they are using AR to communicate high-risk areas where particular care needs to be taken before digging.

Cities, pipeline, and utility companies can all benefit from increased productivity on the job site, improved communication, and data quality confirmation. As infrastructure below the ground continues to deteriorate and become obsolete, proactive stakeholders with underground assets will turn to new technology, such as the Argis Lens, for more effective solutions as they upgrade and improve what is hidden.

DevSummit 2019 Talk: Augmented Reality Paired with Computer Vision

Argis Solutions’ CEO Brady Hustad presented on computer vision and augmented reality and how to use the Open Computer Vision Library with ArcGIS at Esri’s 2019 DevSummit. 

By Alyssa Grant 

At Esri’s 2019 DevSummit held in Palm Springs, CA on March 5-8, machine learning and ArcGIS REST JS created a big buzz. Another hot topic among DevSummit attendees was how developers are switching to 4X JavaScript libraries, which allows for more 3D capability and better functionality. This conference is the annual opportunity for Esri’s expert developers to share their technical knowledge with other software developers to help them write better code, build better systems, and create state-of-the-art apps that utilize ArcGIS mapping technology. While there are 8 rooms simultaneously hosting a full daily schedule of talks, there’s also a little time for dodgeball and beer! 

Argis Solutions’ CEO Brady Hustad had the honor of sharing how to integrate computer vision and Esri, explaining how to use the Open Computer Vision Library (Open CV) with ArcGIS. The presentation covered the technical setup of Open CV, some interesting tips on how to successfully connect it to ArcGIS, and coding tips that will help Open CV function properly for developers interested in using computer vision in their next project. Reflecting on his talk, Hustad remarked, “It was exciting to see how people are getting creative and how computer vision and machine learning are impacting the way they are doing business. People are seeing ways to go beyond doing maps and creating systems software.” 

Computer vision enables a computer to see something and make a decision that a human no longer has to make. For example, if hundreds of miles of roadways need to be analyzed for damage, computer vision could allow the computer to look through thousands of images and determine which images contain road damage. In robotics, computer vision has been one of the hardest problems for programmers to solve. Now with access to libraries such as OpenCV and some some finesse to connect it to ArcGIS, programmers are able to attempt to integrate this complex technology in ways that will shape the future of business. 

Big cities can be GPS black holes—their large volume of concrete and metal throws off GPS service and blocks signals. It can be difficult to get accurate geographic data. Using the rough satellite location of the mobile device, the computer can compare and compute location using two known points, giving improved accuracy in the city. Computer vision could enable the mobile device to detect an asset such as a manhole cover, storm drain, or hydrant. It just needs to be trained to see these known above-ground facilities, enabling the ability to generate accuracy where before none existed. 

Together, AR and computer vision could be used to document missing GIS assets. The computer could be programmed to be passively viewing in the background with the ability to notice an asset that is not documented in the ArcGIS data. The computer would then generate a basic record with spatial location. Its final step would be to ask the end user for further details, for instance: “This asset is not found in your data. Is this correct?” This would be an efficient way to quality check data. 

Computer vision could also be used to train a computer to look at an asset when a field worker is looking at it and then automatically pull up the right manual to work on that asset, streamlining field work. As AR visualizer improves with computer vision and image recognition this will all be possible. Computer vision is a game changer in improving accuracy and processes. 

How do we get there? Spatial referencing is required. Once you know where something is in space by way of dual cameras, you can extrapolate 3D space around it. Most mobile devices now come standard with dual cameras, paving the way for programming these functionalities. Argis Solutions has also made the code for Brady Hustad’s talk available on GITHub for developers interested in building projects using the OpenCV library and ArcGIS

What is needed to program computer vision? A knowledge of a modern programming language like Swift, C++, Java, Python, Kotlin, etc. A developer will also need to be savvy with mobile software like ARCOREARKitOpenCVEsri, etc. Lastly, a project using Open CV and ArcGIS will need decent software and high availability GIS data. If you would like tips and further information on programming, please visit GITHub, where our example is built in Android using Java and Esri’s ArcGIS Runtime SDK for Android.