Asset Data Integrator (ADI)
Integrated data, the enabler for data driven asset management
To create valuable information from spatial data, to view the data or find differences in place or time, the access to the underlying data sources is crucial. At most utility companies this data – both asset data and failure data – are stored in separate databases, set up for a specific goal. Often with dedicated data registration software, tailored to the user entering the data or the application of the collected data.
When a utility has embraced the ambition to base pipe rehabilitation upon data instead of expert judgement, modelling requires data as input for the calculations, and data preparation needs to be undertaken. This comprises of data transformation and integrations, see Figure 1. While doing so, the following questions will (need to) be addressed:
• is data accessible?
• is data complete?
• how to deal with gaps in data or outliers (extreme values)?
• are time stamps defined uniformly?
• can the selected algorithm deal with specific formats in the data, eg maximum length of strings, presence of capitals, etc.
Dutch data science consultancy focusing on the management of underground assets. We combine GIS expertise, with data science and asset management expertise. The 10 staff team represents a century of experience in the Dutch water sector and has a strong passion
to solve the needs of utility companies with data driven solutions. Spatial Insight is considered a leading consultancy in the Dutch home market.
Asset Data Integrator (ADI)
When implementing pipe rehabilitation models, our Dutch clients requested to streamline the spatial data input process for the applied models. That lead to the development of ADI (asset data integrator). Our belief: let the different asset, failure and other data sources, including open data sources, be as they are. Full merging of separate databases can be complex and expensive, but even more important, may lead to the replacement of data registration tools, adding even more costs. ADI mirrors data from existing separate databases and stores them in a uniform and standard format, allowing modelling and exporting in the easiest way. Then we do more. Truly discriminating is the stability of ADI. We run different small scripts that check the new or updated data at every refreshment interval. When data lacks, deviates from expected or when essential steps in the mirroring process fail, a flag is generated and – upon request is made available to the utility’s data manager. A Dutch client uses these flags to identify possible mistakes during data registration.
Your data scientists and external parties will be grateful and will be able to deliver their results faster and at lower cost. Spatial Insight will also be happy and will apply substantial discounts to our other products when ADI is in place.
Data security policy
Data has become an important asset of each utility company, and most utilities apply strict data policies. Since ADI is connected directly (read-only) to internal systems of registration it is usually installed on a server within the customers domain. Spatial Insight supplies the system and assists with installation and configuration. The data is the utility’s property and that’s how we deal with the data.
To set up ADI, the different data-sources need to be accessible. We experience that endusers of pipe rehabilitation models need to convince ICT managers to approach the installation of ADI with a positive mind-set. Some of the steps in data transformation and integration may be similar to activities that normally cause worries so early buy in from internal teams is key to project success.
Limitation of carbon footprint
Spatial Insight intends to limit the carbon footprint of its operations, and therefore we want to limit our travel movements. We can deliver the majority of our work from distance. We propose only to fly in to build trust, which is hard -if not impossible- to do online.
We hope and trust SI-ADI will contribute to the demand for data driven asset management. We are pleased to explore how we can define a proof-of-concept (PoC) project around ADI at any site.
Figure 1. The steps of data transformation and integration (together: data preparation) in the flow from creating data up to information for decision making.