Middleware – the key to asset information management

Tom Joseph, Mott MacDonald

How do you make sense of the flood of data coming your way? Mott MacDonald’s Thomas Joseph has an answer.

The amount of data engineers have to work with is vast and seems to be permanently on the increase. BIM is quickly becoming the norm for planning, analysis, design and construction. GIS data is essential for an understanding of how an asset interacts with its environs. And as the technology becomes more cost effective, we will see increased use of sensors monitoring temperatures, pressures, load, flow rates and stresses within assets, all creating continuous data streams.

"Middleware is a software layer that facilitates communication between data sources and the user interface. Developing good middleware applications is essential for managing large data loads and moving towards real time and smarter decision-making."

Add to this GPS data, weather information and even social media feeds. Managing and making sense of this endless volume of information presents engineers with a significant challenge but also an opportunity.

This is where middleware steps in. Middleware is a software layer that facilitates communication between data sources and the user interface. Developing good middleware applications is essential for managing large data loads and moving towards real time and smarter decision-making.

Working in the water sector, we developed H2KnOw-how, a middleware application which is linked to pressure, flow and water quality sensors that monitor water mains for leaks and wastewater systems for overflows.

In a water system that works well 99% of the time, the job of the middleware package is to ‘watch’ the stream of data and spot anomalies indicating a fault, enabling a maintenance team to be dispatched rapidly to the precise location. In a wastewater system, it provides insight into flow regimes, allowing proactive management of storm water surges, by rerouting or holding back combined sewage runoff to prevent flooding or overflows.

More ambitious plans are taking shape in the transport sector. Sensors are already being installed on road networks to detect traffic density and car speeds, allowing vehicles to be rerouted to avoid congestion. Linking to adjacent data streams such as weather information, would allow drivers to foresee which parts of a road network may be most affected by rain, while data on ticket sales for major sports or entertainment events could help to predict the time and location of traffic hotspots.

Such a system could bring vast efficiencies to cities, with the economy no longer losing millions to lost working hours each year, a reduction in unnecessary carbon emissions and improved road safety. Rolling out equally smart asset management systems to all other sectors and for all major assets would multiply these efficiencies even further.

These data-driven systems require middleware applications in order to make sense of the masses of data being collected, and developing strong middleware is a crucial step on the road to improved asset operation.


What are the elements of good middleware?

Data handling: With so many available sources of data, from sensor-based feeds within an asset to data streams from adjacent assets and other sources, middleware needs to have a powerful data-handling capability.

Data interpretation: Asset information management depends on making use of numerous feeds of unconnected data. For example, data on traffic flow at one road junction is not useful unless it can be linked to data from other parts of the city. Middleware packages have to relate these unconnected data streams in order to provide the decision-making layer with useful information.

Highlighting anomalies: Real time sensors will provide continuous streams of data. Much will be repetitive and of little use – such as regular flows of water through water pipes or routine patterns of road use over the day. Middleware needs to be properly programmed such that it recognises anomalies – such as major spikes and dips in water use – by comparing and contrasting data from multiple sources. These anomalies need to be relayed upwards to a decision-making tool or human operator.

Using historical data: The same principles apply over longer time periods. Good middleware applications should be able to contrast current data with historical data to detect changes in seasonal and annual performance and use patterns, indicating faults, changes in user demand or slow deterioration in asset condition.

Ease of use: Middleware should have a graphical user interface that communicates data in a user friendly way, allowing for smooth decision making. In H2KnOw-how, for example, the user interface presents users with a geographical model of the asset, with the ability to zoom in and click on individual sensor points to retrieve further data. Presenting the full range of data being collected would be overwhelming, so users also need to be able to navigate middleware in order to extract more detailed information where needed.

Functionality: Users should also be able to easily reprogram middleware to take account of new conditions on the ground. For example, trigger levels should be easily changeable, so that warnings are raised sooner or later according to the latest climate science or to optimise asset performance.


Middleware and the future of engineering

In the past we used physical evidence to improve assets and solve problems. A move towards sensor-based systems will see data analytics and statistics playing an ever more important role in optimising assets.

Asset management is evolving into asset information management. The benefits of this include the ability to much more precisely target resources and capital to increase capacity, repair faults and restore functionality – all delivering cost, time, labour and material efficiencies. To realise these changes, we need to develop the strong and dynamic middleware applications that can manage and make sense of the vast levels of data involved.

Thomas Joseph is Mott MacDonald's modelling team leader, working in New Zealand, who developed the H2KnOw-how software.