At Cuurios we are keen to develop software solutions that use your process data to deliver valuable insights. The OSIsoft PI Asset Frameworks (PI AF) provides a hierarchical, asset centric model of data with detailed history, which is indispensable to monitor and further analyse vital production data in the industry. It means millions of records of real-time and historical data. This data is invaluable, provides not only useful insight but also the possibility of improvement for better control and efficiency when processed and displayed right.
Have you chosen wisely?
Having a database filled with vital information is not more than a pile of values, up until the moment when you put that data to work. Every number, every date, every remark can be useless or on the contrary: it should help your company to work smarter, not harder.
Making the right connections, creating useful relations, calculations and transforming your data into concrete actions is what our motto, data to action means. Poor choice of data to process means poor, sometimes even useless insights. Modern technology provides you the tools, to make your company’s job easier – an overflow of data on the other hand will just make everyone’s life harder.
Let’s get technical
During our several encounter with PI AF (using the Web API) we learned some useful lessons and tricks. For those with an IT experience, the fact that the extended use of a framework also means dealing with its limitations and shortcomings is not all too surprising. In case of the PI AF, the main obstacles we had to overcome were related to the limit of search queries, result sets, and performance issues due to the excessive amount of information.
The use-case
One of our use cases were monitoring and handling inhibition values on offshore gas production platforms – more precisely detecting when an inhibition for any checkpoint is turned on, and track it’s changes until it is turned off. The added value lies in a well-managed and monitored safety system.
Finding the PI points
These checkpoints do not just belong to one asset or even one asset type, in fact, they can be found all over the asset tree. This is where the PI AF built in search query comes to the rescue.
https://MyServer/piwebapi/search/query?q=name:*INHIB
The above query gives us almost 6.000 results. Although PI AF has the possibility to set the size of the result set, the built in maximum limit is 1.000 item per page, which means a minimum of 5 REST requests in parallel to process the almost six thousand items.
What makes it complicated to work with PI AF at the very beginning is the fact, that the above query does not provide us values, or any actual data about the PI point other than it’s WebID, the unique identifier used in PI AF. As a next step, querying the WebID gives you direct access to the actual values of a certain checkpoint.
Performance with over 6.000 REST request?
Although now we have all the inhibitions points directly accessible with their unique WebID, the reality is, checking all of their values takes roughly six thousand get requests. If you can’t imagine what that exactly means, below an example:
https://MyServer/piwebapi/streams/yourWebID/value
6.000 of these requests need to be sent and processed. It takes in our particular setup over 5 minutes. Can you imagine pressing a button and waiting for over 5 minutes for the result? Let’s all just be honest – the reality is, most people lose their patience even after a couple of seconds, because even that loading time is unacceptable nowadays, let alone 5+ minutes.
Let’s build stream sets!
Implementing a working Refresh button at this point was the last straw. While the updates where running only hourly in the background, this 5 minutes wasn’t great, but still barely noticeable from a user point of view. Up to the point, when we placed a button on the page for manual refresh purposes. Then it became clear: we need another solution.
After some research we found out, that the PI AF Web API supports bulk data retrieval even for unrelated pi points, the only thing you need is the list of WebID’s. These queries are called stream sets and are providing fast bulk retrieval.
https://MyServer/piwebapi/streamsets/value?webid=yourWebID1&webid=yourWebID2
Only a couple of things to keep in mind:
- the URL has a length limit, in our particular setup we could retrieve the values for approximately 400 WebID’s in one stream set (the length of the webids varies, make sure you test this extensively in your setup);
- it is necessary to adjust your method to handle the response accordingly, as the result is a list of json objects called Items,
- if any of the WebID’s in the stream set runs on error, the whole request runs on error, meaning even one unavailable WebID and the whole request goes down the drain.
- Implementing stream sets meant we could reduce the number of REST request from 6.000 to 15. Additionally, a stream set request response time is not noticeably longer than a simple value request, so our performance got a big boost: from over 5 minutes down to an average of 10 to 15 seconds.
Summary
Building something really meaningful on top of a database with literally millions of available records, if done smart, can improve the way you organize your work. Focusing on what you need to know, what you need to see from those information at first blink means you can put the carefully collected data to work. It is not going to make the decisions for you, but it can help pointing out when and where you need to make decisions, take actions.
