Industrial Analytics and Machine Learning for Asset Maintenance are the Crown Jewels of Industry 4.0. The question we will address in this article is how to conduct a POC to find the most suitable solution for your industrial plant.
The promise of Industry 4.0 and the potential revenue gained from increased uptime has created strong interest in IIoT technologies. For instance, industrial manufacturers that have implemented digitalization strategies were interviewed by PwC and expect revenues to grow by an average of 2.9% and to reduce costs by 3.6% per annum for the next five years. As a result, many analysts predict a surge in spending on IIoT-related technology.
Although there is almost universal agreement that Big Data and Machine Learning can unlock substantial value within the industrial sector, the choice of which solution to purchase is often complex. During dialog with senior management at leading manufacturers and energy producers, we’ve observed the following challenges:
- There is uncertainty about how to select an infrastructure platform for IIoT. Although there are several large vendors, there is still confusion in the marketplace. From a functional perspective, there is no single definition of Industry 4.0. Furthermore, given the dynamic nature of the category, vendors are hesitant about providing meaningful product roadmaps.
- A significant amount of innovation comes from startups that are not widely known by CTO’s and CIO’s. Companies are struggling to keep up with the all the new market entrants and to understand the differences between solution offerings.
- As with every revolution, Industry 4.0 brings disruptive change. In some cases, the position of Chief Digitalization Officer (CDO) has been created. However, other organizations have not created well-defined roles and responsibilities. The result? The process of selecting solution providers can fall through the cracks. Oftentimes, responsibility is given to Information Technology (IT) groups that have less experience with Operational Technology (OT) or vice- versa.
- Even if the organization views the move towards digitalization as strategic, there is still the practical issue of accessing sensor data. Most sensor data that is captured in a production plant is not used. Concerns about data hygiene, the lack of qualified data scientists within the industrial sector and inadequate Business Intelligence tools can slow the adoption of IIoT-based analytics.
Payment for POC
One of the most common topics discussed about POC’s is the issue of payment. Some software vendors charge to conduct a POC whereas others provide this as a free service. The logic is simple: if the POC is part of the vendor’s sales cycle, then why should the potential customer pay?
At SKF Enlight AI, we charge for a POC for the simple reason that there are too many industrial plants that are interested in Industry 4.0 at a strategic level, but have back-end constraints that would prevent deployment of a commercial solution. When a client pays money, it indicates their level of commitment.
For almost all vendors, charging for a POC is not to generate immediate revenue, but for cost recovery.
4–Step Process for Conducting a POC for an Industrial Analytics Solution
Based on our experience on both the client and vendor side, this article provides a framework that can be used for conducting a POC for an Industrial Analytics solution.
Please note that this article is vendor neutral.
Step 1: Scope and Planning Stage (1 week)
During this phase, the internal team is assembled, a project plan is formalized and success criteria are defined.
1.1 Create a Team of Internal Champions
Industrial Analytics software is not a commodity item. In many cases, deciding on an Industrial Analytics solution provider has broader implications. Therefore, the decision-making process should include stakeholders with authority and influence within the organization.
Practitioner’s Tip: Stakeholders should represent both IT and OT
Organizationally, there are still disconnects between IT and OT organization. The causes for these are well known and include the typical turf-war issues (budget, political etc.) inherent in technology disruptions. Furthermore, IT departments will not have the domain expertise in areas such as Machine Learning and Artificial Intelligence. At the same time, IT departments are often the gatekeeper of sensor data stored in a historian database. IT plays a vital role in Data Governance. There are numerous processes for data cleansing, storage and access required for IIoT to be implemented.
IT groups will need to be responsible for setting standards for network access and security protocols. Standards need to be consistent across the organization and should include IIoT related spheres. Without access to clean data in real time, the POC may not be successful. If IT plays a constructive role, it can help create the data infrastructure needed to support the project
1.2 Develop the Project Plan
The purpose of the POC is to reach a decision about vendor selection within a specified time-period. From the outset, all internal stakeholders need to understand the decision criteria that will be used for final selection. This likely requires buy-in from stakeholders with the authority and budget to approve a purchase decision.
This POC is about a significant decision and should be formalized. The project plan does not have to be extensive, but needs to include the following elements:
- List of Requirements / Objectives
- Project Milestones and Deliverables
- Resource Allocation
- Communication Plan and Point of Contact
- Success Criteria (see 1.3 Below)
1.3 Use Data-Driven Success Criteria
No system is 100% accurate and both false positives and false negatives are to be expected. The POC team needs to define the success criteria prior to engaging with potential vendors.
Must-Have Metrics for POC
Your project champions or digitalization team should create specific success criteria based on data performance and of the results as well the functionality of the solution. The following three categories of metrics are recommended:
Accuracy of the results including:
- How many of the findings where meaningful vs. how many were not?
Calculation time including:
- How long does it take to the vendor to provide the results?
- What is the level of vendor’s dependency on customer’s engineers and understanding of the asset?
Product strengths and weaknesses including:
- User interface
- Ease of integration and installation
- Functionality (e.g., strength of vendors’ BI capabilities) and level of support.
- Product roadmap and future features
Keep in mind that a POC gives you the opportunity to work with a vendor and to evaluate whether they are responsive to your requirements and can work with your organization on an ongoing basis.
Step 2: Implementation Phase (3 weeks)
Execution is the core process. Working with the solution provider, we recommend including in the POC historical data analysis. To maximize the efficacy of the pilot, we suggest using months or years of historical data for the training and calibration of the Machine Learning algorithms.
Even before you analyze the performance of the data analysis, it is important to observe the overall process. The specific issues to address include the following:
- Was data provided by resources on the schedule that was agreed upon?
- Were there any internal constraints or roadblocks during the execution?
- Was the vendor able to meet all commitments with respect to the delivery of results?
Based on 2.2 Observations above, the original timelines and scope is changed. Any modifications require the approval of project stakeholders. These should be formalized and communicated to the project team and vendors based on the parameters defined in the Communication Plan. (See 1.2 above)
Practitioner’s Guide: What can go wrong?
Based on our experience at SKF Enlight AI, the following are the most common mistakes made during the POC.
- Failing to secure C-Level support at the onset of the POC
- Excluding IT and procurement from the process
- Misunderstanding the requirements
- Unclear timeframes / weak project plan
- Undefined success criteria
Step 3: Evaluate Success Criteria (1 week)
The next step is to compare the insights of machine performance provided by the solution provider with actual performance in the field. Please refer to 1.3 above for the list of success criteria.
There are many approaches to Industrial Analytics including Supervised, Unsupervised and Semi-Supervised Machine Learning. There are pros and cons to each solution as well as cost differentials. We suggest a review of each methodology as part of the overall POC analysis.
Finally, we recommend conducting both Return on Investment (ROI) and Total Cost of Ownership (TCO) analysis.
Step 4: Vendor Selection (1 week)
The goal of the POC is to make a decision about Industrial Analytics software for Asset Maintenance. The POC is not a theoretical exercise or a learning experience. Both the solution provider and internal stakeholders benefit from clearly defined success criteria and a transparent communication plan to inform the vendor of next steps.
The vendor selection process is optimal when the following occurs:
- There is executive buy-in and support from decision makers with the authority to purchase the software.
- The criteria for vendor selection are established as part of the Project Plan (refer to 1.2 above).
- Vendor notification (go/no-go decision) is based on the mutually agreed upon Communication Plan.