5 Standard Solutions for Data Integration.

 

The datacenter interior
According to the International Data Corporation, 10% of digital data is structured, leaving 90% unused, unstructured, and hidden. 

Data integration - a critical gap in digital transformation

In the times of Industry 4.0, many companies substantially progressed on the path of digital transformation. Fellow procurement colleagues operate their new shiny source-to-contract suites, P2P, and SRM tools. However, one critical aspect of digitalization needs to be addressed or addressed sufficiently. 

The procurement value stream spreads well beyond the Source-to-Contract (S2C) cycle – it starts from planning and goes to supplier performance and risk management. We call it the Plan-to-Pay cycle. 

Plan-to-Pay procurement cycle with all individual elements
Even if your chief architect is a genius who managed to build a fully integrated digital Plan-to-Pay pipeline where data freely flows from one application to another, data integration tends to remain a challenge.

Data integration and its challenges are explained.

Let's define data integration as combining data from different sources for analysis, business intelligence, reporting, and feeding into other digital systems.

In e-procurement, each distinctive module consumes, generates, and feeds data loads. The integrated flow of data is not a given:

  • Planning and budgeting occur in finance systems, 
  • Purchase Requisitions (PRs) come randomly from different systems or even by email to initiate the sourcing process, 
  • the source-to-contract process interrupts upon a supplier award, 
  • contract data entry into the Accounts Payable (AP) database is manual with (maybe) a reference to the sourcing project ID, 
  • Procure-to-Pay (P2P) cycle jumps between different systems (PO in e-procurement, GRN in Material Management, invoicing, and payment in the AP module.) 

Besides the lack of process management, such patchwork installations generate buckets of disintegrated data that need to be manually cleansed, enriched, and analyzed even for such standard tasks as cycle time measuring or spend analysis.    

The process of data integration is far more complex than what's been described above. The e-procurement suite is just a link in the supply chain data network. 

Finance systems are natural data sources for procurement analysis, as their data is structured similarly for ease of exchange within the ERP environment. For example, the AP module will be the reference source of data for the spend analysis.

Further complexity will be observed across other internal systems, where the data format and presentation differ from ERP. 

From experience in aviation, procurement can obtain useful info from Catering on consumption, spare part planning inputs from MRO, cash flow projections from Treasury, digital assets (e.g., licenses) from IT, and more. However, accessing this data from an IT security perspective could be challenging.

The least visible and utilized procurement data is on the supplier side. There is no need to explain how valuable that data could be if there was the most straightforward integration between your systems and one of your partners.

Besides supplier ones, multiple external systems could be integrated to improve procurement's business intelligence and analytical capabilities – supplier financial and risk analysis, global and domestic economic reports and forecasts, open and subscription-based price databases, commodity indexes, etc.

Key highlights on data integration challenges from the "State of Business and IT Innovation" report 

Integration challenges directly impact revenue and customer experiences.


In light of increasing operational inefficiencies, it is not surprising that more than half (54%) of LoB respondents say they are frustrated by the challenge of connecting different IT systems, applications, and data at their organization. Many view this weakness as a threat to their business and the ability to provide connected customer experiences.


  • Siloed systems and data slow down business growth:
LoB employees know the repercussions of failing to connect systems, applications, and data. More than half (59%) agree that failure in this area will hinder business growth and revenue.

  • Behind disconnected experiences are disconnected systems, applications, and data:
Most (59%) of LoB employees agree that an inability to connect systems, applications, and data will negatively impact customer experience — a fundamental prerequisite for business success today.

  • Automation initiatives require integration:
Three in five (60%) respondents admit that failure to connect systems, applications, and data will hinder automation initiatives.  

Digitization vs. digitalization

So many integration failures and disconnected experiences have at least one thing in common. The perceived digital transformation, which created all that data mess, in fact, was digitization.  

Digitization is the process of converting analog information into a digital format. 

Some executives do this when converting their physical tender boxes into new-world e-procurement tools. They carefully translate the status quo from analog to digital format, preserving all outdated business processes.

Alternatively, digitalization uses digital technologies and digitized data to impact how work is done, i.e., transforming business processes and working practices. 

Data integration belongs to digitalization, which transforms legacy data-disjointed processes into integral data streams. 

Standard solutions for data integration

To sort out this spaghetti bowl of internal and external data feeds, procurement can use a few standard solutions:

  1. Manual extraction and analysis or so-called Common Data Interface. This integration happens on a procurement analyst screen, opens multiple windows with different reports, and reconciles and finds logical connections and patterns. Unfortunately, in many companies that claim to undergo digital transformation, this is how far data integration goes.
  2. Uniform Data Access integrates data in a similar format, e.g., between procurement and finance systems.
  3. More sophisticated integration types assume the importance and complexity of the subject and preparedness to invest in appropriate solutions. Data Warehouse stores data from multiple applications and platforms and can be used as a data lake. It is extracted, analyzed, and packaged in custom reports by specialized software.  
  4. Application-based integration is an automated search, extraction, and data processing by a dedicated app. 
  5. In technologically advanced companies, integration occurs on the Middleware level. Internal and external systems communicate via the dedicated software layer, and the data ingestion and format are standardized and usable across all connected platforms.

Data - the oil of Industry 4.0

The experience of many companies who invested in best-in-class procurement and ERP platforms still indicates the need for more attention to the data integration aspect. This leads to procurement operating automated workflows but not utilizing the full potential of analytics and business intelligence and highly sophisticated systems producing arrays of dark data that will never be used. 

The data is called the oil of Industry 4.0. Companies must utilize it consciously and effectively as they manage their fossil fuels and energy.

More information on this and other exciting topics can be found in "The Technology Procurement Handbook." It represents 23 years of experience, billions of dollars worth of sourcing projects, and 1000s hours spent on research, analysis, and content creation for the most demanding professional readers. 

Comments

Post a Comment

Popular Posts

The Importance of Supply Chain Collaboration

Image
  Why is supply chain collaboration essential? Once, I was asked what makes the supply chain important to a company. Our profession may not be rocket science. Other peers around the company overlap with us in their functions - Technology tends to manage suppliers , Finance owns the demand planning and budgeting , and Sales and Marketing monopolize the end-customer value chain.  These and other colleagues do not always appreciate us; some believe they are even better at our job.  Shadow procurement, supply chain, and logistics have become standards for many companies.  However, some critical differentiators - like collaboration - justify the very existence of our profession!  The supply chain is the most collaborative function in the business.  We find peers everywhere, even behind enemy lines. We welcome any opportunity to form a cross-functional team with anyone who can bring business value through supply chain strategies, sales  and operations pl...
Read more

The National Program of IT as the Alter-Ego of Procurement

Image
" One of the worst and most expensive contracting fiascos in the history of the public sector ." Some real-life cases only require a little reflection to understand that the project was predefined for failure well before its launch. Can we make appropriate conclusions to prevent that from happening again?  The first step is to deepen our study of such cases beyond populistic cliches.    NHS IT System (2001-2011) The national program for IT in the health service is the largest civilian computer project in the world. It was spawned in late 2001 and early 2002.  A far-reaching vision set out a program to transform people's healthcare experiences. Hospital admissions and appointments would be booked online—the choose-and-book system. Pharmacists would no longer struggle with GPs' indecipherable handwriting, and drug prescriptions would be handled electronically. Politicians set the duration of the program at two years and nine months from April 2003 until December 2005...
Read more

My Q&A on Agility for "Trusted Magazine" (December 2023)

Image
The source link . How could you describe your career path in a few words?   I worked in procurement for 22 years - telecom, airline, railways, public infrastructure services and consulting industries, and multiple geographies. I’m a Fellow of CIPS and an author of “ The Technology Procurement Handbook ,” where I tried to share my passion for what we used to call “IT Procurement.” This is where I started digging deeper into the Agile agenda , thinking of finding repercussions with our professional life and blogging on related topics.      How do you think agile practices have transformed companies over the past two years?   I assume that Agile practices in procurement are hyped more than used : many companies still mandate the use of the RFx as the sole answer to any emerging sourcing requirement, many fear or don’t appreciate the demand management, and digital transformation frequently happens without any comprehensive strategy and benefits trac...
Read more