White Paper: The Mythical Value of EDI Standards

The Mythical Value of EDI Standards

Alan Chute
October 10, 1996

Introduction

In the 1970s and early 1980s Electronic Data Interchange (EDI) was fueled by the development of standard transaction sets for specific documents such as Invoices, Purchase Orders, Bills of Lading, and Acknowledgments. Because of the high volume/low dollar nature of freight transactions, transportation suppliers led the growth in the use of Standard EDI transaction sets to conduct business electronically. Large carriers and shippers alike achieved productivity gains by switching millions of transactions from paper processing to EDI. However, many expected efficiencies were never realized due to the inability of existing applications to effectively eliminate manual effort for EDI trading partners. In addition, the successes that did occur largely bypassed medium and small companies due to the high cost of standard EDI vs. the low productivity value received.

The key to achieving productivity improvements is an application which can handle the automated interpretation of each supplier’s differentiated trading transaction in a way that eliminates manual effort, yet still delivers the same or better quality output. More emphasis should be placed on broader participation from your business partners. This means companies should be willing to accept any electronic format a supplier can produce consistently and quickly, and expect translation software which is easy to use yet powerful and flexible enough to interpret any format.

The Birth of EDI Standards

In the 1970s the Transportation Data Coordinating Committee (TDCC) developed the initial standard transaction sets for conducting Air, Motor, Rail, and Ocean transportation business electronically. In the early days of electronic commerce, standards were sorely needed to establish an infrastructure for doing business in a radically new way. Early adopters were shippers and carriers who dealt with a high volume of low dollar value transactions which required significant manual effort to process.

At a time when computer technology was expensive and far less advanced than today, dealing with multiple proprietary formats from each carrier was a costly barrier to cost-effective EDI. Achieving industry-wide agreement on standard transaction sets enabled EDI to overcome this hurdle. By the early 1990s, the TDCC standards were absorbed and endorsed by the ANSI.X12 Standards Committee.

Transportation Leads the Way

By the early 1980s, deregulation of the transportation industry in the US broadened the appeal of EDI for major shippers. Carriers were now forced to compete on price in addition to quality of service. Shippers could now negotiate their own pricing and payment terms in valid contracts with carriers. The incentive to accumulate a comprehensive database of actual transportation costs and shipping patterns to negotiate the most favorable rates and make the most cost effective carrier selections on the shipping dock increased dramatically. The least expensive way to accumulate such a database was by switching major carriers to EDI. This mutual incentive on the part of carriers and shippers led to substantial growth in EDI throughout the 1980s and 90s. The standards established by the TDCC helped facilitate this growth.

Unfulfilled Expectations

Despite this growth in the use of standard EDI transactions in transportation, many productivity expectations went unfulfilled. Too many people believed that EDI by itself was the solution to its business problems. Too many EDI implementations only automated the flow of information between trading partners, while manual and/or inefficient processing activities remained for both senders and receivers. Dedicated EDI staff was added to do nothing more than translate information from its native format to Standard EDI.

Many companies used Value Added Networks instead of (or in addition to) dedicated staff to support their new EDI efforts. This incremental cost of doing business electronically could easily offset any productivity gains when the applications and business processes on either side of the EDI transaction did not keep pace. On the receiving end, too many companies simply sent their incoming EDI transactions to the printer while traditional manual processes remained unchanged.

Smaller Companies Left Off the EDI Bandwagon

The incremental cost of translating native formats to standard EDI formats has kept many medium and small companies away from electronic commerce. This prevents these companies from achieving the potential benefits of automating manual business processes. In addition, EDI implementation plans of many large organizations have been derailed by their inability to bring up smaller trading partners on EDI. Many large organizations claim they can automate 60-80% of their transaction volumes and no more. To make matters worse, the 20-40% of remaining manual transactions are often those that have traditionally been the most complex and require the most effort to process. These factors result in disappointing productivity improvements from what would otherwise appear to be impressive EDI implementation statistics.

Standard EDI Not Really Standard

The vast improvement in Cost per unit of processing power - however you choose to measure it - has practically eliminated the economic hurdle that led to the development of EDI Standards in the 1970s. Microprocessors commonly found on desktops at most US companies today can easily process high transaction volumes, and powerful new software can easily and economically deal with unique translation rules for each carrier/shipper pair.

The mythology of EDI standards is that there ever really was a "standard" to begin with. Most trading partners through the years have developed and documented their own interpretations of the "standard" and expected each trading partner to comply. Disputes are decided in favor of the partner with the most economic power in the relationship (usually the customer). The end result is that suppliers typically have to maintain some unique code for every trading partner. What about this business model is "standard"?

Electronic data transfer using the Internet is becoming more prevalent and far more economical than traditional methods of transmitting EDI transactions either directly or through a Value Added Network. Why shouldn’t sending information electronically be as simple and inexpensive as sending e-mail?

While the importance of EDI standards at present is highly debatable because of advances in technology, the plain truth is that they are no longer necessary in today’s economic environment. When Standards were first established, product life cycles were measured in years or decades rather than weeks or months as they are today, and the pace of change was decidedly slower. There was time for standards to evolve through quarterly meetings with industry peers. In today’s competitive environment, companies must launch new products and services at a tremendous rate in certain industries - products which often only survive for a few weeks or months. Can you imagine Federal Express waiting for the next ANSI.X12 conference to launch its "First Overnight" service because there was no standard service code to reflect it in a standard EDI document? While this phenomenon is more pronounced today than ever before, it has probably always been wishful thinking to expect competitors in any industry to completely endorse "standard" transaction sets. Competitors will always seek opportunities to differentiate themselves from their peers which will cause a natural inclination away from any industry standard. How long can you survive in today’s competitive environment by offering "standard" products and services?

Too Much About Format, Not Enough About Content

A day in the life of an EDI Coordinator, responsible for converting suppliers from paper to EDI, is a telling experience. A pattern emerges which indicates decades of too much focus on format, not enough on content. Here are a few excerpts:

Any color you want as long as it’s black...
One supplier provides the standard service code of Miscellaneous ("MSG") for every accessorial charge. Given an example of a billing for Inside Delivery service which should be coded "IDL", the supplier promises to fix the problem. When the next feed is received every accessorial is now coded "IDL." Perfectly legal EDI syntax - perfectly useless for automating your business process.

Freedom - What a Concept...

Another supplier asks to see your "Implementation Requirements Document" for the ANSI 210 invoice transaction format. After years of being strong-armed by customers into accommodating everyone’s unique interpretation of the standard, the carrier responds with stunned silence when told we don’t have one:

Supplier: "How will I know what data to put where?"

Coordinator: "We don’t care what format as long as it contains all the data you provide on your paper invoice."

Supplier: "But, we’ve never done that before. How will we know what to do?"

After years of confinement, this carrier was given some freedom for the first time and the concept was frightening.

Why was I sending you this invoice in the first place?...
Another carrier provided an ANSI 210 invoice feed containing line items for each bill of lading item which were just filled with details about weight, class, pieces, descriptions, and even special comments. Only one small detail was overlooked - the billed amounts due for each line.
They got so carried away providing data in the right looping structure, they forgot why they were sending this invoice document in the first place - to get paid!

Common problems with Standard EDI Implementation Projects

The problems experienced by many companies who have implemented EDI using standard transaction sets cannot be solved by wishing that more companies would start using the standards as we read so often in EDI and Transportation trade journals. Problems can only be solved by addressing the root causes. These are:

Small to Medium size companies can produce electronic documents today, but they resist the "investment" in Standard EDI because they fear the costs will outweigh the potential benefits.
Large company EDI implementations have often fallen short of expectations because they cannot convert the last 20-40% of their trading partners to EDI, causing a high level of manual processing to remain.
Companies often underachieve expected productivity gains even with a high volume of EDI transactions because their applications are not sophisticated enough to truly eliminate manual processing effort.

Breaking Through the Mythology to Achieve Real Productivity

What Business Managers need to focus on instead of EDI Standards is the ability to do business electronically with ALL their business partners. In addition, to finally eliminate manual processing effort, they need powerful Rules-Based processing systems which can be easily maintained and operated by business experts who can make changes immediately to reflect changing business requirements without the need for costly IS departments. To get the productivity improvements promised by EDI, Implementation Managers should take the following actions:

Relax the emphasis on Standard EDI, and encourage the method which is most cost-effective for each supplier to start-up and maintain
As a project manager for implementing an electronic invoice processing system, the last thing I want to hear from a supplier is that they’re just bringing up a "new billing system" which will finally enable them to "do EDI." This typically means delays of 6-12 months. When the first Standard EDI feed is received, it is often in violation of standard syntax rules and missing half the data normally found on the supplier’s paper invoice form. Upon hearing this, they may hire an "EDI Consultant" to help them solve the problem. This automatically adds another 3-6 months before the syntax errors are corrected and the missing data is often never recovered. We should allow suppliers to send electronic trading transactions in any format and transmission method which is economical, easy to implement and maintain, and contains the most complete information.
Identify business process improvements which will make it easier for the last 20-40% of the business, which is the part that uses 80% of your resources, to go electronic
Many companies believe that implementing EDI is by itself a business process improvement. Unless you re-think the underlying assumptions about your audit, validation, and control procedures as you implement EDI, you may never significantly reduce manual effort. Companies often exclude specific processes or programs from EDI because they are believed to be too complex - or because the activities are being performed by another functional organization. Automating the last 20-40% almost always involves cross-functional evaluation of an entire process to eliminate activities that are just no longer necessary, or to re-engineer essential activities to be more cost-effective. Use computer technology to help you research, discover and eliminate process inefficiencies which create artificial barriers to electronic commerce.
Acquire rules-based applications which emulate the thought process of experts to finally automate even the most complex and unique business rules.
Mass Customization is an economic term which describes the competitive pressure to provide goods and services which are customized and personalized to meet each customer’s specific requirements. This trend is inconsistent with a standard model, and is sometimes at odds with efforts to simplify your business processes. Very often, the right approach for your company is to make a simple process more complex to address these competitive realities. To be competitive, you must be flexible enough to adapt to changing market demands, recognizing paradigm shifts and positioning your companies’ processes to respond. You must employ applications which are flexible, able to deal with complex and customized processes, and able to be maintained directly by Knowledge experts without the need for costly and time-consuming MIS support.

Conclusion

Most companies implement EDI by giving their suppliers a black and white choice: Either send us Standard EDI transaction sets or continue to send paper and take the risk that you won’t continue to be our supplier in the long-term. This approach has led to underwhelming results. Small to medium size companies have not participated, business processes have not been truly re-engineered in conjunction with the EDI implementation, and applications have not kept pace with the increased flow of information to enable the automation of manual processing work. Technology has made giant leaps since EDI standards were established twenty years ago. The technical hurdles standards were designed to overcome no longer exist.

The solution which meets with competitive realities of today’s economy of mass customization is to allow your trading partners to send electronic information in any format they prefer as long as it contains the information required to meet processing and information warehousing requirements. Business processes must be re-engineered across functional lines along with an EDI implementation to achieve the expected productivity improvements. Translation and Processing Software must be rules-based, flexible, and easily maintained directly by Knowledge experts to finally eliminate manual effort and automate even the most complex and unique business processes.

Companies who insist on EDI Standards or nothing in their EDI implementations are throwing up roadblocks unnecessarily. In the long-term, the ability to achieve all the potential benefits of electronic commerce will require the acceptance of information in a variety of forms and the use of powerful technology to manage this diversity. For those companies who cannot or will not accept this, the result could be not only the failure of their EDI implementation plans, but the continual decline of their competitive position as a company in an economic era where flexibility is rapidly becoming the most important competitive requirement.