A flawed philosophy is to migrate the legacy data, meet the deadline, call the project green and successful while the users must figure out to how to correct and update the data. These ERP systems are not designed to handle the volume of change to data, provide a simple method to track change, obsolete a record with history view and archive functionality is non-existent.  Another reason this is a flawed philosophy is that purchasing contracts are set up based on the “bad” data, a unit of measure, part number or manufacturer change will void a contract resulting in wasted time of valuable resources and at the end of the day an inability to source the item, this could set in motion a critical manufacturing line shut down. Let face it, an ERP system is designed store a product or service record providing the business a method to transact, not to cleanse a record to a single master version of an accurate classified, verified and technically described Master Record. Therefore the activity of migrating data to a new system is not Master Data Management.

Master Data Management needs to be independently structured and separately managed in the organization not through IT. It is critical that within the Master Data Management organization to properly represent the business assets of the data (engineering, purchasing, customer, etc). The data is the core information used as the foundation to run the operations, sometimes referred to as the BI for the analytics of sound decision making processes. If the data is incorrect in the new systems, how is the BI improved? How is the business case ever calculated and successfully achieved? I can’t even imagine trying to tally up the potential “cost savings” when bad data is migrated to a new system.

Establishing a MDM program will need to have clear and well defined ownership, stake in the end user organizations and representation in the design and schedule of the software roll outs with full participation in all the projects with data involved. They should also participate in the project design strategy for systematically cleansing, classifying and migration of the data to the new system. Strategy should include an audit of data in the legacy system, let’s face it there maybe 20 year old records with no transactional history or balance on hand in inventory. Should this data be moved to the new system? The answer is NO.

An MDM data strategy to support the IT team can encompass a number of options. A simple option is the publishing of a long term schedule establishing adequate time for the data group to meet the data cleansing and classification requirements. This is not always possible, so what about a phased strategy? Some of the possible steps should include

• Evaluation of data to review transactional use 
• Evaluation of the stock on the shelf and confirm that none of the inventory should be obsolete and disposed of.
• Review of data related to the equipment but is not inventoried
• Identify data that should not be moved to the new system
• Establish data priorities for cleansing starting with high transaction use and stock items classified and cleansed first.

An ongoing maintenance and new set up process is imperative to be established with an easy method to request an urgent record during the data migration to support the day to day operations of the business.

We need to get out of the mindset that MDM is simply a data migration to a new system. MDM is a business process to establish the single version of accurate information which is then propagated throughout the organization, part of which is the proper migration of data from legacy systems.

 A key component for Master Data Management (MDM) is data cleansing which has multiple disciplines such as address cleansing or PIM (product information management). My expertise is in PIM, therefore my meetings have been focusing on data in the ERP and Inventory system.

My latest meeting was with an informed Material Manager, he understood the concepts of master data management, after the introduction meeting, he stated that “We had a data cleansing project and it did not work, I ended up going back and correcting the data.” Through the discussion, I came to believe that the data cleansing company, extracted the data and attempted to auto classify a half million records. As a purchaser of these types of services, I asked what was the process for mapping and quality checks?

The business issue is the buying team’s inability to utilize spend analytics and the solution is that the data needs to be referenced to the UNSPSC^® (The United Nations Standard Products and Services Code^®). The scope of the project is mapping the purchasing data to the UNSPSC^®. In my experience, I have identified four general levels of PIM data cleansing, 1) auto mapping 2) auto mapping with a manual review 3) verification and 4) enrichment. The cold hard facts are “buyer beware”.

The detail of the levels are:

1. Auto mapping: if you have a large collection of data, automation is a requirement however there are some issues. First, auto mapping incorrect, incomplete and inconsistent data will result in a system that will still have incorrect, incomplete and inconsistent data. The quality of the auto mapping is dependent on the structure of the data. If the data is structured to a noun or class, the auto mapping process will have high quality rate. If the data is set up as “free text”, the results will be dismal. This method will not address duplication or data quality in your system.
2. Auto mapping with a manual review: this process will take the results of the auto-mapping process and add a step of a manual review of the data. The question of the review, will all records be audited in the review, or is the process to review just the records that when the auto mapping just failed? How will consistency of the audit be managed? Again there are still the inherent issues as described in the auto mapping process.
3. Verification: In order to improve data quality, the data cleansing process requires verification with the manufacturer (service or product). The verification process assures that the purchasing record is set up to the correct manufacturer (referenced to the supplier via the contract), part number for restock ordering, UOM (Purchasing Unit of Measure), description with correctly classified i.e. BEARING, TAPER and the UNSPSC^®. Our process is to request the manufacturer to provide the UNSPSC^®. If the manufacturer cannot provide the UNSPSC^®, the item is correctly classified; the auto map to the UNSPSC^® will be successful. The verification process positions the data to identify duplication, manufacturer obsolescence and inaccurate data requiring additional information from the business to reconcile.
   
4. Enrichment: The fourth level of data cleansing quality, in addition to verifying, the data is enriched, this can be obtaining a price, warranty with the terms, additional description attributes, ECCN (Export Control Classification Number), recommended repair spare part information, eCl@ss, NSN (National Stock Number) or any other data element your business requires.

The conclusion is asking the right questions of how my data cleansing project will be implemented and managed are essential to making it a successful data cleansing project.

The first fundamental rule is that the customer business doesn’t stop. For all you data purists out there that believe that one day a switch to turn on the cleansed database is in the near future, please include me, I would like to see it. Most master data management projects included years and years of legacy data; therefore there is an acceptance to draw a line in the database by last used date. When I design a data cleansing project, I will have a new item setup process referenced to legacy items, this way the client business continues and as the new items are analyzed and setup, we can reference and update the legacy item information. Independently, we will always have the legacy data cleansing parallel the new set up process.

As the data cleansing project is designed, let’s start to explore the data elements and classifications. Every client will have their material types and material status set up but generally during the data / systems assessment there should be a thorough review of industry standards vs. company processes. I find that our clients appreciate the opportunity to bench mark their processes and data structure elements such as material types and status.  We will start with material type and material status.

Material Type

Material types can be as simple as goods and services or as complicated as service, critical spare, spare part, commodity, generic, blueprint, etc. The material type is a critical element to classify which template is used for setup in the downstream legacy systems with an inventory stocking strategy applied.

Obviously a service can be standardized by the class type to describe the service where a cost for the service can be standardized. The definition of the service is described by the properties, for instance a service class of CLEANING, OFFICE can be set up with descriptive elements such as 10,000 square feet, light cleansing (dusting / vacuuming), etc. From a purchasing perspective, the buyer can run the reports globally to determine how much is spent for office cleaning then evaluate the costs and utilize best practice sourcing strategies and other global supply chain processes to lower costs. The purpose of the standard naming conventions of classes and property are to provide enough standardize information to provide the ability to compare and cost services or products.

If a critical spare is being set up for sourcing and inventory, then the part has been evaluated by maintenance or engineering and determined that the spare is critical for production uptime. An inventory plan is developed for stocking the critical spare including an initial buy quantity, plan for stores (inventory) setup of item’s unit of measure (each, assembly, package, etc.), min / max, reorder quality, stocking location, etc.

Material Status

In addition to applying a “material type” to the item records, due to the longevity of materials used in the manufacturing operation, a material status should be utilized as a long term data maintenance process. In dealing with component manufacturers and suppliers, a component may be active from a plant use perspective; however the component manufacturer no longer manufactures the item. How is that possible? A piece of equipment can have a 10 year or a 50 year life span, to maintain a piece of equipment, a list of recommended spare parts is identified and set up for equipment maintenance. If the spare part component is obsolete by the manufacturer but the piece of equipment is still in use on the production line, the material status would be “obsolete active”. A different buy / stock strategy would be implemented, such as purchase all available stock from the manufacturer or another alternative is to source with unconventional methods such as through eBay or maybe contract the item to be built by a local shop.

Typical material statuses that I have experienced are active, inactive item referenced to an active item, obsolete active, obsolete inactive (typically the status to start the disposal process) and archive. The archive status is a classification used by the analysts to allow the viewing of the item information but is not visible to the client or the item record is not exported to the client systems.

I would appreciate any input or better yet a discussion of the different material types and material status used in Product Information Management (PIM) or Master Data Management (MDM). As an industry we inherited material types and material status used in a purchasing system or maintenance systems designed to meet business function but not from the data quality or master data management perspective. What are the proper data requirements for a material type or material status? The MDM or PIM software companies and data quality consultants need to provide input from the data management perspective to provide long term data management functionality.

If we’re talking about a large company then inevitably with this new system comes the monolith project with whole organizations of people and processes, projects and documentation. The compulsion is to be sure that everyone, everywhere who has any relationship to it has their input and their needs accounted for. Along the way, the cost of implementation and other peripheral indirect costs have likely negated a great deal of at least any short term savings. Not to mention the potential increase in continuous maintenance costs and loss in performance. These are a few things I’ve learned from experience and I welcome yours.

Always have a specific objective when planning for development or evaluating software to purchase that overrides all others. Start with something like a mission statement, “We need this new system for….”

Determine the Real Needs. Try to separate the “must haves” from the “nice to haves”. Bells and whistles are great but there needs to be a true benefit. Seek a balance between development time, software performance, hardware performance and user experience. I always try to put special emphasis on the user group which stands to benefit the most. Having many users who can do their job faster and more efficiently can add up to real savings versus the few users who have a special need which bogs down the project and performance.

Change is inevitable. If some requests for additional features come along, evaluate them against the mission objective. There is nothing wrong with listening and investigating ideas for project add-ons as long as the benefits outweigh the costs in time and money, but there needs to be a limit or you’ll never complete the project. Good ideas can always be implemented later if it makes sense then you’ll have the benefit of the research already done, but be quick with the research. Evaluate the impact for doing it now or waiting. Here are some good questions to start with: 1) How much more money?  2) Would this be faster/cheaper for programming to do it now versus waiting and doing a more complicated enhancement?  3) Is the impact to the users great enough to warrant it?

Know the roles. Good ideas can come from anyone. Every project must have a project champion who makes the final decisions (and live with them) and also eliminate roadblocks. You need a user advocate who has done the job and knows what it takes. Have programmers who possess both talent and vision, not just code crunchers, and listen to them.

Have good documentation, and “Good” is subject to interpretation. This is another area where the KISS principle is very often not utilized. If you have to hire ten people to sit in meetings just to maintain your documentation you’ve probably overcomplicated it and certainly increased your project cost. I try to start with these principles:

1. Document the people on the project and their responsibilities. Let there be no question as to who does what.
2. Everyone who has a job to do needs to understand what they need to do and have the documentation to reference.
3. Keep the language simple. Focus on getting the point across. If it takes a rocket scientist to understand it you’ve failed.
4. Of course, document the issues, decisions made, by whom etc. but be sensible. Document enough to cover for the “he said/she said” but content is most important. No bonus points for flash.
5. Know who is supposed to have what done and when. Another obvious one here but I see too often where target dates are determined top down with little or no thought to cost or the tasks. Don’t let the tail wag the dog. Pushing hard to get the job done is fine but be realistic. Listen to the people who know before making bold predictions.