Enterprise System Spectator blog: ERP and enterprise system vendor evaluation, selection, and implementation.

The Enterprise System Spectator

Tuesday, December 16, 2003

Turning software validation into a meaningful exercise

Working with clients in the life science industries, I deal often with the issue of validation of commercial off-the-shelf software. But, because validation is an FDA requirement, companies often treat validation as nothing more than a documentation effort--an exercise in producing a large volume of paperwork that the company can point to during an FDA inspection. So, I've been thinking about what companies can do to make validation a "meaningful exercise," one that adds real value to the system being implemented.

One key point is to understand the meaning of the word validation. Validation is ultimately to ensure that a system meets its requirements. Most validation professionals understand this. But too often the validation team interprets "requirements" as nothing more than "what the users want the system to do." So they conduct a series of interviews to ask users what they want the system to do.

Unfortunately, when users are asked what they want the system to do, most leave out many important things from a regulatory perspective. A better approach, in my opinion, is not to start with what the users want the system to do but what the applicable regulations require the users to do. I have found great value in sitting down with users, walking through the regulations line by line, and asking, "Do you want the system to help you do this?"

For example, FDA regulations for medical device manufacturers require users to ensure that raw materials are only purchased from approved suppliers (21 CFR Part 820.50). If the user wants the system to help him do that, then he probably needs the system to maintain an approved supplier list and prevent production materials from being procured from suppliers that are not approved to supply that material. Once I have established such a system requirement, it is a no-brainer to validate the system against that requirement. I simply test whether I can purchase material from a non-approved supplier. A series of tests such as this, that challenge the system to directly address true requirements, is the only way to uncover design flaws and false assumptions of the developer, implementer, and user.

How much more meaningful is this type of validation in contrast to what I see too often: the validation team writes a long series of test cases in excruciating detail to see whether the user can add, change, and delete a vendor master, whether a user can add, change, and delete a purchase order, whether a user can add, change, and delete a line item, etc., etc.,--but never test to see whether it is possible to order material from an unapproved supplier. No wonder they seldom discover anything interesting. They are not testing the system against its requirements--they are testing it against its design specification. That might be considered a system test, but I would not consider it a meaningful validation.

The lesson: start with the regulations, use the regulations to derive system requirements, and use system requirements to design test cases.

There is much more I could write on this subject, but I'd like some feedback. What is your experience with software validation? How do you ensure that validation is a meaningful exercise?

by Frank Scavo, 12/16/2003 07:38:00 PM | permalink | e-mail this!

 Reader Comments:

I tend to agree globally with your
blog. What matters at the end is
the content and workload involved
in achieving validation.
My experience is that companies
overreact sometimes to the regulations and devise a cumbersome
and extremely long validation for
the Off-the-Shelf SW.
Things shall remain in prespective
and we shall always remember that
we want to develop safe and effective medical devices not validate SW tools.
You are absolutely correct in your example. I work for a validation consulting firm and this is exactly what we do! We create our test scripts based on the design specs and not from the regulation point of view. And yes, we deliver hefty volumes of paperwork and sums it up with "the system is in validated state". Hilarious, just to think about it...
Post a Comment

Links to this post:


Powered by Blogger

(c) 2002-2018, Frank Scavo.

Independent analysis of issues and trends in enterprise applications software and the strengths, weaknesses, advantages, and disadvantages of the vendors that provide them.

About the Enterprise System Spectator.

Frank Scavo Send tips, rumors, gossip, and feedback to Frank Scavo, at .

I'm interested in hearing about best practices, lessons learned, horror stories, and case studies of success or failure.

Selecting a new enterprise system can be a difficult decision. My consulting firm, Strativa, offers assistance that is independent and unbiased. For information on how we can help your organization make and carry out these decisions, write to me.

My IT research firm, Computer Economics provides metrics for IT management, such as IT spending and staffing benchmarks, technology adoption and investment trends, IT management best practices, IT salaries, outsourcing statistics, and more.

Go to latest postings

Search the Spectator!
Join over 1,700 subscribers on the Spectator email list!
Max. 1-2 times/month.
Easy one-click to unsubscribe anytime.

Follow me on Twitter
My RSS feed RSS News Feed

Computer Economics
Outsourcing Statistics
IT Spending and Staffing Benchmarks
IT Staffing Ratios
IT Management Best Practices
Worldwide Technology Trends
IT Salary Report


2014 Best Independent ERP Blog - Winner 2013 Best ERP Writer - Winner Constant Contact 2010 All Star Technobabble Top 100 Analyst Blogs

Key References
Strativa: Business strategy consulting, strategic planning
Strativa: IT strategy consulting
Strativa: Business process improvement, process mapping, consultants
Strativa: IT due diligence
Strativa: ERP software selection consulting and vendor evaluation
Strativa: CRM software selection consulting and vendor evaluation
Strativa: Project management consulting, change management
StreetWolf: Digital creative studio specializing in web, mobile and social applications
Enterprise IT News: diginomica

Spectator Archives
May 2002
June 2002
July 2002
August 2002
September 2002
October 2002
November 2002
December 2002
January 2003
February 2003
March 2003
April 2003
May 2003
June 2003
July 2003
August 2003
September 2003
October 2003
November 2003
December 2003
January 2004
February 2004
March 2004
April 2004
May 2004
June 2004
July 2004
August 2004
September 2004
October 2004
November 2004
December 2004
January 2005
February 2005
March 2005
April 2005
May 2005
June 2005
July 2005
August 2005
September 2005
October 2005
November 2005
December 2005
January 2006
February 2006
March 2006
April 2006
May 2006
June 2006
July 2006
August 2006
September 2006
October 2006
November 2006
December 2006
January 2007
February 2007
March 2007
April 2007
May 2007
June 2007
July 2007
August 2007
September 2007
October 2007
November 2007
December 2007
January 2008
February 2008
March 2008
April 2008
May 2008
June 2008
July 2008
August 2008
September 2008
October 2008
November 2008
December 2008
January 2009
February 2009
March 2009
April 2009
May 2009
June 2009
July 2009
August 2009
September 2009
October 2009
November 2009
December 2009
January 2010
February 2010
March 2010
April 2010
June 2010
July 2010
August 2010
September 2010
October 2010
November 2010
December 2010
January 2011
February 2011
March 2011
April 2011
May 2011
July 2011
August 2011
September 2011
October 2011
November 2011
December 2011
January 2012
February 2012
March 2012
April 2012
May 2012
June 2012
July 2012
September 2012
October 2012
December 2012
January 2013
February 2013
March 2013
May 2013
June 2013
July 2013
September 2013
October 2013
December 2013
January 2014
February 2014
March 2014
April 2014
May 2014
June 2014
July 2014
August 2014
September 2014
October 2014
November 2014
December 2014
February 2015
March 2015
April 2015
May 2015
June 2015
July 2015
September 2015
October 2015
November 2015
February 2016
May 2016
June 2016
July 2016
August 2016
September 2016
October 2016
January 2017
February 2017
May 2017
June 2017
October 2017
January 2018
April 2018
May 2018
Latest postings