Featured Book of the Month:

Title: How to Plan an Audit : ASQ Quality Audit Technical Committee 

Author: Charles B. Robinson (Editor)

Publisher: John Wiley & Sons

Notes: There is a generic planning discipline for auditing office procedures, chemical processing, or manufacturing of a part. The intent of this audit planning guideline is to define the audit planning discipline clearly so that the audit judgment will be a timely, accurate, and meaningful tool. By methodically addressing these guidelines, an organization will efficiently and effectively set the stage for a meaningful audit.

No picture available

You can order this book from internal-auditor.com at: http://www.internal-auditor.com/books.htm

This Month's Feature Article

We've often said that internal auditing is a sampling. We don't audit everything, only bits and pieces. The question that is often asked, is what size should those pieces be? How many records should we examine? How many people do we ask a particular question to? How long do we observe a particular action? All of these questions center around selecting the appropriate sample size for auditing purposes. 

But before we look at sample sizes, let us gain a basic understanding of what a sample is. Let's say you are going to audit purchasing to determine if you are using approved vendors. You might have 5000 individual purchase orders that would apply to the audit. This would be called the "population", the total number of possible audit records (in this case). It might not make sense to check every sing record. So, instead, you might want to audit just a part of the 5000. That would be a "sample". Now the question is how many of the 5000 records are we going to look at? 

First of all, the standard does not give us a specific sample size to consider. In fact, the standard does not even hint that auditing is a sampling. So, what does one use to determine what size of sample to use when auditing? 

• Statistical Validity

• Time

• Complexity

• Criticality

Statistical Validity

There are formulas available which will lead you to determine what would be a statistically valid sample for a given population. In internal auditing, there is no requirement to have a statistically valid sample size. Sample size will, typically, be based on the other factors, rather than statistical validity, and that's okay. 

Time

"Time is the Auditor's worst enemy." That is a phrase we live by, here at Internal-Auditor.com. If time was not a factor, we would be able to examine, interview and observe the entire population. But, time is a factor, so we must determine what we can do in the time allotted. Time is the real reason we only sample, rather than investigate everything. In many cases, the time available will play the largest role in determining sample size. The more time available, the more detailed the audit can be and the larger the sample size may be.

Time can be greatly influenced by various factors. The amount of auditors on the team, the activities to be audited, location of the audit, and other resources available all play a role in determining the allotted time. All of these need to be considered in determining the audit schedule, and included in any audit planning and audit plan.

When determining sample size,  you can also take into account how often the activity is being audited. If the activity is audited monthly, you can adjust the sample size accordingly. Yearly audits will have to be more detailed, because you do not want to wait a year, in event your sample missed a nonconformance. 

Complexity

Some activities, processes and departments we audit will be more complex than others. The more complex the operation, the more factors that need to be determined and the more variables that will need to be considered for auditing. Activity complexity, accompanied with the allotted time is used to determine the relative sample size for an activity, or perhaps even which process variables will be audited and which will be omitted. If any are omitted, then they should be audited during  the next audit cycle.

Criticality

Processes and activities that have the most impact on customer satisfaction should have a larger sampling than those support activities which do not directly relate to meeting customer requirements. This is what the standard refers to "importance of the processes and areas to be audited". The more critical the activity, the greater need to audit more in-depth, and to use a larger sample size. 

Another aspect of criticality is the "status" of the activity or process. This is related to such things as results of previous audits, reject rates, customer complaints, employee turnover, new or modified processes. Anything that could place the activity or process at risk should affect the sample size. It should also affect the audit schedule, but that is another subject. 

Conclusion: 

Because internal auditing involves sampling, there is always the possibility that something major will be overlooked. This creates, as ISO 19011 says: "an element of uncertainty in auditing". We, as internal auditors need to be aware of this uncertainty, and respond accordingly. 

One last thing. Once you set the sample size, you should keep it at that size. You want to avoid the results of the sample to bias the audit. If you check the training records of ten employees and find all ten to be satisfactory, then that is what you found. If all ten were deficient, then that is what you found. Do not keep looking until you found something (good or bad). Just report on the findings, based on the sample. It is up to those performing the corrective actions to determine how bad something is, we just point out that there is something nonconforming.

As always...Good Auditing!

top of page

www.nsf-isr.org

Q – Can I change sample size on the fly?

A – Of course you can, but the question is why would you? There may be times when you might legitimately need to alter the sample size during the audit. However, you need to think about the consequences. It might appear that you are "going fishing" if the sample size were to be enlarged. It might appear that you are giving preferential treatment should you reduce the sample size. Remember we need to maintain credible audits, and changing sample size in the audit might jeopardize that credibility.

top of page

We have mentioned this before, but it is worth repeating. An excellent resource for internal auditors, and Management Reps, and Quality Managers and anyone else, is the Elsmar Cove (formerly Cayman Cove). It is a discussion board that addresses many different aspects of the Quality Profession. Registration is free, but if you wish to download attachments, you need to pay $25 (US). It is well worth the money. Elsmar Cove can be found at: http://elsmar.com/Forums . 

top of page

ISO 9001:2000 Clause 8.2.2 Internal audit

"The audit criteria, scope, frequency and methods shall be defined "

The standard requires these items to be defined. It does not state that they must be in the procedure, nor any other particular location. Audit criteria, scope, frequency and methods may be found is separate locations, and in many cases are. One of the most common places to see these defined is in the individual audit plan. The reason for this, is in many organizations, the complexity of processes and activities being audited require different criteria, or even a different audit scope. Regardless of where they are defined, the procedure (or other documentation) should point to where they are located for easy reference. 

top of page

February 2003’s question:

You are auditing corrective actions. One sample you examine deals with a complaint of product being damaged during shipping. The CA team determined that the crates were not strong enough and the CA was to add additional bracing to the crates. In the report was an email from one of the team members stating he did not feel the CA addressed the root cause. He feels that the root cause is systemic in that the sales department is not determining how the customer, or the trucking companies handle the product. Although this worked on the current complaint, it will not stop the complaint from occurring on other products. There is no record of a response from anyone else on the team, and there is no record of any discussions in any agendas or meeting minutes.

The answer:

This is a typical finding when dealing with corrective actions. It seems that someone always believes that the root cause was either not identified, or misidentified. Whether this is a nonconformance will depend on a couple of things, neither of which are whether we agree with the corrective action or not. The issue is not whether we agree, or not, but whether the corrective action is effective or not, and whether the organization followed their own processes. If the team followed their own procedure, and if the corrective action stopped the product damage, it would be difficult to write a nonconformance. However, an observation questioning the teams response to the validity of the dissenter's comments would be in order. 

 top of page

A Year Gone By

It is customary to look at the previous year, as we look towards the new. But to get some idea of the previous year, we have to go back even farther. 2002 was a fairly tough year for the Bradley family. Ruth Ellen's dad suffered a major heart attack, which required by-pass surgery. He was 81 at the time. His twin brother died of prostrate cancer. We were glad to see the year end. However, 2003 made it clear that it was going to be just as bad as the previous year. I commented on this in the March 2003 newsletter. I just didn't know how bad things were going to get. 

In January, Ruth lost a second uncle to prostrate cancer in January. In February, Ruth was diagnosed with breast cancer. the initial prognosis was not good. It had already spread to her lymph nodes, and there was concern that it had spread throughout the body. Fortunately, it was limited to just the breast and lymph nodes. In May my dad suffered a hemorrhage of the descending aorta. He passed away in June. Now the events became much more than just distractions. I had to deal with the death of my father, while caring for a wife having side effects of chemotherapy. In October, Ruth Ellen underwent surgery for her cancer. In November, she underwent a second surgery, and began radiation. The good news is that she is well on the road to recovery.

There is more good news, the economy is picking up, and jobs are returning to the manufacturing sector. The renewed optimism is showing in the stock market. Things, they are improving, to put a spin on an old song. From a personal standpoint, I hope the new year will be a bit slow. I really need a year with no surprises! From a business standpoint, I pray the year will be a turn-around year, and the economy finally fully recover. I do ask that you pray for Ruth Ellen, myself and this country. 

Dave

...Good auditing!

Copyright notice: internal-auditor.com is fully protected under US and International copyright laws. Copying, or re-transmitting all, or any part of internal-auditor.com is expressly forbidden without prior written consent from Ruth Ellen Carey Communications.
internal-auditor.com is a publication of Ruth Ellen Carey Communications.