Have you ever wondered why the last Internal Audit or External Audit uncovered so many issues even though you thought everything was great? Are most of your nonconformities or corrective actions pointed out by external parties? If the answer to any of these questions is yes then what you may need in order to pass an audit with flying colors is an assessment, a really good assessment done in a very informal way.
The History behind Informal Assessments
It all started when we agreed to help a large company get ISO 9001 certified under a very aggressive timeline. Since time was of essence, our team worked very hard to ensure that the development phase was done quite rapidly to allow plenty of time for implementation. Luckily, everyone in the organization was so committed that the Quality Management System (QMS) development phase went smoothly and the QMS was launched on schedule. This meant that the implementation time allowed remained as planned, actually a total of six weeks before the Internal Audit. Since we wanted to ensure success, we decided to try something new in these six weeks: a series of Informal Assessments.
What is an Informal Assessment?
An Informal Assessment is a term that Mireaux Management Solutions coined as a type of audit, because it differs significantly from the formalities normally seen on other Internal or External Audits. Time, opening/closing meetings and reporting are some of the key differentiators of these two. Let me explain in detail how it works.
Our approach is to use Informal Assessments as a way to train, prepare and test a management system before the Internal Audit. The Informal Assessment would be like an Audit on steroids, where basically time, presentation and reporting would not be constrained by formalities.
Let’s start with time. An Internal Audit usually follows the same number of days as the External Audit. So if your External Audit is 3 days, then the Internal Audit may also last 3 days. Therefore all the processes of the company would be audited within that time frame. In this fashion, the Internal Audit must be conducted expeditiously following the established agenda. You basically have to keep moving or you risk not meeting the schedule. Of course this is predominantly true if you outsource your Internal Audits, although even if you don’t outsource it, nobody likes to have an official internal audit that last more than it has to. An Informal Assessment on the contrary, allocates plenty of time for each of the processes and does not require an opening and closing meeting for the privilege few. So rather than spending 1 or 2 hours looking at a process, you are now spending 4 or more hours, concentrating on critical areas or processes that need most attention.
The Audit report, although really good for formal audits, presented certain constraints we wanted to remove. The Audit Report typically follows an established template and each and every nonconformity must be matched to a clause of the standard or specific requirement of the organization. In the Informal Assessment, we are not limited in our reporting and will not try to match each finding to an ISO clause. Therefore the report provides a detail account of everything found wrong or suspected wrong, whether the requirement was explicitly called out in a procedure or not.
So with all that in mind, we set ourselves to conduct our informal assessment.
Conducting the Informal Assessment
One of the first areas we decided to assess informally was the Team Dispatch process. Our methodology was simple: pick a recent job that was dispatched offshore and check whether all the people that went on that job had completed all the required training and had all their certifications current. We would also look at records and speak with the Training department as well as the Operations Department to see the level of understanding of the requirements.
And so we began our Informal Assessment early in the morning. When I arrived the Operations Manager kindly told me that he had already picked a job for me. I didn’t want to hurt his feelings so I decided to go ahead and go with his pick. I started looking at the job information in detail, including the name of the craft, the assignment date, the day the craft was dispatched, and the team requirements, which included a variety of job positions. I asked how the team was selected and asked to see the actual list of employees and contractors who went on that assignment.
I first verified one by one that they all had their certifications. It was a total of almost 30 people, but that did not stop me, because I was not in a hurry. We reviewed TIC cards, Safe Gulf, Offshore Permits, etc. What we found is that there were a lot of individuals in full compliance but there were also individuals with certain records missing or in some cases with expired certifications.
Then I asked to see all the training records for each and every one of the team members as well as the actual training requirements. Once we started going through the stack of records, it was very obvious here that our exercise was paying its dividends. After searching through all the team members, at least 5 of them had a lot of discrepancies between what was required and what the training records said. We also realized that one of the reasons some of the team members had not been trained was because reporting on who needed training was not issued ahead of the certification expiring. The appropriate managers or individuals themselves did not have visibility on the training status. Since we were dealing with offshore people who go may go offshore for several months, we realized that reporting in this context was of primary importance.
Finally we went through the pertinent dispatch procedures and read line by line each of them to make sure every single procedure was accurate. Along the way, I requested every single record that the procedure asked for, as applicable to the assignment I was assessing. Needless to say, we found some minor inconsistencies that revealed problems not just at the team dispatch level, but at other processes who acted as inputs or outputs to this process.
Another example of an Informal Assessment: Job Dispatch Area
Just like in our previous Informal Assessment we begin the assessment of the Job Dispatch area early in the morning, to ensure that time was not a constrain. We picked the same job we had had picked before and reviewed everything that had been requested and was required to go offshore. In fact, we found it useful on these assessments to keep following the same job. For example we were able to assess the Procurement area for all the supplies and rentals required for the job. We also looked at the craft maintenance records for the same job. So we found it easier to just get a hold of a good job that provided us with various areas from which to assess compliance.
In the Job Dispatch area, we followed the associated procedures line by line and found out that many of the requirements were not being followed. For example, as we read the procedure with a fine comb we realized that the procedure called for a least three signatures for each item loaded in the truck. This was a big issue because hot shots to bring missing items is expensive. We also reviewed all items from the Dispatch sheet for their actual location, to see whether everything looked in order. This was obviously a little hard because the job was a few weeks old, and perhaps some items had already come back and been re-issued. But nevertheless, we looked at the inventory, stockroom and storage areas, checked the inventory and asset codes and looked at how good things were arranged or stored.
We realized in some cases, that items in the stockroom or storage were not in the right place, mainly because the right place was mislabeled or items on the Dispatch sheets did not have the right asset number. Once again we looked item by item, and sifted through at least 50 items. We also discovered, that in some cases even though the Dispatch sheet showed that two items had left, actually more had left, in some cases double the amount, but nobody had gone back to update the files. Although the assignment was completed and everything was fine, discrepancies like this could eventually create a problem because the customer could be charged or not charged enough for the equipment sent out to the job.
Other issue found throughout the process was employees not filling out required fields on forms or filling them out partially. While that may look like a small item, some auditors have an issue with that because a field that is left blank means that either the item was not required, the item was not applicable or the person filling out the form overlooked the need. Obviously if the latter was the case, then you may have a problem.
Finally, we reviewed how everything was received offshore, whether all those items had also been checked off once they arrived to the craft. And to our surprise we found out that in some cases the people on board the craft had written the wrong transportation supplier or the wrong quantity, specially in those cases where there had been last minute changes. This showed that perhaps the person receiving the items had copied from the original documents rather than conducting a physical check.
The results
After the Informal Assessments for these areas were completed, a report was generated containing information on the job analyzed as well as detail recount of all that was found. The report was sent to all involved and a corrective action or CIP (as we call it in the Web QMS) was created to ensure that all issues would be appropriately resolved and also to start getting employees acquainted with root cause analysis.
It is clear that some of these nonconformities may not have been found had it been a simple audit where we would have had an hour or so to look at the whole process. In the Informal Assessment we had plenty of time and we were able to dig and dig and dig. If we would have sampled 3 crew members, as is the norm during an audit, we may have found 1 or no discrepancies at all, so this may have been a small issue or no issue at all. But in the context of a whole job, when you find 5 or more individuals with certification or training problems, regardless of the entire pool of people, you realize that now you may be dealing with a systemic problem. Likewise, if we would have taken a Dispatch sheet and looked at two or three items, most than likely everything would have been fine. In essence, this exercise revealed to us that there were indeed holes and still a lot to do to ensure full QMS compliance. When you are spending 4 or more hours in a single area following all the threads that come up, you are bound to uncover opportunities for improvement. Simply put, there just isn’t enough time during a regular audit to do this; hence we came up with the Informal Assessment concept.
Through the assessment also, we had discussed very informally with employees what an auditor may be looking for, how the records needed to be presented, what level of compliance was expected and how the issues were supposed to be taken care in the corrective action program. The most amazing outcome of these Informal Assessments is that it brought some of the employees and managers that had been standing in the sideline as well as those who did not get it, completely on track. Although some people were not very happy, we all agreed it was a positive way to find opportunities for improvement and an excellent way to get ready for the Internal or External Audit. Overall everyone knew that it was better to find things now rather than later especially because we still had plenty of time to get all issues resolved.
Next stop: Guaranteed Success
Once we had completed all the Informal Assessments, a few more weeks passed by before the Internal Audit was conducted. The Internal Audit uncovered additional nonconformities and a formal report was issued. After those issues were also taken care, we were ready for the External Audit at last, having covered all our bases with no room for failure.
Finally I had to report that our client went through their External Audit, lasting a total of 5 audit man days, ending with only one minor nonconformity and was recommended for ISO certification during the closing meeting. This was another success for Mireaux Management Solutions and highlights the importance of conducting Informal Assessments ahead of the Internal Audit to double and triple check that your Quality Management System is working well, to ensure and guarantee success and to accurately position your company on the path to world class quality.
