by Tim Alcock, Qualimetric
This presentation is designed to give an executive overview of the Quality Management System standards and guidelines applicable to organisations providing forensic science services to the Criminal Justice System in the UK.Organisations can achieve independent recognition to these standards via an accreditation process which confirms compliance. This is generally voluntary, but in the UK, whilst not yet mandated in law, the Forensic Science Regulator has required that Forensic units are accredited to relevant International standards (ie ISO/IEC 17025, 17020).
As the standards mentioned above were not specifically written for forensics applications, supplemental documents have also been published (such as ILAC G-19) and in the UK, the FSR Codes criteria of Practice and Conduct and associated documents as well as specific accreditation body requirements. This presentation provides an overview of the application and relationship between these documents.
Quality Assurance and the ‘process approach’
The principles of quality assurance have been well established over many years in a variety of industries and are based on a premise of controlling processes themselves in order to have confidence in their output and meet customer needs. These principles have clear application in forensics with the aim of ensuring that correct, complete and substantiated findings are presented in court.
Each stage in the process from initial investigation, evidence collection, submission, analysis, reporting, interpretation and presentation are clearly critical and each step can and will profoundly affect the next process in line.
Over the years, Quality Management Standards and guidelines have been developed which define good management principles aimed at providing confidence in the quality of the delivered product or service and form the basis for continual improvement and capture of best practices. One such document (ILAC G19), written specifically for forensics application maps out the ‘core processes’ involved in a forensic investigation as follows
• Initial discussion regarding scene of crime attendance
• Undertaking initial actions at the scene of crime
• Developing a scene of crime strategy
• Undertaking the scene of crime investigation
• Assessment of crime findings and consideration of further examination
• Interpretation and reporting of scene of crime findings
• Examination and testing
• Interpretation of results of examinations and tests
• Reporting of examination and tests including the interpretation of results.
Of course, organisations also need some support functions to ensure that these activities can be conducted, such as recruitment, training and competency assessment, test equipment, kits, software, supplies and consumables, IT support, transport, facilities…. and others such as technical information and methods. All of these processes will affect the ‘core’ business process to some degree and so requirements relating to these activities are also included in quality management standards.
Management also play a role in defining policies, strategies, support and monitoring the operation and requirements are also contained in management standards relating to the management.
Finally, it is accepted things will go wrong and as such there need to be robust processes for recording complaints, mistakes or instances, taking appropriate action and taking action to ensure that they will not reoccur. Risks need to be assessed and mitigated and actions taken to improve the operation of the organisation over time, known as continual improvement. These aspects are also covered by most quality management standards.
Here, we will discuss some of the standards involved.
ISO is The International Organisation for Standardisation which publishes agreed standards for a huge variety of subjects including materials, products and test methods. It also publishes standards relating to quality management systems.
The principle standards which have deemed to be applicable and mandated for use by the Forensic Science Regular are as follows:
• ISO/IEC 17020 – requirements for inspection bodies which is applied at crime scene
• ISO/IEC 17025 – requirements for calibration and test, which is applicable for laboratory applications (including some digital imaging)
• ISO/IEC 17025 covers both testing and supporting calibration services
These two standards are shown here overlayering the ILAC G19 core business process discussed earlier. ISO/IEC 17020 covers scene of crime (ie ‘inspection’ of crime scene’) and ISO/IEC 17025 covers the laboratory function. There is some overlap shown here to cover testing performed at the scene.
Because ISO/IEC 17020 and 17025 were not specifically written with forensic investigation in mind, additional documents have been published which provide additional requirements and interpretations, such as The FSR Codes of Practice and Conduct as well as additional FSR specific guidelines and Accreditation body (UKAS documents) such as RG201 for crime scene investigation
In addition, many organisations including some police forces use a standard called ISO 9001 which can be applied to any type of organisation which also define principles for quality assurance and improvement and covers general principles which overlap the above standards.
Looking at each of the main accreditation standards in more detail, first we will have a look at the structure and content of ISO/IEC 17020.
ISO/IEC 17020 and 17025 both require that the organisation is free from undue pressures, requiring an impartiality risk assessment and associated mitigations to be conducted. They also require maintenance of confidentiality in respect of the activities conducted by the organisation
The standard requires that the organisation be legally accountable and carries some kind of indemnity against claims, it requires that the organisational structure and responsibilities of personnel be clearly defined and that the organisation effectively implements a Management System (ie quality manual, Standard Operating instructions etc).
The following slides show the requirements for the control of the inspection process, in the case of forensics, this would start with a Service Level Agreement, then with the request for investigation, the performance of the investigation including recovery of evidence and generation of contemporaneous notes and then reporting.
Here we see requirements relating to the support functions, such as personnel, equipment, facilities and environment, subcontracting or outsourcing of work and control of documentation to ensure that correct and up to date information is available as well as the retention and storage of records.
Finally the standard covers processes for complaint handling, internal audits, nonconformance and corrective action, preventive action to identify potential problems or risks, user feedback and the periodic review of system effectiveness by the management.
Now we will look at ISO/IEC 17025. As with ISO/IEC 17020 we start with the customer. ISO/IEC 17025 contains similar requirements (although not the same!) relating to the management system, impartiality, confidentiality, organisation and responsibilities as per ISO/IEC 17020 as well as for management review.
Again, the core process starts with the Service Level Agreement and request for work, followed by method development, verification and validation, sampling, which is in the most part already covered by evidence recovery under ISO/IEC 17020, receipt, handling, storage and return/disposal of exhibits, recording of results and technical records, the estimation of measurement uncertainty, the assurance of quality of results or quality control and finally, reporting.
In terms of support activities the standard covers the provision of resources personnel, training and competency, facilities and environment including control of access, equipment maintenance and calibration which includes controls on consumables and reagents, metrological traceability of measurement systems and purchasing as well as outsourcing. IT systems, including support systems, Laboratory Information systems and data backup-recovery are covered here as well as control of documentation and records which has similar requirements to ISO/IEC 17020.
As per ISO/IEC 17020, the standard covers complaints, nonconforming work, internal audits and complaints as well as corrective action, risks and opportunities, customer feedback and improvement.
The following two graphics provide a brief overview of the Forensic Science Regulator Codes of Practice and Conduct. The FSR Codes of Practice and conduct apply to both Crime Scene and Laboratory.
This graphic shows the additional requirements listed in the FSR Codes relating to management aspects which overlap with the ISO standards and specify addition requirements. There cover Business Continuity and additional requirements relating to impartiality and confidentiality…. Non conformance and audits, process requirements such as contract review, ie Service Level Agreements, Requests and review of capability to perform the work, subcontracting, additional requirements relating to packaging and chemicals, complaint handling and….. document control and records.
This slide deals with the technical requirements of the FSR Codes, covering personnel recruitment, code of conduct, training and competence,…. Accommodation and environment, a very comprehensive section on method development, verification and validation… equipment and metrological traceablity, exhibit and case handling.. quality assurance and reporting.
It should noted that the FSR are currently under review and a new version is planned to be published in 2020.
There are also a number of supplemental guidance, codes and protocols published by the regulator which cover specific aspects of forensics, such as method validation in digital forensics, cell site analysis, bloodstain analysis etc some of which are listed here but are subject to continuous review and addition.
It can be seen from the above, there is a great deal of overlap between the various applicable documents and additional requirements specified in some which are not covered in others, so establishing systems and auditing against the requirements is complex.
We will now discuss how organisations gain recognition of their compliance with standards through certification and/or accreditation.
Accreditation is the independent evaluation of conformity assessment bodies (in this case Forensic Laboratories working to ISO/IEC 17025 and crime scene investigation units working to its ‘sister’ standard ISO/IEC 17020). In the UK, accreditation is performed by the United Kingdom Accreditation Service (UKAS). UKAS conduct assessments directly against ‘conformity assessment’ standards such as ISO/IEC 17025 and laboratories and inspection bodies apply directly to them for accreditation.
ISO 9001 (and other standards such as ISO 14001 for environmental management systems) are assessed by ‘Certification Bodies’ who are subject to accreditation themselves by UKAS. In this case the organisation applies to a Certification body of their choice who then assesses the applicant. We have shown here the certification mark of BSI, however there are many bodies accredited to provide certification, a few of which are listed here – a full list can be found on the UKAS website, a link to which is provided at the end of this presentation.
Laboratory accreditation generally differs from ‘Quality Management Certification/Registration’ to standards such as ISO 9001 in that the assessment teams employ subject matter experts who directly evaluate the practical aspects and technical operation of the laboratory which increases the scientific rigor of the assessment. Having said this, being accredited cannot guarantee that mistakes will not happen and there are well documented cases of errors and failures even in accredited facilities. It can however provide confidence that the laboratory operates an effective management system with rigorous requirements for ensuring competency, technical operation and reporting of results and be used as the basis for continuing improvement of the organisation’s management systems.
Accreditation bodies are normally themselves peer assessed through an international system administered by the International Laboratory Accreditation Cooperation (ILAC). Accreditation bodies are established in many countries with the primary purpose of ensuring that conformity assessment bodies are subject to oversight by an authoritative body.
In this section we will discuss briefly the steps involved in establishing a management system and some benefits and perceived roadblocks.
This slide shows some of the key steps in implementing a management system with the aim of gaining accreditation.
• Determination of scope for which accreditation is required
• Training in requirements of applicable standards/codes
• ‘Gap’ assessment of existing systems against requirements
• Create ‘action plan’ and assign responsibilities
• Determine and provide resources (equipment, systems, time)
• Establish quality policy & objectives (Key performance Indicators)
• Develop management system documentation
• Method validation studies and uncertainty assessments
• Train internal auditors
• Perform internal audits
• Apply for accreditation
• Conduct management reviews
This slide shows a draft implementation plan for the roll-out of ISO/IEC 17020 in a small business.
There are many benefits of establishing an effective quality management system and of gaining accreditation. These include:
• Assurance of and confidence in the quality of results and rigor of performance
• Reducing the likelihood of cases being challenged in court
• As a basis for capturing best practice and to improve processes
• To think processes through and ensure their effectiveness
• Management confidence
• Establishing rules for competence etc
• External recognition
Like anything there are also some potential downsides:
• Over-bureaucratic systems (increased paperwork, and form filling)
• Time and cost of development of management systems and writing procedures
• Time and cost of performance of controls (for example method and software validation, internal audits and reviews)
• Time and cost involved in accreditation and ongoing surveillance/reaccreditation
• Often perceived to reduce flexibility and creativity
It is evident from the above that a significant amount of time is typically required to be spent in establishing systems and procedures. Most organisations do not have the luxury of a dedicated team to develop such systems and so the burden of development work falls on the investigation and laboratory team members, typically in addition to their day to day work.
A lot of issues and time spent in writing SOP’s.. If processes are well defined, it is pretty easy to come up with a procedure – if it is not then it becomes much harder. It would be argued that it is precisely these areas where processes are not clear where the most benefit should be gained, so investment here should not be wasted!
I hear a lot of complaints that their woes are ‘because of ISO’. Whilst it is the case that some requirements may be somewhat burdensome and some do not have effective resolution yet (eg duplication of software validation particularly in DF) it is critical that the procedures are practical and implementable and rigorous but not inflexible. It is the organisation’s own procedures that will be assessed during the accreditation process. Of course, systems must ensure the rigorous performance and collection of data to support cases, but care must to bake to not create unnecessary bureaucracy and introducing constraints where they are not required. If procedures are over-specified and overdetailed, they are unlikely to be read and understood and thus counterproductive.
Systems should be set up to meet best practices in line with the target standards and not set out to please the accreditation body. it is your system!
To recap, this presentation has covered
• Principles of Quality Assurance and Management
• Applicable standards and their application
• ISO/IEC 17020
• ISO/IEC 17025
• ISO 9001 in passing
• ILAC G19
• FSR Codes of Practice and Conduct and support documentation
• Accreditation and Certification
• Implementation steps
• Pros and cons of a Quality Management System
We hope that you have found this presentation useful. The next slide provides some links to websites where further information can be found.
Finally, if we can assist further with training or advisory services or would like to give feedback on this presentation, please don’t hesitate to get in touch.
About The Author
Tim Alcock, CQP FCQI MASQ is an IRCA Registered Lead Auditor and Managing Director of Qualimetric Ltd. He has over 30 years’ experience in the application of quality management systems, specialising in Laboratory and Inspection Body accreditation. You can get in touch with Tim on Tim.alcock@qualimetric.co.ukTim.alcock@qualimetric.co.uk or visit Qualimetric’s website at www.qualimetric.co.uk.
