The newly-published ISO standard 5060:2024 Translation services – Evaluation of translation output provides general guidance on the evaluation of output from human translation and post-edited or unedited machine translation.
This blog outlines what guidance ISO 5060 includes, how it is relevant to language service providers and their existing ISO certifications, and what other current initiatives and resources are available in the translation quality evaluation space.
What does ISO 5060 include?
ISO 5060 focuses on an analytic translation evaluation approach – a segment-based comparison of target language content against the source, using an error typology framework with error types and penalty points, to produce an objective error score and quality rating for translation output.
ISO 5060 is a guidance standard, which means that it includes recommendations, not requirements – and as a guidance standard, you cannot get certified to it.
In line with other language services standards, ISO 5060 incorporates professional competences and qualifications for evaluators.
At the heart of the standard is the implementation of an evaluation system, with an error typology framework with weighting and severity levels for:
- Terminology
- Accuracy
- Linguistic conventions
- Style
- Locale conventions
- Audience appropriateness
- Design and markup
The standard includes guidance on the evaluation phase, basic evaluation principles, translation evaluation scorecard, and quality ratings. The standard also includes recommendations for a post-evaluation phase including feedback and dispute resolution.
You can read more about the 5060 standard and purchase a copy through the ISO or BSI. ATC Member Companies can access a read-only version of the standard via the ATC ISO Library in early May 2024.
Why is ISO 5060 relevant for LSPs?
Although ISO 5060 is a guidance, not a requirement standard, it holds significance to quality-orientated language service providers across the world.
ISO 5060 is the first published international standard on the evaluation of translation output, and as such it cements best practices from widely-established and used quality frameworks such as the Multidimensional Quality Metric MQM. It also provides savvy LSPs a way to demonstrate compliance with ISO standards such as ISO 17100 for translation services and ISO 9001 for quality management.
So how could ISO 5060 support LSPs with their processes and in their compliance with requirements standards such as ISO 17100 and 9001?
ISO 17100 for translation services includes requirements for translation service providers to have processes in place to verify translator and reviser competences and for how the translation process is carried out. These are difficult areas to verify without a feedback system or a quality evaluation process, which ISO 5060 can now easily provide.
Likewise, ISO 9001 for quality management provides requirements for an organisation to determine and provide necessary resources to ensure valid and reliable results when monitoring or measuring the conformity of products and services to requirements. Here, ISO 5060 could provide a mechanism for monitoring and measuring the conformity of the translation output – the product. And like ISO 17100, ISO 9001 also includes requirements for demonstrating the competence of persons doing work under the organisation’s control.
What’s going on in quality evaluation otherwise?
Before ISO 5060, there were Multidimensional Quality Metrics – a framework for translation quality evaluation developed by the leading quality experts in language services as a best practice framework, integrating an error typology system capable of handling diverse translation requirements.
In tandem with ISO 5060, there is the ASTM WK46396 – a draft work item developed by the US-based standardisation organisation ASTM, under the title New Practice for Analytic Evaluation of Translation Quality. This new standard will focus on metrics for translation products but also for translation processes and projects.
In the language services industry, standards and standardisation work continue to gain significance as a mechanism for LSPs to develop their business processes and demonstrate their compliance to international best practices, and for the industry as a whole to be seen to endorse and develop international standards that set a benchmark for quality.
You can read more about ISO standards and standards development on the EUATC ISO standards page, and get certified through ATC Certification.
