Value of DQF - Business Intelligence at the Macro Level

As the industry is getting closer and closer to adopting the TAUS Dynamic Quality Framework (DQF) as a standard for evaluating translation productivity and quality, we’ve decided to talk to the heads of the translation department at Dell-EMC about their experience with using DQF. They are a long-time supporter of the vision behind TAUS DQF and, since July 2017, an active user of the integration with GlobalLink (provided by Translations.com).

 

Dell-EMC produces high volumes of translated content – more than 200 million words in more than 34 languages and dialects annually using four different vendors. Since July 2017, 78 million words have run through the DQF integration, with plans to have all volume flowing through the integration within the next year. In addition to that, they work with different content streams and differentiate between several content types, most of which they apply machine translation on. Out of the total generated translation volume that runs through GlobalLink, ~75% of the production data is currently being pushed through DQF and analyzed on the DQF Dashboard.

Here is why:

1. Complex production processes ask for complex measurements

The realization that the LISA QA model they were using was outdated and didn’t take into consideration the differences in content type, purpose of content or its audience, prompted Dell-EMC to seek another solution. Their production reality asked for a multi-faceted evaluation method - one that would not treat all content the same or be mainly focused on the linguistics. And they needed this solution to also track data on translation productivity and edit-distance so that they could evaluate efficiency of their machine translation engine output - at scale, in real-time.

2. Need for alignment and industry benchmarking

The capability of DQF to account for various content types, thanks to its range of advanced evaluation metrics, fulfilled the need for a more dynamic framework. At the same time, an even bigger driver for adoption of DQF was the need for alignment - both internal and external. The importance of sharing the same view on quality and the same metrics across the supply chain became apparent with the merger of Dell and EMC, as the teams from the two companies were brought together. Moreover, DQF offered the ability to benchmark across the industry, speaking to Dell-EMC’s desire to share a single, industry-wide view of how quality is being defined and measured.

3. Source of business intelligence

Currently, Dell-EMC mainly focuses on the productivity and benchmarking data from DQF.

Based on the productivity data that they have analyzed so far, they were able to perform vendor feedback validation on the perceived quality of the machine translation output. In a situation where a vendor raises a question about a seemingly low output quality, they can resort to DQF data and use it to analyze and validate their feedback and avoid arguments: “You need to look at numbers and not impressions”, explains Georg Kirchner. With the numbers from DQF benchmark reports, they can even go a step further as to establishing a high-level productivity degree per language pair.

Next to the automatically produced, high-level reports, DQF allows one to download the actual content that was submitted for analysis. These downloadable reports include the text from before and after the post-editing and help obtain detailed insights that one otherwise wouldn’t have. For Dell-EMC, this possibility enabled them to identify a bug in their MT framework and resolve it timely.

Self-evident choice

DQF came as the first choice for a couple of additional reasons: a) Dell-EMC have been a long-term member of TAUS and have participated in the discussions that led to establishing and shaping the Dynamic Quality Framework, so they were familiar with the vision and the mechanics behind it; b) the potential alternatives did not cut the mustard - the options were to either stick to what they had before or build their own solution that wouldn’t tick all the boxes as the industry standard. As Georg Kirchner pointed out: "The only alternative you would have is to compare yourself against yourself, and that is not terribly interesting."

Conclusion: Industry-level Value

From the Dell-EMC perspective as a buyer, moving to DQF was a great fit for their purpose. The implementation effort is not big and they have the buy-in from all of their vendors, who also see the value in accessing the data. The democratic nature of the model, allowing everyone to see the same data, means that the conversation around pricing becomes less painful, more data-informed and fairer all around.

So what kind of value does DQF add for those who are a part of the supply chain? As Wayne Bourland explains, the value is equally shared across the industry - everyone benefits from understanding what the output quality is, the ability to measure and take action on it: “The ROI for DQF is less about direct savings from internal standardization, while still important – moving away from spreadsheets, manual entry, building homegrown criteria, reporting, and tools, etc. – and more about the value at a macro industry level; aligning the end-to-end supply chain on a single standard and way of talking about and measuring quality, more closely aligning to customer expectations, and differentiating quality based on adequacy to task, plus the value/insights derived from benchmarking.”