Implementation of Machine Translation: What to Consider?

Machine translation (MT) technology has been around for seven decades now. It is praised for its speed and cost-effectiveness, and its quality has gotten a lot better too since the arrival of neural machine translation (NMT). Higher throughput, quicker turnaround time, and the need to reduce overall cost are the main reasons for implementing MT in almost every case. Sounds great, right? Still, to understand how to implement machine translation to meet your translation needs, you should first consider a few factors.

1. Use Case

Today, almost everyone in the industry uses machine translation to boost productivity: from individual language professionals to small and medium LSPs to enterprises. You can trial MT in different business scenarios, but here are some that maximize the potential of this technology:

• When there is no Translation Memory (TM) available to give suggestions based on the previously translated texts, having some input rather than no input at all will have an impact on translation speed, so you can pre-translate the content with MT instead and have translators post-edit it to reach the desired quality.
• For translating large amounts of content that you otherwise wouldn’t be able to. Think of user-generated content or customer support tickets in all different languages. This kind of content is often not a priority for translations, but having this information available in the customer’s native language undoubtedly adds value. You can use MT to fill in the gap until the content gets enough ‘hits’ to be prioritized for translation.
• When the main requirement for translation is understanding text in foreign languages rather than publishing. You might be adamant to run MT on highly visible content, but “gisting” documents to gain an idea of their content is a perfect case for MT.

2. Content Type, its Purpose and Quality Requirements

Not every type of content is equally suitable for MT. Start with a list of all the different types of content that you or your organization handles, including the types of content that have never been translated before.

For each type of content try to assess the following factors:

• Utility: Is the content primarily meant to convey instruction, a task, or a response?
• Time or speed: How fast does the content need to be delivered?
• Sentiment: Is the content meant to emotionally engage the reader about a company or product?

If utility and time are more important than sentiment, then these content types may be more suitable for MT.

Once you are clear on whether your quality requirements are based on utility, time and/or sentiment you can move on to defining the exact quality requirements. TAUS Dynamic Quality Framework (DQF) provides extensive guidance and ready-to-use templates covering various evaluation approaches to help you specify and measure quality.

3. Source Content Suitability

It is also necessary to evaluate whether the source content is a good candidate for MT. The general rule is that the source texts that are hard for people to translate are also likely to yield poor MT output. 

Other points to consider when reviewing the source include:

• Is the source specific to a certain domain? The more the source domain matches the training data used to create the translation engine, the better the MT output. Source texts that cover broad subject matters should be avoided. 

• Does source text contain consistent linguistic features such as vocabulary, syntax, grammar, and style? Less variation in the source generates more consistent machine translation results.

Furthermore, it is essential to check that:

• there are no spelling mistakes;
• punctuation is correct;
• initial capitals are applied correctly;
• And sentences are not unnecessarily long.

4. Language Pairs

The performance of machine translation systems varies greatly depending on the source and target languages involved. Users can expect better performance for similar languages (e.g. translating between the majority of European languages), rather than for languages with high disparity (e.g. English and Asian languages).

Different machine translation systems will perform differently on distinct language pairs and domains. It is important to measure and compare engines in a methodological way.

5. Evaluation of Machine Translation Results

To run a successful MT program (or even just a test), it is essential to have an objective methodology for evaluating and measuring the engines’ performance. There are different metrics to choose from, all of which assign quality scores so they correlate with the human judgment of quality. The main distinction is made between automatic and human evaluation.

Automatic evaluation metrics are used during the development of MT systems to measure the improvement or compare different MT systems, and they all depend on the availability of human reference translations. The most well-known metrics such as BLEU, NIST, METEOR, and TER evaluate the output of MT systems by comparing it to the reference translation. While human evaluation is considered costly and time-consuming, it provides much richer insights and data than automatic evaluation, such as the details on types of errors found in MT output, their severity and more. TAUS DQF offers a real-time solution for MT evaluation.

6. Total Cost of Ownership and Return on Investment

If you want to accurately calculate your return on investment (ROI) you must first be clear on the realistic total cost of ownership by looking at the cost components. Think beyond the initial outlay or cost of Software-as-a-Service and consider other factors, such as:

• cost of training;
• adjusting your workflow;
• additional vendor/client and internal communications;
• potential impacts of improving machine translation quality over time;
• impact of a shift in cost from translation/editing to review.