Documentation Center

Hardware optimization for CPU usage

Hardware optimization enables Neural Machine Translation (NMT) Language Pairs to run on CPUs only.

Benefits

  • Enables NMT state-of-the-art access to customers who do not have access to enterprise GPUs
  • Lowers infrastructure costs
  • Provides flexibility in deployment options
    • Users can work with Language Weaver Edge without GPU and enable GPU once it becomes available
    • Users can operate some Language Pairs on GPU, and some others on CPU, to optimize hardware usage and adjust system performance

Hardware optimization modes

Hardware optimization supports 3 modes of operation for NMT Language Pairs:
  • GPU mode (as before) - This mode is available only if a GPU is installed on the host.
  • CPU mode optimized for Quality
  • CPU mode optimized for Speed

Each NMT Translation Engine will be able to operate in one of the hardware optimization modes. Selection of operation mode is performed when the Translation Engine is created in Language Weaver Edge.

By default, a Language Pair (LP) will operate:

  • In GPU Mode, if a GPU is detected on the host where the LP is installed
  • In CPU mode, optimized for Quality, if no GPU is detected on the host where the LP is installed

Hardware optimization modes: users, function and requirements

ModeNMT 2.0 - GPU modeNMT 2.0 - CPU mode optimized for QualityNMT 2.0 - CPU mode optimized for Speed
Requirements
  • GPU is required
  • Best translation quality available
  • Optimal performance and speed
  • GPU is NOT required
  • Best translation quality available
  • Speed is NOT critical
  • GPU is NOT required
  • High volumes of translatable content when time is critical
  • Short or temporary trade-off on quality to favor translation speed is acceptable
QualityBest QualityBest Quality - in this mode the quality is identical to the GPU modeGreat quality - in this mode, some LPs can experience a short drop in quality
Minimum hardware requirements per 1 PU

(Processing Unit)

1 CPU core

4 GB of RAM

2 GB of HD

1 GPU with 2 GB of GPU RAM

2 CPU cores

4 GB of RAM

2 GB of HD

The host processor must support at least AVX2 in order to reach the maximum throughput per PU.

1 CPU core

4 GB of RAM

2 GB of HD

The required CPU cores are assumed to be logical cores. On OS with Hyper-threading enabled, each physical CPU core will result in 2 logical CPU cores. If Hyper-threading is not enabled, the number of CPU cores are the number of physical CPU cores required.