Hardware optimization for CPU usage
Hardware optimization enables Neural Machine Translation (NMT) Language Pairs to run on CPUs only.
Benefits
- Enables NMT state-of-the-art access to customers who do not have access to enterprise GPUs
- Lowers infrastructure costs
- Provides flexibility in deployment options
- Users can work with Language Weaver Edge without GPU and enable GPU once it becomes available
- Users can operate some Language Pairs on GPU, and some others on CPU, to optimize hardware usage and adjust system performance
Hardware optimization modes
- GPU mode (as before) - This mode is available only if a GPU is installed on the host.
- CPU mode optimized for Quality
- CPU mode optimized for Speed
Each NMT Translation Engine will be able to operate in one of the hardware optimization modes. Selection of operation mode is performed when the Translation Engine is created in Language Weaver Edge.
By default, a Language Pair (LP) will operate:
- In GPU Mode, if a GPU is detected on the host where the LP is installed
- In CPU mode, optimized for Quality, if no GPU is detected on the host where the LP is installed
Hardware optimization modes: users, function and requirements
| Mode | NMT 2.0 - GPU mode | NMT 2.0 - CPU mode optimized for Quality | NMT 2.0 - CPU mode optimized for Speed |
|---|---|---|---|
| Requirements |
|
|
|
| Quality | Best Quality | Best Quality - in this mode the quality is identical to the GPU mode | Great quality - in this mode, some LPs can experience a short drop in quality |
| Minimum hardware requirements per 1 PU (Processing Unit) | 1 CPU core 4 GB of RAM 2 GB of HD 1 GPU with 2 GB of GPU RAM | 2 CPU cores 4 GB of RAM 2 GB of HD The host processor must support at least AVX2 in order to reach the maximum throughput per PU. | 1 CPU core 4 GB of RAM 2 GB of HD |
The required CPU cores are assumed to be logical cores. On OS with Hyper-threading enabled, each physical CPU core will result in 2 logical CPU cores. If Hyper-threading is not enabled, the number of CPU cores are the number of physical CPU cores required.