Using GPUs
This section describes how to use GPUs in a Language Weaver Edge deployment on Kubernetes.
Translation engines and training engines can optionally be deployed on Kubernetes nodes with available GPUs. If an Nvidia GPU is detected, it will be used automatically.
By default, no pods are scheduled on Kubernetes GPU nodes. Translation engines and training engines need to be specifically configured to run on GPU nodes, if required. This is done by turning a flag on. Please see GPU configuration for more information.
For translation engines, the cost of deploying on GPU nodes tends to be comparable with deploying on non-GPU nodes. The additional complexity may not be warranted.
For training engines, GPU deployment offers significant speed gains versus CPU deployment (up to 10x faster). If the language pair takes an unacceptable long time to train, GPU deployment should be taken into consideration.