Documentation Center

Pod scheduling behavior

Procedure

  • Turning the GPU flag on in configuration will perform the following changes in the Kubernetes deployment descriptors for the engine pods:
    spec:
        tolerations:
          ...
          - effect: NoSchedule
            key: nvidia.com/gpu
            operator: Exists
    spec:
      containers:
        resources:
          limits:
            ...
            nvidia.com/gpu: "1"
    If this default configuration is not desired, it can be changed. Please see Advanced GPU configuration for more information.
  • The Kubernetes nodes providing GPUs are expected to be configured like this:
    spec:
      ...
      taints:
      - effect: NoSchedule
        key: nvidia.com/gpu
        value: present
    For hosted Kubernetes clusters, this is usually the default configuration.

Results

  • If an engine (pod) requests GPU resources, the engine will only be scheduled on GPU Kubernetes nodes.
  • Pods that do not request GPU will not be allowed on Kubernetes GPU nodes.