Advanced Scheduling and Pod Affinity and Anti-affinity

    You are viewing documentation for a release that is no longer supported. The latest supported version of version 3 is [3.11]. For the most recent version 4, see

    Pod affinity and pod anti-affinity allow you to specify rules about how pods should be placed relative to other pods. The rules are defined using custom labels on nodes and label selectors specified in pods. Pod affinity/anti-affinity allows a pod to specify an affinity (or anti-affinity) towards a group of pods it can be placed with. The node does not have control over the placement.

    For example, using affinity rules, you could spread or pack pods within a service or relative to pods in other services. Anti-affinity rules allow you to prevent pods of a particular service from scheduling on the same nodes as pods of another service that are known to interfere with the performance of the pods of the first service. Or, you could spread the pods of a service across nodes or availability zones to reduce correlated failures.

    Pod affinity/anti-affinity allows you to constrain which nodes your pod is eligible to be scheduled on based on the labels on other pods. A is a key/value pair.

    • Pod affinity can tell the scheduler to locate a new pod on the same node as other pods if the label selector on the new pod matches the label on the current pod.

    • Pod anti-affinity can prevent the scheduler from locating a new pod on the same node as pods with the same labels if the label selector on the new pod matches the label on the current pod.

    There are two types of pod affinity rules: required and preferred.

    Required rules must be met before a pod can be scheduled on a node. Preferred rules specify that, if the rule is met, the scheduler tries to enforce the rules, but does not guarantee enforcement.

    You configure pod affinity/anti-affinity through the pod specification files. You can specify a required rule, a , or both. If you specify both, the node must first meet the required rule, then attempts to meet the preferred rule.

    The following example shows a pod specification configured for pod affinity and anti-affinity.

    In this example, the pod affinity rule indicates that the pod can schedule onto a node only if that node has at least one already-running pod with a label that has the key and value S1. The pod anti-affinity rule says that the pod prefers to not schedule onto a node if that node is already running a pod with label having key security and value S2.

    Sample pod config file with pod affinity

    Sample pod config file with pod anti-affinity

    1. apiVersion: v1
    2. kind: Pod
    3. metadata:
    4.   name: with-pod-antiaffinity
    5. spec:
    6.   affinity:
    7.     podAntiAffinity: (1)
    8.       preferredDuringSchedulingIgnoredDuringExecution: (2)
    9.       - weight: 100  (3)
    10.         podAffinityTerm:
    11.           labelSelector:
    12.             matchExpressions:
    13.             - key: security (4)
    14.               operator: In (5)
    15.               values:
    16.               - S2
    17.           topologyKey: kubernetes.io/hostname
    18.   containers:
    19.   - name: with-pod-affinity
    20.     image: docker.io/ocpqe/hello-pod

    1. Create a pod with a specific label in the pod specification:

      1. $ cat team4.yaml
      2. apiVersion: v1
      3. kind: Pod
      4. metadata:
      5.   name: security-s1
      6.   labels:
      7.     security: S1
      8. spec:
      9.   containers:
      10.   - name: security-s1
      11.     image: docker.io/ocpqe/hello-pod

    2. When creating other pods, edit the pod specification as follows:

      1. Use the podAffinity stanza to configure the requiredDuringSchedulingIgnoredDuringExecution parameter or preferredDuringSchedulingIgnoredDuringExecution parameter:

      2. Specify the key and value that must be met. If you want the new pod to be scheduled with the other pod, use the same key and value parameters as the label on the first pod.

        1.     podAffinity:
        2.       requiredDuringSchedulingIgnoredDuringExecution:
        3.       - labelSelector:
        4.           matchExpressions:
        5.           - key: security
        6.             operator: In
        7.             values:
        8.             - S1
        9.         topologyKey: failure-domain.beta.kubernetes.io/zone

      3. Specify an operator. The operator can be , NotIn, Exists, or DoesNotExist. For example, use the operator In to require the label to be in the node.

    3. Create the pod.

      1. $ oc create -f <pod-spec>.yaml

    The following steps demonstrate a simple two-pod configuration that creates pod with a label and a pod that uses an anti-affinity preferred rule to attempt to prevent scheduling with that pod.

    1. Create a pod with a specific label in the pod specification:

    2. When creating other pods, edit the pod specification to set the following parameters:

    3. Use the podAffinity stanza to configure the requiredDuringSchedulingIgnoredDuringExecution parameter or preferredDuringSchedulingIgnoredDuringExecution parameter:

      1. Specify a weight for the node, 1-100. The node that with highest weight is preferred.

      2. Specify the key and values that must be met. If you want the new pod to not be scheduled with the other pod, use the same key and value parameters as the label on the first pod.

        1.     podAntiAffinity:
        2.       preferredDuringSchedulingIgnoredDuringExecution:
        3.       - weight: 100
        4.         podAffinityTerm:
        5.           labelSelector:
        6.             matchExpressions:
        7.             - key: security
        8.               operator: In
        9.               values:
        10.               - S2
        11.           topologyKey: kubernetes.io/hostname

      3. For a preferred rule, specify a weight, 1-100.

    4. Specify a topologyKey, which is a prepopulated Kubernetes label that the system uses to denote such a topology domain.

    5. Create the pod.

      1. $ oc create -f <pod-spec>.yaml

    The following examples demonstrate pod affinity and pod anti-affinity.

    The following example demonstrates pod affinity for pods with matching labels and label selectors.

    • The pod team4 has the label team:4.

      1. $ cat team4.yaml
      2. apiVersion: v1
      3. kind: Pod
      4. metadata:
      5.   name: team4
      6.   labels:
      7.      team: "4"
      8. spec:
      9.   containers:
      10.   - name: ocp
      11.     image: docker.io/ocpqe/hello-pod

    • The pod team4a has the label selector team:4 under podAffinity.

      1. $ cat pod-team4a.yaml
      2. apiVersion: v1
      3. kind: Pod
      4. metadata:
      5.   name: team4a
      6. spec:
      7.   affinity:
      8.     podAffinity:
      9.       requiredDuringSchedulingIgnoredDuringExecution:
      10.       - labelSelector:
      11.           matchExpressions:
      12.           - key: team
      13.             operator: In
      14.             values:
      15.             - "4"
      17.   containers:
      18.   - name: pod-affinity
      19.     image: docker.io/ocpqe/hello-pod

    The following example demonstrates pod anti-affinity for pods with matching labels and label selectors.

    • The pod pod-s1 has the label security:s1.

    • The pod pod-s2 has the label selector security:s1 under podAntiAffinity.

      1. cat pod-s2.yaml
      2. apiVersion: v1
      3. kind: Pod
      4. metadata:
      5.   name: pod-s2
      6. spec:
      7.   affinity:
      8.     podAntiAffinity:
      9.       requiredDuringSchedulingIgnoredDuringExecution:
      10.       - labelSelector:
      11.           matchExpressions:
      12.           - key: security
      13.             operator: In
      14.             values:
      15.             - s1
      16.         topologyKey: kubernetes.io/hostname
      17.   containers:
      18.   - name: pod-antiaffinity
      19.     image: docker.io/ocpqe/hello-pod

    • The pod pod-s2 cannot be scheduled on the same node as pod-s1.

    The following example demonstrates pod affinity for pods without matching labels and label selectors.

    • The pod pod-s1 has the label security:s1.

      1. $ cat pod-s1.yaml
      2. apiVersion: v1
      3. kind: Pod
      4. metadata:
      5.   name: pod-s1
      6.   labels:
      7.     security: s1
      8. spec:
      9.   containers:
      10.   - name: ocp
      11.     image: docker.io/ocpqe/hello-pod

    • The pod pod-s2 has the label selector security:s2.

      1. $ cat pod-s2.yaml
      2. apiVersion: v1
      3. kind: Pod
      4. metadata:
      5.   name: pod-s2
      6. spec:
      7.   affinity:
      8.     podAffinity:
      9.       requiredDuringSchedulingIgnoredDuringExecution:
      10.       - labelSelector:
      11.           matchExpressions:
      12.           - key: security
      13.             operator: In
      14.             values:
      15.             - s2
      16.         topologyKey: kubernetes.io/hostname
      17.   containers:
      18.   - name: pod-affinity
      19.     image: docker.io/ocpqe/hello-pod
      1. pod-s2    0/1       Pending   0          32s       <none>