Controlling Pod Placement

    The Pod Node Constraints admission controller ensures that pods are deployed onto only specified node hosts using labels and prevents users without a specific role from using the field to schedule pods.

    Use the Pod Node Constraints admission controller to ensure a pod is deployed onto only a specified node host by assigning it a label and specifying this in the nodeName setting in a pod configuration.

    1. Ensure you have the desired labels (see Updating Labels on Nodes for details) and set up in your environment.

      For example, make sure that your pod configuration features the nodeName value indicating the desired label:

    2. Modify the master configuration file, /etc/origin/master/master-config.yaml, to add PodNodeConstraints to the admissionConfig section:

      1. ...
      2. admissionConfig:
      3.   pluginConfig:
      4.     PodNodeConstraints:
      5.       configuration:
      6.         apiversion: v1
      7.         kind: PodNodeConstraintsConfig
      8. ...

    3. Restart OKD for the changes to take effect.

      1. # master-restart api
      2. # master-restart controllers

    Using node selectors, you can ensure that pods are only placed onto nodes with specific labels. As a cluster administrator, you can use the Pod Node Constraints admission controller to set a policy that prevents users without the pods/binding permission from using node selectors to schedule pods.

    The nodeSelectorLabelBlacklist field of a master configuration file gives you control over the labels that certain roles can specify in a pod configuration’s nodeSelector field. Users, service accounts, and groups that have the pods/binding permission can specify any node selector. Those without the pods/binding permission are prohibited from setting a nodeSelector for any label that appears in nodeSelectorLabelBlacklist.

    For example, an OKD cluster might consist of five data centers spread across two regions. In the U.S., us-east, us-central, and us-west; and in the Asia-Pacific region (APAC), apac-east and apac-west. Each node in each geographical region is labeled accordingly. For example, region: us-east.

    As a cluster administrator, you can create an infrastructure where application developers should be deploying pods only onto the nodes closest to their geographical location. You can create a node selector, grouping the U.S. data centers into superregion: us and the APAC data centers into superregion: apac.

    1. Ensure you have the desired labels (see Updating Labels on Nodes for details) and set up in your environment.

      For example, make sure that your pod configuration features the nodeSelector value indicating the desired label:

      1. apiVersion: v1
      2. kind: Pod
      3. spec:
      4.   nodeSelector:
      5.     <key>: <value>
      6. ...

    2. Modify the master configuration file, /etc/origin/master/master-config.yaml, to add nodeSelectorLabelBlacklist to the section with the labels that are assigned to the node hosts you want to deny pod placement:

      1. ...
      2. admissionConfig:
      3.   pluginConfig:
      4.     PodNodeConstraints:
      5.       configuration:
      6.         apiversion: v1
      7.         kind: PodNodeConstraintsConfig
      8.           - kubernetes.io/hostname
      9.           - <label>
      10. ...

    3. Restart OKD for the changes to take effect.

    The Pod Node Selector admission controller allows you to force pods onto nodes associated with a specific project and prevent pods from being scheduled in those nodes.

    The Pod Node Selector admission controller determines where a pod can be placed using labels on projects and node selectors specified in pods. A new pod will be placed on a node associated with a project only if the node selectors in the pod match the labels in the project.

    After the pod is created, the node selectors are merged into the pod so that the pod specification includes the labels originally included in the specification and any new labels from the node selectors. The example below illustrates the merging effect.

    The Pod Node Selector admission controller also allows you to create a list of labels that are permitted in a specific project. This list acts as a whitelist that lets developers know what labels are acceptable to use in a project and gives administrators greater control over labeling in a cluster.

    To activate the Pod Node Selector admission controller:

    1. Configure the Pod Node Selector admission controller and whitelist, using one of the following methods:

      • Add the following to the master configuration file, /etc/origin/master/master-config.yaml:

        1. admissionConfig:
        2.   pluginConfig:
        3.     PodNodeSelector:
        4.       configuration:
        5.         podNodeSelectorPluginConfig: (1)
        6.           clusterDefaultNodeSelector: "k3=v3" (2)
        7.           ns1: region=west,env=test,infra=fedora,os=fedora (3)
        1Adds the Pod Node Selector admission controller plug-in.
        2Creates default labels for all nodes.
        3Creates a whitelist of permitted labels in the specified project. Here, the project is ns1 and the labels are the key=value pairs that follow.
        1. podNodeSelectorPluginConfig:
        2.     clusterDefaultNodeSelector: "k3=v3"
        3.      ns1: region=west,env=test,infra=fedora,os=fedora

        Then, reference the file in the master configuration:

        1. admissionConfig:
        2.   pluginConfig:
        3.     PodNodeSelector:
        4.       location: <path-to-file>

    1. Restart OKD for the changes to take effect.

      1. # master-restart api
      2. # master-restart controllers

    2. Create a project object that includes the scheduler.alpha.kubernetes.io/node-selector annotation and labels.

      1Annotation to create the labels to match the project label selector. Here, the key/value labels are env=test and infra=fedora.

    3. Create a pod specification that includes the labels in the node selector, for example:

      2. kind: Pod
      3. metadata:
      4.   labels:
      5.     name: hello-pod
      6.   name: hello-pod
      7.   containers:
      8.     - image: "docker.io/ocpqe/hello-pod:latest"
      9.       imagePullPolicy: IfNotPresent
      10.       name: hello-pod
      11.       ports:
      12.         - containerPort: 8080
      13.           protocol: TCP
      14.       resources: {}
      15.       securityContext:
      16.         capabilities: {}
      17.         privileged: false
      18.       terminationMessagePath: /dev/termination-log
      19.   dnsPolicy: ClusterFirst
      20.   restartPolicy: Always
      21.   nodeSelector: (1)
      22.     env: test
      23.     os: fedora
      24.   serviceAccount: ""
      25. status: {}
      1Node selectors to match project labels.

    4. Create the pod in the project:

      1. # oc create -f pod.yaml --namespace=ns1

    5. Check that the node selector labels were added to the pod configuration:

      1. get pod pod1 --namespace=ns1 -o json
      3. nodeSelector": {
      4.  "env": "test",
      5.  "infra": "fedora",
      6.  "os": "fedora"
      7. }

      The node selectors are merged into the pod and the pod should be scheduled in the appropriate project.

    If you create a pod with a label that is not specified in the project specification, the pod is not scheduled on the node.

    For example, here the label env: production is not in any project specification:

    1. nodeSelector:
    2.  "env: production"