Estimating the number of pods your OKD nodes can hold

The cluster capacity tool simulates a sequence of scheduling decisions to determine how many instances of an input pod can be scheduled on the cluster before it is exhausted of resources to provide a more accurate estimation.

You can run the cluster capacity analysis tool as a stand-alone utility from the command line, or as a job in a pod inside an OKD cluster. Running it as job inside of a pod enables you to run it multiple times without intervention.

You can run the OKD cluster capacity tool from the command line to estimate the number of pods that can be scheduled onto your cluster.

Prerequisites

• Run the OpenShift Cluster Capacity Tool, which is available as a container image from the Red Hat Ecosystem Catalog.

• Create a sample spec file, which the tool uses for estimating resource usage. The podspec specifies its resource requirements as limits or requests. The cluster capacity tool takes the pod’s resource requirements into account for its estimation analysis.

  An example of the Pod spec input is:

Procedure

1. From the terminal, log in to the Red Hat Registry:

   $ podman login registry.redhat.io

2. Pull the cluster capacity tool image:

   $ podman pull registry.redhat.io/openshift4/ose-cluster-capacity

3. Run the cluster capacity tool:

   $ podman run -v $HOME/.kube:/kube:Z -v $(pwd):/cc:Z  ose-cluster-capacity \
   /bin/cluster-capacity --kubeconfig /kube/config --podspec /cc/pod-spec.yaml \
   --verbose (1)

   Example output

   small-pod pod requirements:
       - CPU: 150m
       - Memory: 100Mi
   The cluster can schedule 88 instance(s) of the pod small-pod.
   Termination reason: Unschedulable: 0/5 nodes are available: 2 Insufficient cpu,
   3 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't
   tolerate.
   Pod distribution among nodes:
   small-pod
       - 192.168.124.214: 45 instance(s)
       - 192.168.124.120: 43 instance(s)

   In the above example, the number of estimated pods that can be scheduled onto the cluster is 88.

Running the cluster capacity tool as a job inside of a pod has the advantage of being able to be run multiple times without needing user intervention. Running the cluster capacity tool as a job involves using a ConfigMap object.

Prerequisites

Download and install the cluster capacity tool.

Procedure

To run the cluster capacity tool:

1. Example output

   kind: ClusterRole
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     name: cluster-capacity-role
   rules:
   - apiGroups: [""]
     resources: ["pods", "nodes", "persistentvolumeclaims", "persistentvolumes", "services", "replicationcontrollers"]
     verbs: ["get", "watch", "list"]
   - apiGroups: ["apps"]
     resources: ["replicasets", "statefulsets"]
     verbs: ["get", "watch", "list"]
   - apiGroups: ["policy"]
     verbs: ["get", "watch", "list"]
   - apiGroups: ["storage.k8s.io"]
     verbs: ["get", "watch", "list"]
   EOF

2. Create the service account:

   $ oc create sa cluster-capacity-sa

3. Add the role to the service account:

   $ oc adm policy add-cluster-role-to-user cluster-capacity-role \
       system:serviceaccount:default:cluster-capacity-sa

4. Define and create the Pod spec:

   apiVersion: v1
   kind: Pod
   metadata:
     name: small-pod
     labels:
       app: guestbook
       tier: frontend
   spec:
     containers:
     - name: php-redis
       image: gcr.io/google-samples/gb-frontend:v4
       imagePullPolicy: Always
       resources:
         limits:
           cpu: 150m
           memory: 100Mi
         requests:
           cpu: 150m
           memory: 100Mi

5. The cluster capacity analysis is mounted in a volume using a ConfigMap object named cluster-capacity-configmap to mount input pod spec file pod.yaml into a volume test-volume at the path /test-pod.

   If you haven’t created a ConfigMap object, create one before creating the job:

6. Create the job using the below example of a job specification file:

   apiVersion: batch/v1
   kind: Job
   metadata:
     name: cluster-capacity-job
   spec:
     parallelism: 1
     completions: 1
     template:
       spec:
           containers:
           - name: cluster-capacity
             image: openshift/origin-cluster-capacity
             imagePullPolicy: "Always"
             volumeMounts:
             - mountPath: /test-pod
               name: test-volume
             env:
             - name: CC_INCLUSTER (1)
               value: "true"
             command:
             - "/bin/sh"
             - "-ec"
             - |
               /bin/cluster-capacity --podspec=/test-pod/pod.yaml --verbose
           restartPolicy: "Never"
           serviceAccountName: cluster-capacity-sa
           volumes:
           - name: test-volume
             configMap:
               name: cluster-capacity-configmap

7. Run the cluster capacity image as a job in a pod:

   $ oc create -f cluster-capacity-job.yaml

8. Check the job logs to find the number of pods that can be scheduled in the cluster:

   $ oc logs jobs/cluster-capacity-job

   Example output

   small-pod pod requirements:
           - CPU: 150m
           - Memory: 100Mi
   The cluster can schedule 52 instance(s) of the pod small-pod.
   Termination reason: Unschedulable: No nodes are available that match all of the
   following predicates:: Insufficient cpu (2).
   Pod distribution among nodes:
   small-pod
           - 192.168.124.214: 26 instance(s)