Shared Resource CSI Driver Operator

    To enable the Shared Resource CSI Driver, you must enable features using feature gates.

    Storage vendors have traditionally provided storage drivers as part of Kubernetes. With the implementation of the Container Storage Interface (CSI), third-party providers can instead deliver storage plugins using a standard interface without ever having to change the core Kubernetes code.

    CSI Operators give OKD users storage options, such as volume snapshots, that are not possible with in-tree volume plugins.

    Sharing secrets across namespaces

    To share a secret across namespaces in a cluster, you create a SharedSecret custom resource (CR) instance for the Secret object that you want to share.

    Prerequisites

    You must have permission to perform the following actions:

    • Create instances of the sharedsecrets.sharedresource.openshift.io custom resource definition (CRD) at a cluster-scoped level.

    • Manage roles and role bindings across the namespaces in the cluster to control which users can get, list, and watch those instances.

    • Manage roles and role bindings to control whether the service account specified by a pod can mount a Container Storage Interface (CSI) volume that references the SharedSecret CR instance you want to use.

    • Access the namespaces that contain the Secrets you want to share.

    Procedure

    • Create a SharedSecret CR instance for the Secret object you want to share across namespaces in the cluster:

    To access a SharedSecret custom resource (CR) instance from a pod, you grant a given service account RBAC permissions to use that SharedSecret CR instance.

    Prerequisites

    • You have created a SharedSecret CR instance for the secret you want to share across namespaces in the cluster.

    • You must have permission to perform the following actions

      • Discover which SharedSecret CR instances are available by entering the oc get sharedsecrets command and getting a non-empty list back.

      • Determine if the service account your pod specifies is allowed to use the given SharedSecret CR instance. That is, you can run oc adm policy who-can use <identifier of specific SharedSecret> to see if the service account in your namespace is listed.

      • Determine if the service account your pod specifies is allowed to use csi volumes, or if you, as the requesting user who created the pod directly, are allowed to use csi volumes. See “Understanding and managing pod security admission” for details.

    Procedure

    1. Grant a given service account RBAC permissions to use the SharedSecret CR instance in its pod by using oc apply with YAML content:

      Currently, kubectl and oc have hard-coded special case logic restricting the use verb to roles centered around pod security. Therefore, you cannot use oc create role …​ to create the role needed for consuming SharedSecret CR instances.

      1. $ oc apply -f - <<EOF
      2. apiVersion: rbac.authorization.k8s.io/v1
      3. kind: Role
      4. metadata:
      5.   name: shared-resource-my-share
      6.   namespace: my-namespace
      7. rules:
      8.   - apiGroups:
      9.       - sharedresource.openshift.io
      10.     resources:
      11.       - sharedsecrets
      12.     resourceNames:
      13.       - my-share
      14.     verbs:
      15.       - use
      16. EOF

    3. Access the SharedSecret CR instance from a pod:

      1. $ oc apply -f - <<EOF
      2. apiVersion: v1
      3. metadata:
      4.   name: my-app
      5.   namespace: my-namespace
      6. spec:
      7.   serviceAccountName: default
      9. # containers omitted …. Follow standard use of ‘volumeMounts’ for referencing your shared resource volume
      11.     volumes:
      12.     - name: my-csi-volume
      13.       csi:
      15.         driver: csi.sharedresource.openshift.io
      16.         volumeAttributes:
      17.           sharedSecret: my-share
      19. EOF

    Sharing a config map across namespaces

    To share a config map across namespaces in a cluster, you create a SharedConfigMap custom resource (CR) instance for that config map.

    Prerequisites

    You must have permission to perform the following actions:

    • Create instances of the sharedconfigmaps.sharedresource.openshift.io custom resource definition (CRD) at a cluster-scoped level.

    • Manage roles and role bindings across the namespaces in the cluster to control which users can get, list, and watch those instances.

    • Manage roles and role bindings across the namespaces in the cluster to control which service accounts in pods that mount your Container Storage Interface (CSI) volume can use those instances.

    • Access the namespaces that contain the Secrets you want to share.

    Procedure

    1. Create a SharedConfigMap CR instance for the config map that you want to share across namespaces in the cluster:

    Next steps

    To access a SharedConfigMap custom resource (CR) instance from a pod, you grant a given service account RBAC permissions to use that SharedConfigMap CR instance.

    Prerequisites

    • You have created a SharedConfigMap CR instance for the config map that you want to share across namespaces in the cluster.

    • You must have permission to perform the following actions:

      • Discover which SharedConfigMap CR instances are available by entering the oc get sharedconfigmaps command and getting a non-empty list back.

      • Determine if the service account your pod specifies is allowed to use the given SharedSecret CR instance. That is, you can run oc adm policy who-can use <identifier of specific SharedSecret> to see if the service account in your namespace is listed.

      • Determine if the service account your pod specifies is allowed to use csi volumes, or if you, as the requesting user who created the pod directly, are allowed to use csi volumes. See “Understanding and managing pod security admission” for details.

    Procedure

    1. Grant a given service account RBAC permissions to use the SharedConfigMap CR instance in its pod by using oc apply with YAML content.

      Currently, kubectl and oc have hard-coded special case logic restricting the use verb to roles centered around pod security. Therefore, you cannot use oc create role …​ to create the role needed for consuming a SharedConfigMap CR instance.

      1. $ oc apply -f - <<EOF
      2. apiVersion: rbac.authorization.k8s.io/v1
      3. kind: Role
      4. metadata:
      5.   name: shared-resource-my-share
      6.   namespace: my-namespace
      7. rules:
      8.   - apiGroups:
      9.       - sharedresource.openshift.io
      10.     resources:
      11.       - sharedconfigmaps
      12.     resourceNames:
      13.     verbs:
      14.       - use
      15. EOF

    2. Create the RoleBinding associated with the role by using the oc command:

    3. Access the SharedConfigMap CR instance from a pod:

      1. $ oc apply -f - <<EOF
      2. kind: Pod
      3. apiVersion: v1
      4. metadata:
      5.   name: my-app
      6.   namespace: my-namespace
      7. spec:
      8.   serviceAccountName: default
      10. # containers omitted …. Follow standard use of ‘volumeMounts’ for referencing your shared resource volume
      12.     volumes:
      13.     - name: my-csi-volume
      14.       csi:
      15.         readOnly: true
      16.         driver: csi.sharedresource.openshift.io
      17.         volumeAttributes:
      18.           sharedConfigMap: my-share

    Additional support limitations for the Shared Resource CSI Driver

    The Shared Resource CSI Driver has the following noteworthy limitations:

    • The value of the readOnly field must be true. On Pod creation, a validating admission webhook rejects the pod creation if readOnly is false. If for some reason the validating admission webhook cannot be contacted, on volume provisioning during pod startup, the driver returns an error to the kubelet. Requiring readOnly is true is in keeping with proposed best practices for the upstream Kubernetes CSI Driver to apply SELinux labels to associated volumes.

    • The driver ignores the FSType field because it only supports tmpfs volumes.

    • The driver ignores the NodePublishSecretRef field. Instead, it uses SubjectAccessReviews with the use verb to evaluate whether a pod can obtain a volume that contains SharedSecret or SharedConfigMap custom resource (CR) instances.

    • You cannot create SharedSecret or SharedConfigMap custom resource (CR) instances whose names start with openshift.

    The following attributes affect shared resource pod volumes in various ways:

    • The refreshResource attribute in the volumeAttributes properties.

    • The refreshResources attribute in the Shared Resource CSI Driver configuration.

    • The sharedSecret and sharedConfigMap attributes in the volumeAttributes properties.

    The Shared Resource CSI Driver honors the refreshResource attribute in volumeAttributes properties of the volume. This attribute controls whether updates to the contents of the underlying Secret or ConfigMap object are copied to the volume after the volume is initially provisioned as part of pod startup. The default value of refreshResource is true, which means that the contents are updated.

    You can use a global switch to enable or disable refreshing of shared resources. This switch is the refreshResources attribute in the csi-driver-shared-resource-config config map for the Shared Resource CSI Driver, which you can find in the openshift-cluster-csi-drivers namespace. If you set this refreshResources attribute to false, none of the Secret or ConfigMap object-related content stored in the volume is updated after the initial provisioning of the volume.

    Using this Shared Resource CSI Driver configuration to disable refreshing affects all the cluster’s volume mounts that use the Shared Resource CSI Driver, regardless of the refreshResource attribute in the volumeAttributes properties of any of those volumes.

    In the volumeAttributes of a single volume, you must set either a sharedSecret or a sharedConfigMap attribute to the value of a SharedSecret or a SharedConfigMap CS instance. Otherwise, when the volume is provisioned during pod startup, a validation checks the volumeAttributes of that volume and returns an error to the kubelet under the following conditions:

    • Both sharedSecret and sharedConfigMap attributes have specified values.

    • Neither sharedSecret nor sharedConfigMap attributes have specified values.

    • The value of the sharedSecret or sharedConfigMap attribute does not correspond to the name of a SharedSecret or SharedConfigMap CR instance on the cluster.

    Integration between shared resources, Insights Operator, and OKD Builds

    Integration between shared resources, Insights Operator, and OKD Builds makes using Red Hat subscriptions (RHEL entitlements) easier in OKD Builds.

    Previously, in OKD 4.9.x and earlier, you manually imported your credentials and copied them to each project or namespace where you were running builds.

    Now, in OKD 4.10 and later, OKD Builds can use Red Hat subscriptions (RHEL entitlements) by referencing shared resources and the simple content access feature provided by Insights Operator:

    • The simple content access feature imports your subscription credentials to a well-known Secret object. See the links in the following “Additional resources” section.

    • The cluster administrator creates a SharedSecret custom resource (CR) instance around that Secret object and grants permission to particular projects or namespaces. In particular, the cluster administrator gives the builder service account permission to use that SharedSecret CR instance.

    • Builds that run within those projects or namespaces can mount a CSI Volume that references the SharedSecret CR instance and its entitled RHEL content.

    Additional resources