OpenEBS for NuoDB

NuoDB’s distributed SQL database combines the elastic scale and continuous availability of the cloud with the transactional consistency and durability that databases of record demand. NuoDB is deployed usually as a on Kubernetes and requires persistent storage for each instance of NuoDB StorageManager instance. OpenEBS provides persistent volumes on the fly when StorageManagers are scaled up.

Advantages of using OpenEBS for NuoDB database:

• No need to manage the local disks, they are managed by OpenEBS
• Large size PVs can be provisioned by OpenEBS and NuoDB
• Start with small storage and add disks as needed on the fly. Sometimes NuoDB instances are scaled up because of capacity on the nodes. With OpenEBS persistent volumes, capacity can be thin provisioned and disks can be added to OpenEBS on the fly without disruption of service
• If required, take backup of the NuoDB data periodically and back them up to S3 or any object storage so that restoration of the same data is possible to the same or any other Kubernetes cluster

Note: NuoDB can be deployed both as deployment or as statefulset. When NuoDB deployed as statefulset, you don’t need to replicate the data again at OpenEBS level. When NuoDB is deployed as deployment, consider 3 OpenEBS replicas, choose the StorageClass accordingly.

Deployment model

1. Install OpenEBS

   If OpenEBS is not installed in your K8s cluster, this can done from . If OpenEBS is already installed, go to the next step.

2. Connect to Kubera (Optional) : Connecting the Kubernetes cluster to Kubera provides good visibility of storage resources. Kubera has various support options for enterprise customers.

3. Configure cStor Pool

4. Create Storage Class

   You must configure a StorageClass to provision cStor volume on given cStor pool. StorageClass is the interface through which most of the OpenEBS storage policies are defined. In this solution we are using a StorageClass to consume the cStor Pool which is created using external disks attached on the Nodes. Since NuoDB is a statefulset application, it requires only single storage replica. So cStor volume replicaCount is =1. Sample YAML named openebs-sc-disk.yaml to consume cStor pool with cStoveVolume Replica count as 1 is provided in the configuration details below.

5. Download the YAML spec files from OpenEBS litmus repository.

6. Edit the above YAML spec files and update them with correct StorageClass. For example, replace openebs-cstor-sparse with openebs-cstor-disk

7. Launch and test NuoDB

   Create a namespace called testns and apply following YAML files to deploy NuoDB application. Sample YAML files are provided in the Configuration details below.

   kubectl apply -f nuodb.yaml -n testns
   kubectl apply -f nuodb-sm.yaml -n testns
   kubectl apply -f nuodb-te.yaml -n testns

Reference at openebs.ci

Deployment YAML spec files for NuoDB and OpenEBS resources are found

OpenEBS-CI dashboard of NuoDB

Monitor OpenEBS Volume size

It is not seamless to increase the cStor volume size (refer to the roadmap item). Hence, it is recommended that sufficient size is allocated during the initial configuration. However, an alert can be setup for volume size threshold using Kubera.

Monitor cStor Pool size

As in most cases, cStor pool may not be dedicated to just NuoDB database alone. It is recommended to watch the pool capacity and add more disks to the pool before it hits 80% threshold. See

Configuration details

openebs-config.yaml

openebs-sc-disk.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: openebs-cstor-disk
  annotations:
    openebs.io/cas-type: cstor
    cas.openebs.io/config: |
      - name: StoragePoolClaim
        value: "cstor-disk"
      - name: ReplicaCount
        value: "1"       
provisioner: openebs.io/provisioner-iscsi
reclaimPolicy: Delete

nuodb.yaml

nuodb-sm.yaml

---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: sm
  labels:
    app: nuodb
    group: nuodb
  serviceName: sm
  replicas: 1
  selector:
    matchLabels:
      nodetype: sm
      database: nuodb
      app: sm
      group: nuodb
  podManagementPolicy: Parallel
  template:
    metadata:
      labels:
        nodetype: sm
        database: nuodb
        app: sm
        group: nuodb
    spec:
      securityContext:
        runAsUser: 0
        fsGroup: 0
      containers:
      - name: sm
        image: nuodb/nuodb-ce:latest
        imagePullPolicy: IfNotPresent
        args: [ "nuosm", "--servers-ready-timeout", "300" ]
        env:
        - name: NODE_NAME
          valueFrom:
              fieldPath: spec.nodeName
        - name: POD_NAME
          valueFrom:
              fieldPath: metadata.name
        - name: NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - { name: DB_NAME, value: "nuodb"}
        - { name: DB_USER, value: "dba"}
        - { name: DB_PASSWORD, value: "goalie"}
        - { name: NUOCMD_API_SERVER,   value: "admin.testns.svc:8888"}
        - { name: PEER_ADDRESS,        value: "admin.testns.svc" }
        - { name: NUODB_OPTIONS,       value: "mem 1g" }
        ports:
        - containerPort: 48006
          protocol: TCP
        resources:
          requests:
            memory: "1Gi"
            cpu: "200m"
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/opt/nuodb/archive
          name: archive
  volumeClaimTemplates:
  - metadata:
      name: archive
      annotations:
        volume.beta.kubernetes.io/storage-class: openebs-cstor-sparse
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests: