feat(e2e-test): Add e2e-tests for zfs-localpv (#298)

Signed-off-by: w3aman <aman.gupta@mayadata.io>

e2e-tests/experiments/functional/zfspv-raw-block-volume/README.md

@@ -0,0 +1,69 @@
+## About this experiment
+
+This experiment verifies the provision and deprovision of raw block volumes by zfs-localpv. There are some specialized applications that require direct access to a block device because, for example, the file system layer introduces unneeded overhead. The most common case is databases, which prefer to organize their data directly on the underlying storage. In this experiment we are not using any such application for testing, but using a simple busybox application to verify successful provisioning and deprovisioning of raw block volume.
+
+To provisione the Raw Block volume, we should create a storageclass without any fstype as Raw block volume does not have any fstype.
+
+```
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: zfspv-raw-block
+  allowVolumeExpansion: true
+  parameters:
+    poolname: "zfspv-pool"
+    provisioner: zfs.csi.openebs.io
+```    
+Note: For running this experiment above storage-class should be present. This storage class will be created as a part of zfs-localpv provisioner experiment. If zfs-localpv components are not deployed using e2e-test script located at `openebs/zfs-localpv/e2e-tests/experiment/zfs-localpv-provisioiner` please make sure you create the storage class from above mentioned yaml.
+
+## Supported platforms:
+
+K8s : 1.18+
+
+OS : Ubuntu, CentOS
+
+ZFS : 0.7, 0.8
+
+## Entry-Criteria
+
+- K8s cluster should be in healthy state including all desired nodes in ready state.
+- zfs-controller and node-agent daemonset pods should be in running state.
+- storage class without any fstype should be present.
+- a directory should be present on node with name `raw_block_volume`.
+
+## Steps performed
+
+- deploy the busybox application with given a devicePath.
+- verify that application pvc gets bound and application pod is in running state.
+- dump some data into raw block device and take the md5sum of data.
+- restart the application and verify the data consistency.
+- After that update the pvc with double value of previous pvc size, to validate resize support for raw block volumes.
+- when resize is successful, then dump some dummy data into application to use the resized space.
+- At last deprovision the application and check its successful deletion.
+
+## How to run
+
+- This experiment accepts the parameters in form of kubernetes job environmental variables.
+- For running this experiment of zfspv raw block volume creation, first clone openens/zfs-localpv[https://github.com/openebs/zfs-localpv] repo and then apply rbac and crds for e2e-framework.
+```
+kubectl apply -f zfs-localpv/e2e-tests/hack/rbac.yaml
+kubectl apply -f zfs-localpv/e2e-tests/hack/crds.yaml
+```
+then update the needed test specific values in run_e2e_test.yml file and create the kubernetes job.
+```
+kubectl create -f run_e2e_test.yml
+```
+All the env variables description is provided with the comments in the same file.
+After creating kubernetes job, when the job’s pod is instantiated, we can see the logs of that pod which is executing the test-case.
+
+```
+kubectl get pods -n e2e
+kubectl logs -f <zfspv-block-volume-xxxxx-xxxxx> -n e2e
+```
+To get the test-case result, get the corresponding e2e custom-resource `e2eresult` (short name: e2er ) and check its phase (Running or Completed) and result (Pass or Fail).
+
+```
+kubectl get e2er
+kubectl get e2er zfspv-raw-block-volume -n e2e --no-headers -o custom-columns=:.spec.testStatus.phase
+kubectl get e2er zfspv-raw-block-volume -n e2e --no-headers -o custom-columns=:.spec.testStatus.result
+```

e2e-tests/experiments/functional/zfspv-raw-block-volume/busybox.j2

@@ -0,0 +1,53 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: app-busybox
+  labels:
+    app: raw-block-vol
+spec:
+  selector:
+    matchLabels:
+      app: raw-block-vol
+  template:
+    metadata:
+      labels:
+        app: raw-block-vol
+    spec:
+      containers:
+      - name: app-busybox
+        imagePullPolicy: IfNotPresent
+        image: gcr.io/google-containers/busybox
+        command: ["/bin/sh"]
+        args: ["-c", "while true; do sleep 10;done"]
+        env:
+        volumeDevices:
+        - name: data-raw-block-vol
+          devicePath: /dev/sdc
+
+        volumeMounts:
+        - name: data-mount-vol
+          mountPath: /busybox
+
+      volumes:
+      - name: data-raw-block-vol
+        persistentVolumeClaim:
+          claimName: block-vol-pvc
+
+      - name: data-mount-vol
+        hostPath:
+          path: /raw_block_volume
+
+---
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: block-vol-pvc
+spec:
+  volumeMode: Block
+  storageClassName: {{ storage_class }}
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      storage: {{ pvc_size }}

e2e-tests/experiments/functional/zfspv-raw-block-volume/run_e2e_test.yml

@@ -0,0 +1,43 @@
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+  generateName: zfspv-block-volume-
+  namespace: e2e
+spec:
+  template:
+    metadata:
+      labels:
+        test: zfspv-block-volume
+    spec:
+      serviceAccountName: e2e
+      restartPolicy: Never
+
+      containers:
+      - name: ansibletest
+        image: openebs/zfs-localpv-e2e:ci
+        imagePullPolicy: IfNotPresent
+        env:
+          - name: ANSIBLE_STDOUT_CALLBACK
+            value: default
+
+            # This is the namespace where raw block volume consuming application will be deployed
+          - name: APP_NAMESPACE
+            value: 'block-ns'
+            
+            # storage class name for raw block volume
+            # this storage class doesn't have any `fstype` parameter
+          - name: STORAGE_CLASS
+            value: 'zfspv-raw-block'
+   
+            # size of the pvc for `volumeMode: Block`
+          - name: PVC_SIZE
+            value: '5Gi'
+
+            # This is the namespace where zfs-driver creates all its resources.
+            # By default it is `openebs` namespace
+          - name: ZFS_OPERATOR_NAMESPACE
+            value: 'openebs'
+
+        command: ["/bin/bash"]
+        args: ["-c", "ansible-playbook ./e2e-tests/experiments/functional/zfspv-raw-block-volume/test.yml -i /etc/ansible/hosts -vv; exit 0"]

e2e-tests/experiments/functional/zfspv-raw-block-volume/test.yml

@@ -0,0 +1,248 @@
+- hosts: localhost
+  connection: local
+  gather_facts: False
+
+  vars_files:
+    - test_vars.yml
+    
+  tasks:
+    - block:
+
+          ## Generating the testname for zfspv raw block volume test
+        - include_tasks: /e2e-tests/hack/create_testname.yml
+    
+          ## Record SOT (start of test) in e2e result e2e-cr (e2e-custom-resource)
+        - include_tasks: /e2e-tests/hack/update_e2e_result_resource.yml
+          vars:
+            status: 'SOT'
+        
+        - name: Create the namespace for block-volume of zfspv
+          shell: >
+            kubectl create ns {{ app_ns }}
+          args:
+            executable: /bin/bash
+
+        - name: Update the busybox application template with the test specific values
+          template:
+            src: busybox.j2
+            dest: busybox.yml
+
+        - name: Deploy the application using block volume pvc
+          shell: >
+            kubectl create -f busybox.yml -n {{ app_ns }}
+          args:
+            executable: /bin/bash
+
+        - name: Check if the block volume PVC is bound
+          shell: >
+            kubectl get pvc block-vol-pvc -n {{ app_ns }} 
+            --no-headers -o custom-columns=:.status.phase
+          args:
+            executable: /bin/bash
+          register: pvc_status
+          until: "'Bound' in pvc_status.stdout"
+          delay: 5
+          retries: 30
+
+        - name: Get the zvolume name
+          shell: kubectl get pvc block-vol-pvc -n {{ app_ns }} -o custom-columns=:.spec.volumeName
+          args:
+            executable: /bin/bash
+          register: zv_name
+          
+        - name: Get the application pod name
+          shell: >
+            kubectl get pod -n {{ app_ns }} -l app=raw-block-vol
+            --no-headers -o custom-columns=:.metadata.name
+          args:
+            executable: /bin/bash
+          register: app_pod_name
+
+        - name: Check if the application pod is in running state
+          shell: >
+            kubectl get pod {{ app_pod_name.stdout }} -n {{ app_ns }}
+            --no-headers -o custom-columns=:.status.phase
+          register: pod_status
+          until: "'Running' in pod_status.stdout"
+          delay: 5
+          retries: 50
+
+        - name: Create some test data into the raw block device and take the md5sum of data
+          shell: >
+            kubectl exec -ti {{ app_pod_name.stdout }} -n {{ app_ns }}
+            -- sh -c "{{ item }}"
+          args:
+            executable: /bin/bash
+          register: result
+          failed_when: "result.rc != 0"
+          with_items:
+            - "dd if=/dev/urandom of=/dev/sdc bs=4k count=262144"
+            - "md5sum /dev/sdc > /busybox/pre-md5"
+
+        - name: Restart the busybox application
+          shell: kubectl delete pod {{ app_pod_name.stdout }} -n {{ app_ns }}
+          args:
+            executable: /bin/bash
+
+        - name: Get the application pod name after restart
+          shell: >
+            kubectl get pod -n {{ app_ns }} -l app=raw-block-vol
+            --no-headers -o custom-columns=:.metadata.name 
+          args:
+            executable: /bin/bash
+          register: new_pod_name
+
+        - name: Check if the application pod is in running state
+          shell: >
+            kubectl get pod {{ new_pod_name.stdout }} -n {{ app_ns }}
+            --no-headers -o custom-columns=:.status.phase
+          register: pod_status
+          until: "'Running' in pod_status.stdout"
+          delay: 5
+          retries: 50
+
+        - name: Again take the md5sum of the data after restarting the application pod 
+          shell: >
+            kubectl exec -ti {{ new_pod_name.stdout }} -n {{ app_ns }}
+            -- sh -c "md5sum /dev/sdc > /busybox/post-md5"
+          args:
+            executable: /bin/bash
+          register: result
+          failed_when: "result.rc != 0"
+
+        - name: Verify whether data is consistence after restarting the application pod
+          shell: >
+            kubectl exec -ti {{ new_pod_name.stdout }} -n {{ app_ns }}
+            -- sh -c "diff /busybox/pre-md5 /busybox/post-md5"
+          args: 
+            executable: /bin/bash
+          register: result 
+          failed_when: "result.rc != 0 or result.stdout != ''"
+
+        - name: Obtain the mount path for the application
+          shell: > 
+              kubectl get pod {{ new_pod_name.stdout }} -n {{ app_ns }}
+              -o custom-columns=:.spec.containers[].volumeMounts[].mountPath --no-headers
+          args: 
+              executable: /bin/bash
+          register: mount
+
+        - name: Fetch the Storage from PVC using namespace
+          shell: kubectl get pvc -n {{ app_ns }} -o jsonpath={.items[0].spec.resources.requests.storage}
+          args:
+            executable: /bin/bash
+          register: storage_capacity
+
+        - name: Fetch the alphabet(G,M,m,g) from storage capacity
+          shell: echo "{{ storage_capacity.stdout }}" | grep -o -E '[0-9]+'
+          args:
+            executable: /bin/bash
+          register: value_pvc
+
+        ## Here we will resize the volume to double value of present pvc size
+        - set_fact:
+            desired_vol_size: '{{ (value_pvc.stdout | int * 2 | int) |  int }}'
+
+        - name: Obtain the PVC spec
+          shell: >
+            kubectl get pvc block-vol-pvc -n {{ app_ns }}
+            --no-headers -o yaml > pvc.yml
+          args:
+            executable: /bin/bash
+
+        - name: Update the desired capacity in PVC spec
+          replace:
+            path: pvc.yml
+            before: 'storageClassName: {{ storage_class }}'
+            regexp: "storage: {{ pvc_size }}"
+            replace: "storage: {{ desired_vol_size }}Gi"
+
+        - name: Configure PVC with the new capacity
+          shell: kubectl apply -f pvc.yml
+          args:
+            executable: /bin/bash
+          register: result
+          failed_when: "result.rc != 0"
+
+        - name: Check if the desired PVC is bound
+          shell: >
+            kubectl get pvc block-vol-pvc -n {{ app_ns }} --no-headers
+            -o custom-columns=:.status.phase
+          args:
+            executable: /bin/bash
+          register: pvc_status
+          failed_when: "'Bound' not in pvc_status.stdout"
+
+        - name: Check if the storage capacity is updated in PVC
+          shell: >
+              kubectl get pvc block-vol-pvc -n {{ app_ns }} --no-headers 
+              -o custom-columns=:status.capacity.storage
+          args:
+            executable: /bin/bash
+          register: capacity
+          until: "desired_vol_size in capacity.stdout"
+          delay: 10
+          retries: 50
+
+          ## Here we will dump +1Gi data than to previous pvc size
+        - set_fact:
+            value_num: '{{ ( (value_pvc.stdout | int + 1 | int) * 1024) |  int }}'
+
+        - name: Dump some more dummy data in the application mount point for using resized volume
+          shell: >
+              kubectl exec -it "{{ new_pod_name.stdout }}" -n "{{ app_ns }}" 
+              -- sh -c "cd {{ mount.stdout }} && dd if=/dev/urandom of=volume.txt bs=1024k count={{ value_num }}"
+          args:
+              executable: /bin/bash
+          register: load
+          failed_when: "load.rc != 0"
+
+        - name: Deprovision the busybox application
+          shell: kubectl delete -f busybox.yml -n {{ app_ns }}
+          args:
+            executable: /bin/bash
+
+        - name: Verify that busybox application is successfully deprovisioned
+          shell: kubectl get pods -n {{ app_ns }}
+          args:
+            executable: /bin/bash
+          register: app_pod_status
+          until: "'new_app_pod.stdout' not in app_pod_status.stdout"
+          delay: 5
+          retries:
+
+        - name: Verify that pvc is deleted successfully
+          shell: kubectl get pvc -n {{ app_ns }}
+          args:
+            executable: /bin/bash
+          register: pvc_status
+          until: "'block-vol-pvc' not in pvc_status.stdout"
+          delay: 3
+          retries: 30
+
+        - name: Verify the zvolume is deleted successfully
+          shell: kubectl get zv -n {{ zfs_operator_ns }}
+          args:
+            executable: /bin/bash
+          register: zvol_list
+          until: "'zv_name.stdout' not in zvol_list.stdout"
+          delay: 3
+          retries: 30
+
+        - name: Delete the application namespace
+          shell: kubectl delete ns {{ app_ns }}
+          args:
+            executable: /bin/bash
+
+        - set_fact:
+            flag: "Pass"
+        
+      rescue:
+      - set_fact:
+          flag: "Fail"
+        
+      always:
+          ## RECORD END-OF-TEST IN e2e RESULT CR
+        - include_tasks: /e2e-tests/hack/update_e2e_result_resource.yml
+          vars:
+            status: 'EOT'

e2e-tests/experiments/functional/zfspv-raw-block-volume/test_vars.yml

@@ -0,0 +1,9 @@
+test_name: zfspv-raw-block-volume
+
+app_ns: "{{ lookup('env','APP_NAMESPACE') }}"
+
+storage_class: "{{ lookup('env','STORAGE_CLASS') }}"
+
+pvc_size: "{{ lookup('env','PVC_SIZE') }}"
+
+zfs_operator_ns: "{{ lookup('env','ZFS_OPERATOR_NAMESPACE') }}"