zfs-localpv/pkg/scheduler/scheduler.go

100 lines
2.4 KiB
Go
Raw Normal View History

feat(ZFSPV): volume count based scheduler for ZFSPV (#8) This is an initial scheduler implementation for ZFS Local PV. * adding scheduler as a configurable option * adding volumeWeightedScheduler as scheduling logic The volumeWeightedScheduler will go through all the nodes as per topology information and it will pick the node which has less volume provisioned in the given pool. lets say there are 2 nodes node1 and node2 with below pool configuration :- ``` node1 | |-----> pool1 | | | |------> pvc1 | |------> pvc2 |-----> pool2 |------> pvc3 node2 | |-----> pool1 | | | |------> pvc4 |-----> pool2 |------> pvc5 |------> pvc6 ``` So if application is using pool1 as shown in the below storage class, then ZFS driver will schedule it on node2 as it has one volume as compared to node1 which has 2 volumes in pool1. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool1" ``` So if application is using pool2 as shown in the below storage class, then ZFS driver will schedule it on node1 as it has one volume only as compared node2 which has 2 volumes in pool2. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool2" ``` In case of same number of volumes on all the nodes for the given pool, it can pick any node and schedule the PV on that. Signed-off-by: Pawan <pawan@mayadata.io>
2019-11-06 21:20:49 +05:30
/*
Copyright © 2019 The OpenEBS Authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package scheduler
feat(ZFSPV): volume count based scheduler for ZFSPV (#8) This is an initial scheduler implementation for ZFS Local PV. * adding scheduler as a configurable option * adding volumeWeightedScheduler as scheduling logic The volumeWeightedScheduler will go through all the nodes as per topology information and it will pick the node which has less volume provisioned in the given pool. lets say there are 2 nodes node1 and node2 with below pool configuration :- ``` node1 | |-----> pool1 | | | |------> pvc1 | |------> pvc2 |-----> pool2 |------> pvc3 node2 | |-----> pool1 | | | |------> pvc4 |-----> pool2 |------> pvc5 |------> pvc6 ``` So if application is using pool1 as shown in the below storage class, then ZFS driver will schedule it on node2 as it has one volume as compared to node1 which has 2 volumes in pool1. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool1" ``` So if application is using pool2 as shown in the below storage class, then ZFS driver will schedule it on node1 as it has one volume only as compared node2 which has 2 volumes in pool2. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool2" ``` In case of same number of volumes on all the nodes for the given pool, it can pick any node and schedule the PV on that. Signed-off-by: Pawan <pawan@mayadata.io>
2019-11-06 21:20:49 +05:30
import (
"math"
"github.com/container-storage-interface/spec/lib/go/csi"
k8sapi "github.com/openebs/zfs-localpv/pkg/client/k8s/v1alpha1"
feat(ZFSPV): volume count based scheduler for ZFSPV (#8) This is an initial scheduler implementation for ZFS Local PV. * adding scheduler as a configurable option * adding volumeWeightedScheduler as scheduling logic The volumeWeightedScheduler will go through all the nodes as per topology information and it will pick the node which has less volume provisioned in the given pool. lets say there are 2 nodes node1 and node2 with below pool configuration :- ``` node1 | |-----> pool1 | | | |------> pvc1 | |------> pvc2 |-----> pool2 |------> pvc3 node2 | |-----> pool1 | | | |------> pvc4 |-----> pool2 |------> pvc5 |------> pvc6 ``` So if application is using pool1 as shown in the below storage class, then ZFS driver will schedule it on node2 as it has one volume as compared to node1 which has 2 volumes in pool1. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool1" ``` So if application is using pool2 as shown in the below storage class, then ZFS driver will schedule it on node1 as it has one volume only as compared node2 which has 2 volumes in pool2. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool2" ``` In case of same number of volumes on all the nodes for the given pool, it can pick any node and schedule the PV on that. Signed-off-by: Pawan <pawan@mayadata.io>
2019-11-06 21:20:49 +05:30
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/klog"
feat(ZFSPV): volume count based scheduler for ZFSPV (#8) This is an initial scheduler implementation for ZFS Local PV. * adding scheduler as a configurable option * adding volumeWeightedScheduler as scheduling logic The volumeWeightedScheduler will go through all the nodes as per topology information and it will pick the node which has less volume provisioned in the given pool. lets say there are 2 nodes node1 and node2 with below pool configuration :- ``` node1 | |-----> pool1 | | | |------> pvc1 | |------> pvc2 |-----> pool2 |------> pvc3 node2 | |-----> pool1 | | | |------> pvc4 |-----> pool2 |------> pvc5 |------> pvc6 ``` So if application is using pool1 as shown in the below storage class, then ZFS driver will schedule it on node2 as it has one volume as compared to node1 which has 2 volumes in pool1. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool1" ``` So if application is using pool2 as shown in the below storage class, then ZFS driver will schedule it on node1 as it has one volume only as compared node2 which has 2 volumes in pool2. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool2" ``` In case of same number of volumes on all the nodes for the given pool, it can pick any node and schedule the PV on that. Signed-off-by: Pawan <pawan@mayadata.io>
2019-11-06 21:20:49 +05:30
)
// getNodeList gets the nodelist which satisfies the topology info
func getNodeList(topo *csi.TopologyRequirement) ([]string, error) {
var nodelist []string
list, err := k8sapi.ListNodes(metav1.ListOptions{})
if err != nil {
return nil, err
}
for _, node := range list.Items {
for _, prf := range topo.Preferred {
nodeFiltered := false
for key, value := range prf.Segments {
if node.Labels[key] != value {
nodeFiltered = true
break
}
}
if nodeFiltered == false {
nodelist = append(nodelist, node.Name)
break
}
}
}
return nodelist, nil
}
// runScheduler goes through the node mapping
// in the topology and picks the node which is less weighted
func runScheduler(nodelist []string, nmap map[string]int64) string {
feat(ZFSPV): volume count based scheduler for ZFSPV (#8) This is an initial scheduler implementation for ZFS Local PV. * adding scheduler as a configurable option * adding volumeWeightedScheduler as scheduling logic The volumeWeightedScheduler will go through all the nodes as per topology information and it will pick the node which has less volume provisioned in the given pool. lets say there are 2 nodes node1 and node2 with below pool configuration :- ``` node1 | |-----> pool1 | | | |------> pvc1 | |------> pvc2 |-----> pool2 |------> pvc3 node2 | |-----> pool1 | | | |------> pvc4 |-----> pool2 |------> pvc5 |------> pvc6 ``` So if application is using pool1 as shown in the below storage class, then ZFS driver will schedule it on node2 as it has one volume as compared to node1 which has 2 volumes in pool1. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool1" ``` So if application is using pool2 as shown in the below storage class, then ZFS driver will schedule it on node1 as it has one volume only as compared node2 which has 2 volumes in pool2. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool2" ``` In case of same number of volumes on all the nodes for the given pool, it can pick any node and schedule the PV on that. Signed-off-by: Pawan <pawan@mayadata.io>
2019-11-06 21:20:49 +05:30
var selected string
var weight int64 = math.MaxInt64
feat(ZFSPV): volume count based scheduler for ZFSPV (#8) This is an initial scheduler implementation for ZFS Local PV. * adding scheduler as a configurable option * adding volumeWeightedScheduler as scheduling logic The volumeWeightedScheduler will go through all the nodes as per topology information and it will pick the node which has less volume provisioned in the given pool. lets say there are 2 nodes node1 and node2 with below pool configuration :- ``` node1 | |-----> pool1 | | | |------> pvc1 | |------> pvc2 |-----> pool2 |------> pvc3 node2 | |-----> pool1 | | | |------> pvc4 |-----> pool2 |------> pvc5 |------> pvc6 ``` So if application is using pool1 as shown in the below storage class, then ZFS driver will schedule it on node2 as it has one volume as compared to node1 which has 2 volumes in pool1. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool1" ``` So if application is using pool2 as shown in the below storage class, then ZFS driver will schedule it on node1 as it has one volume only as compared node2 which has 2 volumes in pool2. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool2" ``` In case of same number of volumes on all the nodes for the given pool, it can pick any node and schedule the PV on that. Signed-off-by: Pawan <pawan@mayadata.io>
2019-11-06 21:20:49 +05:30
// schedule it on the node which has less weight
for _, node := range nodelist {
if nmap[node] < weight {
feat(ZFSPV): volume count based scheduler for ZFSPV (#8) This is an initial scheduler implementation for ZFS Local PV. * adding scheduler as a configurable option * adding volumeWeightedScheduler as scheduling logic The volumeWeightedScheduler will go through all the nodes as per topology information and it will pick the node which has less volume provisioned in the given pool. lets say there are 2 nodes node1 and node2 with below pool configuration :- ``` node1 | |-----> pool1 | | | |------> pvc1 | |------> pvc2 |-----> pool2 |------> pvc3 node2 | |-----> pool1 | | | |------> pvc4 |-----> pool2 |------> pvc5 |------> pvc6 ``` So if application is using pool1 as shown in the below storage class, then ZFS driver will schedule it on node2 as it has one volume as compared to node1 which has 2 volumes in pool1. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool1" ``` So if application is using pool2 as shown in the below storage class, then ZFS driver will schedule it on node1 as it has one volume only as compared node2 which has 2 volumes in pool2. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool2" ``` In case of same number of volumes on all the nodes for the given pool, it can pick any node and schedule the PV on that. Signed-off-by: Pawan <pawan@mayadata.io>
2019-11-06 21:20:49 +05:30
selected = node
weight = nmap[node]
feat(ZFSPV): volume count based scheduler for ZFSPV (#8) This is an initial scheduler implementation for ZFS Local PV. * adding scheduler as a configurable option * adding volumeWeightedScheduler as scheduling logic The volumeWeightedScheduler will go through all the nodes as per topology information and it will pick the node which has less volume provisioned in the given pool. lets say there are 2 nodes node1 and node2 with below pool configuration :- ``` node1 | |-----> pool1 | | | |------> pvc1 | |------> pvc2 |-----> pool2 |------> pvc3 node2 | |-----> pool1 | | | |------> pvc4 |-----> pool2 |------> pvc5 |------> pvc6 ``` So if application is using pool1 as shown in the below storage class, then ZFS driver will schedule it on node2 as it has one volume as compared to node1 which has 2 volumes in pool1. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool1" ``` So if application is using pool2 as shown in the below storage class, then ZFS driver will schedule it on node1 as it has one volume only as compared node2 which has 2 volumes in pool2. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool2" ``` In case of same number of volumes on all the nodes for the given pool, it can pick any node and schedule the PV on that. Signed-off-by: Pawan <pawan@mayadata.io>
2019-11-06 21:20:49 +05:30
}
}
return selected
}
// Scheduler schedules the PV as per topology constraints for
// the given node weight.
func Scheduler(req *csi.CreateVolumeRequest, nmap map[string]int64) string {
topo := req.AccessibilityRequirements
if topo == nil ||
len(topo.Preferred) == 0 {
klog.Errorf("scheduler: topology information not provided")
return ""
}
nodelist, err := getNodeList(topo)
if err != nil {
klog.Errorf("scheduler: can not get the nodelist err : %v", err.Error())
return ""
} else if len(nodelist) == 0 {
klog.Errorf("scheduler: nodelist is empty")
feat(ZFSPV): volume count based scheduler for ZFSPV (#8) This is an initial scheduler implementation for ZFS Local PV. * adding scheduler as a configurable option * adding volumeWeightedScheduler as scheduling logic The volumeWeightedScheduler will go through all the nodes as per topology information and it will pick the node which has less volume provisioned in the given pool. lets say there are 2 nodes node1 and node2 with below pool configuration :- ``` node1 | |-----> pool1 | | | |------> pvc1 | |------> pvc2 |-----> pool2 |------> pvc3 node2 | |-----> pool1 | | | |------> pvc4 |-----> pool2 |------> pvc5 |------> pvc6 ``` So if application is using pool1 as shown in the below storage class, then ZFS driver will schedule it on node2 as it has one volume as compared to node1 which has 2 volumes in pool1. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool1" ``` So if application is using pool2 as shown in the below storage class, then ZFS driver will schedule it on node1 as it has one volume only as compared node2 which has 2 volumes in pool2. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool2" ``` In case of same number of volumes on all the nodes for the given pool, it can pick any node and schedule the PV on that. Signed-off-by: Pawan <pawan@mayadata.io>
2019-11-06 21:20:49 +05:30
return ""
}
feat(ZFSPV): volume count based scheduler for ZFSPV (#8) This is an initial scheduler implementation for ZFS Local PV. * adding scheduler as a configurable option * adding volumeWeightedScheduler as scheduling logic The volumeWeightedScheduler will go through all the nodes as per topology information and it will pick the node which has less volume provisioned in the given pool. lets say there are 2 nodes node1 and node2 with below pool configuration :- ``` node1 | |-----> pool1 | | | |------> pvc1 | |------> pvc2 |-----> pool2 |------> pvc3 node2 | |-----> pool1 | | | |------> pvc4 |-----> pool2 |------> pvc5 |------> pvc6 ``` So if application is using pool1 as shown in the below storage class, then ZFS driver will schedule it on node2 as it has one volume as compared to node1 which has 2 volumes in pool1. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool1" ``` So if application is using pool2 as shown in the below storage class, then ZFS driver will schedule it on node1 as it has one volume only as compared node2 which has 2 volumes in pool2. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool2" ``` In case of same number of volumes on all the nodes for the given pool, it can pick any node and schedule the PV on that. Signed-off-by: Pawan <pawan@mayadata.io>
2019-11-06 21:20:49 +05:30
// if there is a single node, schedule it on that
if len(nodelist) == 1 {
return nodelist[0]
feat(ZFSPV): volume count based scheduler for ZFSPV (#8) This is an initial scheduler implementation for ZFS Local PV. * adding scheduler as a configurable option * adding volumeWeightedScheduler as scheduling logic The volumeWeightedScheduler will go through all the nodes as per topology information and it will pick the node which has less volume provisioned in the given pool. lets say there are 2 nodes node1 and node2 with below pool configuration :- ``` node1 | |-----> pool1 | | | |------> pvc1 | |------> pvc2 |-----> pool2 |------> pvc3 node2 | |-----> pool1 | | | |------> pvc4 |-----> pool2 |------> pvc5 |------> pvc6 ``` So if application is using pool1 as shown in the below storage class, then ZFS driver will schedule it on node2 as it has one volume as compared to node1 which has 2 volumes in pool1. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool1" ``` So if application is using pool2 as shown in the below storage class, then ZFS driver will schedule it on node1 as it has one volume only as compared node2 which has 2 volumes in pool2. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool2" ``` In case of same number of volumes on all the nodes for the given pool, it can pick any node and schedule the PV on that. Signed-off-by: Pawan <pawan@mayadata.io>
2019-11-06 21:20:49 +05:30
}
return runScheduler(nodelist, nmap)
feat(ZFSPV): volume count based scheduler for ZFSPV (#8) This is an initial scheduler implementation for ZFS Local PV. * adding scheduler as a configurable option * adding volumeWeightedScheduler as scheduling logic The volumeWeightedScheduler will go through all the nodes as per topology information and it will pick the node which has less volume provisioned in the given pool. lets say there are 2 nodes node1 and node2 with below pool configuration :- ``` node1 | |-----> pool1 | | | |------> pvc1 | |------> pvc2 |-----> pool2 |------> pvc3 node2 | |-----> pool1 | | | |------> pvc4 |-----> pool2 |------> pvc5 |------> pvc6 ``` So if application is using pool1 as shown in the below storage class, then ZFS driver will schedule it on node2 as it has one volume as compared to node1 which has 2 volumes in pool1. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool1" ``` So if application is using pool2 as shown in the below storage class, then ZFS driver will schedule it on node1 as it has one volume only as compared node2 which has 2 volumes in pool2. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-zfspv provisioner: zfs.csi.openebs.io parameters: blocksize: "4k" compression: "on" dedup: "on" thinprovision: "yes" poolname: "pool2" ``` In case of same number of volumes on all the nodes for the given pool, it can pick any node and schedule the PV on that. Signed-off-by: Pawan <pawan@mayadata.io>
2019-11-06 21:20:49 +05:30
}