Cluster API 檢索從未如此簡單
?Clusterpedia 是一個 CNCF 沙箱項目，用于跨集群復(fù)雜的資源檢索。
名字源于 Wikipedia，寓意是打造多集群的百科全書，可以與多個集群同步資源，并在與 Kubernetes OpenAPI 兼容的基礎(chǔ)上，提供更強大的搜索功能，以幫助您快速、簡便、有效地獲取任何多集群資源。?
項目地址：https://github.com/clusterpedia-io/clusterpedia
0.4.0 后，Clusterpedia 提供了更加友好的接入多云平臺的方式，用戶在多云平臺創(chuàng)建或者納管集群后，便可以直接使用 kubectl 來檢索這些集群內(nèi)的資源。
我們在Clusterpedia 倉庫[1]中維護(hù)了各個多云平臺的 ClusterImportPolicy[2]。非常歡迎大家提交用于對接其他多云平臺的 ClusterImportPolicy。?
用戶在安裝 Clusterpedia 后，創(chuàng)建合適的 ClusterImportPolicy 即可，用戶也可以根據(jù)自己的需求來創(chuàng)建新的 ClusterImportPolicy[3]。
Cluster API 的 ClusterImportPolicy 已經(jīng)在 clusterpedia#288[4]中提交，在 Cluster API 中創(chuàng)建集群后，可以直接使用 Clusterpedia 來對這些集群內(nèi)的資源進(jìn)行復(fù)雜檢索。
$ kubectl get cluster
NAME PHASE AGE VERSION
capi-quickstart Provisioned 10m v1.24.2
capi-quickstart-2 Provisioned 118s v1.24.2
$ kubectl get kubeadmcontrolplane
NAME CLUSTER INITIALIZED API SERVER AVAILABLE REPLICAS READY UPDATED UNAVAILABLE AGE VERSION
capi-quickstart-2-ctm9k capi-quickstart-2 true 1 1 1 10m v1.24.2
capi-quickstart-2xcsz capi-quickstart true 1 1 1 19m v1.24.2
$ # pediacluster 會根據(jù) cluster 資源自動創(chuàng)建，更新和刪除
$ kubectl get pediacluster -o wide
NAME READY VERSION APISERVER VALIDATED SYNCHRORUNNING CLUSTERHEALTHY
default-capi-quickstart True v1.24.2 Validated Running Healthy
default-capi-quickstart-2 True v1.24.2 Validated Running Healthy
$ kubectl --cluster clusterpedia get no
CLUSTER NAME STATUS ROLES AGE VERSION
default-capi-quickstart-2 capi-quickstart-2-ctm9k-g2m87 NotReady control-plane 12m v1.24.2
default-capi-quickstart-2 capi-quickstart-2-md-0-s8hbx-7bd44554b5-kzcb6 NotReady <none> 11m v1.24.2
default-capi-quickstart capi-quickstart-2xcsz-fxrrk NotReady control-plane 21m v1.24.2
default-capi-quickstart capi-quickstart-md-0-9tw2g-b8b4f46cf-gggvq NotReady <none> 20m v1.24.2
01示例環(huán)境
如何快速部署一套 Cluster API And Clusterpedia 的示例環(huán)境？
預(yù)備條件
- 安裝 kubectl[5] 到本地環(huán)境
- 安裝 Kind[6] and Docker[7]
- 安裝 clusterctl[8]
Minimum kind supported version: v0.14.0
創(chuàng)鍵管理集群并部署 Cluster API
部署 Cluster API 也可以參考 https://cluster-api.sigs.k8s.io/user/quick-start.html
$ cat > kind-cluster-with-extramounts.yaml <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraMounts:
- hostPath: /var/run/docker.sock
containerPath: /var/run/docker.sock
EOF
$ kind create cluster --name capi-sample --config kind-cluster-with-extramounts.yaml
$ export CLUSTER_TOPOLOGY=true
$ clusterctl init --infrastructure docker
部署 Clusterpedia
$ git clone https://github.com/clusterpedia-io/clusterpedia.git && cd clusterpedia/charts
$ helm install clusterpedia . \
--namespace clusterpedia-system \
--create-namespace \
--set installCRDs=true \
# --set persistenceMatchNode={{ LOCAL_PV_NODE }}
--set persistenceMatchNode=capi-sample-control-plane
clusterpedia charts 提供了 Local PV，需要創(chuàng)建 LOCAL PV 綁定的節(jié)點。如果不需要 charts 來創(chuàng)建 LOCAL PV，可以使用 --set persistenceMatchNode=None.詳見[9]
創(chuàng)建用于接入 Cluster API 的集群自動導(dǎo)入策略[10]
$ kubectl apply -f https://raw.githubusercontent.com/Iceber/clusterpedia/add_cluster_api_clusterimportpolicy/deploy/clusterimportpolicy/cluster_api.yaml
Clusterpedia 可以接入任何的多云管理平臺，接入方式可以參考 Interfacing to Multi-Cloud Platforms[11]
生成 kubectl cluster shortcut[12]，如果使用 client-go 或者 OpenAPI 來訪問，可以省略該步驟。
$ curl -sfL https://raw.githubusercontent.com/clusterpedia-io/clusterpedia/main/hack/gen-clusterconfigs.sh | sh -
$ # 使用 kubectl 檢索多集群資源，當(dāng)前 Cluster API 未創(chuàng)建集群，所以返回空
$ kubectl --cluster clusterpedia api-resources
02使用 Cluster API創(chuàng)建集群
使用示例環(huán)境的 Docker Provider 來創(chuàng)建集群時，需要添加 --flavor development。
$ clusterctl generate cluster capi-quickstart --flavor development \
--kubernetes-version v1.24.2 \
--control-plane-machine-count=1 \
--worker-machine-count=1 \
> capi-quickstart.yaml
$ kubectl apply -f ./capi-quickstart.yaml
觀察集群創(chuàng)建情況
$ kubectl get cluster
NAME PHASE AGE VERSION
capi-quickstart Provisioned 8s v1.24.2
$ kubectl get kubeadmcontrolplane -w
NAME CLUSTER INITIALIZED API SERVER AVAILABLE REPLICAS READY UPDATED UNAVAILABLE AGE VERSION
capi-quickstart-2xcsz capi-quickstart true
當(dāng) kubeadmcontrolplane 的 Initialized 為 True 后，clusterpedia 會自動同步該集群內(nèi)的資源，可以使用 kubectl --cluster clusterpedia get po -A 來查看資源。
$ kubectl get pediacluster
NAME READY VERSION APISERVER
default-capi-quickstart True v1.24.2
$ kubectl --cluster clusterpedia get pod -A
NAMESPACE CLUSTER NAME READY STATUS RESTARTS AGE
kube-system default-capi-quickstart kube-apiserver-capi-quickstart-2xcsz-fxrrk 1/1 Running 0 2m32s
kube-system default-capi-quickstart kube-scheduler-capi-quickstart-2xcsz-fxrrk 1/1 Running 0 2m31s
kube-system default-capi-quickstart coredns-6d4b75cb6d-lrwj4 0/1 Pending 0 2m20s
kube-system default-capi-quickstart kube-proxy-p8v9m 1/1 Running 0 2m20s
kube-system default-capi-quickstart kube-controller-manager-capi-quickstart-2xcsz-fxrrk 1/1 Running 0 2m32s
kube-system default-capi-quickstart etcd-capi-quickstart-2xcsz-fxrrk 1/1 Running 0 2m32s
kube-system default-capi-quickstart kube-proxy-2ln2w 1/1 Running 0 105s
kube-system default-capi-quickstart coredns-6d4b75cb6d-2hcmz 0/1 Pending 0 2m20s
自動創(chuàng)建的 pediacluster 默認(rèn)的同步資源在 cluster-api clusterimportpolicy 中設(shè)置[13]，用戶也可以手動修改 pediacluster 中同步的配置， Synchronize Cluster Resources[14]。
在 Cluster API 中刪除集群時，Clusterpedia 也同步刪除 PeidaCluster，不會繼續(xù)同步該集群。
03對多個集群的資源檢索
使用上述步驟創(chuàng)建多個集群。
$ kubectl get cluster
NAME PHASE AGE VERSION
capi-quickstart Provisioned 10m v1.24.2
capi-quickstart-2 Provisioned 118s v1.24.2
$ kubectl get kubeadmcontrolplane
NAME CLUSTER INITIALIZED API SERVER AVAILABLE REPLICAS READY UPDATED UNAVAILABLE AGE VERSION
capi-quickstart-2-ctm9k capi-quickstart-2 true 1 1 1 10m v1.24.2
capi-quickstart-2xcsz capi-quickstart true 1 1 1 19m v1.24.2
$ # pediacluster 會根據(jù) cluster 資源自動創(chuàng)建
$ kubectl get pediacluster -o wide
NAME READY VERSION APISERVER VALIDATED SYNCHRORUNNING CLUSTERHEALTHY
default-capi-quickstart True v1.24.2 Validated Running Healthy
default-capi-quickstart-2 True v1.24.2 Validated Running Healthy
$ kubectl --cluster clusterpedia get no
CLUSTER NAME STATUS ROLES AGE VERSION
default-capi-quickstart-2 capi-quickstart-2-ctm9k-g2m87 NotReady control-plane 12m v1.24.2
default-capi-quickstart-2 capi-quickstart-2-md-0-s8hbx-7bd44554b5-kzcb6 NotReady <none> 11m v1.24.2
default-capi-quickstart capi-quickstart-2xcsz-fxrrk NotReady control-plane 21m v1.24.2
default-capi-quickstart capi-quickstart-md-0-9tw2g-b8b4f46cf-gggvq NotReady <none> 20m v1.24.2
clusterpedia 提供了兩種資源檢索方式
兼容 Kubernetes OpenAPI 的資源檢索[15]
$ kubectl --cluster clusterpedia get cm -A
NAMESPACE CLUSTER NAME DATA AGE
kube-system default-capi-quickstart extension-apiserver-authentication 6 19m
kube-system default-capi-quickstart kubeadm-config 1 19m
kube-public default-capi-quickstart cluster-info 2 19m
kube-system default-capi-quickstart kube-proxy 2 19m
kube-node-lease default-capi-quickstart kube-root-ca.crt 1 19m
kube-system default-capi-quickstart-2 extension-apiserver-authentication 6 10m
kube-system default-capi-quickstart kubelet-config 1 19m
kube-system default-capi-quickstart coredns 1 19m
kube-system default-capi-quickstart kube-root-ca.crt 1 19m
kube-public default-capi-quickstart kube-root-ca.crt 1 19m
kube-system default-capi-quickstart-2 coredns 1 10m
default default-capi-quickstart kube-root-ca.crt 1 19m
kube-system default-capi-quickstart-2 kube-proxy 2 10m
kube-system default-capi-quickstart-2 kubeadm-config 1 10m
kube-system default-capi-quickstart-2 kubelet-config 1 10m
kube-system default-capi-quickstart-2 kube-root-ca.crt 1 10m
kube-node-lease default-capi-quickstart-2 kube-root-ca.crt 1 10m
kube-public default-capi-quickstart-2 cluster-info 3 10m
kube-public default-capi-quickstart-2 kube-root-ca.crt 1 10m
default default-capi-quickstart-2 kube-root-ca.crt 1 10m
$ # gen cluster shortcuts
$ curl -sfL https://raw.githubusercontent.com/clusterpedia-io/clusterpedia/main/hack/gen-clusterconfigs.sh | sh -
$ kubectl --cluster default-capi-quickstart get cm -n kube-system
Collection Resource?[16]
$ kubectl get collectionresources
NAME RESOURCES
any *
workloads apps.deployments,apps.daemonsets,apps.statefulsets
kuberesources .*,admission.k8s.io.*,admissionregistration.k8s.io.*,apiextensions.k8s.io.*,apps.*,authentication.k8s.io.*,authorization.k8s.io.*,autoscaling.*,batch.*,certificates.k8s.io.*,coordination.k8s.io.*,discovery.k8s.io.*,events.k8s.io.*,extensions.*,flowcontrol.apiserver.k8s.io.*,imagepolicy.k8s.io.*,internal.apiserver.k8s.io.*,networking.k8s.io.*,node.k8s.io.*,policy.*,rbac.authorization.k8s.io.*,scheduling.k8s.io.*,storage.k8s.io.*
$ kubectl get collectionresources workloads
檢索條件
- 元信息過濾 (資源名稱，命名空間，集群，創(chuàng)建時間區(qū)間)[17]
$ kubectl --cluster clusterpedia get cm -A -l \
"search.clusterpedia.io/clusters in (default-capi-quickstart,default-capi-quickstart-2),\
search.clusterpedia.io/namespaces in (kube-system,default)"
NAMESPACE CLUSTER NAME DATA AGE
kube-system default-capi-quickstart extension-apiserver-authentication 6 23m
kube-system default-capi-quickstart kubeadm-config 1 23m
kube-system default-capi-quickstart kube-proxy 2 23m
kube-system default-capi-quickstart-2 extension-apiserver-authentication 6 14m
kube-system default-capi-quickstart kubelet-config 1 23m
kube-system default-capi-quickstart coredns 1 23m
kube-system default-capi-quickstart kube-root-ca.crt 1 23m
kube-system default-capi-quickstart-2 coredns 1 14m
default default-capi-quickstart kube-root-ca.crt 1 23m
kube-system default-capi-quickstart-2 kube-proxy 2 14m
kube-system default-capi-quickstart-2 kubeadm-config 1 14m
kube-system default-capi-quickstart-2 kubelet-config 1 14m
kube-system default-capi-quickstart-2 kube-root-ca.crt 1 14m
default default-capi-quickstart-2 kube-root-ca.crt 1 14m
- 模糊搜索[18]
- 增強的 Field Selector[19]
- 根據(jù)父輩或者祖輩 Owner 檢索[20]
- 分頁和排序[21]
- 自定義條件搜索[22]?
資料鏈接
[1] Clusterpedia 倉庫: https://github.com/clusterpedia-io/clusterpedia/tree/main/deploy/clusterimportpolicy
[2] ClusterImportPolicy: https://clusterpedia.io/zh-cn/docs/concepts/cluster-import-policy/
[3] 創(chuàng)建新的 ClusterImportPolicy: https://clusterpedia.io/docs/usage/interfacing-to-multi-cloud-platforms/#new-clusterimportpolicy
[4] clusterpedia#288: https://github.com/clusterpedia-io/clusterpedia/pull/288
[5] kubectl: https://kubernetes.io/docs/tasks/tools/install-kubectl/
[6] Kind: https://kind.sigs.k8s.io/
[7] Docker: https://www.docker.com/
[8] clusterctl: https://cluster-api.sigs.k8s.io/user/quick-start.html#install-clusterctl
[9] 詳見: https://github.com/clusterpedia-io/clusterpedia/tree/main/charts
[10] 集群自動導(dǎo)入策略: https://clusterpedia.io/docs/concepts/cluster-import-policy/
[11] Interfacing to Multi-Cloud Platforms: https://clusterpedia.io/docs/usage/interfacing-to-multi-cloud-platforms/
[12] 生成 kubectl cluster shortcut: https://clusterpedia.io/docs/usage/access-clusterpedia/#configure-the-cluster-shortcut-for-kubectl
[13] clusterimportpolicy 中設(shè)置: https://clusterpedia.io/docs/concepts/cluster-import-policy/#pediacluster-template
[14] Synchronize Cluster Resources: https://clusterpedia.io/docs/usage/sync-resources/
[15] 兼容 Kubernetes OpenAPI 的資源檢索: https://clusterpedia.io/zh-cn/docs/usage/access-clusterpedia/#%E8%AE%BF%E9%97%AE-clusterpedia-%E8%B5%84%E6%BA%90
[16] Collection Resource: https://clusterpedia.io/zh-cn/docs/concepts/collection-resource/
[17] 元信息過濾 (資源名稱，命名空間，集群，創(chuàng)建時間區(qū)間): https://clusterpedia.io/docs/usage/search/#search-by-metadata
[18] 模糊搜索: https://clusterpedia.io/docs/usage/search/multi-cluster/#fuzzy-search
[19] 增強的 Field Selector: https://clusterpedia.io/docs/usage/search/multi-cluster/#field-selector
[20] 根據(jù)父輩或者祖輩 Owner 檢索: https://clusterpedia.io/docs/usage/search/multi-cluster/#search-by-parent-or-ancestor-owner
[21] 分頁和排序: https://clusterpedia.io/docs/usage/search/multi-cluster/#paging-and-sorting
[22] 自定義條件搜索: https://clusterpedia.io/docs/us?age/search/#advanced-searchcustom-conditional-search
