Our Beta is live - Join Now for Free
Tech

How to change a ClusterRole (without breaking the cluster)

Dylan Ratcliffe
October 26, 2023

In Kubernetes, a ClusterRole and a Role define authorization rules to access Kubernetes resources either on a cluster level (ClusterRole) or on a particular namespace (Role).

A ClusterRole is a non-namespaced resource that allows you to define permissions across the entire cluster. It can grant access to resources like nodes, namespaces, or persistent volumes that exist at the cluster level, independent of any particular namespace. On the other hand, a Role operates within the boundary of a particular namespace and is used to grant permissions on resources such as Pods, Services, or ConfigMaps that exist within that namespace.

This ClusterRole for example allows read access to all pods in the cluster:

kind: ClusterRoleapiVersion: rbac.authorization.k8s.io/v1metadata:  name: pod-readerrules:- apiGroups: [""]  resources: ["pods"]  verbs: ["get", "watch", "list"]

And this Role allows writing (creating, updating, deleting) to pods in a particular namespace (my-namespace for this instance):

kind: RoleapiVersion: rbac.authorization.k8s.io/v1metadata:  namespace: my-namespace  name: pod-writerrules:- apiGroups: [""]  resources: ["pods"]  verbs: ["create", "update", "delete"]

In order for a workload (Pod) to be able to use these permissions:

  • That pod must have a ServiceAccount
  • The ServiceAccount must have a RoleBinding/ClusterRoleBinding that reference the Role/ClusterRole

However Pods aren’t used directly in Kubernetes (usually), they are instead controlled by Deployments, ReplicaSets, DaemonSets, Jobs or CronJobs and therefore in order to truly understand the impact of changing a Role/ClusterRole, we need to link all the way back to these controlling resources. This means that the whole relationship looks like this:

This means that in order to work out the true potential impact of changing a Role/ClusterRole, we must follow all of the relationships in this diagram. Here are the commands you need:

Get the details of a role:

kubectl describe clusterrole {role-name}
Name:         srcman-proxy-role
Labels:       {none}
Annotations:  {none}
PolicyRule:
  Resources                                  Non-Resource URLs  Resource Names  Verbs
  ---------                                  -----------------  --------------  -----
  tokenreviews.authentication.k8s.io         []                 []              [create]
  subjectaccessreviews.authorization.k8s.io  []                 []              [create] 

Get the bindings that refer to that role

kubectl get clusterrolebinding -o custom-columns=NAME:.metadata.name,ROLE:.roleRef.name --all-namespaces | grep {role-name}srcman-proxy-rolebinding                               srcman-proxy-role 

Get the ServiceAccounts that use a given ClusterRoleBinding bindings

kubectl get clusterrolebinding  -o=jsonpath='{range .subjects[?(@.kind=="ServiceAccount")]}{@.name}{"\n"}{end}' --all-namespacessrcman-controller-manager 

Get the Pods that use that ServiceAccount

❯ kubectl get pods --all-namespaces -o=jsonpath='{range .items[?(@.spec.serviceAccountName=="")]}namespace={.metadata.namespace} pod={.metadata.name} ownerKind={.metadata.ownerReferences[0].kind} ownerName={.metadata.ownerReferences[0].name}{"\n"}{end}'
namespace=srcman-system pod=srcman-controller-manager-767496f48-fvbgh ownerKind=ReplicaSet ownerName=srcman-controller-manager-767496f48 

Note that in the above command we produce the ownerKind and ownerName columns. These show what type of resource owns this pod, e.g. a ReplicaSet, DaemonSet, Job or CronJob

Get the CronJob that controls a given Job:

kubectl get job  -n  -o=jsonpath='{.metadata.ownerReferences[0].name}{"\n"}'srcman-controller-cronjob 

Get the CronJob that controls a given Job:

kubectl get job  -n  -o=jsonpath='{.metadata.ownerReferences[0].name}{"\n"}'srcman-controller-cronjob 

Get the Deployment that controls a ReplicaSet:

kubectl get rs  -n  -o=jsonpath='{.metadata.ownerReferences[0].name}{"\n"}'srcman-controller-manager 

Get the details of a deployment:

❯ kubectl describe deployment  -n 
Name:                   srcman-controller-manager
Namespace:              srcman-system
CreationTimestamp:      Fri, 16 Jun 2023 15:23:36 +0100
Labels:                 control-plane=controller-manager
Annotations:            deployment.kubernetes.io/revision: 2
Selector:               control-plane=controller-manager
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:           control-plane=controller-manager
  Service Account:  srcman-controller-manager
  Containers:
   kube-rbac-proxy:
    Image:      gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0
    Port:       8443/TCP
    Host Port:  0/TCP
    Args:
      --secure-listen-address=0.0.0.0:8443
      --upstream=http://127.0.0.1:8080/
      --logtostderr=true
      --v=10
    Environment:  
    Mounts:       
   manager:
    Image:      ghcr.io/overmindtech/srcman:0.13.0
    Port:       
    Host Port:  
    Command:
      /manager
    Args:
      --health-probe-bind-address=:8081
      --metrics-bind-address=127.0.0.1:8080
      --leader-elect
    Limits:
      cpu:     100m
      memory:  30Mi
    Requests:
      cpu:      100m
      memory:   20Mi
    Liveness:   http-get http://:8081/healthz delay=15s timeout=1s period=20s #success=1 #failure=3
    Readiness:  http-get http://:8081/readyz delay=5s timeout=1s period=10s #success=1 #failure=3
    Environment Variables from:
      tracing-keys  Secret  Optional: true
    Environment:    
    Mounts:         
  Volumes:          
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Progressing    True    NewReplicaSetAvailable
  Available      True    MinimumReplicasAvailable
OldReplicaSets:  srcman-controller-manager-8485cff9d6 (0/0 replicas created)
NewReplicaSet:   srcman-controller-manager-767496f48 (1/1 replicas created)
Events:            

With Overmind

Overmind allows you to understand the potential impact of changes to ClusterRoles, Deployments and any other Kubernetes or AWS resource before you make them, much more quickly than using the methods above. In order to calculate the impact of a change to a ClusterRole, I simply create a new change, and select the relevant ClusterRole as the expected change. From there Overmind automatically determines the potential impact in real time:

Overmind Kubernetes Graph

Related Blogs