Using Templates to Implement Policy-Based Governance at Scale
As mentioned in this weblog post, Red Hat Advanced Cluster Management for Kubernetes (RHACM) governance affords an effective coverage framework for governing configurations to the preferred nation wanted for high-quality practices throughout your fleet of controlled clusters. A ConfigurationPolicy object, wherein configurations are targeted to the coverage framework, turned into formerly static and the identical definition turned into implemented throughout all controlled clusters to the coverage turned into deployed. With the advent of the templates feature, you could now outline bendy configuration guidelines with variable content. Templatization permits coverage-primarily based totally governance to be implemented at scale as an unmarried coverage is customized, as opposed to having to create more than one guideline for controlled clusters that require barely one-of-a-kind behavior.
In this post, I show off the blessings of templates and the use of diverse template capabilities examples.
Templates in coverage definitions have to comply with the Go textual content template specification. You can specify the templates to be processed at the controlled cluster or the hub cluster.
- Introduced in RHACM 2.3, templates delimited via way of means of ‘’ are processed at the goal controlled cluster via way of means of the configuration coverage controller simply earlier than the configuration is enforced or monitored.
- Added in RHACM 2.4, templates delimited via way of means of ‘’ are processed at the hub cluster via way of means of the coverage controller simply earlier than it’s far propagated to the controlled clusters for enforcement.
Along with the cap potential to encompass Go templates, the coverage framework additionally gives diverse custom template capabilities for inclusion withinside the coverage definition. View the subsequent attributes of the definition:
- Resource unique capabilities like ‘fromConfigMap(ns, name, field)’ and others to reference the contents of unique aid gadgets at the cluster.
- A generic ‘lookup’ characteristic to reference any item at the cluster.
- Several application capabilities e.g. base64enc, base64dec, etc.
- Objects referenced in custom capabilities are retrieved from the cluster in which the template is being processed.
- Any item at the controlled cluster may be referenced withinside the templates, however, at the hub for getting right of entry to control, templates in a Policy item can best reference gadgets which are withinside the equal namespaces because of the Policy item.
- For extra facts approximately the coverage templates function and the entire listing of custom capabilities, see the product documentation.
Customizing Configuration to the Target Managed Cluster
Consider an instance situation wherein a Deployment item for Nginx pods needs to be enforced at the controlled cluster. The contents of the Deployment, however, rely upon the goal-controlled cluster configuration (i.e.,):
- The Nginx field photo model relies upon at the Red Hat OpenShift model of the controlled cluster.
- The quantity of replicas to configure withinside the Deployment relies upon the surroundings form of the cluster e.g. production, dev, or take a look at the cluster, and so on.
Without templates, this requires (a) understanding the controlled cluster configuration upfront, (b) hardcoding the field photo and replicas values withinside the coverage definition, and (c) defining a separate coverage for every goal-controlled cluster with a one-of-a-kind configuration.
SOLUTION WITH TEMPLATES
Typically, the Red Hat OpenShift model and surroundings kind are to be had as labels at the controlled cluster useful resource item. Every such label outcomes in a cluster.open-cluster-management.io.ClusterClaim item at the controlled cluster. The coverage for the Nginx Deployment may be described by the use of templates as follows to dynamically install the values of the controlled cluster particular configuration:
- Include ManagedCluster templates to get the right of entry to the ClusterClaims as those items are at the controlled cluster.
- Use the fromClusterClaim “model.openshift” custom template characteristic to get the model fee of the goal-controlled cluster.
- Use the fromClusterClaim “surroundings” custom template characteristic to get the surroundings kind of the goal-controlled cluster.
- If/else template constructs helps to select among custom and default values.
… apiVersion: policy.open-cluster-management.io/v1 kind: ConfigurationPolicy metadata: name: policy-nginx-deployment spec: remediationAction: enforce … object-templates: - complianceType: musthave objectDefinition: apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment-templatized … spec: # if label env == prod, set replicas to 3 else 1 replicas: ' 3 1 | toInt }}' … spec: containers: - name: nginx # if ocp version is 4.9.13 set nginx to v 1.21.4 else v 1.20.2 image: ' nginx:1.21.4 nginx:1.20.2 ' ports: - containerPort: 80 …
In the preceding instance, templates may be used to keep away from hardcoding the values and create coverage definitions custom designed to the goal cluster. The whole coverage may be discovered here. Another instance of this state of affairs is coverage that configures the OpenShift LogForwarder, wherein controlled cluster unique records have likewise described the use of templates.
Managing the Configuration of a Large Fleet of Managed Clusters: Improving Scalability
Consider the state of affairs wherein the configuration of a community’s useful resource need to be controlled on a big fleet of a thousand controlled clusters. The configuration is often equal throughout the fleet besides for values of some settings that change for every cluster (i.e.,):
- A SriovNetwork item needs to be enforced throughout the complete fleet of clusters.
- Network VLAN IDs and useful resource names range for every controlled cluster.
Without templates, reaching this calls for growing a separate coverage for every controlled cluster with its personal unique values for VLAN ID and useful resource name (i.e., for a thousand controlled clusters); it calls for deploying a thousand coverage gadgets and an extra a thousand placement assets to be created at the hub and propagated to the controlled clusters only for the enforcement of an unmarried SriovNetwork item.
SOLUTION WITH TEMPLATES
This may be completed via unmarried coverage through storing variable configuration values of VLAN ID and useful resource calls at the hub cluster and such as hub templates withinside the coverage definition to retrieve the values in line with a controlled cluster at runtime.
- On the hub cluster, create a ConfigMap with all of the variable statistics for every goal cluster:
# Configmap with values of variable data for each target cluster kind: ConfigMap apiVersion: v1 metadata: name: sno-master-config namespace: sites-sub data: cluster0001-interface: "ens5f0" cluster0001-resourceName: "du_fh" cluster0001-vlan: "3620" cluster0002-interface: "ens5f0" cluster0002-resourceName: "du_mh" cluster0002-vlan: "3621" …
- Define the coverage which includes Hub templates to retrieve managed-cluster unique values for the VLAN ID and aid call from the previous configmap.
- The destination managed cluster’s name, .managedClusterName, is a template context value that is set at runtime and processed by the hub cluster controller for propagation.
- Create a Policy object in the same namespace as the ConfigMap due to access control restrictions:
# policy that enforces the network resource apiVersion: policy.open-cluster-management.io/v1 kind: Policy metadata: name: policy-site-nw-templatized namespace: sites-sub … spec: remediationAction: enforce policy-templates: - objectDefinition: apiVersion: policy.open-cluster-management.io/v1 kind: ConfigurationPolicy metadata: name: policy-site-nw-templatized spec: remediationAction: enforce … object-templates: - complianceType: musthave objectDefinition: apiVersion: sriovnetwork.openshift.io/v1 kind: SriovNetwork metadata: name: sriov-nw namespace: openshift-sriov-network-operator spec: networkNamespace: openshift-sriov-network-operator # use hub templates with fromConfigMap custom function templates to retrieve values from site-config configmap resourceName: '' vlan: '' ...
As proven withinside the preceding pattern, the use of templates notably reduces the wide variety of aid items required to manipulate the configuration from a thousand Policy items to one, and a thousand Placement items to one, thereby making it clean to create and hold them and additionally substantially lowering the weight at the system. The whole coverage definition may be located here, and every other instance of this situation is the coverage-autoscaler-templatized.yaml pattern coverage that demonstrates configuring ClusterAutoScaler.
Securely Managing and Distributing Sensitive Data
Consider a state of affairs in which the configuration of a few touchy records should be controlled at the far-off clusters, for example:
- Database credentials are saved externally in a mystery control gadget like HashiCorp Vault.
- These credentials should be allotted for your fleet of controlled clusters to be used with the aid of using a utility workload strolling at the clusters.
Without templates, such touchy records can’t be deployed to controlled clusters the usage of regulations, as Policy gadgets aren’t ETCD encrypted, and it isn’t encouraged to shop such touchy records in plain-textual content inside regulations saved withinside the Git repository and deployed via GitOps. Also, standard protection pleasant exercise is to dynamically retrieve touchy records from a secrets and techniques control gadget.
SOLUTION WITH TEMPLATES
This may be finished via way of means of making the touchy records to be had at the hub cluster as a Kubernetes Secret item and the usage of hub cluster templates withinside the coverage definition to retrieve the values at runtime and securely propagate secrets and techniques to the controlled clusters.
- Using an operator like external secrets and techniques, retrieve the database credentials from Hashicorp Vault and shop them at the Hub as a Secret useful resource item.
- Use the form mystery template feature to reference the values of the database credentials Secret item from the preceding step.
- At runtime, the hub cluster retrieves the database credentials from the Secret useful resource item, encrypts the touchy record’s string, replaces the template with the encrypted string, and propagates the up-to-date Policy definition to the controlled cluster.
- On the controlled cluster, touchy records withinside the Policy definition are decrypted earlier than being enforced locally.
View the subsequent Secret example:
# example secret resource that needs to be delivered to the remote managed cluster … apiVersion: v1 kind: Secret metadata: name: db-creds … type: Opaque data: db_username: XXX db_password: YYY … # policy that delivers and enforces the secret on the remote managed cluster apiVersion: policy.open-cluster-management.io/v1 kind: Policy metadata: name: policy-securesecret … spec: … policy-templates: - objectDefinition: apiVersion: policy.open-cluster-management.io/v1 kind: ConfigurationPolicy metadata: name: policy-securesecret spec: … object-templates: - complianceType: musthave objectDefinition: apiVersion: v1 kind: Secret metadata: name: cloudconnectioncreds namespace: default type: Opaque data: db_USER: '' db_PASS: ''
In the previous example, the Policy definition shows how to securely distribute secrets from external secret management synced with the hub cluster to the managed clusters. The application workloads on the cluster can then access the database using this secret. Additionally, any modifications to the private information synced from outside secret management systems after the Policy object is generated are not automatically refreshed on the managed cluster until the Policy object is updated on the hub cluster.
As demonstrated in this post, templates have a number of advantages, including preventing hardcoding of values in policy definitions, tailoring rules to the target managed cluster, and safely referencing and dispersing sensitive data in your policies. In all circumstances, it lowers the number of Policy objects required to manage configuration in systems with a sizable fleet of managed clusters, enhancing usability and allowing the widespread implementation of policy-based governance.
For further examples of policies that make use of the functionality of the template, visit the repository for the policy collection.