Gatekeeper
6 分钟阅读
简要概述
各公司针对各自 K8S 集群都有一些合规政策,为了满足运维、安全、合规等要求。如果试图通过手动确保合规性就有可能会出现错误,所以需通过自动化方式执行确保一致性,即时反馈、并允许开发人员独立运作而无需牺牲合规性来提高灵活性。
Kubernetes 是允许通过 Admission Controller Webhooks 准入控制器将策略决策与 API Server 的内部工作解耦,这些 Webhooks 在每次创建、更新或删除资源时执行。如果各个规则均需要编写一个对应的准入控制器,难免有些复杂不够灵活,所以产生了通用的策略服务组件,也就是 Gatekeeper。
Gatekeeper
它支持验证性质的准入 Webhook ValidatingAdmissionWebhook 与变更性质的准入 Webhook MutatingAdmissionWebhook,通过由 Open Policy Agent(OPA)执行预先编写的 rego 策略规则,基于 CRD 的方式数据持久存储在 etcd 中。而 OPA 是一个在云原生环境中执行的策略引擎,由 CNCF 作为一个已毕业项目进行托管。
它的基础是基于 OPA Constraint Framework 框架实现,主要包含以下概念:
- ConstraintTemplate:是 Gatekeeper 中的一个自定义资源,用于定义一组策略规则,每个模版包含了用于验证和强制执行策略的 Rego 代码和相应的参数模式。
- Constraint:是 ConstraintTemplate 的实例,用于具体应用策略,每个对象定义了将被验证的 Kubernetes 对象类型、相应的参数和用于验证的 Rego 代码,当然必须遵循 ConstraintTemplate 中定义的 crd 属性与规则。
其中 Constraint 也是以 crd 在 k8s 集群中体现,它是由 ConstraintTemplate 模版自动创建的。待 Gatekeeper 安装后默认存在 crd 列表如下:
assign.mutations.gatekeeper.sh
assignimage.mutations.gatekeeper.sh
assignmetadata.mutations.gatekeeper.sh
configs.config.gatekeeper.sh
constraintpodstatuses.status.gatekeeper.sh
constrainttemplatepodstatuses.status.gatekeeper.sh
constrainttemplates.templates.gatekeeper.sh
expansiontemplate.expansion.gatekeeper.sh
expansiontemplatepodstatuses.status.gatekeeper.sh
modifyset.mutations.gatekeeper.sh
mutatorpodstatuses.status.gatekeeper.sh
providers.externaldata.gatekeeper.sh
其中来自于 OPA Constraint Framework 框架有 “expansiontemplate.expansion.gatekeeper.sh” 以及使用 “constraints.gatekeeper.sh” 后缀的 crd,如:
k8srequiredlabels.constraints.gatekeeper.sh
配置示例
ConstraintTemplate
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
name: foosystemrequiredlabels
spec:
# 自动创建 foosystemrequiredlabels.constraints.gatekeeper.sh 类型
crd:
spec:
names:
kind: FooSystemRequiredLabels
validation:
# 这里的属性会添加到自动创建 crd 的 parameters 参数下
openAPIV3Schema:
type: object
properties:
labels:
type: array
items:
type: string
targets:
- target: admission.k8s.gatekeeper.sh
libs:
- |
package lib.helpers
make_message(missing) = msg {
msg := sprintf("you must provide labels: %v", [missing])
}
rego: |
package foosystemrequiredlabels
import data.lib.helpers
violation[{"msg": msg, "details": {"missing_labels": missing}}] {
provided := {label | input.request.object.metadata.labels[label]}
required := {label | label := input.parameters.labels[_]}
missing := required - provided
count(missing) > 0
msg := helpers.make_message(missing)
}
Constraint
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
name: ns-must-have-gk
spec:
match:
kinds:
- apiGroups: [""]
kinds: ["Namespace"]
parameters:
labels: ["gatekeeper"]
enforcementAction: "deny"
Rego 引用变量
请求评估的对象
input.review
对象存储正在评估的 Admission 请求,数据结构可参考 AdmissionRequest。
用户自定义属性
input.parameters
对象存储了用户在 “ConstraintTemplate” 中定义的属性,既 “crd.spec.validation.openAPIV3Schema.properties” 这配置的值。
数据结构
ConstraintTemplate
在代码仓库 github.com/open-policy-agent/frameworks
中定义:
// ConstraintTemplate is the Schema for the constrainttemplates API
// +k8s:openapi-gen=true
// +k8s:conversion-gen-external-types=github.com/open-policy-agent/frameworks/constraint/pkg/apis/templates
type ConstraintTemplate struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec ConstraintTemplateSpec `json:"spec,omitempty"`
Status ConstraintTemplateStatus `json:"status,omitempty"`
}
// ConstraintTemplateSpec defines the desired state of ConstraintTemplate.
type ConstraintTemplateSpec struct {
CRD CRD `json:"crd,omitempty"`
Targets []Target `json:"targets,omitempty"`
}
type CRD struct {
Spec CRDSpec `json:"spec,omitempty"`
}
type CRDSpec struct {
Names Names `json:"names,omitempty"`
// +kubebuilder:default={legacySchema: false}
Validation *Validation `json:"validation,omitempty"`
}
type Names struct {
Kind string `json:"kind,omitempty"`
ShortNames []string `json:"shortNames,omitempty"`
}
type Validation struct {
// +kubebuilder:validation:Schemaless
// +kubebuilder:validation:Type=object
// +kubebuilder:pruning:PreserveUnknownFields
// +k8s:conversion-gen=false
OpenAPIV3Schema *apiextensionsv1.JSONSchemaProps `json:"openAPIV3Schema,omitempty"`
// +kubebuilder:default=false
LegacySchema *bool `json:"legacySchema,omitempty"` // *bool allows for "unset" state which we need to apply appropriate defaults
}
type Target struct {
Target string `json:"target,omitempty"`
Rego string `json:"rego,omitempty"`
Libs []string `json:"libs,omitempty"`
// The source code options for the constraint template. "Rego" can only
// be specified in one place (either here or in the "rego" field)
// +listType=map
// +listMapKey=engine
// +kubebuilder:validation:Required
Code []Code `json:"code,omitempty"`
}
type Code struct {
// The engine used to evaluate the code. Example: "Rego". Required.
// +kubebuilder:validation:Required
Engine string `json:"engine"`
// +kubebuilder:validation:Required
// +kubebuilder:validation:Schemaless
// +kubebuilder:pruning:PreserveUnknownFields
// The source code for the template. Required.
Source *templates.Anything `json:"source"`
}
Constraint
是定义在 “ConstraintTemplate” 中自动生成的 CRD,查看该集群存在多少可通过以下指令:
kubectl get constraints
创建该资源 crd 的逻辑在 github.com/open-policy-agent/frameworks/constraint/pkg/client/crds/crds.go
中 “CreateCRD“ 根据 “ConstraintTemplate” 中的 “openAPIV3Schema” 定义并把参数添加到 “parameters” 里面,然后在集群在把对应的 “Constraint” CRD 创建出来,大致设计以下代码片段:
github.com/open-policy-agent/frameworks/constraint/pkg/client/crds/schema.go -> CreateSchema -> apiextensions.JSONSchemaProps
github.com/open-policy-agent/frameworks/constraint/pkg/client/crds/crds.go -> CreateCRD -> apiextensions.CustomResourceDefinition
apiextensions.JSONSchemaProps -> apiextensions.CustomResourceDefinition
每个 “Constraint” CRD 都存在以下三个参数,在 CreateSchema
方法中定义:
参数 | 类型 |
---|---|
match | target.MatchSchema() |
enforcementAction | string |
parameters | ConstraintTemplate.Spec.CRD.Spec.Validation.OpenAPIV3Schema |
“match” 字段定义了约束将被应用于哪些资源,它支持以下类型的匹配器:
- “kinds” 接受一个包含 apiGroups 和 kinds 字段的对象列表,列出了约束将应用于的对象的组或种类。如果指定了多个组或种类,则只需要一个匹配项,资源就会处于范围内。
- “scope” 确定是否匹配集群范围或命名空间范围的资源。接受 *, Cluster 或 Namespaced。(默认为 *)
- “namespaces” 是一个命名空间名称的列表。如果定义了,约束仅适用于列出的命名空间中的资源。Namespaces 还支持基于前缀的通配符。例如,namespaces: [kube-*] 匹配 kube-system 和 kube-public。
- “excludedNamespaces” 是一个命名空间名称的列表。如果定义了,约束仅适用于不在列出的命名空间中的资源。excludedNamespaces 也支持基于前缀的通配符。例如,excludedNamespaces: [kube-*] 匹配不在 kube-system 和 kube-public 中的资源。
- “labelSelector” 是两个可选字段的组合:matchLabels 和 matchExpressions。这两个字段提供了基于对象元数据中包含的标签键和值选择或排除 Kubernetes 对象的不同方法。所有选择表达式都会进行 AND 运算,以确定对象是否符合选择器的累积要求。
- “namespaceSelector” 是针对对象所在的命名空间或对象本身(如果对象是一个命名空间)的标签选择器。
- “name” 是 Kubernetes 对象的名称。如果定义了,它将与具有指定名称的对象匹配。Name 也支持基于前缀的通配符。例如,name: pod-* 匹配 pod-a 和 pod-b。
“enforcementAction” 字段定义了处理约束违规的操作,支持 [deny, dryrun, warn] 三个取值,默认情况下被设置为 deny,因为默认行为是拒绝任何违规的 Admission 请求。
“parameters” 字段描述了约束的意图,它可以被 “ConstraintTemplate” 的 Rego 源代码中 input.parameters
引用。
- CreateSchema
github.com/open-policy-agent/frameworks/constraint/pkg/client/crds/schema.go
// CreateSchema combines the schema of the match target and the ConstraintTemplate parameters
// to form the schema of the actual constraint resource.
func CreateSchema(templ *templates.ConstraintTemplate, target MatchSchemaProvider) *apiextensions.JSONSchemaProps {
props := map[string]apiextensions.JSONSchemaProps{
"match": target.MatchSchema(),
"enforcementAction": {Type: "string"},
}
if templ.Spec.CRD.Spec.Validation != nil && templ.Spec.CRD.Spec.Validation.OpenAPIV3Schema != nil {
internalSchema := *templ.Spec.CRD.Spec.Validation.OpenAPIV3Schema.DeepCopy()
props["parameters"] = internalSchema
}
schema := &apiextensions.JSONSchemaProps{
Type: "object",
Properties: map[string]apiextensions.JSONSchemaProps{
"metadata": {
Type: "object",
Properties: map[string]apiextensions.JSONSchemaProps{
"name": {
Type: "string",
MaxLength: pointer.Int64(63),
},
},
},
"spec": {
Type: "object",
Properties: props,
},
"status": {
XPreserveUnknownFields: pointer.Bool(true),
},
},
}
return schema
}
- target.MatchSchema
github.com/open-policy-agent/gatekeeper/pkg/mutation/match/match_types.go github.com/open-policy-agent/gatekeeper/pkg/target/matchcrd_constant.go
// Match selects which objects are in scope.
// +kubebuilder:object:generate=true
type Match struct {
// Source determines whether generated or original resources are matched.
// Accepts `Generated`|`Original`|`All` (defaults to `All`). A value of
// `Generated` will only match generated resources, while `Original` will only
// match regular resources.
// +kubebuilder:validation:Enum=All;Generated;Original
Source string `json:"source,omitempty"`
Kinds []Kinds `json:"kinds,omitempty"`
// Scope determines if cluster-scoped and/or namespaced-scoped resources
// are matched. Accepts `*`, `Cluster`, or `Namespaced`. (defaults to `*`)
Scope apiextensionsv1.ResourceScope `json:"scope,omitempty"`
// Namespaces is a list of namespace names. If defined, a constraint only
// applies to resources in a listed namespace. Namespaces also supports a
// prefix or suffix based glob. For example, `namespaces: [kube-*]` matches both
// `kube-system` and `kube-public`, and `namespaces: [*-system]` matches both
// `kube-system` and `gatekeeper-system`.
Namespaces []wildcard.Wildcard `json:"namespaces,omitempty"`
// ExcludedNamespaces is a list of namespace names. If defined, a
// constraint only applies to resources not in a listed namespace.
// ExcludedNamespaces also supports a prefix or suffix based glob. For example,
// `excludedNamespaces: [kube-*]` matches both `kube-system` and
// `kube-public`, and `excludedNamespaces: [*-system]` matches both `kube-system` and
// `gatekeeper-system`.
ExcludedNamespaces []wildcard.Wildcard `json:"excludedNamespaces,omitempty"`
// LabelSelector is the combination of two optional fields: `matchLabels`
// and `matchExpressions`. These two fields provide different methods of
// selecting or excluding k8s objects based on the label keys and values
// included in object metadata. All selection expressions from both
// sections are ANDed to determine if an object meets the cumulative
// requirements of the selector.
LabelSelector *metav1.LabelSelector `json:"labelSelector,omitempty"`
// NamespaceSelector is a label selector against an object's containing
// namespace or the object itself, if the object is a namespace.
NamespaceSelector *metav1.LabelSelector `json:"namespaceSelector,omitempty"`
// Name is the name of an object. If defined, it will match against objects with the specified
// name. Name also supports a prefix or suffix glob. For example, `name: pod-*` would match
// both `pod-a` and `pod-b`, and `name: *-pod` would match both `a-pod` and `b-pod`.
Name wildcard.Wildcard `json:"name,omitempty"`
}
// Kinds accepts a list of objects with apiGroups and kinds fields
// that list the groups/kinds of objects to which the mutation will apply.
// If multiple groups/kinds objects are specified,
// only one match is needed for the resource to be in scope.
// +kubebuilder:object:generate=true
type Kinds struct {
// APIGroups is the API groups the resources belong to. '*' is all groups.
// If '*' is present, the length of the slice must be one.
// Required.
APIGroups []string `json:"apiGroups,omitempty" protobuf:"bytes,1,rep,name=apiGroups"`
Kinds []string `json:"kinds,omitempty"`
}
apiextensionsv1.JSONSchemaProps
TODO;
templates.Anything
TODO;