Gatekeeper

简要概述

各公司针对各自 K8S 集群都有一些合规政策,为了满足运维、安全、合规等要求。如果试图通过手动确保合规性就有可能会出现错误,所以需通过自动化方式执行确保一致性,即时反馈、并允许开发人员独立运作而无需牺牲合规性来提高灵活性。

Kubernetes 是允许通过 Admission Controller Webhooks 准入控制器将策略决策与 API Server 的内部工作解耦,这些 Webhooks 在每次创建、更新或删除资源时执行。如果各个规则均需要编写一个对应的准入控制器,难免有些复杂不够灵活,所以产生了通用的策略服务组件,也就是 Gatekeeper。

Gatekeeper

它支持验证性质的准入 Webhook ValidatingAdmissionWebhook 与变更性质的准入 Webhook MutatingAdmissionWebhook,通过由 Open Policy Agent(OPA)执行预先编写的 rego 策略规则,基于 CRD 的方式数据持久存储在 etcd 中。而 OPA 是一个在云原生环境中执行的策略引擎,由 CNCF 作为一个已毕业项目进行托管。

它的基础是基于 OPA Constraint Framework 框架实现,主要包含以下概念:

  1. ConstraintTemplate:是 Gatekeeper 中的一个自定义资源,用于定义一组策略规则,每个模版包含了用于验证和强制执行策略的 Rego 代码和相应的参数模式。
  2. Constraint:是 ConstraintTemplate 的实例,用于具体应用策略,每个对象定义了将被验证的 Kubernetes 对象类型、相应的参数和用于验证的 Rego 代码,当然必须遵循 ConstraintTemplate 中定义的 crd 属性与规则。

其中 Constraint 也是以 crd 在 k8s 集群中体现,它是由 ConstraintTemplate 模版自动创建的。待 Gatekeeper 安装后默认存在 crd 列表如下:

assign.mutations.gatekeeper.sh
assignimage.mutations.gatekeeper.sh
assignmetadata.mutations.gatekeeper.sh
configs.config.gatekeeper.sh
constraintpodstatuses.status.gatekeeper.sh
constrainttemplatepodstatuses.status.gatekeeper.sh
constrainttemplates.templates.gatekeeper.sh
expansiontemplate.expansion.gatekeeper.sh
expansiontemplatepodstatuses.status.gatekeeper.sh
modifyset.mutations.gatekeeper.sh
mutatorpodstatuses.status.gatekeeper.sh
providers.externaldata.gatekeeper.sh

其中来自于 OPA Constraint Framework 框架有 “expansiontemplate.expansion.gatekeeper.sh” 以及使用 “constraints.gatekeeper.sh” 后缀的 crd,如:

k8srequiredlabels.constraints.gatekeeper.sh

配置示例

ConstraintTemplate

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: foosystemrequiredlabels
spec:
  # 自动创建 foosystemrequiredlabels.constraints.gatekeeper.sh 类型
  crd:
    spec:
      names:
        kind: FooSystemRequiredLabels
      validation:
        # 这里的属性会添加到自动创建 crd 的 parameters 参数下
        openAPIV3Schema:
          type: object
          properties:
            labels:
              type: array
              items:
                type: string

  targets:
  - target: admission.k8s.gatekeeper.sh
    libs:
      - |
        package lib.helpers

        make_message(missing) = msg {
          msg := sprintf("you must provide labels: %v", [missing])
        }        

    rego: |
      package foosystemrequiredlabels

      import data.lib.helpers

      violation[{"msg": msg, "details": {"missing_labels": missing}}] {
        provided := {label | input.request.object.metadata.labels[label]}
        required := {label | label := input.parameters.labels[_]}
        missing := required - provided
        count(missing) > 0
        msg := helpers.make_message(missing)
      }      

Constraint

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: ns-must-have-gk
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Namespace"]
  parameters:
    labels: ["gatekeeper"]
  enforcementAction: "deny"

Rego 引用变量

请求评估的对象

input.review 

对象存储正在评估的 Admission 请求,数据结构可参考 AdmissionRequest

用户自定义属性

input.parameters

对象存储了用户在 “ConstraintTemplate” 中定义的属性,既 “crd.spec.validation.openAPIV3Schema.properties” 这配置的值。

数据结构

ConstraintTemplate

在代码仓库 github.com/open-policy-agent/frameworks 中定义:

https://github.com/open-policy-agent/frameworks/blob/master/constraint/pkg/apis/templates/v1/constrainttemplate_types.go

// ConstraintTemplate is the Schema for the constrainttemplates API
// +k8s:openapi-gen=true
// +k8s:conversion-gen-external-types=github.com/open-policy-agent/frameworks/constraint/pkg/apis/templates
type ConstraintTemplate struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`

    Spec   ConstraintTemplateSpec   `json:"spec,omitempty"`
    Status ConstraintTemplateStatus `json:"status,omitempty"`
}

// ConstraintTemplateSpec defines the desired state of ConstraintTemplate.
type ConstraintTemplateSpec struct {
    CRD     CRD      `json:"crd,omitempty"`
    Targets []Target `json:"targets,omitempty"`
}

type CRD struct {
    Spec CRDSpec `json:"spec,omitempty"`
}

type CRDSpec struct {
    Names Names `json:"names,omitempty"`
    // +kubebuilder:default={legacySchema: false}
    Validation *Validation `json:"validation,omitempty"`
}

type Names struct {
    Kind       string   `json:"kind,omitempty"`
    ShortNames []string `json:"shortNames,omitempty"`
}

type Validation struct {
    // +kubebuilder:validation:Schemaless
    // +kubebuilder:validation:Type=object
    // +kubebuilder:pruning:PreserveUnknownFields
    // +k8s:conversion-gen=false
    OpenAPIV3Schema *apiextensionsv1.JSONSchemaProps `json:"openAPIV3Schema,omitempty"`
    // +kubebuilder:default=false
    LegacySchema *bool `json:"legacySchema,omitempty"` // *bool allows for "unset" state which we need to apply appropriate defaults
}

type Target struct {
    Target string   `json:"target,omitempty"`
    Rego   string   `json:"rego,omitempty"`
    Libs   []string `json:"libs,omitempty"`
    // The source code options for the constraint template. "Rego" can only
    // be specified in one place (either here or in the "rego" field)
    // +listType=map
    // +listMapKey=engine
    // +kubebuilder:validation:Required
    Code []Code `json:"code,omitempty"`
}

type Code struct {
    // The engine used to evaluate the code. Example: "Rego". Required.
    // +kubebuilder:validation:Required
    Engine string `json:"engine"`

    // +kubebuilder:validation:Required
    // +kubebuilder:validation:Schemaless
    // +kubebuilder:pruning:PreserveUnknownFields
    // The source code for the template. Required.
    Source *templates.Anything `json:"source"`
}

Constraint

是定义在 “ConstraintTemplate” 中自动生成的 CRD,查看该集群存在多少可通过以下指令:

kubectl get constraints

创建该资源 crd 的逻辑在 github.com/open-policy-agent/frameworks/constraint/pkg/client/crds/crds.go 中 “CreateCRD“ 根据 “ConstraintTemplate” 中的 “openAPIV3Schema” 定义并把参数添加到 “parameters” 里面,然后在集群在把对应的 “Constraint” CRD 创建出来,大致设计以下代码片段:

github.com/open-policy-agent/frameworks/constraint/pkg/client/crds/schema.go -> CreateSchema -> apiextensions.JSONSchemaProps
github.com/open-policy-agent/frameworks/constraint/pkg/client/crds/crds.go -> CreateCRD -> apiextensions.CustomResourceDefinition

apiextensions.JSONSchemaProps -> apiextensions.CustomResourceDefinition

每个 “Constraint” CRD 都存在以下三个参数,在 CreateSchema 方法中定义:

参数 类型
match target.MatchSchema()
enforcementAction string
parameters ConstraintTemplate.Spec.CRD.Spec.Validation.OpenAPIV3Schema

“match” 字段定义了约束将被应用于哪些资源,它支持以下类型的匹配器:

  1. “kinds” 接受一个包含 apiGroups 和 kinds 字段的对象列表,列出了约束将应用于的对象的组或种类。如果指定了多个组或种类,则只需要一个匹配项,资源就会处于范围内。
  2. “scope” 确定是否匹配集群范围或命名空间范围的资源。接受 *, Cluster 或 Namespaced。(默认为 *)
  3. “namespaces” 是一个命名空间名称的列表。如果定义了,约束仅适用于列出的命名空间中的资源。Namespaces 还支持基于前缀的通配符。例如,namespaces: [kube-*] 匹配 kube-system 和 kube-public。
  4. “excludedNamespaces” 是一个命名空间名称的列表。如果定义了,约束仅适用于不在列出的命名空间中的资源。excludedNamespaces 也支持基于前缀的通配符。例如,excludedNamespaces: [kube-*] 匹配不在 kube-system 和 kube-public 中的资源。
  5. “labelSelector” 是两个可选字段的组合:matchLabels 和 matchExpressions。这两个字段提供了基于对象元数据中包含的标签键和值选择或排除 Kubernetes 对象的不同方法。所有选择表达式都会进行 AND 运算,以确定对象是否符合选择器的累积要求。
  6. “namespaceSelector” 是针对对象所在的命名空间或对象本身(如果对象是一个命名空间)的标签选择器。
  7. “name” 是 Kubernetes 对象的名称。如果定义了,它将与具有指定名称的对象匹配。Name 也支持基于前缀的通配符。例如,name: pod-* 匹配 pod-a 和 pod-b。

“enforcementAction” 字段定义了处理约束违规的操作,支持 [deny, dryrun, warn] 三个取值,默认情况下被设置为 deny,因为默认行为是拒绝任何违规的 Admission 请求。

“parameters” 字段描述了约束的意图,它可以被 “ConstraintTemplate” 的 Rego 源代码中 input.parameters 引用。

  • CreateSchema

github.com/open-policy-agent/frameworks/constraint/pkg/client/crds/schema.go

// CreateSchema combines the schema of the match target and the ConstraintTemplate parameters
// to form the schema of the actual constraint resource.
func CreateSchema(templ *templates.ConstraintTemplate, target MatchSchemaProvider) *apiextensions.JSONSchemaProps {
    props := map[string]apiextensions.JSONSchemaProps{
        "match":             target.MatchSchema(),
        "enforcementAction": {Type: "string"},
    }

    if templ.Spec.CRD.Spec.Validation != nil && templ.Spec.CRD.Spec.Validation.OpenAPIV3Schema != nil {
        internalSchema := *templ.Spec.CRD.Spec.Validation.OpenAPIV3Schema.DeepCopy()
        props["parameters"] = internalSchema
    }

    schema := &apiextensions.JSONSchemaProps{
        Type: "object",
        Properties: map[string]apiextensions.JSONSchemaProps{
            "metadata": {
                Type: "object",
                Properties: map[string]apiextensions.JSONSchemaProps{
                    "name": {
                        Type:      "string",
                        MaxLength: pointer.Int64(63),
                    },
                },
            },
            "spec": {
                Type:       "object",
                Properties: props,
            },
            "status": {
                XPreserveUnknownFields: pointer.Bool(true),
            },
        },
    }

    return schema
}
  • target.MatchSchema

github.com/open-policy-agent/gatekeeper/pkg/mutation/match/match_types.go github.com/open-policy-agent/gatekeeper/pkg/target/matchcrd_constant.go

// Match selects which objects are in scope.
// +kubebuilder:object:generate=true
type Match struct {
    // Source determines whether generated or original resources are matched.
    // Accepts `Generated`|`Original`|`All` (defaults to `All`). A value of
    // `Generated` will only match generated resources, while `Original` will only
    // match regular resources.
    // +kubebuilder:validation:Enum=All;Generated;Original
    Source string  `json:"source,omitempty"`
    Kinds  []Kinds `json:"kinds,omitempty"`
    // Scope determines if cluster-scoped and/or namespaced-scoped resources
    // are matched.  Accepts `*`, `Cluster`, or `Namespaced`. (defaults to `*`)
    Scope apiextensionsv1.ResourceScope `json:"scope,omitempty"`
    // Namespaces is a list of namespace names. If defined, a constraint only
    // applies to resources in a listed namespace.  Namespaces also supports a
    // prefix or suffix based glob.  For example, `namespaces: [kube-*]` matches both
    // `kube-system` and `kube-public`, and `namespaces: [*-system]` matches both
    // `kube-system` and `gatekeeper-system`.
    Namespaces []wildcard.Wildcard `json:"namespaces,omitempty"`
    // ExcludedNamespaces is a list of namespace names. If defined, a
    // constraint only applies to resources not in a listed namespace.
    // ExcludedNamespaces also supports a prefix or suffix based glob.  For example,
    // `excludedNamespaces: [kube-*]` matches both `kube-system` and
    // `kube-public`, and `excludedNamespaces: [*-system]` matches both `kube-system` and
    // `gatekeeper-system`.
    ExcludedNamespaces []wildcard.Wildcard `json:"excludedNamespaces,omitempty"`
    // LabelSelector is the combination of two optional fields: `matchLabels`
    // and `matchExpressions`.  These two fields provide different methods of
    // selecting or excluding k8s objects based on the label keys and values
    // included in object metadata.  All selection expressions from both
    // sections are ANDed to determine if an object meets the cumulative
    // requirements of the selector.
    LabelSelector *metav1.LabelSelector `json:"labelSelector,omitempty"`
    // NamespaceSelector is a label selector against an object's containing
    // namespace or the object itself, if the object is a namespace.
    NamespaceSelector *metav1.LabelSelector `json:"namespaceSelector,omitempty"`
    // Name is the name of an object.  If defined, it will match against objects with the specified
    // name.  Name also supports a prefix or suffix glob.  For example, `name: pod-*` would match
    // both `pod-a` and `pod-b`, and `name: *-pod` would match both `a-pod` and `b-pod`.
    Name wildcard.Wildcard `json:"name,omitempty"`
}

// Kinds accepts a list of objects with apiGroups and kinds fields
// that list the groups/kinds of objects to which the mutation will apply.
// If multiple groups/kinds objects are specified,
// only one match is needed for the resource to be in scope.
// +kubebuilder:object:generate=true
type Kinds struct {
    // APIGroups is the API groups the resources belong to. '*' is all groups.
    // If '*' is present, the length of the slice must be one.
    // Required.
    APIGroups []string `json:"apiGroups,omitempty" protobuf:"bytes,1,rep,name=apiGroups"`
    Kinds     []string `json:"kinds,omitempty"`
}

apiextensionsv1.JSONSchemaProps

TODO;

templates.Anything

TODO;




最后修改 2024.01.04: docs: update gatekeeper (817a465)