1 - Client Authentication (v1)

Resource Types

ExecCredential

ExecCredential is used by exec-based plugins to communicate credentials to HTTP transports.

FieldDescription
apiVersion
string
client.authentication.k8s.io/v1
kind
string
ExecCredential
spec [Required]
ExecCredentialSpec

Spec holds information passed to the plugin by the transport.

status
ExecCredentialStatus

Status is filled in by the plugin and holds the credentials that the transport should use to contact the API.

Cluster

Appears in:

Cluster contains information to allow an exec plugin to communicate with the kubernetes cluster being authenticated to.

To ensure that this struct contains everything someone would need to communicate with a kubernetes cluster (just like they would via a kubeconfig), the fields should shadow "k8s.io/client-go/tools/clientcmd/api/v1".Cluster, with the exception of CertificateAuthority, since CA data will always be passed to the plugin as bytes.

FieldDescription
server [Required]
string

Server is the address of the kubernetes cluster (https://hostname:port).

tls-server-name
string

TLSServerName is passed to the server for SNI and is used in the client to check server certificates against. If ServerName is empty, the hostname used to contact the server is used.

insecure-skip-tls-verify
bool

InsecureSkipTLSVerify skips the validity check for the server's certificate. This will make your HTTPS connections insecure.

certificate-authority-data
[]byte

CAData contains PEM-encoded certificate authority certificates. If empty, system roots should be used.

proxy-url
string

ProxyURL is the URL to the proxy to be used for all requests to this cluster.

disable-compression
bool

DisableCompression allows client to opt-out of response compression for all requests to the server. This is useful to speed up requests (specifically lists) when client-server network bandwidth is ample, by saving time on compression (server-side) and decompression (client-side): https://github.com/kubernetes/kubernetes/issues/112296.

config
k8s.io/apimachinery/pkg/runtime.RawExtension

Config holds additional config data that is specific to the exec plugin with regards to the cluster being authenticated to.

This data is sourced from the clientcmd Cluster object's extensions[client.authentication.k8s.io/exec] field:

clusters:

  • name: my-cluster cluster: ... extensions:
    • name: client.authentication.k8s.io/exec # reserved extension name for per cluster exec config extension: audience: 06e3fbd18de8 # arbitrary config

In some environments, the user config may be exactly the same across many clusters (i.e. call this exec plugin) minus some details that are specific to each cluster such as the audience. This field allows the per cluster config to be directly specified with the cluster info. Using this field to store secret data is not recommended as one of the prime benefits of exec plugins is that no secrets need to be stored directly in the kubeconfig.

ExecCredentialSpec

Appears in:

ExecCredentialSpec holds request and runtime specific information provided by the transport.

FieldDescription
cluster
Cluster

Cluster contains information to allow an exec plugin to communicate with the kubernetes cluster being authenticated to. Note that Cluster is non-nil only when provideClusterInfo is set to true in the exec provider config (i.e., ExecConfig.ProvideClusterInfo).

interactive [Required]
bool

Interactive declares whether stdin has been passed to this exec plugin.

ExecCredentialStatus

Appears in:

ExecCredentialStatus holds credentials for the transport to use.

Token and ClientKeyData are sensitive fields. This data should only be transmitted in-memory between client and exec plugin process. Exec plugin itself should at least be protected via file permissions.

FieldDescription
expirationTimestamp
meta/v1.Time

ExpirationTimestamp indicates a time when the provided credentials expire.

token [Required]
string

Token is a bearer token used by the client for request authentication.

clientCertificateData [Required]
string

PEM-encoded client TLS certificates (including intermediates, if any).

clientKeyData [Required]
string

PEM-encoded private key for the above certificate.

2 - Client Authentication (v1beta1)

Resource Types

ExecCredential

ExecCredential is used by exec-based plugins to communicate credentials to HTTP transports.

FieldDescription
apiVersion
string
client.authentication.k8s.io/v1beta1
kind
string
ExecCredential
spec [Required]
ExecCredentialSpec

Spec holds information passed to the plugin by the transport.

status
ExecCredentialStatus

Status is filled in by the plugin and holds the credentials that the transport should use to contact the API.

Cluster

Appears in:

Cluster contains information to allow an exec plugin to communicate with the kubernetes cluster being authenticated to.

To ensure that this struct contains everything someone would need to communicate with a kubernetes cluster (just like they would via a kubeconfig), the fields should shadow "k8s.io/client-go/tools/clientcmd/api/v1".Cluster, with the exception of CertificateAuthority, since CA data will always be passed to the plugin as bytes.

FieldDescription
server [Required]
string

Server is the address of the kubernetes cluster (https://hostname:port).

tls-server-name
string

TLSServerName is passed to the server for SNI and is used in the client to check server certificates against. If ServerName is empty, the hostname used to contact the server is used.

insecure-skip-tls-verify
bool

InsecureSkipTLSVerify skips the validity check for the server's certificate. This will make your HTTPS connections insecure.

certificate-authority-data
[]byte

CAData contains PEM-encoded certificate authority certificates. If empty, system roots should be used.

proxy-url
string

ProxyURL is the URL to the proxy to be used for all requests to this cluster.

disable-compression
bool

DisableCompression allows client to opt-out of response compression for all requests to the server. This is useful to speed up requests (specifically lists) when client-server network bandwidth is ample, by saving time on compression (server-side) and decompression (client-side): https://github.com/kubernetes/kubernetes/issues/112296.

config
k8s.io/apimachinery/pkg/runtime.RawExtension

Config holds additional config data that is specific to the exec plugin with regards to the cluster being authenticated to.

This data is sourced from the clientcmd Cluster object's extensions[client.authentication.k8s.io/exec] field:

clusters:

  • name: my-cluster cluster: ... extensions:
    • name: client.authentication.k8s.io/exec # reserved extension name for per cluster exec config extension: audience: 06e3fbd18de8 # arbitrary config

In some environments, the user config may be exactly the same across many clusters (i.e. call this exec plugin) minus some details that are specific to each cluster such as the audience. This field allows the per cluster config to be directly specified with the cluster info. Using this field to store secret data is not recommended as one of the prime benefits of exec plugins is that no secrets need to be stored directly in the kubeconfig.

ExecCredentialSpec

Appears in:

ExecCredentialSpec holds request and runtime specific information provided by the transport.

FieldDescription
cluster
Cluster

Cluster contains information to allow an exec plugin to communicate with the kubernetes cluster being authenticated to. Note that Cluster is non-nil only when provideClusterInfo is set to true in the exec provider config (i.e., ExecConfig.ProvideClusterInfo).

interactive [Required]
bool

Interactive declares whether stdin has been passed to this exec plugin.

ExecCredentialStatus

Appears in:

ExecCredentialStatus holds credentials for the transport to use.

Token and ClientKeyData are sensitive fields. This data should only be transmitted in-memory between client and exec plugin process. Exec plugin itself should at least be protected via file permissions.

FieldDescription
expirationTimestamp
meta/v1.Time

ExpirationTimestamp indicates a time when the provided credentials expire.

token [Required]
string

Token is a bearer token used by the client for request authentication.

clientCertificateData [Required]
string

PEM-encoded client TLS certificates (including intermediates, if any).

clientKeyData [Required]
string

PEM-encoded private key for the above certificate.

3 - Event Rate Limit Configuration (v1alpha1)

Resource Types

Configuration

Configuration provides configuration for the EventRateLimit admission controller.

FieldDescription
apiVersion
string
eventratelimit.admission.k8s.io/v1alpha1
kind
string
Configuration
limits [Required]
[]Limit

limits are the limits to place on event queries received. Limits can be placed on events received server-wide, per namespace, per user, and per source+object. At least one limit is required.

Limit

Appears in:

Limit is the configuration for a particular limit type

FieldDescription
type [Required]
LimitType

type is the type of limit to which this configuration applies

qps [Required]
int32

qps is the number of event queries per second that are allowed for this type of limit. The qps and burst fields are used together to determine if a particular event query is accepted. The qps determines how many queries are accepted once the burst amount of queries has been exhausted.

burst [Required]
int32

burst is the burst number of event queries that are allowed for this type of limit. The qps and burst fields are used together to determine if a particular event query is accepted. The burst determines the maximum size of the allowance granted for a particular bucket. For example, if the burst is 10 and the qps is 3, then the admission control will accept 10 queries before blocking any queries. Every second, 3 more queries will be allowed. If some of that allowance is not used, then it will roll over to the next second, until the maximum allowance of 10 is reached.

cacheSize
int32

cacheSize is the size of the LRU cache for this type of limit. If a bucket is evicted from the cache, then the allowance for that bucket is reset. If more queries are later received for an evicted bucket, then that bucket will re-enter the cache with a clean slate, giving that bucket a full allowance of burst queries.

The default cache size is 4096.

If limitType is 'server', then cacheSize is ignored.

LimitType

(Alias of string)

Appears in:

LimitType is the type of the limit (e.g., per-namespace)

4 - Image Policy API (v1alpha1)

Resource Types

ImageReview

ImageReview checks if the set of images in a pod are allowed.

FieldDescription
apiVersion
string
imagepolicy.k8s.io/v1alpha1
kind
string
ImageReview
metadata
meta/v1.ObjectMeta

Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata

Refer to the Kubernetes API documentation for the fields of the metadata field.
spec [Required]
ImageReviewSpec

Spec holds information about the pod being evaluated

status
ImageReviewStatus

Status is filled in by the backend and indicates whether the pod should be allowed.

ImageReviewContainerSpec

Appears in:

ImageReviewContainerSpec is a description of a container within the pod creation request.

FieldDescription
image
string

This can be in the form image:tag or image@SHA:012345679abcdef.

ImageReviewSpec

Appears in:

ImageReviewSpec is a description of the pod creation request.

FieldDescription
containers
[]ImageReviewContainerSpec

Containers is a list of a subset of the information in each container of the Pod being created.

annotations
map[string]string

Annotations is a list of key-value pairs extracted from the Pod's annotations. It only includes keys which match the pattern *.image-policy.k8s.io/*. It is up to each webhook backend to determine how to interpret these annotations, if at all.

namespace
string

Namespace is the namespace the pod is being created in.

ImageReviewStatus

Appears in:

ImageReviewStatus is the result of the review for the pod creation request.

FieldDescription
allowed [Required]
bool

Allowed indicates that all images were allowed to be run.

reason
string

Reason should be empty unless Allowed is false in which case it may contain a short description of what is wrong. Kubernetes may truncate excessively long errors when displaying to the user.

auditAnnotations
map[string]string

AuditAnnotations will be added to the attributes object of the admission controller request using 'AddAnnotation'. The keys should be prefix-less (i.e., the admission controller will add an appropriate prefix).

5 - kube-apiserver Admission (v1)

Resource Types

AdmissionReview

AdmissionReview describes an admission review request/response.

FieldDescription
apiVersion
string
admission.k8s.io/v1
kind
string
AdmissionReview
request
AdmissionRequest

Request describes the attributes for the admission request.

response
AdmissionResponse

Response describes the attributes for the admission response.

AdmissionRequest

Appears in:

AdmissionRequest describes the admission.Attributes for the admission request.

FieldDescription
uid [Required]
k8s.io/apimachinery/pkg/types.UID

UID is an identifier for the individual request/response. It allows us to distinguish instances of requests which are otherwise identical (parallel requests, requests when earlier requests did not modify etc) The UID is meant to track the round trip (request/response) between the KAS and the WebHook, not the user request. It is suitable for correlating log entries between the webhook and apiserver, for either auditing or debugging.

kind [Required]
meta/v1.GroupVersionKind

Kind is the fully-qualified type of object being submitted (for example, v1.Pod or autoscaling.v1.Scale)

resource [Required]
meta/v1.GroupVersionResource

Resource is the fully-qualified resource being requested (for example, v1.pods)

subResource
string

SubResource is the subresource being requested, if any (for example, "status" or "scale")

requestKind
meta/v1.GroupVersionKind

RequestKind is the fully-qualified type of the original API request (for example, v1.Pod or autoscaling.v1.Scale). If this is specified and differs from the value in "kind", an equivalent match and conversion was performed.

For example, if deployments can be modified via apps/v1 and apps/v1beta1, and a webhook registered a rule of apiGroups:["apps"], apiVersions:["v1"], resources: ["deployments"] and matchPolicy: Equivalent, an API request to apps/v1beta1 deployments would be converted and sent to the webhook with kind: {group:"apps", version:"v1", kind:"Deployment"} (matching the rule the webhook registered for), and requestKind: {group:"apps", version:"v1beta1", kind:"Deployment"} (indicating the kind of the original API request).

See documentation for the "matchPolicy" field in the webhook configuration type for more details.

requestResource
meta/v1.GroupVersionResource

RequestResource is the fully-qualified resource of the original API request (for example, v1.pods). If this is specified and differs from the value in "resource", an equivalent match and conversion was performed.

For example, if deployments can be modified via apps/v1 and apps/v1beta1, and a webhook registered a rule of apiGroups:["apps"], apiVersions:["v1"], resources: ["deployments"] and matchPolicy: Equivalent, an API request to apps/v1beta1 deployments would be converted and sent to the webhook with resource: {group:"apps", version:"v1", resource:"deployments"} (matching the resource the webhook registered for), and requestResource: {group:"apps", version:"v1beta1", resource:"deployments"} (indicating the resource of the original API request).

See documentation for the "matchPolicy" field in the webhook configuration type.

requestSubResource
string

RequestSubResource is the name of the subresource of the original API request, if any (for example, "status" or "scale") If this is specified and differs from the value in "subResource", an equivalent match and conversion was performed. See documentation for the "matchPolicy" field in the webhook configuration type.

name
string

Name is the name of the object as presented in the request. On a CREATE operation, the client may omit name and rely on the server to generate the name. If that is the case, this field will contain an empty string.

namespace
string

Namespace is the namespace associated with the request (if any).

operation [Required]
Operation

Operation is the operation being performed. This may be different than the operation requested. e.g. a patch can result in either a CREATE or UPDATE Operation.

userInfo [Required]
authentication/v1.UserInfo

UserInfo is information about the requesting user

object
k8s.io/apimachinery/pkg/runtime.RawExtension

Object is the object from the incoming request.

oldObject
k8s.io/apimachinery/pkg/runtime.RawExtension

OldObject is the existing object. Only populated for DELETE and UPDATE requests.

dryRun
bool

DryRun indicates that modifications will definitely not be persisted for this request. Defaults to false.

options
k8s.io/apimachinery/pkg/runtime.RawExtension

Options is the operation option structure of the operation being performed. e.g. meta.k8s.io/v1.DeleteOptions or meta.k8s.io/v1.CreateOptions. This may be different than the options the caller provided. e.g. for a patch request the performed Operation might be a CREATE, in which case the Options will a meta.k8s.io/v1.CreateOptions even though the caller provided meta.k8s.io/v1.PatchOptions.

AdmissionResponse

Appears in:

AdmissionResponse describes an admission response.

FieldDescription
uid [Required]
k8s.io/apimachinery/pkg/types.UID

UID is an identifier for the individual request/response. This must be copied over from the corresponding AdmissionRequest.

allowed [Required]
bool

Allowed indicates whether or not the admission request was permitted.

status
meta/v1.Status

Result contains extra details into why an admission request was denied. This field IS NOT consulted in any way if "Allowed" is "true".

patch
[]byte

The patch body. Currently we only support "JSONPatch" which implements RFC 6902.

patchType
PatchType

The type of Patch. Currently we only allow "JSONPatch".

auditAnnotations
map[string]string

AuditAnnotations is an unstructured key value map set by remote admission controller (e.g. error=image-blacklisted). MutatingAdmissionWebhook and ValidatingAdmissionWebhook admission controller will prefix the keys with admission webhook name (e.g. imagepolicy.example.com/error=image-blacklisted). AuditAnnotations will be provided by the admission webhook to add additional context to the audit log for this request.

warnings
[]string

warnings is a list of warning messages to return to the requesting API client. Warning messages describe a problem the client making the API request should correct or be aware of. Limit warnings to 120 characters if possible. Warnings over 256 characters and large numbers of warnings may be truncated.

Operation

(Alias of string)

Appears in:

Operation is the type of resource operation being checked for admission control

PatchType

(Alias of string)

Appears in:

PatchType is the type of patch being used to represent the mutated object

6 - kube-apiserver Audit Configuration (v1)

Resource Types

Event

Appears in:

Event captures all the information that can be included in an API audit log.

FieldDescription
apiVersion
string
audit.k8s.io/v1
kind
string
Event
level [Required]
Level

AuditLevel at which event was generated

auditID [Required]
k8s.io/apimachinery/pkg/types.UID

Unique audit ID, generated for each request.

stage [Required]
Stage

Stage of the request handling when this event instance was generated.

requestURI [Required]
string

RequestURI is the request URI as sent by the client to a server.

verb [Required]
string

Verb is the kubernetes verb associated with the request. For non-resource requests, this is the lower-cased HTTP method.

user [Required]
authentication/v1.UserInfo

Authenticated user information.

impersonatedUser
authentication/v1.UserInfo

Impersonated user information.

sourceIPs
[]string

Source IPs, from where the request originated and intermediate proxies. The source IPs are listed from (in order):

  1. X-Forwarded-For request header IPs
  2. X-Real-Ip header, if not present in the X-Forwarded-For list
  3. The remote address for the connection, if it doesn't match the last IP in the list up to here (X-Forwarded-For or X-Real-Ip). Note: All but the last IP can be arbitrarily set by the client.
userAgent
string

UserAgent records the user agent string reported by the client. Note that the UserAgent is provided by the client, and must not be trusted.

objectRef
ObjectReference

Object reference this request is targeted at. Does not apply for List-type requests, or non-resource requests.

responseStatus
meta/v1.Status

The response status, populated even when the ResponseObject is not a Status type. For successful responses, this will only include the Code and StatusSuccess. For non-status type error responses, this will be auto-populated with the error Message.

requestObject
k8s.io/apimachinery/pkg/runtime.Unknown

API object from the request, in JSON format. The RequestObject is recorded as-is in the request (possibly re-encoded as JSON), prior to version conversion, defaulting, admission or merging. It is an external versioned object type, and may not be a valid object on its own. Omitted for non-resource requests. Only logged at Request Level and higher.

responseObject
k8s.io/apimachinery/pkg/runtime.Unknown

API object returned in the response, in JSON. The ResponseObject is recorded after conversion to the external type, and serialized as JSON. Omitted for non-resource requests. Only logged at Response Level.

requestReceivedTimestamp
meta/v1.MicroTime

Time the request reached the apiserver.

stageTimestamp
meta/v1.MicroTime

Time the request reached current audit stage.

annotations
map[string]string

Annotations is an unstructured key value map stored with an audit event that may be set by plugins invoked in the request serving chain, including authentication, authorization and admission plugins. Note that these annotations are for the audit event, and do not correspond to the metadata.annotations of the submitted object. Keys should uniquely identify the informing component to avoid name collisions (e.g. podsecuritypolicy.admission.k8s.io/policy). Values should be short. Annotations are included in the Metadata level.

EventList

EventList is a list of audit Events.

FieldDescription
apiVersion
string
audit.k8s.io/v1
kind
string
EventList
metadata
meta/v1.ListMeta
No description provided.
items [Required]
[]Event
No description provided.

Policy

Appears in:

Policy defines the configuration of audit logging, and the rules for how different request categories are logged.

FieldDescription
apiVersion
string
audit.k8s.io/v1
kind
string
Policy
metadata
meta/v1.ObjectMeta

ObjectMeta is included for interoperability with API infrastructure.

Refer to the Kubernetes API documentation for the fields of the metadata field.
rules [Required]
[]PolicyRule

Rules specify the audit Level a request should be recorded at. A request may match multiple rules, in which case the FIRST matching rule is used. The default audit level is None, but can be overridden by a catch-all rule at the end of the list. PolicyRules are strictly ordered.

omitStages
[]Stage

OmitStages is a list of stages for which no events are created. Note that this can also be specified per rule in which case the union of both are omitted.

omitManagedFields
bool

OmitManagedFields indicates whether to omit the managed fields of the request and response bodies from being written to the API audit log. This is used as a global default - a value of 'true' will omit the managed fileds, otherwise the managed fields will be included in the API audit log. Note that this can also be specified per rule in which case the value specified in a rule will override the global default.

PolicyList

PolicyList is a list of audit Policies.

FieldDescription
apiVersion
string
audit.k8s.io/v1
kind
string
PolicyList
metadata
meta/v1.ListMeta
No description provided.
items [Required]
[]Policy
No description provided.

GroupResources

Appears in:

GroupResources represents resource kinds in an API group.

FieldDescription
group
string

Group is the name of the API group that contains the resources. The empty string represents the core API group.

resources
[]string

Resources is a list of resources this rule applies to.

For example:

  • pods matches pods.
  • pods/log matches the log subresource of pods.
  • * matches all resources and their subresources.
  • pods/* matches all subresources of pods.
  • */scale matches all scale subresources.

If wildcard is present, the validation rule will ensure resources do not overlap with each other.

An empty list implies all resources and subresources in this API groups apply.

resourceNames
[]string

ResourceNames is a list of resource instance names that the policy matches. Using this field requires Resources to be specified. An empty list implies that every instance of the resource is matched.

Level

(Alias of string)

Appears in:

Level defines the amount of information logged during auditing

ObjectReference

Appears in:

ObjectReference contains enough information to let you inspect or modify the referred object.

FieldDescription
resource
string
No description provided.
namespace
string
No description provided.
name
string
No description provided.
uid
k8s.io/apimachinery/pkg/types.UID
No description provided.
apiGroup
string

APIGroup is the name of the API group that contains the referred object. The empty string represents the core API group.

apiVersion
string

APIVersion is the version of the API group that contains the referred object.

resourceVersion
string
No description provided.
subresource
string
No description provided.

PolicyRule

Appears in:

PolicyRule maps requests based off metadata to an audit Level. Requests must match the rules of every field (an intersection of rules).

FieldDescription
level [Required]
Level

The Level that requests matching this rule are recorded at.

users
[]string

The users (by authenticated user name) this rule applies to. An empty list implies every user.

userGroups
[]string

The user groups this rule applies to. A user is considered matching if it is a member of any of the UserGroups. An empty list implies every user group.

verbs
[]string

The verbs that match this rule. An empty list implies every verb.

resources
[]GroupResources

Resources that this rule matches. An empty list implies all kinds in all API groups.

namespaces
[]string

Namespaces that this rule matches. The empty string "" matches non-namespaced resources. An empty list implies every namespace.

nonResourceURLs
[]string

NonResourceURLs is a set of URL paths that should be audited. *s are allowed, but only as the full, final step in the path. Examples:

  • /metrics - Log requests for apiserver metrics
  • /healthz* - Log all health checks
omitStages
[]Stage

OmitStages is a list of stages for which no events are created. Note that this can also be specified policy wide in which case the union of both are omitted. An empty list means no restrictions will apply.

omitManagedFields
bool

OmitManagedFields indicates whether to omit the managed fields of the request and response bodies from being written to the API audit log.

  • a value of 'true' will drop the managed fields from the API audit log
  • a value of 'false' indicates that the managed fileds should be included in the API audit log Note that the value, if specified, in this rule will override the global default If a value is not specified then the global default specified in Policy.OmitManagedFields will stand.

Stage

(Alias of string)

Appears in:

Stage defines the stages in request handling that audit events may be generated.

7 - kube-apiserver Configuration (v1)

Package v1 is the v1 version of the API.

Resource Types

AdmissionConfiguration

AdmissionConfiguration provides versioned configuration for admission controllers.

FieldDescription
apiVersion
string
apiserver.config.k8s.io/v1
kind
string
AdmissionConfiguration
plugins
[]AdmissionPluginConfiguration

Plugins allows specifying a configuration per admission control plugin.

EncryptionConfiguration

EncryptionConfiguration stores the complete configuration for encryption providers. It also allows the use of wildcards to specify the resources that should be encrypted. Use '*.<group>' to encrypt all resources within a group or '*.*' to encrypt all resources. '*.' can be used to encrypt all resource in the core group. '*.*' will encrypt all resources, even custom resources that are added after API server start. Use of wildcards that overlap within the same resource list or across multiple entries are not allowed since part of the configuration would be ineffective. Resource lists are processed in order, with earlier lists taking precedence.

Example:

kind: EncryptionConfiguration
apiVersion: apiserver.config.k8s.io/v1
resources:
- resources:
  - events
  providers:
  - identity: {}  # do not encrypt events even though *.* is specified below
- resources:
  - secrets
  - configmaps
  - pandas.awesome.bears.example
  providers:
  - aescbc:
      keys:
      - name: key1
        secret: c2VjcmV0IGlzIHNlY3VyZQ==
- resources:
  - '*.apps'
  providers:
  - aescbc:
      keys:
      - name: key2
        secret: c2VjcmV0IGlzIHNlY3VyZSwgb3IgaXMgaXQ/Cg==
- resources:
  - '*.*'
  providers:
  - aescbc:
      keys:
      - name: key3
        secret: c2VjcmV0IGlzIHNlY3VyZSwgSSB0aGluaw==
FieldDescription
apiVersion
string
apiserver.config.k8s.io/v1
kind
string
EncryptionConfiguration
resources [Required]
[]ResourceConfiguration

resources is a list containing resources, and their corresponding encryption providers.

AESConfiguration

Appears in:

AESConfiguration contains the API configuration for an AES transformer.

FieldDescription
keys [Required]
[]Key

keys is a list of keys to be used for creating the AES transformer. Each key has to be 32 bytes long for AES-CBC and 16, 24 or 32 bytes for AES-GCM.

AdmissionPluginConfiguration

Appears in:

AdmissionPluginConfiguration provides the configuration for a single plug-in.

FieldDescription
name [Required]
string

Name is the name of the admission controller. It must match the registered admission plugin name.

path
string

Path is the path to a configuration file that contains the plugin's configuration

configuration
k8s.io/apimachinery/pkg/runtime.Unknown

Configuration is an embedded configuration object to be used as the plugin's configuration. If present, it will be used instead of the path to the configuration file.

IdentityConfiguration

Appears in:

IdentityConfiguration is an empty struct to allow identity transformer in provider configuration.

KMSConfiguration

Appears in:

KMSConfiguration contains the name, cache size and path to configuration file for a KMS based envelope transformer.

FieldDescription
apiVersion
string

apiVersion of KeyManagementService

name [Required]
string

name is the name of the KMS plugin to be used.

cachesize
int32

cachesize is the maximum number of secrets which are cached in memory. The default value is 1000. Set to a negative value to disable caching. This field is only allowed for KMS v1 providers.

endpoint [Required]
string

endpoint is the gRPC server listening address, for example "unix:///var/run/kms-provider.sock".

timeout
meta/v1.Duration

timeout for gRPC calls to kms-plugin (ex. 5s). The default is 3 seconds.

Key

Appears in:

Key contains name and secret of the provided key for a transformer.

FieldDescription
name [Required]
string

name is the name of the key to be used while storing data to disk.

secret [Required]
string

secret is the actual key, encoded in base64.

ProviderConfiguration

Appears in:

ProviderConfiguration stores the provided configuration for an encryption provider.

FieldDescription
aesgcm [Required]
AESConfiguration

aesgcm is the configuration for the AES-GCM transformer.

aescbc [Required]
AESConfiguration

aescbc is the configuration for the AES-CBC transformer.

secretbox [Required]
SecretboxConfiguration

secretbox is the configuration for the Secretbox based transformer.

identity [Required]
IdentityConfiguration

identity is the (empty) configuration for the identity transformer.

kms [Required]
KMSConfiguration

kms contains the name, cache size and path to configuration file for a KMS based envelope transformer.

ResourceConfiguration

Appears in:

ResourceConfiguration stores per resource configuration.

FieldDescription
resources [Required]
[]string

resources is a list of kubernetes resources which have to be encrypted. The resource names are derived from resource or resource.group of the group/version/resource. eg: pandas.awesome.bears.example is a custom resource with 'group': awesome.bears.example, 'resource': pandas. Use '*.*' to encrypt all resources and '*.<group>' to encrypt all resources in a specific group. eg: '*.awesome.bears.example' will encrypt all resources in the group 'awesome.bears.example'. eg: '*.' will encrypt all resources in the core group (such as pods, configmaps, etc).

providers [Required]
[]ProviderConfiguration

providers is a list of transformers to be used for reading and writing the resources to disk. eg: aesgcm, aescbc, secretbox, identity, kms.

SecretboxConfiguration

Appears in:

SecretboxConfiguration contains the API configuration for an Secretbox transformer.

FieldDescription
keys [Required]
[]Key

keys is a list of keys to be used for creating the Secretbox transformer. Each key has to be 32 bytes long.

8 - kube-apiserver Configuration (v1alpha1)

Package v1alpha1 is the v1alpha1 version of the API.

Resource Types

TracingConfiguration

Appears in:

TracingConfiguration provides versioned configuration for OpenTelemetry tracing clients.

FieldDescription
endpoint
string

Endpoint of the collector this component will report traces to. The connection is insecure, and does not currently support TLS. Recommended is unset, and endpoint is the otlp grpc default, localhost:4317.

samplingRatePerMillion
int32

SamplingRatePerMillion is the number of samples to collect per million spans. Recommended is unset. If unset, sampler respects its parent span's sampling rate, but otherwise never samples.

AdmissionConfiguration

AdmissionConfiguration provides versioned configuration for admission controllers.

FieldDescription
apiVersion
string
apiserver.k8s.io/v1alpha1
kind
string
AdmissionConfiguration
plugins
[]AdmissionPluginConfiguration

Plugins allows specifying a configuration per admission control plugin.

AuthenticationConfiguration

AuthenticationConfiguration provides versioned configuration for authentication.

FieldDescription
apiVersion
string
apiserver.k8s.io/v1alpha1
kind
string
AuthenticationConfiguration
jwt [Required]
[]JWTAuthenticator

jwt is a list of authenticator to authenticate Kubernetes users using JWT compliant tokens. The authenticator will attempt to parse a raw ID token, verify it's been signed by the configured issuer. The public key to verify the signature is discovered from the issuer's public endpoint using OIDC discovery. For an incoming token, each JWT authenticator will be attempted in the order in which it is specified in this list. Note however that other authenticators may run before or after the JWT authenticators. The specific position of JWT authenticators in relation to other authenticators is neither defined nor stable across releases. Since each JWT authenticator must have a unique issuer URL, at most one JWT authenticator will attempt to cryptographically validate the token.

The minimum valid JWT payload must contain the following claims: { "iss": "https://issuer.example.com", "aud": ["audience"], "exp": 1234567890, "<username claim>": "username" }

AuthorizationConfiguration

FieldDescription
apiVersion
string
apiserver.k8s.io/v1alpha1
kind
string
AuthorizationConfiguration
authorizers [Required]
[]AuthorizerConfiguration

Authorizers is an ordered list of authorizers to authorize requests against. This is similar to the --authorization-modes kube-apiserver flag Must be at least one.

EgressSelectorConfiguration

EgressSelectorConfiguration provides versioned configuration for egress selector clients.

FieldDescription
apiVersion
string
apiserver.k8s.io/v1alpha1
kind
string
EgressSelectorConfiguration
egressSelections [Required]
[]EgressSelection

connectionServices contains a list of egress selection client configurations

TracingConfiguration

TracingConfiguration provides versioned configuration for tracing clients.

FieldDescription
apiVersion
string
apiserver.k8s.io/v1alpha1
kind
string
TracingConfiguration
TracingConfiguration [Required]
TracingConfiguration
(Members of TracingConfiguration are embedded into this type.)

Embed the component config tracing configuration struct

AdmissionPluginConfiguration

Appears in:

AdmissionPluginConfiguration provides the configuration for a single plug-in.

FieldDescription
name [Required]
string

Name is the name of the admission controller. It must match the registered admission plugin name.

path
string

Path is the path to a configuration file that contains the plugin's configuration

configuration
k8s.io/apimachinery/pkg/runtime.Unknown

Configuration is an embedded configuration object to be used as the plugin's configuration. If present, it will be used instead of the path to the configuration file.

AudienceMatchPolicyType

(Alias of string)

Appears in:

AudienceMatchPolicyType is a set of valid values for issuer.audienceMatchPolicy

AuthorizerConfiguration

Appears in:

FieldDescription
type [Required]
string

Type refers to the type of the authorizer "Webhook" is supported in the generic API server Other API servers may support additional authorizer types like Node, RBAC, ABAC, etc.

name [Required]
string

Name used to describe the webhook This is explicitly used in monitoring machinery for metrics Note: Names must be DNS1123 labels like myauthorizername or subdomains like myauthorizer.example.domain Required, with no default

webhook [Required]
WebhookConfiguration

Webhook defines the configuration for a Webhook authorizer Must be defined when Type=Webhook Must not be defined when Type!=Webhook

ClaimMappings

Appears in:

ClaimMappings provides the configuration for claim mapping

FieldDescription
username [Required]
PrefixedClaimOrExpression

username represents an option for the username attribute. The claim's value must be a singular string. Same as the --oidc-username-claim and --oidc-username-prefix flags. If username.expression is set, the expression must produce a string value. If username.expression uses 'claims.email', then 'claims.email_verified' must be used in username.expression or extra[*].valueExpression or claimValidationRules[*].expression. An example claim validation rule expression that matches the validation automatically applied when username.claim is set to 'email' is 'claims.?email_verified.orValue(true)'.

In the flag based approach, the --oidc-username-claim and --oidc-username-prefix are optional. If --oidc-username-claim is not set, the default value is "sub". For the authentication config, there is no defaulting for claim or prefix. The claim and prefix must be set explicitly. For claim, if --oidc-username-claim was not set with legacy flag approach, configure username.claim="sub" in the authentication config. For prefix: (1) --oidc-username-prefix="-", no prefix was added to the username. For the same behavior using authentication config, set username.prefix="" (2) --oidc-username-prefix="" and --oidc-username-claim != "email", prefix was "<value of --oidc-issuer-url>#". For the same behavior using authentication config, set username.prefix="<value of issuer.url>#" (3) --oidc-username-prefix="<value>". For the same behavior using authentication config, set username.prefix="<value>"

groups
PrefixedClaimOrExpression

groups represents an option for the groups attribute. The claim's value must be a string or string array claim. If groups.claim is set, the prefix must be specified (and can be the empty string). If groups.expression is set, the expression must produce a string or string array value. "", [], and null values are treated as the group mapping not being present.

uid
ClaimOrExpression

uid represents an option for the uid attribute. Claim must be a singular string claim. If uid.expression is set, the expression must produce a string value.

extra
[]ExtraMapping

extra represents an option for the extra attribute. expression must produce a string or string array value. If the value is empty, the extra mapping will not be present.

hard-coded extra key/value

  • key: "foo" valueExpression: "'bar'" This will result in an extra attribute - foo: ["bar"]

hard-coded key, value copying claim value

  • key: "foo" valueExpression: "claims.some_claim" This will result in an extra attribute - foo: [value of some_claim]

hard-coded key, value derived from claim value

  • key: "admin" valueExpression: '(has(claims.is_admin) && claims.is_admin) ? "true":""' This will result in:
  • if is_admin claim is present and true, extra attribute - admin: ["true"]
  • if is_admin claim is present and false or is_admin claim is not present, no extra attribute will be added

ClaimOrExpression

Appears in:

ClaimOrExpression provides the configuration for a single claim or expression.

FieldDescription
claim
string

claim is the JWT claim to use. Either claim or expression must be set. Mutually exclusive with expression.

expression
string

expression represents the expression which will be evaluated by CEL.

CEL expressions have access to the contents of the token claims, organized into CEL variable:

  • 'claims' is a map of claim names to claim values. For example, a variable named 'sub' can be accessed as 'claims.sub'. Nested claims can be accessed using dot notation, e.g. 'claims.foo.bar'.

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

Mutually exclusive with claim.

ClaimValidationRule

Appears in:

ClaimValidationRule provides the configuration for a single claim validation rule.

FieldDescription
claim
string

claim is the name of a required claim. Same as --oidc-required-claim flag. Only string claim keys are supported. Mutually exclusive with expression and message.

requiredValue
string

requiredValue is the value of a required claim. Same as --oidc-required-claim flag. Only string claim values are supported. If claim is set and requiredValue is not set, the claim must be present with a value set to the empty string. Mutually exclusive with expression and message.

expression
string

expression represents the expression which will be evaluated by CEL. Must produce a boolean.

CEL expressions have access to the contents of the token claims, organized into CEL variable:

  • 'claims' is a map of claim names to claim values. For example, a variable named 'sub' can be accessed as 'claims.sub'. Nested claims can be accessed using dot notation, e.g. 'claims.foo.bar'. Must return true for the validation to pass.

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

Mutually exclusive with claim and requiredValue.

message
string

message customizes the returned error message when expression returns false. message is a literal string. Mutually exclusive with claim and requiredValue.

Connection

Appears in:

Connection provides the configuration for a single egress selection client.

FieldDescription
proxyProtocol [Required]
ProtocolType

Protocol is the protocol used to connect from client to the konnectivity server.

transport
Transport

Transport defines the transport configurations we use to dial to the konnectivity server. This is required if ProxyProtocol is HTTPConnect or GRPC.

EgressSelection

Appears in:

EgressSelection provides the configuration for a single egress selection client.

FieldDescription
name [Required]
string

name is the name of the egress selection. Currently supported values are "controlplane", "master", "etcd" and "cluster" The "master" egress selector is deprecated in favor of "controlplane"

connection [Required]
Connection

connection is the exact information used to configure the egress selection

ExtraMapping

Appears in:

ExtraMapping provides the configuration for a single extra mapping.

FieldDescription
key [Required]
string

key is a string to use as the extra attribute key. key must be a domain-prefix path (e.g. example.org/foo). All characters before the first "/" must be a valid subdomain as defined by RFC 1123. All characters trailing the first "/" must be valid HTTP Path characters as defined by RFC 3986. key must be lowercase. Required to be unique.

valueExpression [Required]
string

valueExpression is a CEL expression to extract extra attribute value. valueExpression must produce a string or string array value. "", [], and null values are treated as the extra mapping not being present. Empty string values contained within a string array are filtered out.

CEL expressions have access to the contents of the token claims, organized into CEL variable:

  • 'claims' is a map of claim names to claim values. For example, a variable named 'sub' can be accessed as 'claims.sub'. Nested claims can be accessed using dot notation, e.g. 'claims.foo.bar'.

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

Issuer

Appears in:

Issuer provides the configuration for an external provider's specific settings.

FieldDescription
url [Required]
string

url points to the issuer URL in a format https://url or https://url/path. This must match the "iss" claim in the presented JWT, and the issuer returned from discovery. Same value as the --oidc-issuer-url flag. Discovery information is fetched from "{url}/.well-known/openid-configuration" unless overridden by discoveryURL. Required to be unique across all JWT authenticators. Note that egress selection configuration is not used for this network connection.

discoveryURL
string

discoveryURL, if specified, overrides the URL used to fetch discovery information instead of using "{url}/.well-known/openid-configuration". The exact value specified is used, so "/.well-known/openid-configuration" must be included in discoveryURL if needed.

The "issuer" field in the fetched discovery information must match the "issuer.url" field in the AuthenticationConfiguration and will be used to validate the "iss" claim in the presented JWT. This is for scenarios where the well-known and jwks endpoints are hosted at a different location than the issuer (such as locally in the cluster).

Example: A discovery url that is exposed using kubernetes service 'oidc' in namespace 'oidc-namespace' and discovery information is available at '/.well-known/openid-configuration'. discoveryURL: "https://oidc.oidc-namespace/.well-known/openid-configuration" certificateAuthority is used to verify the TLS connection and the hostname on the leaf certificate must be set to 'oidc.oidc-namespace'.

curl https://oidc.oidc-namespace/.well-known/openid-configuration (.discoveryURL field) { issuer: "https://oidc.example.com" (.url field) }

discoveryURL must be different from url. Required to be unique across all JWT authenticators. Note that egress selection configuration is not used for this network connection.

certificateAuthority
string

certificateAuthority contains PEM-encoded certificate authority certificates used to validate the connection when fetching discovery information. If unset, the system verifier is used. Same value as the content of the file referenced by the --oidc-ca-file flag.

audiences [Required]
[]string

audiences is the set of acceptable audiences the JWT must be issued to. At least one of the entries must match the "aud" claim in presented JWTs. Same value as the --oidc-client-id flag (though this field supports an array). Required to be non-empty.

audienceMatchPolicy
AudienceMatchPolicyType

audienceMatchPolicy defines how the "audiences" field is used to match the "aud" claim in the presented JWT. Allowed values are:

  1. "MatchAny" when multiple audiences are specified and
  2. empty (or unset) or "MatchAny" when a single audience is specified.
  • MatchAny: the "aud" claim in the presented JWT must match at least one of the entries in the "audiences" field. For example, if "audiences" is ["foo", "bar"], the "aud" claim in the presented JWT must contain either "foo" or "bar" (and may contain both).

  • "": The match policy can be empty (or unset) when a single audience is specified in the "audiences" field. The "aud" claim in the presented JWT must contain the single audience (and may contain others).

For more nuanced audience validation, use claimValidationRules. example: claimValidationRule[].expression: 'sets.equivalent(claims.aud, ["bar", "foo", "baz"])' to require an exact match.

JWTAuthenticator

Appears in:

JWTAuthenticator provides the configuration for a single JWT authenticator.

FieldDescription
issuer [Required]
Issuer

issuer contains the basic OIDC provider connection options.

claimValidationRules
[]ClaimValidationRule

claimValidationRules are rules that are applied to validate token claims to authenticate users.

claimMappings [Required]
ClaimMappings

claimMappings points claims of a token to be treated as user attributes.

userValidationRules
[]UserValidationRule

userValidationRules are rules that are applied to final user before completing authentication. These allow invariants to be applied to incoming identities such as preventing the use of the system: prefix that is commonly used by Kubernetes components. The validation rules are logically ANDed together and must all return true for the validation to pass.

PrefixedClaimOrExpression

Appears in:

PrefixedClaimOrExpression provides the configuration for a single prefixed claim or expression.

FieldDescription
claim
string

claim is the JWT claim to use. Mutually exclusive with expression.

prefix
string

prefix is prepended to claim's value to prevent clashes with existing names. prefix needs to be set if claim is set and can be the empty string. Mutually exclusive with expression.

expression
string

expression represents the expression which will be evaluated by CEL.

CEL expressions have access to the contents of the token claims, organized into CEL variable:

  • 'claims' is a map of claim names to claim values. For example, a variable named 'sub' can be accessed as 'claims.sub'. Nested claims can be accessed using dot notation, e.g. 'claims.foo.bar'.

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

Mutually exclusive with claim and prefix.

ProtocolType

(Alias of string)

Appears in:

ProtocolType is a set of valid values for Connection.ProtocolType

TCPTransport

Appears in:

TCPTransport provides the information to connect to konnectivity server via TCP

FieldDescription
url [Required]
string

URL is the location of the konnectivity server to connect to. As an example it might be "https://127.0.0.1:8131"

tlsConfig
TLSConfig

TLSConfig is the config needed to use TLS when connecting to konnectivity server

TLSConfig

Appears in:

TLSConfig provides the authentication information to connect to konnectivity server Only used with TCPTransport

FieldDescription
caBundle
string

caBundle is the file location of the CA to be used to determine trust with the konnectivity server. Must be absent/empty if TCPTransport.URL is prefixed with http:// If absent while TCPTransport.URL is prefixed with https://, default to system trust roots.

clientKey
string

clientKey is the file location of the client key to be used in mtls handshakes with the konnectivity server. Must be absent/empty if TCPTransport.URL is prefixed with http:// Must be configured if TCPTransport.URL is prefixed with https://

clientCert
string

clientCert is the file location of the client certificate to be used in mtls handshakes with the konnectivity server. Must be absent/empty if TCPTransport.URL is prefixed with http:// Must be configured if TCPTransport.URL is prefixed with https://

Transport

Appears in:

Transport defines the transport configurations we use to dial to the konnectivity server

FieldDescription
tcp
TCPTransport

TCP is the TCP configuration for communicating with the konnectivity server via TCP ProxyProtocol of GRPC is not supported with TCP transport at the moment Requires at least one of TCP or UDS to be set

uds
UDSTransport

UDS is the UDS configuration for communicating with the konnectivity server via UDS Requires at least one of TCP or UDS to be set

UDSTransport

Appears in:

UDSTransport provides the information to connect to konnectivity server via UDS

FieldDescription
udsName [Required]
string

UDSName is the name of the unix domain socket to connect to konnectivity server This does not use a unix:// prefix. (Eg: /etc/srv/kubernetes/konnectivity-server/konnectivity-server.socket)

UserValidationRule

Appears in:

UserValidationRule provides the configuration for a single user info validation rule.

FieldDescription
expression [Required]
string

expression represents the expression which will be evaluated by CEL. Must return true for the validation to pass.

CEL expressions have access to the contents of UserInfo, organized into CEL variable:

  • 'user' - authentication.k8s.io/v1, Kind=UserInfo object Refer to https://github.com/kubernetes/api/blob/release-1.28/authentication/v1/types.go#L105-L122 for the definition. API documentation: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#userinfo-v1-authentication-k8s-io

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

message
string

message customizes the returned error message when rule returns false. message is a literal string.

WebhookConfiguration

Appears in:

FieldDescription
authorizedTTL [Required]
meta/v1.Duration

The duration to cache 'authorized' responses from the webhook authorizer. Same as setting --authorization-webhook-cache-authorized-ttl flag Default: 5m0s

unauthorizedTTL [Required]
meta/v1.Duration

The duration to cache 'unauthorized' responses from the webhook authorizer. Same as setting --authorization-webhook-cache-unauthorized-ttl flag Default: 30s

timeout [Required]
meta/v1.Duration

Timeout for the webhook request Maximum allowed value is 30s. Required, no default value.

subjectAccessReviewVersion [Required]
string

The API version of the authorization.k8s.io SubjectAccessReview to send to and expect from the webhook. Same as setting --authorization-webhook-version flag Valid values: v1beta1, v1 Required, no default value

matchConditionSubjectAccessReviewVersion [Required]
string

MatchConditionSubjectAccessReviewVersion specifies the SubjectAccessReview version the CEL expressions are evaluated against Valid values: v1 Required, no default value

failurePolicy [Required]
string

Controls the authorization decision when a webhook request fails to complete or returns a malformed response or errors evaluating matchConditions. Valid values:

  • NoOpinion: continue to subsequent authorizers to see if one of them allows the request
  • Deny: reject the request without consulting subsequent authorizers Required, with no default.
connectionInfo [Required]
WebhookConnectionInfo

ConnectionInfo defines how we talk to the webhook

matchConditions [Required]
[]WebhookMatchCondition

matchConditions is a list of conditions that must be met for a request to be sent to this webhook. An empty list of matchConditions matches all requests. There are a maximum of 64 match conditions allowed.

The exact matching logic is (in order):

  1. If at least one matchCondition evaluates to FALSE, then the webhook is skipped.
  2. If ALL matchConditions evaluate to TRUE, then the webhook is called.
  3. If at least one matchCondition evaluates to an error (but none are FALSE):
    • If failurePolicy=Deny, then the webhook rejects the request
    • If failurePolicy=NoOpinion, then the error is ignored and the webhook is skipped

WebhookConnectionInfo

Appears in:

FieldDescription
type [Required]
string

Controls how the webhook should communicate with the server. Valid values:

  • KubeConfigFile: use the file specified in kubeConfigFile to locate the server.
  • InClusterConfig: use the in-cluster configuration to call the SubjectAccessReview API hosted by kube-apiserver. This mode is not allowed for kube-apiserver.
kubeConfigFile [Required]
string

Path to KubeConfigFile for connection info Required, if connectionInfo.Type is KubeConfig

WebhookMatchCondition

Appears in:

FieldDescription
expression [Required]
string

expression represents the expression which will be evaluated by CEL. Must evaluate to bool. CEL expressions have access to the contents of the SubjectAccessReview in v1 version. If version specified by subjectAccessReviewVersion in the request variable is v1beta1, the contents would be converted to the v1 version before evaluating the CEL expression.

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

9 - kube-apiserver Configuration (v1beta1)

Package v1beta1 is the v1beta1 version of the API.

Resource Types

TracingConfiguration

Appears in:

TracingConfiguration provides versioned configuration for OpenTelemetry tracing clients.

FieldDescription
endpoint
string

Endpoint of the collector this component will report traces to. The connection is insecure, and does not currently support TLS. Recommended is unset, and endpoint is the otlp grpc default, localhost:4317.

samplingRatePerMillion
int32

SamplingRatePerMillion is the number of samples to collect per million spans. Recommended is unset. If unset, sampler respects its parent span's sampling rate, but otherwise never samples.

AuthenticationConfiguration

AuthenticationConfiguration provides versioned configuration for authentication.

FieldDescription
apiVersion
string
apiserver.k8s.io/v1beta1
kind
string
AuthenticationConfiguration
jwt [Required]
[]JWTAuthenticator

jwt is a list of authenticator to authenticate Kubernetes users using JWT compliant tokens. The authenticator will attempt to parse a raw ID token, verify it's been signed by the configured issuer. The public key to verify the signature is discovered from the issuer's public endpoint using OIDC discovery. For an incoming token, each JWT authenticator will be attempted in the order in which it is specified in this list. Note however that other authenticators may run before or after the JWT authenticators. The specific position of JWT authenticators in relation to other authenticators is neither defined nor stable across releases. Since each JWT authenticator must have a unique issuer URL, at most one JWT authenticator will attempt to cryptographically validate the token.

The minimum valid JWT payload must contain the following claims: { "iss": "https://issuer.example.com", "aud": ["audience"], "exp": 1234567890, "<username claim>": "username" }

AuthorizationConfiguration

FieldDescription
apiVersion
string
apiserver.k8s.io/v1beta1
kind
string
AuthorizationConfiguration
authorizers [Required]
[]AuthorizerConfiguration

Authorizers is an ordered list of authorizers to authorize requests against. This is similar to the --authorization-modes kube-apiserver flag Must be at least one.

EgressSelectorConfiguration

EgressSelectorConfiguration provides versioned configuration for egress selector clients.

FieldDescription
apiVersion
string
apiserver.k8s.io/v1beta1
kind
string
EgressSelectorConfiguration
egressSelections [Required]
[]EgressSelection

connectionServices contains a list of egress selection client configurations

TracingConfiguration

TracingConfiguration provides versioned configuration for tracing clients.

FieldDescription
apiVersion
string
apiserver.k8s.io/v1beta1
kind
string
TracingConfiguration
TracingConfiguration [Required]
TracingConfiguration
(Members of TracingConfiguration are embedded into this type.)

Embed the component config tracing configuration struct

AudienceMatchPolicyType

(Alias of string)

Appears in:

AudienceMatchPolicyType is a set of valid values for issuer.audienceMatchPolicy

AuthorizerConfiguration

Appears in:

FieldDescription
type [Required]
string

Type refers to the type of the authorizer "Webhook" is supported in the generic API server Other API servers may support additional authorizer types like Node, RBAC, ABAC, etc.

name [Required]
string

Name used to describe the webhook This is explicitly used in monitoring machinery for metrics Note: Names must be DNS1123 labels like myauthorizername or subdomains like myauthorizer.example.domain Required, with no default

webhook [Required]
WebhookConfiguration

Webhook defines the configuration for a Webhook authorizer Must be defined when Type=Webhook Must not be defined when Type!=Webhook

ClaimMappings

Appears in:

ClaimMappings provides the configuration for claim mapping

FieldDescription
username [Required]
PrefixedClaimOrExpression

username represents an option for the username attribute. The claim's value must be a singular string. Same as the --oidc-username-claim and --oidc-username-prefix flags. If username.expression is set, the expression must produce a string value. If username.expression uses 'claims.email', then 'claims.email_verified' must be used in username.expression or extra[*].valueExpression or claimValidationRules[*].expression. An example claim validation rule expression that matches the validation automatically applied when username.claim is set to 'email' is 'claims.?email_verified.orValue(true)'.

In the flag based approach, the --oidc-username-claim and --oidc-username-prefix are optional. If --oidc-username-claim is not set, the default value is "sub". For the authentication config, there is no defaulting for claim or prefix. The claim and prefix must be set explicitly. For claim, if --oidc-username-claim was not set with legacy flag approach, configure username.claim="sub" in the authentication config. For prefix: (1) --oidc-username-prefix="-", no prefix was added to the username. For the same behavior using authentication config, set username.prefix="" (2) --oidc-username-prefix="" and --oidc-username-claim != "email", prefix was "<value of --oidc-issuer-url>#". For the same behavior using authentication config, set username.prefix="<value of issuer.url>#" (3) --oidc-username-prefix="<value>". For the same behavior using authentication config, set username.prefix="<value>"

groups
PrefixedClaimOrExpression

groups represents an option for the groups attribute. The claim's value must be a string or string array claim. If groups.claim is set, the prefix must be specified (and can be the empty string). If groups.expression is set, the expression must produce a string or string array value. "", [], and null values are treated as the group mapping not being present.

uid
ClaimOrExpression

uid represents an option for the uid attribute. Claim must be a singular string claim. If uid.expression is set, the expression must produce a string value.

extra
[]ExtraMapping

extra represents an option for the extra attribute. expression must produce a string or string array value. If the value is empty, the extra mapping will not be present.

hard-coded extra key/value

  • key: "foo" valueExpression: "'bar'" This will result in an extra attribute - foo: ["bar"]

hard-coded key, value copying claim value

  • key: "foo" valueExpression: "claims.some_claim" This will result in an extra attribute - foo: [value of some_claim]

hard-coded key, value derived from claim value

  • key: "admin" valueExpression: '(has(claims.is_admin) && claims.is_admin) ? "true":""' This will result in:
  • if is_admin claim is present and true, extra attribute - admin: ["true"]
  • if is_admin claim is present and false or is_admin claim is not present, no extra attribute will be added

ClaimOrExpression

Appears in:

ClaimOrExpression provides the configuration for a single claim or expression.

FieldDescription
claim
string

claim is the JWT claim to use. Either claim or expression must be set. Mutually exclusive with expression.

expression
string

expression represents the expression which will be evaluated by CEL.

CEL expressions have access to the contents of the token claims, organized into CEL variable:

  • 'claims' is a map of claim names to claim values. For example, a variable named 'sub' can be accessed as 'claims.sub'. Nested claims can be accessed using dot notation, e.g. 'claims.foo.bar'.

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

Mutually exclusive with claim.

ClaimValidationRule

Appears in:

ClaimValidationRule provides the configuration for a single claim validation rule.

FieldDescription
claim
string

claim is the name of a required claim. Same as --oidc-required-claim flag. Only string claim keys are supported. Mutually exclusive with expression and message.

requiredValue
string

requiredValue is the value of a required claim. Same as --oidc-required-claim flag. Only string claim values are supported. If claim is set and requiredValue is not set, the claim must be present with a value set to the empty string. Mutually exclusive with expression and message.

expression
string

expression represents the expression which will be evaluated by CEL. Must produce a boolean.

CEL expressions have access to the contents of the token claims, organized into CEL variable:

  • 'claims' is a map of claim names to claim values. For example, a variable named 'sub' can be accessed as 'claims.sub'. Nested claims can be accessed using dot notation, e.g. 'claims.foo.bar'. Must return true for the validation to pass.

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

Mutually exclusive with claim and requiredValue.

message
string

message customizes the returned error message when expression returns false. message is a literal string. Mutually exclusive with claim and requiredValue.

Connection

Appears in:

Connection provides the configuration for a single egress selection client.

FieldDescription
proxyProtocol [Required]
ProtocolType

Protocol is the protocol used to connect from client to the konnectivity server.

transport
Transport

Transport defines the transport configurations we use to dial to the konnectivity server. This is required if ProxyProtocol is HTTPConnect or GRPC.

EgressSelection

Appears in:

EgressSelection provides the configuration for a single egress selection client.

FieldDescription
name [Required]
string

name is the name of the egress selection. Currently supported values are "controlplane", "master", "etcd" and "cluster" The "master" egress selector is deprecated in favor of "controlplane"

connection [Required]
Connection

connection is the exact information used to configure the egress selection

ExtraMapping

Appears in:

ExtraMapping provides the configuration for a single extra mapping.

FieldDescription
key [Required]
string

key is a string to use as the extra attribute key. key must be a domain-prefix path (e.g. example.org/foo). All characters before the first "/" must be a valid subdomain as defined by RFC 1123. All characters trailing the first "/" must be valid HTTP Path characters as defined by RFC 3986. key must be lowercase. Required to be unique.

valueExpression [Required]
string

valueExpression is a CEL expression to extract extra attribute value. valueExpression must produce a string or string array value. "", [], and null values are treated as the extra mapping not being present. Empty string values contained within a string array are filtered out.

CEL expressions have access to the contents of the token claims, organized into CEL variable:

  • 'claims' is a map of claim names to claim values. For example, a variable named 'sub' can be accessed as 'claims.sub'. Nested claims can be accessed using dot notation, e.g. 'claims.foo.bar'.

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

Issuer

Appears in:

Issuer provides the configuration for an external provider's specific settings.

FieldDescription
url [Required]
string

url points to the issuer URL in a format https://url or https://url/path. This must match the "iss" claim in the presented JWT, and the issuer returned from discovery. Same value as the --oidc-issuer-url flag. Discovery information is fetched from "{url}/.well-known/openid-configuration" unless overridden by discoveryURL. Required to be unique across all JWT authenticators. Note that egress selection configuration is not used for this network connection.

discoveryURL
string

discoveryURL, if specified, overrides the URL used to fetch discovery information instead of using "{url}/.well-known/openid-configuration". The exact value specified is used, so "/.well-known/openid-configuration" must be included in discoveryURL if needed.

The "issuer" field in the fetched discovery information must match the "issuer.url" field in the AuthenticationConfiguration and will be used to validate the "iss" claim in the presented JWT. This is for scenarios where the well-known and jwks endpoints are hosted at a different location than the issuer (such as locally in the cluster).

Example: A discovery url that is exposed using kubernetes service 'oidc' in namespace 'oidc-namespace' and discovery information is available at '/.well-known/openid-configuration'. discoveryURL: "https://oidc.oidc-namespace/.well-known/openid-configuration" certificateAuthority is used to verify the TLS connection and the hostname on the leaf certificate must be set to 'oidc.oidc-namespace'.

curl https://oidc.oidc-namespace/.well-known/openid-configuration (.discoveryURL field) { issuer: "https://oidc.example.com" (.url field) }

discoveryURL must be different from url. Required to be unique across all JWT authenticators. Note that egress selection configuration is not used for this network connection.

certificateAuthority
string

certificateAuthority contains PEM-encoded certificate authority certificates used to validate the connection when fetching discovery information. If unset, the system verifier is used. Same value as the content of the file referenced by the --oidc-ca-file flag.

audiences [Required]
[]string

audiences is the set of acceptable audiences the JWT must be issued to. At least one of the entries must match the "aud" claim in presented JWTs. Same value as the --oidc-client-id flag (though this field supports an array). Required to be non-empty.

audienceMatchPolicy
AudienceMatchPolicyType

audienceMatchPolicy defines how the "audiences" field is used to match the "aud" claim in the presented JWT. Allowed values are:

  1. "MatchAny" when multiple audiences are specified and
  2. empty (or unset) or "MatchAny" when a single audience is specified.
  • MatchAny: the "aud" claim in the presented JWT must match at least one of the entries in the "audiences" field. For example, if "audiences" is ["foo", "bar"], the "aud" claim in the presented JWT must contain either "foo" or "bar" (and may contain both).

  • "": The match policy can be empty (or unset) when a single audience is specified in the "audiences" field. The "aud" claim in the presented JWT must contain the single audience (and may contain others).

For more nuanced audience validation, use claimValidationRules. example: claimValidationRule[].expression: 'sets.equivalent(claims.aud, ["bar", "foo", "baz"])' to require an exact match.

JWTAuthenticator

Appears in:

JWTAuthenticator provides the configuration for a single JWT authenticator.

FieldDescription
issuer [Required]
Issuer

issuer contains the basic OIDC provider connection options.

claimValidationRules
[]ClaimValidationRule

claimValidationRules are rules that are applied to validate token claims to authenticate users.

claimMappings [Required]
ClaimMappings

claimMappings points claims of a token to be treated as user attributes.

userValidationRules
[]UserValidationRule

userValidationRules are rules that are applied to final user before completing authentication. These allow invariants to be applied to incoming identities such as preventing the use of the system: prefix that is commonly used by Kubernetes components. The validation rules are logically ANDed together and must all return true for the validation to pass.

PrefixedClaimOrExpression

Appears in:

PrefixedClaimOrExpression provides the configuration for a single prefixed claim or expression.

FieldDescription
claim
string

claim is the JWT claim to use. Mutually exclusive with expression.

prefix
string

prefix is prepended to claim's value to prevent clashes with existing names. prefix needs to be set if claim is set and can be the empty string. Mutually exclusive with expression.

expression
string

expression represents the expression which will be evaluated by CEL.

CEL expressions have access to the contents of the token claims, organized into CEL variable:

  • 'claims' is a map of claim names to claim values. For example, a variable named 'sub' can be accessed as 'claims.sub'. Nested claims can be accessed using dot notation, e.g. 'claims.foo.bar'.

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

Mutually exclusive with claim and prefix.

ProtocolType

(Alias of string)

Appears in:

ProtocolType is a set of valid values for Connection.ProtocolType

TCPTransport

Appears in:

TCPTransport provides the information to connect to konnectivity server via TCP

FieldDescription
url [Required]
string

URL is the location of the konnectivity server to connect to. As an example it might be "https://127.0.0.1:8131"

tlsConfig
TLSConfig

TLSConfig is the config needed to use TLS when connecting to konnectivity server

TLSConfig

Appears in:

TLSConfig provides the authentication information to connect to konnectivity server Only used with TCPTransport

FieldDescription
caBundle
string

caBundle is the file location of the CA to be used to determine trust with the konnectivity server. Must be absent/empty if TCPTransport.URL is prefixed with http:// If absent while TCPTransport.URL is prefixed with https://, default to system trust roots.

clientKey
string

clientKey is the file location of the client key to be used in mtls handshakes with the konnectivity server. Must be absent/empty if TCPTransport.URL is prefixed with http:// Must be configured if TCPTransport.URL is prefixed with https://

clientCert
string

clientCert is the file location of the client certificate to be used in mtls handshakes with the konnectivity server. Must be absent/empty if TCPTransport.URL is prefixed with http:// Must be configured if TCPTransport.URL is prefixed with https://

Transport

Appears in:

Transport defines the transport configurations we use to dial to the konnectivity server

FieldDescription
tcp
TCPTransport

TCP is the TCP configuration for communicating with the konnectivity server via TCP ProxyProtocol of GRPC is not supported with TCP transport at the moment Requires at least one of TCP or UDS to be set

uds
UDSTransport

UDS is the UDS configuration for communicating with the konnectivity server via UDS Requires at least one of TCP or UDS to be set

UDSTransport

Appears in:

UDSTransport provides the information to connect to konnectivity server via UDS

FieldDescription
udsName [Required]
string

UDSName is the name of the unix domain socket to connect to konnectivity server This does not use a unix:// prefix. (Eg: /etc/srv/kubernetes/konnectivity-server/konnectivity-server.socket)

UserValidationRule

Appears in:

UserValidationRule provides the configuration for a single user info validation rule.

FieldDescription
expression [Required]
string

expression represents the expression which will be evaluated by CEL. Must return true for the validation to pass.

CEL expressions have access to the contents of UserInfo, organized into CEL variable:

  • 'user' - authentication.k8s.io/v1, Kind=UserInfo object Refer to https://github.com/kubernetes/api/blob/release-1.28/authentication/v1/types.go#L105-L122 for the definition. API documentation: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#userinfo-v1-authentication-k8s-io

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

message
string

message customizes the returned error message when rule returns false. message is a literal string.

WebhookConfiguration

Appears in:

FieldDescription
authorizedTTL [Required]
meta/v1.Duration

The duration to cache 'authorized' responses from the webhook authorizer. Same as setting --authorization-webhook-cache-authorized-ttl flag Default: 5m0s

unauthorizedTTL [Required]
meta/v1.Duration

The duration to cache 'unauthorized' responses from the webhook authorizer. Same as setting --authorization-webhook-cache-unauthorized-ttl flag Default: 30s

timeout [Required]
meta/v1.Duration

Timeout for the webhook request Maximum allowed value is 30s. Required, no default value.

subjectAccessReviewVersion [Required]
string

The API version of the authorization.k8s.io SubjectAccessReview to send to and expect from the webhook. Same as setting --authorization-webhook-version flag Valid values: v1beta1, v1 Required, no default value

matchConditionSubjectAccessReviewVersion [Required]
string

MatchConditionSubjectAccessReviewVersion specifies the SubjectAccessReview version the CEL expressions are evaluated against Valid values: v1 Required, no default value

failurePolicy [Required]
string

Controls the authorization decision when a webhook request fails to complete or returns a malformed response or errors evaluating matchConditions. Valid values:

  • NoOpinion: continue to subsequent authorizers to see if one of them allows the request
  • Deny: reject the request without consulting subsequent authorizers Required, with no default.
connectionInfo [Required]
WebhookConnectionInfo

ConnectionInfo defines how we talk to the webhook

matchConditions [Required]
[]WebhookMatchCondition

matchConditions is a list of conditions that must be met for a request to be sent to this webhook. An empty list of matchConditions matches all requests. There are a maximum of 64 match conditions allowed.

The exact matching logic is (in order):

  1. If at least one matchCondition evaluates to FALSE, then the webhook is skipped.
  2. If ALL matchConditions evaluate to TRUE, then the webhook is called.
  3. If at least one matchCondition evaluates to an error (but none are FALSE):
    • If failurePolicy=Deny, then the webhook rejects the request
    • If failurePolicy=NoOpinion, then the error is ignored and the webhook is skipped

WebhookConnectionInfo

Appears in:

FieldDescription
type [Required]
string

Controls how the webhook should communicate with the server. Valid values:

  • KubeConfigFile: use the file specified in kubeConfigFile to locate the server.
  • InClusterConfig: use the in-cluster configuration to call the SubjectAccessReview API hosted by kube-apiserver. This mode is not allowed for kube-apiserver.
kubeConfigFile [Required]
string

Path to KubeConfigFile for connection info Required, if connectionInfo.Type is KubeConfig

WebhookMatchCondition

Appears in:

FieldDescription
expression [Required]
string

expression represents the expression which will be evaluated by CEL. Must evaluate to bool. CEL expressions have access to the contents of the SubjectAccessReview in v1 version. If version specified by subjectAccessReviewVersion in the request variable is v1beta1, the contents would be converted to the v1 version before evaluating the CEL expression.

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

10 - kube-controller-manager Configuration (v1alpha1)

Resource Types

NodeControllerConfiguration

Appears in:

NodeControllerConfiguration contains elements describing NodeController.

FieldDescription
ConcurrentNodeSyncs [Required]
int32

ConcurrentNodeSyncs is the number of workers concurrently synchronizing nodes

ServiceControllerConfiguration

Appears in:

ServiceControllerConfiguration contains elements describing ServiceController.

FieldDescription
ConcurrentServiceSyncs [Required]
int32

concurrentServiceSyncs is the number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load.

CloudControllerManagerConfiguration

CloudControllerManagerConfiguration contains elements describing cloud-controller manager.

FieldDescription
apiVersion
string
cloudcontrollermanager.config.k8s.io/v1alpha1
kind
string
CloudControllerManagerConfiguration
Generic [Required]
GenericControllerManagerConfiguration

Generic holds configuration for a generic controller-manager

KubeCloudShared [Required]
KubeCloudSharedConfiguration

KubeCloudSharedConfiguration holds configuration for shared related features both in cloud controller manager and kube-controller manager.

NodeController [Required]
NodeControllerConfiguration

NodeController holds configuration for node controller related features.

ServiceController [Required]
ServiceControllerConfiguration

ServiceControllerConfiguration holds configuration for ServiceController related features.

NodeStatusUpdateFrequency [Required]
meta/v1.Duration

NodeStatusUpdateFrequency is the frequency at which the controller updates nodes' status

Webhook [Required]
WebhookConfiguration

Webhook is the configuration for cloud-controller-manager hosted webhooks

CloudProviderConfiguration

Appears in:

CloudProviderConfiguration contains basically elements about cloud provider.

FieldDescription
Name [Required]
string

Name is the provider for cloud services.

CloudConfigFile [Required]
string

cloudConfigFile is the path to the cloud provider configuration file.

KubeCloudSharedConfiguration

Appears in:

KubeCloudSharedConfiguration contains elements shared by both kube-controller manager and cloud-controller manager, but not genericconfig.

FieldDescription
CloudProvider [Required]
CloudProviderConfiguration

CloudProviderConfiguration holds configuration for CloudProvider related features.

ExternalCloudVolumePlugin [Required]
string

externalCloudVolumePlugin specifies the plugin to use when cloudProvider is "external". It is currently used by the in repo cloud providers to handle node and volume control in the KCM.

UseServiceAccountCredentials [Required]
bool

useServiceAccountCredentials indicates whether controllers should be run with individual service account credentials.

AllowUntaggedCloud [Required]
bool

run with untagged cloud instances

RouteReconciliationPeriod [Required]
meta/v1.Duration

routeReconciliationPeriod is the period for reconciling routes created for Nodes by cloud provider..

NodeMonitorPeriod [Required]
meta/v1.Duration

nodeMonitorPeriod is the period for syncing NodeStatus in NodeController.

ClusterName [Required]
string

clusterName is the instance prefix for the cluster.

ClusterCIDR [Required]
string

clusterCIDR is CIDR Range for Pods in cluster.

AllocateNodeCIDRs [Required]
bool

AllocateNodeCIDRs enables CIDRs for Pods to be allocated and, if ConfigureCloudRoutes is true, to be set on the cloud provider.

CIDRAllocatorType [Required]
string

CIDRAllocatorType determines what kind of pod CIDR allocator will be used.

ConfigureCloudRoutes [Required]
bool

configureCloudRoutes enables CIDRs allocated with allocateNodeCIDRs to be configured on the cloud provider.

NodeSyncPeriod [Required]
meta/v1.Duration

nodeSyncPeriod is the period for syncing nodes from cloudprovider. Longer periods will result in fewer calls to cloud provider, but may delay addition of new nodes to cluster.

WebhookConfiguration

Appears in:

WebhookConfiguration contains configuration related to cloud-controller-manager hosted webhooks

FieldDescription
Webhooks [Required]
[]string

Webhooks is the list of webhooks to enable or disable '*' means "all enabled by default webhooks" 'foo' means "enable 'foo'" '-foo' means "disable 'foo'" first item for a particular name wins

LeaderMigrationConfiguration

Appears in:

LeaderMigrationConfiguration provides versioned configuration for all migrating leader locks.

FieldDescription
apiVersion
string
controllermanager.config.k8s.io/v1alpha1
kind
string
LeaderMigrationConfiguration
leaderName [Required]
string

LeaderName is the name of the leader election resource that protects the migration E.g. 1-20-KCM-to-1-21-CCM

resourceLock [Required]
string

ResourceLock indicates the resource object type that will be used to lock Should be "leases" or "endpoints"

controllerLeaders [Required]
[]ControllerLeaderConfiguration

ControllerLeaders contains a list of migrating leader lock configurations

ControllerLeaderConfiguration

Appears in:

ControllerLeaderConfiguration provides the configuration for a migrating leader lock.

FieldDescription
name [Required]
string

Name is the name of the controller being migrated E.g. service-controller, route-controller, cloud-node-controller, etc

component [Required]
string

Component is the name of the component in which the controller should be running. E.g. kube-controller-manager, cloud-controller-manager, etc Or '*' meaning the controller can be run under any component that participates in the migration

GenericControllerManagerConfiguration

Appears in:

GenericControllerManagerConfiguration holds configuration for a generic controller-manager.

FieldDescription
Port [Required]
int32

port is the port that the controller-manager's http service runs on.

Address [Required]
string

address is the IP address to serve on (set to 0.0.0.0 for all interfaces).

MinResyncPeriod [Required]
meta/v1.Duration

minResyncPeriod is the resync period in reflectors; will be random between minResyncPeriod and 2*minResyncPeriod.

ClientConnection [Required]
ClientConnectionConfiguration

ClientConnection specifies the kubeconfig file and client connection settings for the proxy server to use when communicating with the apiserver.

ControllerStartInterval [Required]
meta/v1.Duration

How long to wait between starting controller managers

LeaderElection [Required]
LeaderElectionConfiguration

leaderElection defines the configuration of leader election client.

Controllers [Required]
[]string

Controllers is the list of controllers to enable or disable '*' means "all enabled by default controllers" 'foo' means "enable 'foo'" '-foo' means "disable 'foo'" first item for a particular name wins

Debugging [Required]
DebuggingConfiguration

DebuggingConfiguration holds configuration for Debugging related features.

LeaderMigrationEnabled [Required]
bool

LeaderMigrationEnabled indicates whether Leader Migration should be enabled for the controller manager.

LeaderMigration [Required]
LeaderMigrationConfiguration

LeaderMigration holds the configuration for Leader Migration.

KubeControllerManagerConfiguration

KubeControllerManagerConfiguration contains elements describing kube-controller manager.

FieldDescription
apiVersion
string
kubecontrollermanager.config.k8s.io/v1alpha1
kind
string
KubeControllerManagerConfiguration
Generic [Required]
GenericControllerManagerConfiguration

Generic holds configuration for a generic controller-manager

KubeCloudShared [Required]
KubeCloudSharedConfiguration

KubeCloudSharedConfiguration holds configuration for shared related features both in cloud controller manager and kube-controller manager.

AttachDetachController [Required]
AttachDetachControllerConfiguration

AttachDetachControllerConfiguration holds configuration for AttachDetachController related features.

CSRSigningController [Required]
CSRSigningControllerConfiguration

CSRSigningControllerConfiguration holds configuration for CSRSigningController related features.

DaemonSetController [Required]
DaemonSetControllerConfiguration

DaemonSetControllerConfiguration holds configuration for DaemonSetController related features.

DeploymentController [Required]
DeploymentControllerConfiguration

DeploymentControllerConfiguration holds configuration for DeploymentController related features.

StatefulSetController [Required]
StatefulSetControllerConfiguration

StatefulSetControllerConfiguration holds configuration for StatefulSetController related features.

DeprecatedController [Required]
DeprecatedControllerConfiguration

DeprecatedControllerConfiguration holds configuration for some deprecated features.

EndpointController [Required]
EndpointControllerConfiguration

EndpointControllerConfiguration holds configuration for EndpointController related features.

EndpointSliceController [Required]
EndpointSliceControllerConfiguration

EndpointSliceControllerConfiguration holds configuration for EndpointSliceController related features.

EndpointSliceMirroringController [Required]
EndpointSliceMirroringControllerConfiguration

EndpointSliceMirroringControllerConfiguration holds configuration for EndpointSliceMirroringController related features.

EphemeralVolumeController [Required]
EphemeralVolumeControllerConfiguration

EphemeralVolumeControllerConfiguration holds configuration for EphemeralVolumeController related features.

GarbageCollectorController [Required]
GarbageCollectorControllerConfiguration

GarbageCollectorControllerConfiguration holds configuration for GarbageCollectorController related features.

HPAController [Required]
HPAControllerConfiguration

HPAControllerConfiguration holds configuration for HPAController related features.

JobController [Required]
JobControllerConfiguration

JobControllerConfiguration holds configuration for JobController related features.

CronJobController [Required]
CronJobControllerConfiguration

CronJobControllerConfiguration holds configuration for CronJobController related features.

LegacySATokenCleaner [Required]
LegacySATokenCleanerConfiguration

LegacySATokenCleanerConfiguration holds configuration for LegacySATokenCleaner related features.

NamespaceController [Required]
NamespaceControllerConfiguration

NamespaceControllerConfiguration holds configuration for NamespaceController related features.

NodeIPAMController [Required]
NodeIPAMControllerConfiguration

NodeIPAMControllerConfiguration holds configuration for NodeIPAMController related features.

NodeLifecycleController [Required]
NodeLifecycleControllerConfiguration

NodeLifecycleControllerConfiguration holds configuration for NodeLifecycleController related features.

PersistentVolumeBinderController [Required]
PersistentVolumeBinderControllerConfiguration

PersistentVolumeBinderControllerConfiguration holds configuration for PersistentVolumeBinderController related features.

PodGCController [Required]
PodGCControllerConfiguration

PodGCControllerConfiguration holds configuration for PodGCController related features.

ReplicaSetController [Required]
ReplicaSetControllerConfiguration

ReplicaSetControllerConfiguration holds configuration for ReplicaSet related features.

ReplicationController [Required]
ReplicationControllerConfiguration

ReplicationControllerConfiguration holds configuration for ReplicationController related features.

ResourceQuotaController [Required]
ResourceQuotaControllerConfiguration

ResourceQuotaControllerConfiguration holds configuration for ResourceQuotaController related features.

SAController [Required]
SAControllerConfiguration

SAControllerConfiguration holds configuration for ServiceAccountController related features.

ServiceController [Required]
ServiceControllerConfiguration

ServiceControllerConfiguration holds configuration for ServiceController related features.

TTLAfterFinishedController [Required]
TTLAfterFinishedControllerConfiguration

TTLAfterFinishedControllerConfiguration holds configuration for TTLAfterFinishedController related features.

ValidatingAdmissionPolicyStatusController [Required]
ValidatingAdmissionPolicyStatusControllerConfiguration

ValidatingAdmissionPolicyStatusControllerConfiguration holds configuration for ValidatingAdmissionPolicyStatusController related features.

AttachDetachControllerConfiguration

Appears in:

AttachDetachControllerConfiguration contains elements describing AttachDetachController.

FieldDescription
DisableAttachDetachReconcilerSync [Required]
bool

Reconciler runs a periodic loop to reconcile the desired state of the with the actual state of the world by triggering attach detach operations. This flag enables or disables reconcile. Is false by default, and thus enabled.

ReconcilerSyncLoopPeriod [Required]
meta/v1.Duration

ReconcilerSyncLoopPeriod is the amount of time the reconciler sync states loop wait between successive executions. Is set to 60 sec by default.

disableForceDetachOnTimeout [Required]
bool

DisableForceDetachOnTimeout disables force detach when the maximum unmount time is exceeded. Is false by default, and thus force detach on unmount is enabled.

CSRSigningConfiguration

Appears in:

CSRSigningConfiguration holds information about a particular CSR signer

FieldDescription
CertFile [Required]
string

certFile is the filename containing a PEM-encoded X509 CA certificate used to issue certificates

KeyFile [Required]
string

keyFile is the filename containing a PEM-encoded RSA or ECDSA private key used to issue certificates

CSRSigningControllerConfiguration

Appears in:

CSRSigningControllerConfiguration contains elements describing CSRSigningController.

FieldDescription
ClusterSigningCertFile [Required]
string

clusterSigningCertFile is the filename containing a PEM-encoded X509 CA certificate used to issue cluster-scoped certificates

ClusterSigningKeyFile [Required]
string

clusterSigningCertFile is the filename containing a PEM-encoded RSA or ECDSA private key used to issue cluster-scoped certificates

KubeletServingSignerConfiguration [Required]
CSRSigningConfiguration

kubeletServingSignerConfiguration holds the certificate and key used to issue certificates for the kubernetes.io/kubelet-serving signer

KubeletClientSignerConfiguration [Required]
CSRSigningConfiguration

kubeletClientSignerConfiguration holds the certificate and key used to issue certificates for the kubernetes.io/kube-apiserver-client-kubelet

KubeAPIServerClientSignerConfiguration [Required]
CSRSigningConfiguration

kubeAPIServerClientSignerConfiguration holds the certificate and key used to issue certificates for the kubernetes.io/kube-apiserver-client

LegacyUnknownSignerConfiguration [Required]
CSRSigningConfiguration

legacyUnknownSignerConfiguration holds the certificate and key used to issue certificates for the kubernetes.io/legacy-unknown

ClusterSigningDuration [Required]
meta/v1.Duration

clusterSigningDuration is the max length of duration signed certificates will be given. Individual CSRs may request shorter certs by setting spec.expirationSeconds.

CronJobControllerConfiguration

Appears in:

CronJobControllerConfiguration contains elements describing CrongJob2Controller.

FieldDescription
ConcurrentCronJobSyncs [Required]
int32

concurrentCronJobSyncs is the number of job objects that are allowed to sync concurrently. Larger number = more responsive jobs, but more CPU (and network) load.

DaemonSetControllerConfiguration

Appears in:

DaemonSetControllerConfiguration contains elements describing DaemonSetController.

FieldDescription
ConcurrentDaemonSetSyncs [Required]
int32

concurrentDaemonSetSyncs is the number of daemonset objects that are allowed to sync concurrently. Larger number = more responsive daemonset, but more CPU (and network) load.

DeploymentControllerConfiguration

Appears in:

DeploymentControllerConfiguration contains elements describing DeploymentController.

FieldDescription
ConcurrentDeploymentSyncs [Required]
int32

concurrentDeploymentSyncs is the number of deployment objects that are allowed to sync concurrently. Larger number = more responsive deployments, but more CPU (and network) load.

DeprecatedControllerConfiguration

Appears in:

DeprecatedControllerConfiguration contains elements be deprecated.

EndpointControllerConfiguration

Appears in:

EndpointControllerConfiguration contains elements describing EndpointController.

FieldDescription
ConcurrentEndpointSyncs [Required]
int32

concurrentEndpointSyncs is the number of endpoint syncing operations that will be done concurrently. Larger number = faster endpoint updating, but more CPU (and network) load.

EndpointUpdatesBatchPeriod [Required]
meta/v1.Duration

EndpointUpdatesBatchPeriod describes the length of endpoint updates batching period. Processing of pod changes will be delayed by this duration to join them with potential upcoming updates and reduce the overall number of endpoints updates.

EndpointSliceControllerConfiguration

Appears in:

EndpointSliceControllerConfiguration contains elements describing EndpointSliceController.

FieldDescription
ConcurrentServiceEndpointSyncs [Required]
int32

concurrentServiceEndpointSyncs is the number of service endpoint syncing operations that will be done concurrently. Larger number = faster endpoint slice updating, but more CPU (and network) load.

MaxEndpointsPerSlice [Required]
int32

maxEndpointsPerSlice is the maximum number of endpoints that will be added to an EndpointSlice. More endpoints per slice will result in fewer and larger endpoint slices, but larger resources.

EndpointUpdatesBatchPeriod [Required]
meta/v1.Duration

EndpointUpdatesBatchPeriod describes the length of endpoint updates batching period. Processing of pod changes will be delayed by this duration to join them with potential upcoming updates and reduce the overall number of endpoints updates.

EndpointSliceMirroringControllerConfiguration

Appears in:

EndpointSliceMirroringControllerConfiguration contains elements describing EndpointSliceMirroringController.

FieldDescription
MirroringConcurrentServiceEndpointSyncs [Required]
int32

mirroringConcurrentServiceEndpointSyncs is the number of service endpoint syncing operations that will be done concurrently. Larger number = faster endpoint slice updating, but more CPU (and network) load.

MirroringMaxEndpointsPerSubset [Required]
int32

mirroringMaxEndpointsPerSubset is the maximum number of endpoints that will be mirrored to an EndpointSlice for an EndpointSubset.

MirroringEndpointUpdatesBatchPeriod [Required]
meta/v1.Duration

mirroringEndpointUpdatesBatchPeriod can be used to batch EndpointSlice updates. All updates triggered by EndpointSlice changes will be delayed by up to 'mirroringEndpointUpdatesBatchPeriod'. If other addresses in the same Endpoints resource change in that period, they will be batched to a single EndpointSlice update. Default 0 value means that each Endpoints update triggers an EndpointSlice update.

EphemeralVolumeControllerConfiguration

Appears in:

EphemeralVolumeControllerConfiguration contains elements describing EphemeralVolumeController.

FieldDescription
ConcurrentEphemeralVolumeSyncs [Required]
int32

ConcurrentEphemeralVolumeSyncseSyncs is the number of ephemeral volume syncing operations that will be done concurrently. Larger number = faster ephemeral volume updating, but more CPU (and network) load.

GarbageCollectorControllerConfiguration

Appears in:

GarbageCollectorControllerConfiguration contains elements describing GarbageCollectorController.

FieldDescription
EnableGarbageCollector [Required]
bool

enables the generic garbage collector. MUST be synced with the corresponding flag of the kube-apiserver. WARNING: the generic garbage collector is an alpha feature.

ConcurrentGCSyncs [Required]
int32

concurrentGCSyncs is the number of garbage collector workers that are allowed to sync concurrently.

GCIgnoredResources [Required]
[]GroupResource

gcIgnoredResources is the list of GroupResources that garbage collection should ignore.

GroupResource

Appears in:

GroupResource describes an group resource.

FieldDescription
Group [Required]
string

group is the group portion of the GroupResource.

Resource [Required]
string

resource is the resource portion of the GroupResource.

HPAControllerConfiguration

Appears in:

HPAControllerConfiguration contains elements describing HPAController.

FieldDescription
ConcurrentHorizontalPodAutoscalerSyncs [Required]
int32

ConcurrentHorizontalPodAutoscalerSyncs is the number of HPA objects that are allowed to sync concurrently. Larger number = more responsive HPA processing, but more CPU (and network) load.

HorizontalPodAutoscalerSyncPeriod [Required]
meta/v1.Duration

HorizontalPodAutoscalerSyncPeriod is the period for syncing the number of pods in horizontal pod autoscaler.

HorizontalPodAutoscalerUpscaleForbiddenWindow [Required]
meta/v1.Duration

HorizontalPodAutoscalerUpscaleForbiddenWindow is a period after which next upscale allowed.

HorizontalPodAutoscalerDownscaleStabilizationWindow [Required]
meta/v1.Duration

HorizontalPodAutoscalerDowncaleStabilizationWindow is a period for which autoscaler will look backwards and not scale down below any recommendation it made during that period.

HorizontalPodAutoscalerDownscaleForbiddenWindow [Required]
meta/v1.Duration

HorizontalPodAutoscalerDownscaleForbiddenWindow is a period after which next downscale allowed.

HorizontalPodAutoscalerTolerance [Required]
float64

HorizontalPodAutoscalerTolerance is the tolerance for when resource usage suggests upscaling/downscaling

HorizontalPodAutoscalerCPUInitializationPeriod [Required]
meta/v1.Duration

HorizontalPodAutoscalerCPUInitializationPeriod is the period after pod start when CPU samples might be skipped.

HorizontalPodAutoscalerInitialReadinessDelay [Required]
meta/v1.Duration

HorizontalPodAutoscalerInitialReadinessDelay is period after pod start during which readiness changes are treated as readiness being set for the first time. The only effect of this is that HPA will disregard CPU samples from unready pods that had last readiness change during that period.

JobControllerConfiguration

Appears in:

JobControllerConfiguration contains elements describing JobController.

FieldDescription
ConcurrentJobSyncs [Required]
int32

concurrentJobSyncs is the number of job objects that are allowed to sync concurrently. Larger number = more responsive jobs, but more CPU (and network) load.

LegacySATokenCleanerConfiguration

Appears in:

LegacySATokenCleanerConfiguration contains elements describing LegacySATokenCleaner

FieldDescription
CleanUpPeriod [Required]
meta/v1.Duration

CleanUpPeriod is the period of time since the last usage of an auto-generated service account token before it can be deleted.

NamespaceControllerConfiguration

Appears in:

NamespaceControllerConfiguration contains elements describing NamespaceController.

FieldDescription
NamespaceSyncPeriod [Required]
meta/v1.Duration

namespaceSyncPeriod is the period for syncing namespace life-cycle updates.

ConcurrentNamespaceSyncs [Required]
int32

concurrentNamespaceSyncs is the number of namespace objects that are allowed to sync concurrently.

NodeIPAMControllerConfiguration

Appears in:

NodeIPAMControllerConfiguration contains elements describing NodeIpamController.

FieldDescription
ServiceCIDR [Required]
string

serviceCIDR is CIDR Range for Services in cluster.

SecondaryServiceCIDR [Required]
string

secondaryServiceCIDR is CIDR Range for Services in cluster. This is used in dual stack clusters. SecondaryServiceCIDR must be of different IP family than ServiceCIDR

NodeCIDRMaskSize [Required]
int32

NodeCIDRMaskSize is the mask size for node cidr in cluster.

NodeCIDRMaskSizeIPv4 [Required]
int32

NodeCIDRMaskSizeIPv4 is the mask size for node cidr in dual-stack cluster.

NodeCIDRMaskSizeIPv6 [Required]
int32

NodeCIDRMaskSizeIPv6 is the mask size for node cidr in dual-stack cluster.

NodeLifecycleControllerConfiguration

Appears in:

NodeLifecycleControllerConfiguration contains elements describing NodeLifecycleController.

FieldDescription
NodeEvictionRate [Required]
float32

nodeEvictionRate is the number of nodes per second on which pods are deleted in case of node failure when a zone is healthy

SecondaryNodeEvictionRate [Required]
float32

secondaryNodeEvictionRate is the number of nodes per second on which pods are deleted in case of node failure when a zone is unhealthy

NodeStartupGracePeriod [Required]
meta/v1.Duration

nodeStartupGracePeriod is the amount of time which we allow starting a node to be unresponsive before marking it unhealthy.

NodeMonitorGracePeriod [Required]
meta/v1.Duration

nodeMontiorGracePeriod is the amount of time which we allow a running node to be unresponsive before marking it unhealthy. Must be N times more than kubelet's nodeStatusUpdateFrequency, where N means number of retries allowed for kubelet to post node status.

PodEvictionTimeout [Required]
meta/v1.Duration

podEvictionTimeout is the grace period for deleting pods on failed nodes.

LargeClusterSizeThreshold [Required]
int32

secondaryNodeEvictionRate is implicitly overridden to 0 for clusters smaller than or equal to largeClusterSizeThreshold

UnhealthyZoneThreshold [Required]
float32

Zone is treated as unhealthy in nodeEvictionRate and secondaryNodeEvictionRate when at least unhealthyZoneThreshold (no less than 3) of Nodes in the zone are NotReady

PersistentVolumeBinderControllerConfiguration

Appears in:

PersistentVolumeBinderControllerConfiguration contains elements describing PersistentVolumeBinderController.

FieldDescription
PVClaimBinderSyncPeriod [Required]
meta/v1.Duration

pvClaimBinderSyncPeriod is the period for syncing persistent volumes and persistent volume claims.

VolumeConfiguration [Required]
VolumeConfiguration

volumeConfiguration holds configuration for volume related features.

VolumeHostCIDRDenylist [Required]
[]string

DEPRECATED: VolumeHostCIDRDenylist is a list of CIDRs that should not be reachable by the controller from plugins.

VolumeHostAllowLocalLoopback [Required]
bool

DEPRECATED: VolumeHostAllowLocalLoopback indicates if local loopback hosts (127.0.0.1, etc) should be allowed from plugins.

PersistentVolumeRecyclerConfiguration

Appears in:

PersistentVolumeRecyclerConfiguration contains elements describing persistent volume plugins.

FieldDescription
MaximumRetry [Required]
int32

maximumRetry is number of retries the PV recycler will execute on failure to recycle PV.

MinimumTimeoutNFS [Required]
int32

minimumTimeoutNFS is the minimum ActiveDeadlineSeconds to use for an NFS Recycler pod.

PodTemplateFilePathNFS [Required]
string

podTemplateFilePathNFS is the file path to a pod definition used as a template for NFS persistent volume recycling

IncrementTimeoutNFS [Required]
int32

incrementTimeoutNFS is the increment of time added per Gi to ActiveDeadlineSeconds for an NFS scrubber pod.

PodTemplateFilePathHostPath [Required]
string

podTemplateFilePathHostPath is the file path to a pod definition used as a template for HostPath persistent volume recycling. This is for development and testing only and will not work in a multi-node cluster.

MinimumTimeoutHostPath [Required]
int32

minimumTimeoutHostPath is the minimum ActiveDeadlineSeconds to use for a HostPath Recycler pod. This is for development and testing only and will not work in a multi-node cluster.

IncrementTimeoutHostPath [Required]
int32

incrementTimeoutHostPath is the increment of time added per Gi to ActiveDeadlineSeconds for a HostPath scrubber pod. This is for development and testing only and will not work in a multi-node cluster.

PodGCControllerConfiguration

Appears in:

PodGCControllerConfiguration contains elements describing PodGCController.

FieldDescription
TerminatedPodGCThreshold [Required]
int32

terminatedPodGCThreshold is the number of terminated pods that can exist before the terminated pod garbage collector starts deleting terminated pods. If <= 0, the terminated pod garbage collector is disabled.

ReplicaSetControllerConfiguration

Appears in:

ReplicaSetControllerConfiguration contains elements describing ReplicaSetController.

FieldDescription
ConcurrentRSSyncs [Required]
int32

concurrentRSSyncs is the number of replica sets that are allowed to sync concurrently. Larger number = more responsive replica management, but more CPU (and network) load.

ReplicationControllerConfiguration

Appears in:

ReplicationControllerConfiguration contains elements describing ReplicationController.

FieldDescription
ConcurrentRCSyncs [Required]
int32

concurrentRCSyncs is the number of replication controllers that are allowed to sync concurrently. Larger number = more responsive replica management, but more CPU (and network) load.

ResourceQuotaControllerConfiguration

Appears in:

ResourceQuotaControllerConfiguration contains elements describing ResourceQuotaController.

FieldDescription
ResourceQuotaSyncPeriod [Required]
meta/v1.Duration

resourceQuotaSyncPeriod is the period for syncing quota usage status in the system.

ConcurrentResourceQuotaSyncs [Required]
int32

concurrentResourceQuotaSyncs is the number of resource quotas that are allowed to sync concurrently. Larger number = more responsive quota management, but more CPU (and network) load.

SAControllerConfiguration

Appears in:

SAControllerConfiguration contains elements describing ServiceAccountController.

FieldDescription
ServiceAccountKeyFile [Required]
string

serviceAccountKeyFile is the filename containing a PEM-encoded private RSA key used to sign service account tokens.

ConcurrentSATokenSyncs [Required]
int32

concurrentSATokenSyncs is the number of service account token syncing operations that will be done concurrently.

RootCAFile [Required]
string

rootCAFile is the root certificate authority will be included in service account's token secret. This must be a valid PEM-encoded CA bundle.

StatefulSetControllerConfiguration

Appears in:

StatefulSetControllerConfiguration contains elements describing StatefulSetController.

FieldDescription
ConcurrentStatefulSetSyncs [Required]
int32

concurrentStatefulSetSyncs is the number of statefulset objects that are allowed to sync concurrently. Larger number = more responsive statefulsets, but more CPU (and network) load.

TTLAfterFinishedControllerConfiguration

Appears in:

TTLAfterFinishedControllerConfiguration contains elements describing TTLAfterFinishedController.

FieldDescription
ConcurrentTTLSyncs [Required]
int32

concurrentTTLSyncs is the number of TTL-after-finished collector workers that are allowed to sync concurrently.

ValidatingAdmissionPolicyStatusControllerConfiguration

Appears in:

ValidatingAdmissionPolicyStatusControllerConfiguration contains elements describing ValidatingAdmissionPolicyStatusController.

FieldDescription
ConcurrentPolicySyncs [Required]
int32

ConcurrentPolicySyncs is the number of policy objects that are allowed to sync concurrently. Larger number = quicker type checking, but more CPU (and network) load. The default value is 5.

VolumeConfiguration

Appears in:

VolumeConfiguration contains all enumerated flags meant to configure all volume plugins. From this config, the controller-manager binary will create many instances of volume.VolumeConfig, each containing only the configuration needed for that plugin which are then passed to the appropriate plugin. The ControllerManager binary is the only part of the code which knows what plugins are supported and which flags correspond to each plugin.

FieldDescription
EnableHostPathProvisioning [Required]
bool

enableHostPathProvisioning enables HostPath PV provisioning when running without a cloud provider. This allows testing and development of provisioning features. HostPath provisioning is not supported in any way, won't work in a multi-node cluster, and should not be used for anything other than testing or development.

EnableDynamicProvisioning [Required]
bool

enableDynamicProvisioning enables the provisioning of volumes when running within an environment that supports dynamic provisioning. Defaults to true.

PersistentVolumeRecyclerConfiguration [Required]
PersistentVolumeRecyclerConfiguration

persistentVolumeRecyclerConfiguration holds configuration for persistent volume plugins.

FlexVolumePluginDir [Required]
string

volumePluginDir is the full path of the directory in which the flex volume plugin should search for additional third party volume plugins

11 - kube-proxy Configuration (v1alpha1)

Resource Types

ClientConnectionConfiguration

Appears in:

ClientConnectionConfiguration contains details for constructing a client.

FieldDescription
kubeconfig [Required]
string

kubeconfig is the path to a KubeConfig file.

acceptContentTypes [Required]
string

acceptContentTypes defines the Accept header sent by clients when connecting to a server, overriding the default value of 'application/json'. This field will control all connections to the server used by a particular client.

contentType [Required]
string

contentType is the content type used when sending data to the server from this client.

qps [Required]
float32

qps controls the number of queries per second allowed for this connection.

burst [Required]
int32

burst allows extra queries to accumulate when a client is exceeding its rate.

DebuggingConfiguration

Appears in:

DebuggingConfiguration holds configuration for Debugging related features.

FieldDescription
enableProfiling [Required]
bool

enableProfiling enables profiling via web interface host:port/debug/pprof/

enableContentionProfiling [Required]
bool

enableContentionProfiling enables block profiling, if enableProfiling is true.

LeaderElectionConfiguration

Appears in:

LeaderElectionConfiguration defines the configuration of leader election clients for components that can run with leader election enabled.

FieldDescription
leaderElect [Required]
bool

leaderElect enables a leader election client to gain leadership before executing the main loop. Enable this when running replicated components for high availability.

leaseDuration [Required]
meta/v1.Duration

leaseDuration is the duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled.

renewDeadline [Required]
meta/v1.Duration

renewDeadline is the interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than or equal to the lease duration. This is only applicable if leader election is enabled.

retryPeriod [Required]
meta/v1.Duration

retryPeriod is the duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled.

resourceLock [Required]
string

resourceLock indicates the resource object type that will be used to lock during leader election cycles.

resourceName [Required]
string

resourceName indicates the name of resource object that will be used to lock during leader election cycles.

resourceNamespace [Required]
string

resourceName indicates the namespace of resource object that will be used to lock during leader election cycles.

KubeProxyConfiguration

KubeProxyConfiguration contains everything necessary to configure the Kubernetes proxy server.

FieldDescription
apiVersion
string
kubeproxy.config.k8s.io/v1alpha1
kind
string
KubeProxyConfiguration
featureGates [Required]
map[string]bool

featureGates is a map of feature names to bools that enable or disable alpha/experimental features.

clientConnection [Required]
ClientConnectionConfiguration

clientConnection specifies the kubeconfig file and client connection settings for the proxy server to use when communicating with the apiserver.

logging [Required]
LoggingConfiguration

logging specifies the options of logging. Refer to Logs Options for more information.

hostnameOverride [Required]
string

hostnameOverride, if non-empty, will be used as the name of the Node that kube-proxy is running on. If unset, the node name is assumed to be the same as the node's hostname.

bindAddress [Required]
string

bindAddress can be used to override kube-proxy's idea of what its node's primary IP is. Note that the name is a historical artifact, and kube-proxy does not actually bind any sockets to this IP.

healthzBindAddress [Required]
string

healthzBindAddress is the IP address and port for the health check server to serve on, defaulting to "0.0.0.0:10256" (if bindAddress is unset or IPv4), or "[::]:10256" (if bindAddress is IPv6).

metricsBindAddress [Required]
string

metricsBindAddress is the IP address and port for the metrics server to serve on, defaulting to "127.0.0.1:10249" (if bindAddress is unset or IPv4), or "[::1]:10249" (if bindAddress is IPv6). (Set to "0.0.0.0:10249" / "[::]:10249" to bind on all interfaces.)

bindAddressHardFail [Required]
bool

bindAddressHardFail, if true, tells kube-proxy to treat failure to bind to a port as fatal and exit

enableProfiling [Required]
bool

enableProfiling enables profiling via web interface on /debug/pprof handler. Profiling handlers will be handled by metrics server.

showHiddenMetricsForVersion [Required]
string

showHiddenMetricsForVersion is the version for which you want to show hidden metrics.

mode [Required]
ProxyMode

mode specifies which proxy mode to use.

iptables [Required]
KubeProxyIPTablesConfiguration

iptables contains iptables-related configuration options.

ipvs [Required]
KubeProxyIPVSConfiguration

ipvs contains ipvs-related configuration options.

nftables [Required]
KubeProxyNFTablesConfiguration

nftables contains nftables-related configuration options.

winkernel [Required]
KubeProxyWinkernelConfiguration

winkernel contains winkernel-related configuration options.

detectLocalMode [Required]
LocalMode

detectLocalMode determines mode to use for detecting local traffic, defaults to LocalModeClusterCIDR

detectLocal [Required]
DetectLocalConfiguration

detectLocal contains optional configuration settings related to DetectLocalMode.

clusterCIDR [Required]
string

clusterCIDR is the CIDR range of the pods in the cluster. (For dual-stack clusters, this can be a comma-separated dual-stack pair of CIDR ranges.). When DetectLocalMode is set to LocalModeClusterCIDR, kube-proxy will consider traffic to be local if its source IP is in this range. (Otherwise it is not used.)

nodePortAddresses [Required]
[]string

nodePortAddresses is a list of CIDR ranges that contain valid node IPs. If set, connections to NodePort services will only be accepted on node IPs in one of the indicated ranges. If unset, NodePort connections will be accepted on all local IPs.

oomScoreAdj [Required]
int32

oomScoreAdj is the oom-score-adj value for kube-proxy process. Values must be within the range [-1000, 1000]

conntrack [Required]
KubeProxyConntrackConfiguration

conntrack contains conntrack-related configuration options.

configSyncPeriod [Required]
meta/v1.Duration

configSyncPeriod is how often configuration from the apiserver is refreshed. Must be greater than 0.

portRange [Required]
string

portRange was previously used to configure the userspace proxy, but is now unused.

DetectLocalConfiguration

Appears in:

DetectLocalConfiguration contains optional settings related to DetectLocalMode option

FieldDescription
bridgeInterface [Required]
string

bridgeInterface is a bridge interface name. When DetectLocalMode is set to LocalModeBridgeInterface, kube-proxy will consider traffic to be local if it originates from this bridge.

interfaceNamePrefix [Required]
string

interfaceNamePrefix is an interface name prefix. When DetectLocalMode is set to LocalModeInterfaceNamePrefix, kube-proxy will consider traffic to be local if it originates from any interface whose name begins with this prefix.

KubeProxyConntrackConfiguration

Appears in:

KubeProxyConntrackConfiguration contains conntrack settings for the Kubernetes proxy server.

FieldDescription
maxPerCore [Required]
int32

maxPerCore is the maximum number of NAT connections to track per CPU core (0 to leave the limit as-is and ignore min).

min [Required]
int32

min is the minimum value of connect-tracking records to allocate, regardless of maxPerCore (set maxPerCore=0 to leave the limit as-is).

tcpEstablishedTimeout [Required]
meta/v1.Duration

tcpEstablishedTimeout is how long an idle TCP connection will be kept open (e.g. '2s'). Must be greater than 0 to set.

tcpCloseWaitTimeout [Required]
meta/v1.Duration

tcpCloseWaitTimeout is how long an idle conntrack entry in CLOSE_WAIT state will remain in the conntrack table. (e.g. '60s'). Must be greater than 0 to set.

tcpBeLiberal [Required]
bool

tcpBeLiberal, if true, kube-proxy will configure conntrack to run in liberal mode for TCP connections and packets with out-of-window sequence numbers won't be marked INVALID.

udpTimeout [Required]
meta/v1.Duration

udpTimeout is how long an idle UDP conntrack entry in UNREPLIED state will remain in the conntrack table (e.g. '30s'). Must be greater than 0 to set.

udpStreamTimeout [Required]
meta/v1.Duration

udpStreamTimeout is how long an idle UDP conntrack entry in ASSURED state will remain in the conntrack table (e.g. '300s'). Must be greater than 0 to set.

KubeProxyIPTablesConfiguration

Appears in:

KubeProxyIPTablesConfiguration contains iptables-related configuration details for the Kubernetes proxy server.

FieldDescription
masqueradeBit [Required]
int32

masqueradeBit is the bit of the iptables fwmark space to use for SNAT if using the iptables or ipvs proxy mode. Values must be within the range [0, 31].

masqueradeAll [Required]
bool

masqueradeAll tells kube-proxy to SNAT all traffic sent to Service cluster IPs, when using the iptables or ipvs proxy mode. This may be required with some CNI plugins.

localhostNodePorts [Required]
bool

localhostNodePorts, if false, tells kube-proxy to disable the legacy behavior of allowing NodePort services to be accessed via localhost. (Applies only to iptables mode and IPv4; localhost NodePorts are never allowed with other proxy modes or with IPv6.)

syncPeriod [Required]
meta/v1.Duration

syncPeriod is an interval (e.g. '5s', '1m', '2h22m') indicating how frequently various re-synchronizing and cleanup operations are performed. Must be greater than 0.

minSyncPeriod [Required]
meta/v1.Duration

minSyncPeriod is the minimum period between iptables rule resyncs (e.g. '5s', '1m', '2h22m'). A value of 0 means every Service or EndpointSlice change will result in an immediate iptables resync.

KubeProxyIPVSConfiguration

Appears in:

KubeProxyIPVSConfiguration contains ipvs-related configuration details for the Kubernetes proxy server.

FieldDescription
syncPeriod [Required]
meta/v1.Duration

syncPeriod is an interval (e.g. '5s', '1m', '2h22m') indicating how frequently various re-synchronizing and cleanup operations are performed. Must be greater than 0.

minSyncPeriod [Required]
meta/v1.Duration

minSyncPeriod is the minimum period between IPVS rule resyncs (e.g. '5s', '1m', '2h22m'). A value of 0 means every Service or EndpointSlice change will result in an immediate IPVS resync.

scheduler [Required]
string

scheduler is the IPVS scheduler to use

excludeCIDRs [Required]
[]string

excludeCIDRs is a list of CIDRs which the ipvs proxier should not touch when cleaning up ipvs services.

strictARP [Required]
bool

strictARP configures arp_ignore and arp_announce to avoid answering ARP queries from kube-ipvs0 interface

tcpTimeout [Required]
meta/v1.Duration

tcpTimeout is the timeout value used for idle IPVS TCP sessions. The default value is 0, which preserves the current timeout value on the system.

tcpFinTimeout [Required]
meta/v1.Duration

tcpFinTimeout is the timeout value used for IPVS TCP sessions after receiving a FIN. The default value is 0, which preserves the current timeout value on the system.

udpTimeout [Required]
meta/v1.Duration

udpTimeout is the timeout value used for IPVS UDP packets. The default value is 0, which preserves the current timeout value on the system.

KubeProxyNFTablesConfiguration

Appears in:

KubeProxyNFTablesConfiguration contains nftables-related configuration details for the Kubernetes proxy server.

FieldDescription
masqueradeBit [Required]
int32

masqueradeBit is the bit of the iptables fwmark space to use for SNAT if using the nftables proxy mode. Values must be within the range [0, 31].

masqueradeAll [Required]
bool

masqueradeAll tells kube-proxy to SNAT all traffic sent to Service cluster IPs, when using the nftables mode. This may be required with some CNI plugins.

syncPeriod [Required]
meta/v1.Duration

syncPeriod is an interval (e.g. '5s', '1m', '2h22m') indicating how frequently various re-synchronizing and cleanup operations are performed. Must be greater than 0.

minSyncPeriod [Required]
meta/v1.Duration

minSyncPeriod is the minimum period between iptables rule resyncs (e.g. '5s', '1m', '2h22m'). A value of 0 means every Service or EndpointSlice change will result in an immediate iptables resync.

KubeProxyWinkernelConfiguration

Appears in:

KubeProxyWinkernelConfiguration contains Windows/HNS settings for the Kubernetes proxy server.

FieldDescription
networkName [Required]
string

networkName is the name of the network kube-proxy will use to create endpoints and policies

sourceVip [Required]
string

sourceVip is the IP address of the source VIP endpoint used for NAT when loadbalancing

enableDSR [Required]
bool

enableDSR tells kube-proxy whether HNS policies should be created with DSR

rootHnsEndpointName [Required]
string

rootHnsEndpointName is the name of hnsendpoint that is attached to l2bridge for root network namespace

forwardHealthCheckVip [Required]
bool

forwardHealthCheckVip forwards service VIP for health check port on Windows

LocalMode

(Alias of string)

Appears in:

LocalMode represents modes to detect local traffic from the node

ProxyMode

(Alias of string)

Appears in:

ProxyMode represents modes used by the Kubernetes proxy server.

Currently, two modes of proxy are available on Linux platforms: 'iptables' and 'ipvs'. One mode of proxy is available on Windows platforms: 'kernelspace'.

If the proxy mode is unspecified, the best-available proxy mode will be used (currently this is iptables on Linux and kernelspace on Windows). If the selected proxy mode cannot be used (due to lack of kernel support, missing userspace components, etc) then kube-proxy will exit with an error.

12 - kube-scheduler Configuration (v1)

Resource Types

ClientConnectionConfiguration

Appears in:

ClientConnectionConfiguration contains details for constructing a client.

FieldDescription
kubeconfig [Required]
string

kubeconfig is the path to a KubeConfig file.

acceptContentTypes [Required]
string

acceptContentTypes defines the Accept header sent by clients when connecting to a server, overriding the default value of 'application/json'. This field will control all connections to the server used by a particular client.

contentType [Required]
string

contentType is the content type used when sending data to the server from this client.

qps [Required]
float32

qps controls the number of queries per second allowed for this connection.

burst [Required]
int32

burst allows extra queries to accumulate when a client is exceeding its rate.

DebuggingConfiguration

Appears in:

DebuggingConfiguration holds configuration for Debugging related features.

FieldDescription
enableProfiling [Required]
bool

enableProfiling enables profiling via web interface host:port/debug/pprof/

enableContentionProfiling [Required]
bool

enableContentionProfiling enables block profiling, if enableProfiling is true.

LeaderElectionConfiguration

Appears in:

LeaderElectionConfiguration defines the configuration of leader election clients for components that can run with leader election enabled.

FieldDescription
leaderElect [Required]
bool

leaderElect enables a leader election client to gain leadership before executing the main loop. Enable this when running replicated components for high availability.

leaseDuration [Required]
meta/v1.Duration

leaseDuration is the duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled.

renewDeadline [Required]
meta/v1.Duration

renewDeadline is the interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than or equal to the lease duration. This is only applicable if leader election is enabled.

retryPeriod [Required]
meta/v1.Duration

retryPeriod is the duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled.

resourceLock [Required]
string

resourceLock indicates the resource object type that will be used to lock during leader election cycles.

resourceName [Required]
string

resourceName indicates the name of resource object that will be used to lock during leader election cycles.

resourceNamespace [Required]
string

resourceName indicates the namespace of resource object that will be used to lock during leader election cycles.

DefaultPreemptionArgs

DefaultPreemptionArgs holds arguments used to configure the DefaultPreemption plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1
kind
string
DefaultPreemptionArgs
minCandidateNodesPercentage [Required]
int32

MinCandidateNodesPercentage is the minimum number of candidates to shortlist when dry running preemption as a percentage of number of nodes. Must be in the range [0, 100]. Defaults to 10% of the cluster size if unspecified.

minCandidateNodesAbsolute [Required]
int32

MinCandidateNodesAbsolute is the absolute minimum number of candidates to shortlist. The likely number of candidates enumerated for dry running preemption is given by the formula: numCandidates = max(numNodes * minCandidateNodesPercentage, minCandidateNodesAbsolute) We say "likely" because there are other factors such as PDB violations that play a role in the number of candidates shortlisted. Must be at least 0 nodes. Defaults to 100 nodes if unspecified.

InterPodAffinityArgs

InterPodAffinityArgs holds arguments used to configure the InterPodAffinity plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1
kind
string
InterPodAffinityArgs
hardPodAffinityWeight [Required]
int32

HardPodAffinityWeight is the scoring weight for existing pods with a matching hard affinity to the incoming pod.

ignorePreferredTermsOfExistingPods [Required]
bool

IgnorePreferredTermsOfExistingPods configures the scheduler to ignore existing pods' preferred affinity rules when scoring candidate nodes, unless the incoming pod has inter-pod affinities.

KubeSchedulerConfiguration

KubeSchedulerConfiguration configures a scheduler

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1
kind
string
KubeSchedulerConfiguration
parallelism [Required]
int32

Parallelism defines the amount of parallelism in algorithms for scheduling a Pods. Must be greater than 0. Defaults to 16

leaderElection [Required]
LeaderElectionConfiguration

LeaderElection defines the configuration of leader election client.

clientConnection [Required]
ClientConnectionConfiguration

ClientConnection specifies the kubeconfig file and client connection settings for the proxy server to use when communicating with the apiserver.

DebuggingConfiguration [Required]
DebuggingConfiguration
(Members of DebuggingConfiguration are embedded into this type.)

DebuggingConfiguration holds configuration for Debugging related features TODO: We might wanna make this a substruct like Debugging componentbaseconfigv1alpha1.DebuggingConfiguration

percentageOfNodesToScore [Required]
int32

PercentageOfNodesToScore is the percentage of all nodes that once found feasible for running a pod, the scheduler stops its search for more feasible nodes in the cluster. This helps improve scheduler's performance. Scheduler always tries to find at least "minFeasibleNodesToFind" feasible nodes no matter what the value of this flag is. Example: if the cluster size is 500 nodes and the value of this flag is 30, then scheduler stops finding further feasible nodes once it finds 150 feasible ones. When the value is 0, default percentage (5%--50% based on the size of the cluster) of the nodes will be scored. It is overridden by profile level PercentageofNodesToScore.

podInitialBackoffSeconds [Required]
int64

PodInitialBackoffSeconds is the initial backoff for unschedulable pods. If specified, it must be greater than 0. If this value is null, the default value (1s) will be used.

podMaxBackoffSeconds [Required]
int64

PodMaxBackoffSeconds is the max backoff for unschedulable pods. If specified, it must be greater than podInitialBackoffSeconds. If this value is null, the default value (10s) will be used.

profiles [Required]
[]KubeSchedulerProfile

Profiles are scheduling profiles that kube-scheduler supports. Pods can choose to be scheduled under a particular profile by setting its associated scheduler name. Pods that don't specify any scheduler name are scheduled with the "default-scheduler" profile, if present here.

extenders [Required]
[]Extender

Extenders are the list of scheduler extenders, each holding the values of how to communicate with the extender. These extenders are shared by all scheduler profiles.

delayCacheUntilActive [Required]
bool

DelayCacheUntilActive specifies when to start caching. If this is true and leader election is enabled, the scheduler will wait to fill informer caches until it is the leader. Doing so will have slower failover with the benefit of lower memory overhead while waiting to become leader. Defaults to false.

NodeAffinityArgs

NodeAffinityArgs holds arguments to configure the NodeAffinity plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1
kind
string
NodeAffinityArgs
addedAffinity
core/v1.NodeAffinity

AddedAffinity is applied to all Pods additionally to the NodeAffinity specified in the PodSpec. That is, Nodes need to satisfy AddedAffinity AND .spec.NodeAffinity. AddedAffinity is empty by default (all Nodes match). When AddedAffinity is used, some Pods with affinity requirements that match a specific Node (such as Daemonset Pods) might remain unschedulable.

NodeResourcesBalancedAllocationArgs

NodeResourcesBalancedAllocationArgs holds arguments used to configure NodeResourcesBalancedAllocation plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1
kind
string
NodeResourcesBalancedAllocationArgs
resources [Required]
[]ResourceSpec

Resources to be managed, the default is "cpu" and "memory" if not specified.

NodeResourcesFitArgs

NodeResourcesFitArgs holds arguments used to configure the NodeResourcesFit plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1
kind
string
NodeResourcesFitArgs
ignoredResources [Required]
[]string

IgnoredResources is the list of resources that NodeResources fit filter should ignore. This doesn't apply to scoring.

ignoredResourceGroups [Required]
[]string

IgnoredResourceGroups defines the list of resource groups that NodeResources fit filter should ignore. e.g. if group is ["example.com"], it will ignore all resource names that begin with "example.com", such as "example.com/aaa" and "example.com/bbb". A resource group name can't contain '/'. This doesn't apply to scoring.

scoringStrategy [Required]
ScoringStrategy

ScoringStrategy selects the node resource scoring strategy. The default strategy is LeastAllocated with an equal "cpu" and "memory" weight.

PodTopologySpreadArgs

PodTopologySpreadArgs holds arguments used to configure the PodTopologySpread plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1
kind
string
PodTopologySpreadArgs
defaultConstraints
[]core/v1.TopologySpreadConstraint

DefaultConstraints defines topology spread constraints to be applied to Pods that don't define any in pod.spec.topologySpreadConstraints. .defaultConstraints[*].labelSelectors must be empty, as they are deduced from the Pod's membership to Services, ReplicationControllers, ReplicaSets or StatefulSets. When not empty, .defaultingType must be "List".

defaultingType
PodTopologySpreadConstraintsDefaulting

DefaultingType determines how .defaultConstraints are deduced. Can be one of "System" or "List".

  • "System": Use kubernetes defined constraints that spread Pods among Nodes and Zones.
  • "List": Use constraints defined in .defaultConstraints.

Defaults to "System".

VolumeBindingArgs

VolumeBindingArgs holds arguments used to configure the VolumeBinding plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1
kind
string
VolumeBindingArgs
bindTimeoutSeconds [Required]
int64

BindTimeoutSeconds is the timeout in seconds in volume binding operation. Value must be non-negative integer. The value zero indicates no waiting. If this value is nil, the default value (600) will be used.

shape
[]UtilizationShapePoint

Shape specifies the points defining the score function shape, which is used to score nodes based on the utilization of statically provisioned PVs. The utilization is calculated by dividing the total requested storage of the pod by the total capacity of feasible PVs on each node. Each point contains utilization (ranges from 0 to 100) and its associated score (ranges from 0 to 10). You can turn the priority by specifying different scores for different utilization numbers. The default shape points are:

  1. 0 for 0 utilization
  2. 10 for 100 utilization All points must be sorted in increasing order by utilization.

Extender

Appears in:

Extender holds the parameters used to communicate with the extender. If a verb is unspecified/empty, it is assumed that the extender chose not to provide that extension.

FieldDescription
urlPrefix [Required]
string

URLPrefix at which the extender is available

filterVerb [Required]
string

Verb for the filter call, empty if not supported. This verb is appended to the URLPrefix when issuing the filter call to extender.

preemptVerb [Required]
string

Verb for the preempt call, empty if not supported. This verb is appended to the URLPrefix when issuing the preempt call to extender.

prioritizeVerb [Required]
string

Verb for the prioritize call, empty if not supported. This verb is appended to the URLPrefix when issuing the prioritize call to extender.

weight [Required]
int64

The numeric multiplier for the node scores that the prioritize call generates. The weight should be a positive integer

bindVerb [Required]
string

Verb for the bind call, empty if not supported. This verb is appended to the URLPrefix when issuing the bind call to extender. If this method is implemented by the extender, it is the extender's responsibility to bind the pod to apiserver. Only one extender can implement this function.

enableHTTPS [Required]
bool

EnableHTTPS specifies whether https should be used to communicate with the extender

tlsConfig [Required]
ExtenderTLSConfig

TLSConfig specifies the transport layer security config

httpTimeout [Required]
meta/v1.Duration

HTTPTimeout specifies the timeout duration for a call to the extender. Filter timeout fails the scheduling of the pod. Prioritize timeout is ignored, k8s/other extenders priorities are used to select the node.

nodeCacheCapable [Required]
bool

NodeCacheCapable specifies that the extender is capable of caching node information, so the scheduler should only send minimal information about the eligible nodes assuming that the extender already cached full details of all nodes in the cluster

managedResources
[]ExtenderManagedResource

ManagedResources is a list of extended resources that are managed by this extender.

  • A pod will be sent to the extender on the Filter, Prioritize and Bind (if the extender is the binder) phases iff the pod requests at least one of the extended resources in this list. If empty or unspecified, all pods will be sent to this extender.
  • If IgnoredByScheduler is set to true for a resource, kube-scheduler will skip checking the resource in predicates.
ignorable [Required]
bool

Ignorable specifies if the extender is ignorable, i.e. scheduling should not fail when the extender returns an error or is not reachable.

ExtenderManagedResource

Appears in:

ExtenderManagedResource describes the arguments of extended resources managed by an extender.

FieldDescription
name [Required]
string

Name is the extended resource name.

ignoredByScheduler [Required]
bool

IgnoredByScheduler indicates whether kube-scheduler should ignore this resource when applying predicates.

ExtenderTLSConfig

Appears in:

ExtenderTLSConfig contains settings to enable TLS with extender

FieldDescription
insecure [Required]
bool

Server should be accessed without verifying the TLS certificate. For testing only.

serverName [Required]
string

ServerName is passed to the server for SNI and is used in the client to check server certificates against. If ServerName is empty, the hostname used to contact the server is used.

certFile [Required]
string

Server requires TLS client certificate authentication

keyFile [Required]
string

Server requires TLS client certificate authentication

caFile [Required]
string

Trusted root certificates for server

certData [Required]
[]byte

CertData holds PEM-encoded bytes (typically read from a client certificate file). CertData takes precedence over CertFile

keyData [Required]
[]byte

KeyData holds PEM-encoded bytes (typically read from a client certificate key file). KeyData takes precedence over KeyFile

caData [Required]
[]byte

CAData holds PEM-encoded bytes (typically read from a root certificates bundle). CAData takes precedence over CAFile

KubeSchedulerProfile

Appears in:

KubeSchedulerProfile is a scheduling profile.

FieldDescription
schedulerName [Required]
string

SchedulerName is the name of the scheduler associated to this profile. If SchedulerName matches with the pod's "spec.schedulerName", then the pod is scheduled with this profile.

percentageOfNodesToScore [Required]
int32

PercentageOfNodesToScore is the percentage of all nodes that once found feasible for running a pod, the scheduler stops its search for more feasible nodes in the cluster. This helps improve scheduler's performance. Scheduler always tries to find at least "minFeasibleNodesToFind" feasible nodes no matter what the value of this flag is. Example: if the cluster size is 500 nodes and the value of this flag is 30, then scheduler stops finding further feasible nodes once it finds 150 feasible ones. When the value is 0, default percentage (5%--50% based on the size of the cluster) of the nodes will be scored. It will override global PercentageOfNodesToScore. If it is empty, global PercentageOfNodesToScore will be used.

plugins [Required]
Plugins

Plugins specify the set of plugins that should be enabled or disabled. Enabled plugins are the ones that should be enabled in addition to the default plugins. Disabled plugins are any of the default plugins that should be disabled. When no enabled or disabled plugin is specified for an extension point, default plugins for that extension point will be used if there is any. If a QueueSort plugin is specified, the same QueueSort Plugin and PluginConfig must be specified for all profiles.

pluginConfig [Required]
[]PluginConfig

PluginConfig is an optional set of custom plugin arguments for each plugin. Omitting config args for a plugin is equivalent to using the default config for that plugin.

Plugin

Appears in:

Plugin specifies a plugin name and its weight when applicable. Weight is used only for Score plugins.

FieldDescription
name [Required]
string

Name defines the name of plugin

weight [Required]
int32

Weight defines the weight of plugin, only used for Score plugins.

PluginConfig

Appears in:

PluginConfig specifies arguments that should be passed to a plugin at the time of initialization. A plugin that is invoked at multiple extension points is initialized once. Args can have arbitrary structure. It is up to the plugin to process these Args.

FieldDescription
name [Required]
string

Name defines the name of plugin being configured

args [Required]
k8s.io/apimachinery/pkg/runtime.RawExtension

Args defines the arguments passed to the plugins at the time of initialization. Args can have arbitrary structure.

PluginSet

Appears in:

PluginSet specifies enabled and disabled plugins for an extension point. If an array is empty, missing, or nil, default plugins at that extension point will be used.

FieldDescription
enabled [Required]
[]Plugin

Enabled specifies plugins that should be enabled in addition to default plugins. If the default plugin is also configured in the scheduler config file, the weight of plugin will be overridden accordingly. These are called after default plugins and in the same order specified here.

disabled [Required]
[]Plugin

Disabled specifies default plugins that should be disabled. When all default plugins need to be disabled, an array containing only one "*" should be provided.

Plugins

Appears in:

Plugins include multiple extension points. When specified, the list of plugins for a particular extension point are the only ones enabled. If an extension point is omitted from the config, then the default set of plugins is used for that extension point. Enabled plugins are called in the order specified here, after default plugins. If they need to be invoked before default plugins, default plugins must be disabled and re-enabled here in desired order.

FieldDescription
preEnqueue [Required]
PluginSet

PreEnqueue is a list of plugins that should be invoked before adding pods to the scheduling queue.

queueSort [Required]
PluginSet

QueueSort is a list of plugins that should be invoked when sorting pods in the scheduling queue.

preFilter [Required]
PluginSet

PreFilter is a list of plugins that should be invoked at "PreFilter" extension point of the scheduling framework.

filter [Required]
PluginSet

Filter is a list of plugins that should be invoked when filtering out nodes that cannot run the Pod.

postFilter [Required]
PluginSet

PostFilter is a list of plugins that are invoked after filtering phase, but only when no feasible nodes were found for the pod.

preScore [Required]
PluginSet

PreScore is a list of plugins that are invoked before scoring.

score [Required]
PluginSet

Score is a list of plugins that should be invoked when ranking nodes that have passed the filtering phase.

reserve [Required]
PluginSet

Reserve is a list of plugins invoked when reserving/unreserving resources after a node is assigned to run the pod.

permit [Required]
PluginSet

Permit is a list of plugins that control binding of a Pod. These plugins can prevent or delay binding of a Pod.

preBind [Required]
PluginSet

PreBind is a list of plugins that should be invoked before a pod is bound.

bind [Required]
PluginSet

Bind is a list of plugins that should be invoked at "Bind" extension point of the scheduling framework. The scheduler call these plugins in order. Scheduler skips the rest of these plugins as soon as one returns success.

postBind [Required]
PluginSet

PostBind is a list of plugins that should be invoked after a pod is successfully bound.

multiPoint [Required]
PluginSet

MultiPoint is a simplified config section to enable plugins for all valid extension points. Plugins enabled through MultiPoint will automatically register for every individual extension point the plugin has implemented. Disabling a plugin through MultiPoint disables that behavior. The same is true for disabling "*" through MultiPoint (no default plugins will be automatically registered). Plugins can still be disabled through their individual extension points.

In terms of precedence, plugin config follows this basic hierarchy

  1. Specific extension points
  2. Explicitly configured MultiPoint plugins
  3. The set of default plugins, as MultiPoint plugins This implies that a higher precedence plugin will run first and overwrite any settings within MultiPoint. Explicitly user-configured plugins also take a higher precedence over default plugins. Within this hierarchy, an Enabled setting takes precedence over Disabled. For example, if a plugin is set in both multiPoint.Enabled and multiPoint.Disabled, the plugin will be enabled. Similarly, including multiPoint.Disabled = '*' and multiPoint.Enabled = pluginA will still register that specific plugin through MultiPoint. This follows the same behavior as all other extension point configurations.

PodTopologySpreadConstraintsDefaulting

(Alias of string)

Appears in:

PodTopologySpreadConstraintsDefaulting defines how to set default constraints for the PodTopologySpread plugin.

RequestedToCapacityRatioParam

Appears in:

RequestedToCapacityRatioParam define RequestedToCapacityRatio parameters

FieldDescription
shape [Required]
[]UtilizationShapePoint

Shape is a list of points defining the scoring function shape.

ResourceSpec

Appears in:

ResourceSpec represents a single resource.

FieldDescription
name [Required]
string

Name of the resource.

weight [Required]
int64

Weight of the resource.

ScoringStrategy

Appears in:

ScoringStrategy define ScoringStrategyType for node resource plugin

FieldDescription
type [Required]
ScoringStrategyType

Type selects which strategy to run.

resources [Required]
[]ResourceSpec

Resources to consider when scoring. The default resource set includes "cpu" and "memory" with an equal weight. Allowed weights go from 1 to 100. Weight defaults to 1 if not specified or explicitly set to 0.

requestedToCapacityRatio [Required]
RequestedToCapacityRatioParam

Arguments specific to RequestedToCapacityRatio strategy.

ScoringStrategyType

(Alias of string)

Appears in:

ScoringStrategyType the type of scoring strategy used in NodeResourcesFit plugin.

UtilizationShapePoint

Appears in:

UtilizationShapePoint represents single point of priority function shape.

FieldDescription
utilization [Required]
int32

Utilization (x axis). Valid values are 0 to 100. Fully utilized node maps to 100.

score [Required]
int32

Score assigned to given utilization (y axis). Valid values are 0 to 10.

13 - kubeadm Configuration (v1beta3)

Overview

Package v1beta3 defines the v1beta3 version of the kubeadm configuration file format. This version improves on the v1beta2 format by fixing some minor issues and adding a few new fields.

A list of changes since v1beta2:

  • The deprecated "ClusterConfiguration.useHyperKubeImage" field has been removed. Kubeadm no longer supports the hyperkube image.
  • The "ClusterConfiguration.DNS.Type" field has been removed since CoreDNS is the only supported DNS server type by kubeadm.
  • Include "datapolicy" tags on the fields that hold secrets. This would result in the field values to be omitted when API structures are printed with klog.
  • Add "InitConfiguration.SkipPhases", "JoinConfiguration.SkipPhases" to allow skipping a list of phases during kubeadm init/join command execution.
  • Add "InitConfiguration.NodeRegistration.ImagePullPolicy" and "JoinConfiguration.NodeRegistration.ImagePullPolicy" to allow specifying the images pull policy during kubeadm "init" and "join". The value must be one of "Always", "Never" or "IfNotPresent". "IfNotPresent" is the default, which has been the existing behavior prior to this addition.
  • Add "InitConfiguration.Patches.Directory", "JoinConfiguration.Patches.Directory" to allow the user to configure a directory from which to take patches for components deployed by kubeadm.
  • Move the BootstrapToken* API and related utilities out of the "kubeadm" API group to a new group "bootstraptoken". The kubeadm API version v1beta3 no longer contains the BootstrapToken* structures.

Migration from old kubeadm config versions

  • kubeadm v1.15.x and newer can be used to migrate from v1beta1 to v1beta2.
  • kubeadm v1.22.x and newer no longer support v1beta1 and older APIs, but can be used to migrate v1beta2 to v1beta3.
  • kubeadm v1.27.x and newer no longer support v1beta2 and older APIs,

Basics

The preferred way to configure kubeadm is to pass an YAML configuration file with the --config option. Some of the configuration options defined in the kubeadm config file are also available as command line flags, but only the most common/simple use case are supported with this approach.

A kubeadm config file could contain multiple configuration types separated using three dashes (---).

kubeadm supports the following configuration types:

apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration

apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration

apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration

apiVersion: kubeadm.k8s.io/v1beta3
kind: JoinConfiguration

To print the defaults for "init" and "join" actions use the following commands:

kubeadm config print init-defaults
kubeadm config print join-defaults

The list of configuration types that must be included in a configuration file depends by the action you are performing (init or join) and by the configuration options you are going to use (defaults or advanced customization).

If some configuration types are not provided, or provided only partially, kubeadm will use default values; defaults provided by kubeadm includes also enforcing consistency of values across components when required (e.g. --cluster-cidr flag on controller manager and clusterCIDR on kube-proxy).

Users are always allowed to override default values, with the only exception of a small subset of setting with relevance for security (e.g. enforce authorization-mode Node and RBAC on api server).

If the user provides a configuration types that is not expected for the action you are performing, kubeadm will ignore those types and print a warning.

Kubeadm init configuration types

When executing kubeadm init with the --config option, the following configuration types could be used: InitConfiguration, ClusterConfiguration, KubeProxyConfiguration, KubeletConfiguration, but only one between InitConfiguration and ClusterConfiguration is mandatory.

apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
bootstrapTokens:
  ...
nodeRegistration:
  ...

The InitConfiguration type should be used to configure runtime settings, that in case of kubeadm init are the configuration of the bootstrap token and all the setting which are specific to the node where kubeadm is executed, including:

  • NodeRegistration, that holds fields that relate to registering the new node to the cluster; use it to customize the node name, the CRI socket to use or any other settings that should apply to this node only (e.g. the node ip).

  • LocalAPIEndpoint, that represents the endpoint of the instance of the API server to be deployed on this node; use it e.g. to customize the API server advertise address.

apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
networking:
  ...
etcd:
  ...
apiServer:
  extraArgs:
    ...
  extraVolumes:
    ...
...

The ClusterConfiguration type should be used to configure cluster-wide settings, including settings for:

  • networking that holds configuration for the networking topology of the cluster; use it e.g. to customize Pod subnet or services subnet.

  • etcd: use it e.g. to customize the local etcd or to configure the API server for using an external etcd cluster.

  • kube-apiserver, kube-scheduler, kube-controller-manager configurations; use it to customize control-plane components by adding customized setting or overriding kubeadm default settings.

apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
  ...

The KubeProxyConfiguration type should be used to change the configuration passed to kube-proxy instances deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults.

See https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/ or https://pkg.go.dev/k8s.io/kube-proxy/config/v1alpha1#KubeProxyConfiguration for kube-proxy official documentation.

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
  ...

The KubeletConfiguration type should be used to change the configurations that will be passed to all kubelet instances deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults.

See https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ or https://pkg.go.dev/k8s.io/kubelet/config/v1beta1#KubeletConfiguration for kubelet official documentation.

Here is a fully populated example of a single YAML file containing multiple configuration types to be used during a kubeadm init run.

apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
bootstrapTokens:
  - token: "9a08jv.c0izixklcxtmnze7"
    description: "kubeadm bootstrap token"
    ttl: "24h"
  - token: "783bde.3f89s0fje9f38fhf"
    description: "another bootstrap token"
    usages:
      - authentication
      - signing
    groups:
      - system:bootstrappers:kubeadm:default-node-token
nodeRegistration:
  name: "ec2-10-100-0-1"
  criSocket: "/var/run/dockershim.sock"
  taints:
    - key: "kubeadmNode"
      value: "someValue"
      effect: "NoSchedule"
  kubeletExtraArgs:
    v: 4
  ignorePreflightErrors:
    - IsPrivilegedUser
  imagePullPolicy: "IfNotPresent"
localAPIEndpoint:
  advertiseAddress: "10.100.0.1"
  bindPort: 6443
certificateKey: "e6a2eb8581237ab72a4f494f30285ec12a9694d750b9785706a83bfcbbbd2204"
skipPhases:
  - addon/kube-proxy
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
etcd:
  # one of local or external
  local:
    imageRepository: "registry.k8s.io"
    imageTag: "3.2.24"
    dataDir: "/var/lib/etcd"
    extraArgs:
      listen-client-urls: "http://10.100.0.1:2379"
    serverCertSANs:
      -  "ec2-10-100-0-1.compute-1.amazonaws.com"
    peerCertSANs:
      - "10.100.0.1"
  # external:
    # endpoints:
    # - "10.100.0.1:2379"
    # - "10.100.0.2:2379"
    # caFile: "/etcd/kubernetes/pki/etcd/etcd-ca.crt"
    # certFile: "/etcd/kubernetes/pki/etcd/etcd.crt"
    # keyFile: "/etcd/kubernetes/pki/etcd/etcd.key"
networking:
  serviceSubnet: "10.96.0.0/16"
  podSubnet: "10.244.0.0/24"
  dnsDomain: "cluster.local"
kubernetesVersion: "v1.21.0"
controlPlaneEndpoint: "10.100.0.1:6443"
apiServer:
  extraArgs:
    authorization-mode: "Node,RBAC"
  extraVolumes:
    - name: "some-volume"
      hostPath: "/etc/some-path"
      mountPath: "/etc/some-pod-path"
      readOnly: false
      pathType: File
  certSANs:
    - "10.100.1.1"
    - "ec2-10-100-0-1.compute-1.amazonaws.com"
  timeoutForControlPlane: 4m0s
controllerManager:
  extraArgs:
    "node-cidr-mask-size": "20"
  extraVolumes:
    - name: "some-volume"
      hostPath: "/etc/some-path"
      mountPath: "/etc/some-pod-path"
      readOnly: false
      pathType: File
scheduler:
  extraArgs:
    bind-address: "10.100.0.1"
  extraVolumes:
    - name: "some-volume"
      hostPath: "/etc/some-path"
      mountPath: "/etc/some-pod-path"
      readOnly: false
      pathType: File
certificatesDir: "/etc/kubernetes/pki"
imageRepository: "registry.k8s.io"
clusterName: "example-cluster"
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
# kubelet specific options here
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
# kube-proxy specific options here

Kubeadm join configuration types

When executing kubeadm join with the --config option, the JoinConfiguration type should be provided.

apiVersion: kubeadm.k8s.io/v1beta3
kind: JoinConfiguration
  ...

The JoinConfiguration type should be used to configure runtime settings, that in case of kubeadm join are the discovery method used for accessing the cluster info and all the setting which are specific to the node where kubeadm is executed, including:

  • nodeRegistration, that holds fields that relate to registering the new node to the cluster; use it to customize the node name, the CRI socket to use or any other settings that should apply to this node only (e.g. the node ip).

  • apiEndpoint, that represents the endpoint of the instance of the API server to be eventually deployed on this node.

Resource Types

BootstrapToken

Appears in:

BootstrapToken describes one bootstrap token, stored as a Secret in the cluster

FieldDescription
token [Required]
BootstrapTokenString

token is used for establishing bidirectional trust between nodes and control-planes. Used for joining nodes in the cluster.

description
string

description sets a human-friendly message why this token exists and what it's used for, so other administrators can know its purpose.

ttl
meta/v1.Duration

ttl defines the time to live for this token. Defaults to 24h. expires and ttl are mutually exclusive.

expires
meta/v1.Time

expires specifies the timestamp when this token expires. Defaults to being set dynamically at runtime based on the ttl. expires and ttl are mutually exclusive.

usages
[]string

usages describes the ways in which this token can be used. Can by default be used for establishing bidirectional trust, but that can be changed here.

groups
[]string

groups specifies the extra groups that this token will authenticate as when/if used for authentication

BootstrapTokenString

Appears in:

BootstrapTokenString is a token of the format abcdef.abcdef0123456789 that is used for both validation of the practically of the API server from a joining node's point of view and as an authentication method for the node in the bootstrap phase of "kubeadm join". This token is and should be short-lived.

FieldDescription
- [Required]
string
No description provided.
- [Required]
string
No description provided.

ClusterConfiguration

ClusterConfiguration contains cluster-wide configuration for a kubeadm cluster.

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta3
kind
string
ClusterConfiguration
etcd
Etcd

etcd holds the configuration for etcd.

networking
Networking

networking holds configuration for the networking topology of the cluster.

kubernetesVersion
string

kubernetesVersion is the target version of the control plane.

controlPlaneEndpoint
string

controlPlaneEndpoint sets a stable IP address or DNS name for the control plane. It can be a valid IP address or a RFC-1123 DNS subdomain, both with optional TCP port. In case the controlPlaneEndpoint is not specified, the advertiseAddress + bindPort are used; in case the controlPlaneEndpoint is specified but without a TCP port, the bindPort is used. Possible usages are:

  • In a cluster with more than one control plane instances, this field should be assigned the address of the external load balancer in front of the control plane instances.
  • In environments with enforced node recycling, the controlPlaneEndpoint could be used for assigning a stable DNS to the control plane.
apiServer
APIServer

apiServer contains extra settings for the API server.

controllerManager
ControlPlaneComponent

controllerManager contains extra settings for the controller manager.

scheduler
ControlPlaneComponent

scheduler contains extra settings for the scheduler.

dns
DNS

dns defines the options for the DNS add-on installed in the cluster.

certificatesDir
string

certificatesDir specifies where to store or look for all required certificates.

imageRepository
string

imageRepository sets the container registry to pull images from. If empty, registry.k8s.io will be used by default. In case of kubernetes version is a CI build (kubernetes version starts with ci/) gcr.io/k8s-staging-ci-images will be used as a default for control plane components and for kube-proxy, while registry.k8s.io will be used for all the other images.

featureGates
map[string]bool

featureGates contains the feature gates enabled by the user.

clusterName
string

The cluster name.

InitConfiguration

InitConfiguration contains a list of elements that is specific "kubeadm init"-only runtime information. kubeadm init-only information. These fields are solely used the first time kubeadm init runs. After that, the information in the fields IS NOT uploaded to the kubeadm-config ConfigMap that is used by kubeadm upgrade for instance. These fields must be omitempty.

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta3
kind
string
InitConfiguration
bootstrapTokens
[]BootstrapToken

bootstrapTokens is respected at kubeadm init time and describes a set of Bootstrap Tokens to create. This information IS NOT uploaded to the kubeadm cluster configmap, partly because of its sensitive nature

nodeRegistration
NodeRegistrationOptions

nodeRegistration holds fields that relate to registering the new control-plane node to the cluster.

localAPIEndpoint
APIEndpoint

localAPIEndpoint represents the endpoint of the API server instance that's deployed on this control plane node. In HA setups, this differs from ClusterConfiguration.controlPlaneEndpoint in the sense that controlPlaneEndpoint is the global endpoint for the cluster, which then load-balances the requests to each individual API server. This configuration object lets you customize what IP/DNS name and port the local API server advertises it's accessible on. By default, kubeadm tries to auto-detect the IP of the default interface and use that, but in case that process fails you may set the desired value here.

certificateKey
string

certificateKey sets the key with which certificates and keys are encrypted prior to being uploaded in a Secret in the cluster during the uploadcerts init phase. The certificate key is a hex encoded string that is an AES key of size 32 bytes.

skipPhases
[]string

skipPhases is a list of phases to skip during command execution. The list of phases can be obtained with the kubeadm init --help command. The flag "--skip-phases" takes precedence over this field.

patches
Patches

patches contains options related to applying patches to components deployed by kubeadm during kubeadm init.

JoinConfiguration

JoinConfiguration contains elements describing a particular node.

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta3
kind
string
JoinConfiguration
nodeRegistration
NodeRegistrationOptions

nodeRegistration holds fields that relate to registering the new control-plane node to the cluster.

caCertPath
string

caCertPath is the path to the SSL certificate authority used to secure comunications between a node and the control-plane. Defaults to "/etc/kubernetes/pki/ca.crt".

discovery [Required]
Discovery

discovery specifies the options for the kubelet to use during the TLS bootstrap process.

controlPlane
JoinControlPlane

controlPlane defines the additional control plane instance to be deployed on the joining node. If nil, no additional control plane instance will be deployed.

skipPhases
[]string

skipPhases is a list of phases to skip during command execution. The list of phases can be obtained with the kubeadm join --help command. The flag --skip-phases takes precedence over this field.

patches
Patches

patches contains options related to applying patches to components deployed by kubeadm during kubeadm join.

APIEndpoint

Appears in:

APIEndpoint struct contains elements of API server instance deployed on a node.

FieldDescription
advertiseAddress
string

advertiseAddress sets the IP address for the API server to advertise.

bindPort
int32

bindPort sets the secure port for the API Server to bind to. Defaults to 6443.

APIServer

Appears in:

APIServer holds settings necessary for API server deployments in the cluster

FieldDescription
ControlPlaneComponent [Required]
ControlPlaneComponent
(Members of ControlPlaneComponent are embedded into this type.) No description provided.
certSANs
[]string

certSANs sets extra Subject Alternative Names (SANs) for the API Server signing certificate.

timeoutForControlPlane
meta/v1.Duration

timeoutForControlPlane controls the timeout that we wait for API server to appear.

BootstrapTokenDiscovery

Appears in:

BootstrapTokenDiscovery is used to set the options for bootstrap token based discovery.

FieldDescription
token [Required]
string

token is a token used to validate cluster information fetched from the control-plane.

apiServerEndpoint
string

apiServerEndpoint is an IP or domain name to the API server from which information will be fetched.

caCertHashes
[]string

caCertHashes specifies a set of public key pins to verify when token-based discovery is used. The root CA found during discovery must match one of these values. Specifying an empty set disables root CA pinning, which can be unsafe. Each hash is specified as <type>:<value>, where the only currently supported type is "sha256". This is a hex-encoded SHA-256 hash of the Subject Public Key Info (SPKI) object in DER-encoded ASN.1. These hashes can be calculated using, for example, OpenSSL.

unsafeSkipCAVerification
bool

unsafeSkipCAVerification allows token-based discovery without CA verification via caCertHashes. This can weaken the security of kubeadm since other nodes can impersonate the control-plane.

ControlPlaneComponent

Appears in:

ControlPlaneComponent holds settings common to control plane component of the cluster

FieldDescription
extraArgs
map[string]string

extraArgs is an extra set of flags to pass to the control plane component. A key in this map is the flag name as it appears on the command line except without leading dash(es).

extraVolumes
[]HostPathMount

extraVolumes is an extra set of host volumes, mounted to the control plane component.

DNS

Appears in:

DNS defines the DNS addon that should be used in the cluster

FieldDescription
ImageMeta [Required]
ImageMeta
(Members of ImageMeta are embedded into this type.)

imageMeta allows to customize the image used for the DNS component.

Discovery

Appears in:

Discovery specifies the options for the kubelet to use during the TLS Bootstrap process.

FieldDescription
bootstrapToken
BootstrapTokenDiscovery

bootstrapToken is used to set the options for bootstrap token based discovery. bootstrapToken and file are mutually exclusive.

file
FileDiscovery

file is used to specify a file or URL to a kubeconfig file from which to load cluster information. bootstrapToken and file are mutually exclusive.

tlsBootstrapToken
string

tlsBootstrapToken is a token used for TLS bootstrapping. If bootstrapToken is set, this field is defaulted to .bootstrapToken.token, but can be overridden. If file is set, this field must be set in case the KubeConfigFile does not contain any other authentication information

timeout
meta/v1.Duration

timeout modifies the discovery timeout.

Etcd

Appears in:

Etcd contains elements describing Etcd configuration.

FieldDescription
local
LocalEtcd

local provides configuration knobs for configuring the local etcd instance. local and external are mutually exclusive.

external
ExternalEtcd

external describes how to connect to an external etcd cluster. local and external are mutually exclusive.

ExternalEtcd

Appears in:

ExternalEtcd describes an external etcd cluster. Kubeadm has no knowledge of where certificate files live and they must be supplied.

FieldDescription
endpoints [Required]
[]string

endpoints contains the list of etcd members.

caFile [Required]
string

caFile is an SSL Certificate Authority (CA) file used to secure etcd communication. Required if using a TLS connection.

certFile [Required]
string

certFile is an SSL certification file used to secure etcd communication. Required if using a TLS connection.

keyFile [Required]
string

keyFile is an SSL key file used to secure etcd communication. Required if using a TLS connection.

FileDiscovery

Appears in:

FileDiscovery is used to specify a file or URL to a kubeconfig file from which to load cluster information.

FieldDescription
kubeConfigPath [Required]
string

kubeConfigPath is used to specify the actual file path or URL to the kubeconfig file from which to load cluster information.

HostPathMount

Appears in:

HostPathMount contains elements describing volumes that are mounted from the host.

FieldDescription
name [Required]
string

name is the name of the volume inside the Pod template.

hostPath [Required]
string

hostPath is the path in the host that will be mounted inside the Pod.

mountPath [Required]
string

mountPath is the path inside the Pod where hostPath will be mounted.

readOnly
bool

readOnly controls write access to the volume.

pathType
core/v1.HostPathType

pathType is the type of the hostPath.

ImageMeta

Appears in:

ImageMeta allows to customize the image used for components that are not originated from the Kubernetes/Kubernetes release process

FieldDescription
imageRepository
string

imageRepository sets the container registry to pull images from. If not set, the imageRepository defined in ClusterConfiguration will be used instead.

imageTag
string

imageTag allows to specify a tag for the image. In case this value is set, kubeadm does not change automatically the version of the above components during upgrades.

JoinControlPlane

Appears in:

JoinControlPlane contains elements describing an additional control plane instance to be deployed on the joining node.

FieldDescription
localAPIEndpoint
APIEndpoint

localAPIEndpoint represents the endpoint of the API server instance to be deployed on this node.

certificateKey
string

certificateKey is the key that is used for decryption of certificates after they are downloaded from the secret upon joining a new control plane node. The corresponding encryption key is in the InitConfiguration. The certificate key is a hex encoded string that is an AES key of size 32 bytes.

LocalEtcd

Appears in:

LocalEtcd describes that kubeadm should run an etcd cluster locally.

FieldDescription
ImageMeta [Required]
ImageMeta
(Members of ImageMeta are embedded into this type.)

ImageMeta allows to customize the container used for etcd.

dataDir [Required]
string

dataDir is the directory etcd will place its data. Defaults to "/var/lib/etcd".

extraArgs
map[string]string

extraArgs are extra arguments provided to the etcd binary when run inside a static Pod. A key in this map is the flag name as it appears on the command line except without leading dash(es).

serverCertSANs
[]string

serverCertSANs sets extra Subject Alternative Names (SANs) for the etcd server signing certificate.

peerCertSANs
[]string

peerCertSANs sets extra Subject Alternative Names (SANs) for the etcd peer signing certificate.

Networking

Appears in:

Networking contains elements describing cluster's networking configuration.

FieldDescription
serviceSubnet
string

serviceSubnet is the subnet used by Kubernetes Services. Defaults to "10.96.0.0/12".

podSubnet
string

podSubnet is the subnet used by Pods.

dnsDomain
string

dnsDomain is the DNS domain used by Kubernetes Services. Defaults to "cluster.local".

NodeRegistrationOptions

Appears in:

NodeRegistrationOptions holds fields that relate to registering a new control-plane or node to the cluster, either via kubeadm init or kubeadm join.

FieldDescription
name
string

name is the .metadata.name field of the Node API object that will be created in this kubeadm init or kubeadm join operation. This field is also used in the CommonName field of the kubelet's client certificate to the API server. Defaults to the hostname of the node if not provided.

criSocket
string

criSocket is used to retrieve container runtime info. This information will be annotated to the Node API object, for later re-use.

taints [Required]
[]core/v1.Taint

taints specifies the taints the Node API object should be registered with. If this field is unset, i.e. nil, it will be defaulted with a control-plane taint for control-plane nodes. If you don't want to taint your control-plane node, set this field to an empty list, i.e. taints: [] in the YAML file. This field is solely used for Node registration.

kubeletExtraArgs
map[string]string

kubeletExtraArgs passes through extra arguments to the kubelet. The arguments here are passed to the kubelet command line via the environment file kubeadm writes at runtime for the kubelet to source. This overrides the generic base-level configuration in the kubelet-config ConfigMap. Flags have higher priority when parsing. These values are local and specific to the node kubeadm is executing on. A key in this map is the flag name as it appears on the command line except without leading dash(es).

ignorePreflightErrors
[]string

ignorePreflightErrors provides a list of pre-flight errors to be ignored when the current node is registered, e.g. IsPrevilegedUser,Swap. Value all ignores errors from all checks.

imagePullPolicy
core/v1.PullPolicy

imagePullPolicy specifies the policy for image pulling during kubeadm "init" and "join" operations. The value of this field must be one of "Always", "IfNotPresent" or "Never". If this field is not set, kubeadm will default it to "IfNotPresent", or pull the required images if not present on the host.

Patches

Appears in:

Patches contains options related to applying patches to components deployed by kubeadm.

FieldDescription
directory
string

directory is a path to a directory that contains files named "target[suffix][+patchtype].extension". For example, "kube-apiserver0+merge.yaml" or just "etcd.json". "target" can be one of "kube-apiserver", "kube-controller-manager", "kube-scheduler", "etcd". "patchtype" can be one of "strategic" "merge" or "json" and they match the patch formats supported by kubectl. The default "patchtype" is "strategic". "extension" must be either "json" or "yaml". "suffix" is an optional string that can be used to determine which patches are applied first alpha-numerically.

14 - kubeadm Configuration (v1beta4)

Overview

Package v1beta4 defines the v1beta4 version of the kubeadm configuration file format. This version improves on the v1beta3 format by fixing some minor issues and adding a few new fields.

A list of changes since v1beta3:

  • TODO https://github.com/kubernetes/kubeadm/issues/2890
  • Support custom environment variables in control plane components under ClusterConfiguration. Use APIServer.ExtraEnvs, ControllerManager.ExtraEnvs, Scheduler.ExtraEnvs, Etcd.Local.ExtraEnvs.
  • The ResetConfiguration API type is now supported in v1beta4. Users are able to reset a node by passing a --config file to kubeadm reset.
  • dry-run mode is now configureable in InitConfiguration and JoinConfiguration config files.
  • Replace the existing string/string extra argument maps with structured extra arguments that support duplicates. The change applies to ClusterConfiguration - APIServer.ExtraArgs, ControllerManager.ExtraArgs, Scheduler.ExtraArgs. Also to NodeRegistrationOptions.KubeletExtraArgs.
  • Add ClusterConfiguration.EncryptionAlgorithm that can be used to set the asymmetric encryption algorithm used for this cluster's keys and certificates. Can be one of "RSA-2048" (default), "RSA-3072", "RSA-4096" or "ECDSA-P256".
  • Add ClusterConfiguration.DNS.Disabled and ClusterConfiguration.Proxy.Disabled that can be used to disable the CoreDNS and kube-proxy addons during cluster initialization. Skipping the related addons phases, during cluster creation will set the same fields to false.
  • Add the NodeRegistration.ImagePullSerial field in 'InitConfigurationandJoinConfiguration`, which can be used to control if kubeadm pulls images serially or in parallel.
  • The UpgradeConfiguration kubeadm API is now supported in v1beta4 when passing --config to kubeadm upgrade subcommands. Usage of component configuration for kubelet and kube-proxy, InitConfiguration and ClusterConfiguration is deprecated and will be ignored when passing --config to upgrade subcommands.
  • Add a Timeouts structure to InitConfiguration, JoinConfiguration, ResetConfiguration and UpgradeConfiguration that can be used to configure various timeouts.

Migration from old kubeadm config versions

  • kubeadm v1.15.x and newer can be used to migrate from v1beta1 to v1beta2.
  • kubeadm v1.22.x and newer no longer support v1beta1 and older APIs, but can be used to migrate v1beta2 to v1beta3.
  • kubeadm v1.27.x and newer no longer support v1beta2 and older APIs.
  • TODO: https://github.com/kubernetes/kubeadm/issues/2890 add version that can be used to convert to v1beta4

Basics

The preferred way to configure kubeadm is to pass an YAML configuration file with the `--config“ option. Some of the configuration options defined in the kubeadm config file are also available as command line flags, but only the most common/simple use case are supported with this approach.

A kubeadm config file could contain multiple configuration types separated using three dashes (---).

kubeadm supports the following configuration types:

apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration

apiVersion: kubeadm.k8s.io/v1beta4 kind: ClusterConfiguration

apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration

apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration

apiVersion: kubeadm.k8s.io/v1beta4 kind: JoinConfiguration

To print the defaults for "init" and "join" actions use the following commands:

kubeadm config print init-defaults
kubeadm config print join-defaults

The list of configuration types that must be included in a configuration file depends by the action you are performing (init or join`) and by the configuration options you are going to use (defaults or advanced customization).

If some configuration types are not provided, or provided only partially, kubeadm will use default values; defaults provided by kubeadm includes also enforcing consistency of values across components when required (e.g. --cluster-cidr flag on controller manager and clusterCIDR on kube-proxy).

Users are always allowed to override default values, with the only exception of a small subset of setting with relevance for security (e.g. enforce authorization-mode Node and RBAC on api server).

If the user provides a configuration types that is not expected for the action you are performing, kubeadm will ignore those types and print a warning.

Kubeadm init configuration types

When executing kubeadm init with the `--config“ option, the following configuration types could be used: InitConfiguration, ClusterConfiguration, KubeProxyConfiguration, KubeletConfiguration, but only one between InitConfiguration and ClusterConfiguration is mandatory.

apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
bootstrapTokens:

	...

nodeRegistration:

	...

The InitConfiguration type should be used to configure runtime settings, that in case of kubeadm init are the configuration of the bootstrap token and all the setting which are specific to the node where kubeadm is executed, including:

  • NodeRegistration, that holds fields that relate to registering the new node to the cluster; use it to customize the node name, the CRI socket to use or any other settings that should apply to this node only (e.g. the node ip).

  • LocalAPIEndpoint, that represents the endpoint of the instance of the API server to be deployed on this node; use it e.g. to customize the API server advertise address.

apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
networking:

	...

etcd:

	...

apiServer:

	extraArgs:
	  ...
	extraVolumes:
	  ...

...

The ClusterConfiguration type should be used to configure cluster-wide settings, including settings for:

  • networking that holds configuration for the networking topology of the cluster; use it e.g. to customize Pod subnet or services subnet.

  • etcd: use it e.g. to customize the local etcd or to configure the API server for using an external etcd cluster.

  • kube-apiserver, kube-scheduler, kube-controller-manager configurations; use it to customize control-plane components by adding customized setting or overriding kubeadm default settings.

apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration

	...

The KubeProxyConfiguration type should be used to change the configuration passed to kube-proxy instances deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults.

See https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/ or https://pkg.go.dev/k8s.io/kube-proxy/config/v1alpha1#KubeProxyConfiguration for kube-proxy official documentation.

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration

	...

The KubeletConfiguration type should be used to change the configurations that will be passed to all kubelet instances deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults.

See https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ or https://pkg.go.dev/k8s.io/kubelet/config/v1beta1#KubeletConfiguration for kubelet official documentation.

Here is a fully populated example of a single YAML file containing multiple configuration types to be used during a kubeadm init run.

apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
bootstrapTokens:
  - token: "9a08jv.c0izixklcxtmnze7"
    description: "kubeadm bootstrap token"
    ttl: "24h"
  - token: "783bde.3f89s0fje9f38fhf"
    description: "another bootstrap token"
    usages:
  - authentication
  - signing
    groups:
  - system:bootstrappers:kubeadm:default-node-token

nodeRegistration:

	name: "ec2-10-100-0-1"
	criSocket: "unix:///var/run/containerd/containerd.sock"
	taints:
	  - key: "kubeadmNode"
	    value: "someValue"
	    effect: "NoSchedule"
	kubeletExtraArgs:
	  v: 4
	ignorePreflightErrors:
	  - IsPrivilegedUser
	imagePullPolicy: "IfNotPresent"

localAPIEndpoint:

	advertiseAddress: "10.100.0.1"
	bindPort: 6443

certificateKey: "e6a2eb8581237ab72a4f494f30285ec12a9694d750b9785706a83bfcbbbd2204"
skipPhases:
  - addon/kube-proxy

---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
etcd:

	# one of local or external
	local:
	  imageRepository: "registry.k8s.io"
	  imageTag: "3.2.24"
	  dataDir: "/var/lib/etcd"
	  extraArgs:
	    listen-client-urls: "http://10.100.0.1:2379"
	  serverCertSANs:
	    -  "ec2-10-100-0-1.compute-1.amazonaws.com"
	  peerCertSANs:
	    - "10.100.0.1"
	# external:
	  # endpoints:
	  # - "10.100.0.1:2379"
	  # - "10.100.0.2:2379"
	  # caFile: "/etcd/kubernetes/pki/etcd/etcd-ca.crt"
	  # certFile: "/etcd/kubernetes/pki/etcd/etcd.crt"
	  # keyFile: "/etcd/kubernetes/pki/etcd/etcd.key"

networking:

	serviceSubnet: "10.96.0.0/16"
	podSubnet: "10.244.0.0/24"
	dnsDomain: "cluster.local"

kubernetesVersion: "v1.21.0"
controlPlaneEndpoint: "10.100.0.1:6443"
apiServer:

	extraArgs:
	  authorization-mode: "Node,RBAC"
	extraVolumes:
	  - name: "some-volume"
	    hostPath: "/etc/some-path"
	    mountPath: "/etc/some-pod-path"
	    readOnly: false
	    pathType: File
	certSANs:
	  - "10.100.1.1"
	  - "ec2-10-100-0-1.compute-1.amazonaws.com"
	timeoutForControlPlane: 4m0s

controllerManager:

	extraArgs:
	  "node-cidr-mask-size": "20"
	extraVolumes:
	  - name: "some-volume"
	    hostPath: "/etc/some-path"
	    mountPath: "/etc/some-pod-path"
	    readOnly: false
	    pathType: File

scheduler:

	extraArgs:
	  address: "10.100.0.1"
	extraVolumes:
	  - name: "some-volume"
	    hostPath: "/etc/some-path"
	    mountPath: "/etc/some-pod-path"
	    readOnly: false
	    pathType: File

certificatesDir: "/etc/kubernetes/pki"
imageRepository: "registry.k8s.io"
clusterName: "example-cluster"
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
# kubelet specific options here
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
# kube-proxy specific options here

Kubeadm join configuration types

When executing kubeadm join with the --config option, the JoinConfiguration type should be provided.

apiVersion: kubeadm.k8s.io/v1beta4
kind: JoinConfiguration

	...

The JoinConfiguration type should be used to configure runtime settings, that in case of kubeadm join are the discovery method used for accessing the cluster info and all the setting which are specific to the node where kubeadm is executed, including:

  • nodeRegistration, that holds fields that relate to registering the new node to the cluster; use it to customize the node name, the CRI socket to use or any other settings that should apply to this node only (e.g. the node ip).

  • `apiEndpoint“, that represents the endpoint of the instance of the API server to be eventually deployed on this node.

Resource Types

BootstrapToken

Appears in:

BootstrapToken describes one bootstrap token, stored as a Secret in the cluster

FieldDescription
token [Required]
BootstrapTokenString

token is used for establishing bidirectional trust between nodes and control-planes. Used for joining nodes in the cluster.

description
string

description sets a human-friendly message why this token exists and what it's used for, so other administrators can know its purpose.

ttl
meta/v1.Duration

ttl defines the time to live for this token. Defaults to 24h. expires and ttl are mutually exclusive.

expires
meta/v1.Time

expires specifies the timestamp when this token expires. Defaults to being set dynamically at runtime based on the ttl. expires and ttl are mutually exclusive.

usages
[]string

usages describes the ways in which this token can be used. Can by default be used for establishing bidirectional trust, but that can be changed here.

groups
[]string

groups specifies the extra groups that this token will authenticate as when/if used for authentication

BootstrapTokenString

Appears in:

BootstrapTokenString is a token of the format abcdef.abcdef0123456789 that is used for both validation of the practically of the API server from a joining node's point of view and as an authentication method for the node in the bootstrap phase of "kubeadm join". This token is and should be short-lived.

FieldDescription
- [Required]
string
No description provided.
- [Required]
string
No description provided.

ClusterConfiguration

ClusterConfiguration contains cluster-wide configuration for a kubeadm cluster.

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta4
kind
string
ClusterConfiguration
etcd
Etcd

etcd holds the configuration for etcd.

networking
Networking

networking holds configuration for the networking topology of the cluster.

kubernetesVersion
string

kubernetesVersion is the target version of the control plane.

controlPlaneEndpoint
string

controlPlaneEndpoint sets a stable IP address or DNS name for the control plane; It can be a valid IP address or a RFC-1123 DNS subdomain, both with optional TCP port. In case the controlPlaneEndpoint is not specified, the advertiseAddress + bindPort are used; in case the controlPlaneEndpoint is specified but without a TCP port, the bindPort is used. Possible usages are:

  • In a cluster with more than one control plane instances, this field should be assigned the address of the external load balancer in front of the control plane instances.
  • In environments with enforced node recycling, the controlPlaneEndpoint could be used for assigning a stable DNS to the control plane.
apiServer
APIServer

apiServer contains extra settings for the API server.

controllerManager
ControlPlaneComponent

controllerManager contains extra settings for the controller manager.

scheduler
ControlPlaneComponent

scheduler contains extra settings for the scheduler.

dns
DNS

dns defines the options for the DNS add-on installed in the cluster.

proxy [Required]
Proxy

proxy defines the options for the proxy add-on installed in the cluster.

certificatesDir
string

certificatesDir specifies where to store or look for all required certificates.

imageRepository
string

imageRepository sets the container registry to pull images from. If empty, registry.k8s.io will be used by default. In case of kubernetes version is a CI build (kubernetes version starts with ci/) gcr.io/k8s-staging-ci-images will be used as a default for control plane components and for kube-proxy, while registry.k8s.io will be used for all the other images.

featureGates
map[string]bool

featureGates contains the feature gates enabled by the user.

clusterName
string

The cluster name.

encryptionAlgorithm
EncryptionAlgorithmType

encryptionAlgorithm holds the type of asymmetric encryption algorithm used for keys and certificates. Can be "RSA" (default algorithm, key size is 2048) or "ECDSA" (uses the P-256 elliptic curve).

InitConfiguration

InitConfiguration contains a list of elements that is specific "kubeadm init"-only runtime information. kubeadm init-only information. These fields are solely used the first time kubeadm init runs. After that, the information in the fields IS NOT uploaded to the kubeadm-config ConfigMap that is used by kubeadm upgrade for instance. These fields must be omitempty.

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta4
kind
string
InitConfiguration
bootstrapTokens
[]BootstrapToken

bootstrapTokens is respected at kubeadm init time and describes a set of Bootstrap Tokens to create. This information IS NOT uploaded to the kubeadm cluster configmap, partly because of its sensitive nature

dryRun [Required]
bool

dryRun tells if the dry run mode is enabled, don't apply any change in dry run mode, just out put what would be done.

nodeRegistration
NodeRegistrationOptions

nodeRegistration holds fields that relate to registering the new control-plane node to the cluster.

localAPIEndpoint
APIEndpoint

localAPIEndpoint represents the endpoint of the API server instance that's deployed on this control plane node. In HA setups, this differs from ClusterConfiguration.controlPlaneEndpoint in the sense that controlPlaneEndpoint is the global endpoint for the cluster, which then loadbalances the requests to each individual API server. This configuration object lets you customize what IP/DNS name and port the local API server advertises it's accessible on. By default, kubeadm tries to auto-detect the IP of the default interface and use that, but in case that process fails you may set the desired value here.

certificateKey
string

certificateKey sets the key with which certificates and keys are encrypted prior to being uploaded in a Secret in the cluster during the uploadcerts init phase. The certificate key is a hex encoded string that is an AES key of size 32 bytes.

skipPhases
[]string

skipPhases is a list of phases to skip during command execution. The list of phases can be obtained with the kubeadm init --help command. The flag --skip-phases takes precedence over this field.

patches
Patches

patches contains options related to applying patches to components deployed by kubeadm during kubeadm init.

JoinConfiguration

JoinConfiguration contains elements describing a particular node.

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta4
kind
string
JoinConfiguration
dryRun
bool

dryRun tells if the dry run mode is enabled, don't apply any change if it is set, just output what would be done.

nodeRegistration
NodeRegistrationOptions

nodeRegistration holds fields that relate to registering the new control-plane node to the cluster

caCertPath
string

caCertPath is the path to the SSL certificate authority used to secure comunications between node and control-plane. Defaults to "/etc/kubernetes/pki/ca.crt".

discovery [Required]
Discovery

discovery specifies the options for the kubelet to use during the TLS bootstrap process.

controlPlane
JoinControlPlane

controlPlane defines the additional control plane instance to be deployed on the joining node. If nil, no additional control plane instance will be deployed.

skipPhases
[]string

skipPhases is a list of phases to skip during command execution. The list of phases can be obtained with the kubeadm join --help command. The flag --skip-phases takes precedence over this field.

patches
Patches

patches contains options related to applying patches to components deployed by kubeadm during kubeadm join.

ResetConfiguration

ResetConfiguration contains a list of fields that are specifically kubeadm reset-only runtime information.

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta4
kind
string
ResetConfiguration
cleanupTmpDir
bool

cleanupTmpDir specifies whether the "/etc/kubernetes/tmp" directory should be cleaned during the reset process.

certificatesDir
string

certificatesDir specifies the directory where the certificates are stored. If specified, it will be cleaned during the reset process.

criSocket
string

criSocket is used to retrieve container runtime inforomation and used for the removal of the containers. If criSocket is not specified by flag or config file, kubeadm will try to detect one valid CRI socket instead.

dryRun
bool

dryRun tells if the dry run mode is enabled, don't apply any change if it is set and just output what would be done.

force
bool

The force flag instructs kubeadm to reset the node without prompting for confirmation.

ignorePreflightErrors
[]string

ignorePreflightErrors provides a list of pre-flight errors to be ignored during the reset process, e.g. IsPrivilegedUser,Swap. Value all ignores errors from all checks.

skipPhases
[]string

skipPhases is a list of phases to skip during command execution. The list of phases can be obtained with the kubeadm reset phase --help command.

UpgradeConfiguration

UpgradeConfiguration contains a list of options that are specific to kubeadm upgrade subcommands.

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta4
kind
string
UpgradeConfiguration
apply
UpgradeApplyConfiguration

apply holds a list of options that are specific to the kubeadm upgrade apply command.

diff
UpgradeDiffConfiguration

diff holds a list of options that are specific to the kubeadm upgrade diff command.

node
UpgradeNodeConfiguration

node holds a list of options that are specific to the kubeadm upgrade node command.

plan
UpgradePlanConfiguration

plan holds a list of options that are specific to the kubeadm upgrade plan command.

timeouts
Timeouts

timeouts holds various timeouts that apply to kubeadm commands.

APIEndpoint

Appears in:

APIEndpoint struct contains elements of API server instance deployed on a node.

FieldDescription
advertiseAddress
string

dvertiseAddress sets the IP address for the API server to advertise.

bindPort
int32

bindPort sets the secure port for the API Server to bind to. Defaults to 6443.

APIServer

Appears in:

APIServer holds settings necessary for API server deployments in the cluster

FieldDescription
ControlPlaneComponent [Required]
ControlPlaneComponent
(Members of ControlPlaneComponent are embedded into this type.) No description provided.
certSANs
[]string

certSANs sets extra Subject Alternative Names (SANs) for the API Server signing certificate.

timeoutForControlPlane
meta/v1.Duration

timeoutForControlPlane controls the timeout that we use for API server to appear.

Arg

Appears in:

Arg represents an argument with a name and a value.

FieldDescription
name [Required]
string

The name of the argument.

value [Required]
string

The value of the argument.

BootstrapTokenDiscovery

Appears in:

BootstrapTokenDiscovery is used to set the options for bootstrap token based discovery.

FieldDescription
token [Required]
string

token is a token used to validate cluster information fetched from the control-plane.

apiServerEndpoint
string

apiServerEndpoint is an IP or domain name to the API server from which information will be fetched.

caCertHashes
[]string

caCertHashes specifies a set of public key pins to verify when token-based discovery is used. The root CA found during discovery must match one of these values. Specifying an empty set disables root CA pinning, which can be unsafe. Each hash is specified as <type>:<value>, where the only currently supported type is "sha256". This is a hex-encoded SHA-256 hash of the Subject Public Key Info (SPKI) object in DER-encoded ASN.1. These hashes can be // calculated using, for example, OpenSSL.

unsafeSkipCAVerification
bool

unsafeSkipCAVerification allows token-based discovery without CA verification via caCertHashes. This can weaken the security of kubeadm since other nodes can impersonate the control-plane.

ControlPlaneComponent

Appears in:

ControlPlaneComponent holds settings common to control plane component of the cluster

FieldDescription
extraArgs
[]Arg

extraArgs is an extra set of flags to pass to the control plane component. An argument name in this list is the flag name as it appears on the command line except without leading dash(es). Extra arguments will override existing default arguments. Duplicate extra arguments are allowed.

extraVolumes
[]HostPathMount

extraVolumes is an extra set of host volumes, mounted to the control plane component.

extraEnvs
[]EnvVar

extraEnvs is an extra set of environment variables to pass to the control plane component. Environment variables passed using extraEnvs will override any existing environment variables, or *_proxy environment variables that kubeadm adds by default.

DNS

Appears in:

DNS defines the DNS addon that should be used in the cluster

FieldDescription
ImageMeta [Required]
ImageMeta
(Members of ImageMeta are embedded into this type.)

imageMeta allows to customize the image used for the DNS addon.

disabled [Required]
bool

disabled specifies whether to disable this addon in the cluster.

Discovery

Appears in:

Discovery specifies the options for the kubelet to use during the TLS Bootstrap process

FieldDescription
bootstrapToken
BootstrapTokenDiscovery

bootstrapToken is used to set the options for bootstrap token based discovery. bootstrapToken and file are mutually exclusive.

file
FileDiscovery

file is used to specify a file or URL to a kubeconfig file from which to load cluster information. bootstrapToken and file are mutually exclusive.

tlsBootstrapToken
string

tlsBootstrapToken is a token used for TLS bootstrapping. If bootstrapToken is set, this field is defaulted to bootstrapToken.token, but can be overridden. If file is set, this field must be set in case the KubeConfigFile does not contain any other authentication information.

timeout
meta/v1.Duration

timeout modifies the discovery timeout.

EncryptionAlgorithmType

(Alias of string)

Appears in:

EncryptionAlgorithmType can define an asymmetric encryption algorithm type.

EnvVar

Appears in:

EnvVar represents an environment variable present in a Container.

FieldDescription
EnvVar [Required]
core/v1.EnvVar
(Members of EnvVar are embedded into this type.) No description provided.

Etcd

Appears in:

Etcd contains elements describing Etcd configuration.

FieldDescription
local
LocalEtcd

local provides configuration knobs for configuring the local etcd instance. local and external are mutually exclusive.

external
ExternalEtcd

external describes how to connect to an external etcd cluster. local and external are mutually exclusive.

ExternalEtcd

Appears in:

ExternalEtcd describes an external etcd cluster. Kubeadm has no knowledge of where certificate files live and they must be supplied.

FieldDescription
endpoints [Required]
[]string

endpoints contains the list of etcd members.

caFile [Required]
string

caFile is an SSL Certificate Authority (CA) file used to secure etcd communication. Required if using a TLS connection.

certFile [Required]
string

certFile is an SSL certification file used to secure etcd communication. Required if using a TLS connection.

keyFile [Required]
string

keyFile is an SSL key file used to secure etcd communication. Required if using a TLS connection.

FileDiscovery

Appears in:

FileDiscovery is used to specify a file or URL to a kubeconfig file from which to load cluster information.

FieldDescription
kubeConfigPath [Required]
string

kubeConfigPath is used to specify the actual file path or URL to the kubeconfig file from which to load cluster information.

HostPathMount

Appears in:

HostPathMount contains elements describing volumes that are mounted from the host.

FieldDescription
name [Required]
string

name is the name of the volume inside the Pod template.

hostPath [Required]
string

hostPath is the path in the host that will be mounted inside the Pod.

mountPath [Required]
string

mountPath is the path inside the Pod where hostPath will be mounted.

readOnly
bool

readOnly controls write access to the volume.

pathType
core/v1.HostPathType

pathType is the type of the hostPath.

ImageMeta

Appears in:

ImageMeta allows to customize the image used for components that are not originated from the Kubernetes/Kubernetes release process

FieldDescription
imageRepository
string

imageRepository sets the container registry to pull images from. if not set, the imageRepository defined in ClusterConfiguration will be used instead.

imageTag
string

imageTag allows to specify a tag for the image. In case this value is set, kubeadm does not change automatically the version of the above components during upgrades.

JoinControlPlane

Appears in:

JoinControlPlane contains elements describing an additional control plane instance to be deployed on the joining node.

FieldDescription
localAPIEndpoint
APIEndpoint

localAPIEndpoint represents the endpoint of the API server instance to be deployed on this node.

certificateKey
string

certificateKey is the key that is used for decryption of certificates after they are downloaded from the Secret upon joining a new control plane node. The corresponding encryption key is in the InitConfiguration. The certificate key is a hex encoded string that is an AES key of size 32 bytes.

LocalEtcd

Appears in:

LocalEtcd describes that kubeadm should run an etcd cluster locally.

FieldDescription
ImageMeta [Required]
ImageMeta
(Members of ImageMeta are embedded into this type.)

ImageMeta allows to customize the container used for etcd

dataDir [Required]
string

dataDir is the directory etcd will place its data. Defaults to "/var/lib/etcd".

extraArgs [Required]
[]Arg

extraArgs are extra arguments provided to the etcd binary when run inside a static Pod. An argument name in this list is the flag name as it appears on the command line except without leading dash(es). Extra arguments will override existing default arguments. Duplicate extra arguments are allowed.

extraEnvs
[]EnvVar

extraEnvs is an extra set of environment variables to pass to the control plane component. Environment variables passed using extraEnvs will override any existing environment variables, or *_proxy environment variables that kubeadm adds by default.

serverCertSANs
[]string

serverCertSANs sets extra Subject Alternative Names (SANs) for the etcd server signing certificate.

peerCertSANs
[]string

peerCertSANs sets extra Subject Alternative Names (SANs) for the etcd peer signing certificate.

Networking

Appears in:

Networking contains elements describing cluster's networking configuration.

FieldDescription
serviceSubnet
string

serviceSubnet is the subnet used by Kubernetes Services. Defaults to "10.96.0.0/12".

podSubnet
string

podSubnet is the subnet used by Pods.

dnsDomain
string

dnsDomain is the dns domain used by Kubernetes Services. Defaults to "cluster.local".

NodeRegistrationOptions

Appears in:

NodeRegistrationOptions holds fields that relate to registering a new control-plane or node to the cluster, either via kubeadm init or kubeadm join.

FieldDescription
name
string

name is the .Metadata.Name field of the Node API object that will be created in this kubeadm init or kubeadm join operation. This field is also used in the CommonName field of the kubelet's client certificate to the API server. Defaults to the hostname of the node if not provided.

criSocket
string

criSocket is used to retrieve container runtime info. This information will be annotated to the Node API object, for later re-use.

taints [Required]
[]core/v1.Taint

taints specifies the taints the Node API object should be registered with. If this field is unset, i.e. nil, it will be defaulted with a control-plane taint for control-plane nodes. If you don't want to taint your control-plane node, set this field to an empty list, i.e. taints: [] in the YAML file. This field is solely used for Node registration.

kubeletExtraArgs
[]Arg

kubeletExtraArgs passes through extra arguments to the kubelet. The arguments here are passed to the kubelet command line via the environment file kubeadm writes at runtime for the kubelet to source. This overrides the generic base-level configuration in the kubelet-config ConfigMap. Flags have higher priority when parsing. These values are local and specific to the node kubeadm is executing on. An argument name in this list is the flag name as it appears on the command line except without leading dash(es). Extra arguments will override existing default arguments. Duplicate extra arguments are allowed.

ignorePreflightErrors
[]string

ignorePreflightErrors provides a slice of pre-flight errors to be ignored when the current node is registered, e.g. 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.

imagePullPolicy
core/v1.PullPolicy

imagePullPolicy specifies the policy for image pulling during kubeadm init and join operations. The value of this field must be one of "Always", "IfNotPresent" or "Never". If this field is unset kubeadm will default it to "IfNotPresent", or pull the required images if not present on the host.

Patches

Appears in:

Patches contains options related to applying patches to components deployed by kubeadm.

FieldDescription
directory
string

directory is a path to a directory that contains files named "target[suffix][+patchtype].extension". For example, "kube-apiserver0+merge.yaml" or just "etcd.json". "target" can be one of "kube-apiserver", "kube-controller-manager", "kube-scheduler", "etcd", "kubeletconfiguration". "patchtype" can be one of "strategic", "merge" or "json" and they match the patch formats supported by kubectl. The default "patchtype" is "strategic". "extension" must be either "json" or "yaml". "suffix" is an optional string that can be used to determine which patches are applied first alpha-numerically.

Proxy

Appears in:

Proxy defines the proxy addon that should be used in the cluster.

FieldDescription
disabled [Required]
bool

disabled specifies whether to disable this addon in the cluster.

Timeouts

Appears in:

Timeouts holds various timeouts that apply to kubeadm commands.

FieldDescription
controlPlaneComponentHealthCheck
meta/v1.Duration

controlPlaneComponentHealthCheck is the amount of time to wait for a control plane component, such as the API server, to be healthy during kubeadm init and kubeadm join. Default: 4m

kubeletHealthCheck
meta/v1.Duration

kubeletHealthCheck is the amount of time to wait for the kubelet to be healthy during kubeadm init and kubeadm join. Default: 4m

kubernetesAPICall
meta/v1.Duration

kubernetesAPICall is the amount of time to wait for the kubeadm client to complete a request to the API server. This applies to all types of methods (GET, POST, etc). Default: 1m

etcdAPICall
meta/v1.Duration

etcdAPICall is the amount of time to wait for the kubeadm etcd client to complete a request to the etcd cluster. Default: 2m

tlsBootstrap
meta/v1.Duration

tlsBootstrap is the amount of time to wait for the kubelet to complete TLS bootstrap for a joining node. Default: 5m

discovery
meta/v1.Duration

discovery is the amount of time to wait for kubeadm to validate the API server identity for a joining node. Default: 5m

upgradeManifests [Required]
meta/v1.Duration

upgradeManifests is the timeout for upgradring static Pod manifests Default: 5m

UpgradeApplyConfiguration

Appears in:

UpgradeApplyConfiguration contains a list of configurable options which are specific to the "kubeadm upgrade apply" command.

FieldDescription
kubernetesVersion
string

kubernetesVersion is the target version of the control plane.

allowExperimentalUpgrades
bool

allowExperimentalUpgrades instructs kubeadm to show unstable versions of Kubernetes as an upgrade alternative and allows upgrading to an alpha/beta/release candidate version of Kubernetes. Default: false

allowRCUpgrades
bool

Enable allowRCUpgrades will show release candidate versions of Kubernetes as an upgrade alternative and allows upgrading to a release candidate version of Kubernetes.

certificateRenewal
bool

certificateRenewal instructs kubeadm to execute certificate renewal during upgrades. Defaults to true.

dryRun
bool

dryRun tells if the dry run mode is enabled, don't apply any change if it is and just output what would be done.

etcdUpgrade
bool

etcdUpgrade instructs kubeadm to execute etcd upgrade during upgrades. Defaults to true.

forceUpgrade
bool

forceUpgrade flag instructs kubeadm to upgrade the cluster without prompting for confirmation.

ignorePreflightErrors
[]string

ignorePreflightErrors provides a slice of pre-flight errors to be ignored during the upgrade process, e.g. 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.

patches
Patches

patches contains options related to applying patches to components deployed by kubeadm during "kubeadm upgrade".

printConfig
bool

printConfig specifies whether the configuration file that will be used in the upgrade should be printed or not.

SkipPhases [Required]
[]string

skipPhases is a list of phases to skip during command execution. NOTE: This field is currently ignored for "kubeadm upgrade apply", but in the future it will be supported.

UpgradeDiffConfiguration

Appears in:

UpgradeDiffConfiguration contains a list of configurable options which are specific to the "kubeadm upgrade diff" command.

FieldDescription
kubernetesVersion
string

kubernetesVersion is the target version of the control plane.

contextLines
int

diffContextLines is the number of lines of context in the diff.

UpgradeNodeConfiguration

Appears in:

UpgradeNodeConfiguration contains a list of configurable options which are specific to the "kubeadm upgrade node" command.

FieldDescription
certificateRenewal
bool

certificateRenewal instructs kubeadm to execute certificate renewal during upgrades. Defaults to true.

dryRun
bool

dryRun tells if the dry run mode is enabled, don't apply any change if it is and just output what would be done.

etcdUpgrade
bool

etcdUpgrade instructs kubeadm to execute etcd upgrade during upgrades. Defaults to true.

ignorePreflightErrors
[]string

ignorePreflightErrors provides a slice of pre-flight errors to be ignored during the upgrade process, e.g. 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.

skipPhases
[]string

skipPhases is a list of phases to skip during command execution. The list of phases can be obtained with the kubeadm upgrade node phase --help command.

patches
Patches

patches contains options related to applying patches to components deployed by kubeadm during kubeadm upgrade.

UpgradePlanConfiguration

Appears in:

UpgradePlanConfiguration contains a list of configurable options which are specific to the "kubeadm upgrade plan" command.

FieldDescription
kubernetesVersion [Required]
string

kubernetesVersion is the target version of the control plane.

allowExperimentalUpgrades
bool

allowExperimentalUpgrades instructs kubeadm to show unstable versions of Kubernetes as an upgrade alternative and allows upgrading to an alpha/beta/release candidate version of Kubernetes. Default: false

allowRCUpgrades
bool

Enable allowRCUpgrades will show release candidate versions of Kubernetes as an upgrade alternative and allows upgrading to a release candidate version of Kubernetes.

dryRun
bool

dryRun tells if the dry run mode is enabled, don't apply any change if it is and just output what would be done.

ignorePreflightErrors
[]string

ignorePreflightErrors provides a slice of pre-flight errors to be ignored during the upgrade process, e.g. 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.

printConfig
bool

printConfig specifies whether the configuration file that will be used in the upgrade should be printed or not.

15 - kubeconfig (v1)

Resource Types

Config

Config holds the information needed to build connect to remote kubernetes clusters as a given user

FieldDescription
apiVersion
string
/v1
kind
string
Config
kind
string

Legacy field from pkg/api/types.go TypeMeta. TODO(jlowdermilk): remove this after eliminating downstream dependencies.

apiVersion
string

Legacy field from pkg/api/types.go TypeMeta. TODO(jlowdermilk): remove this after eliminating downstream dependencies.

preferences [Required]
Preferences

Preferences holds general information to be use for cli interactions

clusters [Required]
[]NamedCluster

Clusters is a map of referencable names to cluster configs

users [Required]
[]NamedAuthInfo

AuthInfos is a map of referencable names to user configs

contexts [Required]
[]NamedContext

Contexts is a map of referencable names to context configs

current-context [Required]
string

CurrentContext is the name of the context that you would like to use by default

extensions
[]NamedExtension

Extensions holds additional information. This is useful for extenders so that reads and writes don't clobber unknown fields

AuthInfo

Appears in:

AuthInfo contains information that describes identity information. This is use to tell the kubernetes cluster who you are.

FieldDescription
client-certificate
string

ClientCertificate is the path to a client cert file for TLS.

client-certificate-data
[]byte

ClientCertificateData contains PEM-encoded data from a client cert file for TLS. Overrides ClientCertificate

client-key
string

ClientKey is the path to a client key file for TLS.

client-key-data
[]byte

ClientKeyData contains PEM-encoded data from a client key file for TLS. Overrides ClientKey

token
string

Token is the bearer token for authentication to the kubernetes cluster.

tokenFile
string

TokenFile is a pointer to a file that contains a bearer token (as described above). If both Token and TokenFile are present, Token takes precedence.

as
string

Impersonate is the username to impersonate. The name matches the flag.

as-uid
string

ImpersonateUID is the uid to impersonate.

as-groups
[]string

ImpersonateGroups is the groups to impersonate.

as-user-extra
map[string][]string

ImpersonateUserExtra contains additional information for impersonated user.

username
string

Username is the username for basic authentication to the kubernetes cluster.

password
string

Password is the password for basic authentication to the kubernetes cluster.

auth-provider
AuthProviderConfig

AuthProvider specifies a custom authentication plugin for the kubernetes cluster.

exec
ExecConfig

Exec specifies a custom exec-based authentication plugin for the kubernetes cluster.

extensions
[]NamedExtension

Extensions holds additional information. This is useful for extenders so that reads and writes don't clobber unknown fields

AuthProviderConfig

Appears in:

AuthProviderConfig holds the configuration for a specified auth provider.

FieldDescription
name [Required]
string
No description provided.
config [Required]
map[string]string
No description provided.

Cluster

Appears in:

Cluster contains information about how to communicate with a kubernetes cluster

FieldDescription
server [Required]
string

Server is the address of the kubernetes cluster (https://hostname:port).

tls-server-name
string

TLSServerName is used to check server certificate. If TLSServerName is empty, the hostname used to contact the server is used.

insecure-skip-tls-verify
bool

InsecureSkipTLSVerify skips the validity check for the server's certificate. This will make your HTTPS connections insecure.

certificate-authority
string

CertificateAuthority is the path to a cert file for the certificate authority.

certificate-authority-data
[]byte

CertificateAuthorityData contains PEM-encoded certificate authority certificates. Overrides CertificateAuthority

proxy-url
string

ProxyURL is the URL to the proxy to be used for all requests made by this client. URLs with "http", "https", and "socks5" schemes are supported. If this configuration is not provided or the empty string, the client attempts to construct a proxy configuration from http_proxy and https_proxy environment variables. If these environment variables are not set, the client does not attempt to proxy requests.

socks5 proxying does not currently support spdy streaming endpoints (exec, attach, port forward).

disable-compression
bool

DisableCompression allows client to opt-out of response compression for all requests to the server. This is useful to speed up requests (specifically lists) when client-server network bandwidth is ample, by saving time on compression (server-side) and decompression (client-side): https://github.com/kubernetes/kubernetes/issues/112296.

extensions
[]NamedExtension

Extensions holds additional information. This is useful for extenders so that reads and writes don't clobber unknown fields

Context

Appears in:

Context is a tuple of references to a cluster (how do I communicate with a kubernetes cluster), a user (how do I identify myself), and a namespace (what subset of resources do I want to work with)

FieldDescription
cluster [Required]
string

Cluster is the name of the cluster for this context

user [Required]
string

AuthInfo is the name of the authInfo for this context

namespace
string

Namespace is the default namespace to use on unspecified requests

extensions
[]NamedExtension

Extensions holds additional information. This is useful for extenders so that reads and writes don't clobber unknown fields

ExecConfig

Appears in:

ExecConfig specifies a command to provide client credentials. The command is exec'd and outputs structured stdout holding credentials.

See the client.authentication.k8s.io API group for specifications of the exact input and output format

FieldDescription
command [Required]
string

Command to execute.

args
[]string

Arguments to pass to the command when executing it.

env
[]ExecEnvVar

Env defines additional environment variables to expose to the process. These are unioned with the host's environment, as well as variables client-go uses to pass argument to the plugin.

apiVersion [Required]
string

Preferred input version of the ExecInfo. The returned ExecCredentials MUST use the same encoding version as the input.

installHint [Required]
string

This text is shown to the user when the executable doesn't seem to be present. For example, brew install foo-cli might be a good InstallHint for foo-cli on Mac OS systems.

provideClusterInfo [Required]
bool

ProvideClusterInfo determines whether or not to provide cluster information, which could potentially contain very large CA data, to this exec plugin as a part of the KUBERNETES_EXEC_INFO environment variable. By default, it is set to false. Package k8s.io/client-go/tools/auth/exec provides helper methods for reading this environment variable.

interactiveMode
ExecInteractiveMode

InteractiveMode determines this plugin's relationship with standard input. Valid values are "Never" (this exec plugin never uses standard input), "IfAvailable" (this exec plugin wants to use standard input if it is available), or "Always" (this exec plugin requires standard input to function). See ExecInteractiveMode values for more details.

If APIVersion is client.authentication.k8s.io/v1alpha1 or client.authentication.k8s.io/v1beta1, then this field is optional and defaults to "IfAvailable" when unset. Otherwise, this field is required.

ExecEnvVar

Appears in:

ExecEnvVar is used for setting environment variables when executing an exec-based credential plugin.

FieldDescription
name [Required]
string
No description provided.
value [Required]
string
No description provided.

ExecInteractiveMode

(Alias of string)

Appears in:

ExecInteractiveMode is a string that describes an exec plugin's relationship with standard input.

NamedAuthInfo

Appears in:

NamedAuthInfo relates nicknames to auth information

FieldDescription
name [Required]
string

Name is the nickname for this AuthInfo

user [Required]
AuthInfo

AuthInfo holds the auth information

NamedCluster

Appears in:

NamedCluster relates nicknames to cluster information

FieldDescription
name [Required]
string

Name is the nickname for this Cluster

cluster [Required]
Cluster

Cluster holds the cluster information

NamedContext

Appears in:

NamedContext relates nicknames to context information

FieldDescription
name [Required]
string

Name is the nickname for this Context

context [Required]
Context

Context holds the context information

NamedExtension

Appears in:

NamedExtension relates nicknames to extension information

FieldDescription
name [Required]
string

Name is the nickname for this Extension

extension [Required]
k8s.io/apimachinery/pkg/runtime.RawExtension

Extension holds the extension information

Preferences

Appears in:

FieldDescription
colors
bool
No description provided.
extensions
[]NamedExtension

Extensions holds additional information. This is useful for extenders so that reads and writes don't clobber unknown fields

16 - Kubelet Configuration (v1)

Resource Types

CredentialProviderConfig

CredentialProviderConfig is the configuration containing information about each exec credential provider. Kubelet reads this configuration from disk and enables each provider as specified by the CredentialProvider type.

FieldDescription
apiVersion
string
kubelet.config.k8s.io/v1
kind
string
CredentialProviderConfig
providers [Required]
[]CredentialProvider

providers is a list of credential provider plugins that will be enabled by the kubelet. Multiple providers may match against a single image, in which case credentials from all providers will be returned to the kubelet. If multiple providers are called for a single image, the results are combined. If providers return overlapping auth keys, the value from the provider earlier in this list is used.

CredentialProvider

Appears in:

CredentialProvider represents an exec plugin to be invoked by the kubelet. The plugin is only invoked when an image being pulled matches the images handled by the plugin (see matchImages).

FieldDescription
name [Required]
string

name is the required name of the credential provider. It must match the name of the provider executable as seen by the kubelet. The executable must be in the kubelet's bin directory (set by the --image-credential-provider-bin-dir flag).

matchImages [Required]
[]string

matchImages is a required list of strings used to match against images in order to determine if this provider should be invoked. If one of the strings matches the requested image from the kubelet, the plugin will be invoked and given a chance to provide credentials. Images are expected to contain the registry domain and URL path.

Each entry in matchImages is a pattern which can optionally contain a port and a path. Globs can be used in the domain, but not in the port or the path. Globs are supported as subdomains like '*.k8s.io' or 'k8s.*.io', and top-level-domains such as 'k8s.*'. Matching partial subdomains like 'app*.k8s.io' is also supported. Each glob can only match a single subdomain segment, so '*.io' does not match '*.k8s.io'.

A match exists between an image and a matchImage when all of the below are true:

  • Both contain the same number of domain parts and each part matches.
  • The URL path of an imageMatch must be a prefix of the target image URL path.
  • If the imageMatch contains a port, then the port must match in the image as well.

Example values of matchImages:

  • 123456789.dkr.ecr.us-east-1.amazonaws.com
  • *.azurecr.io
  • gcr.io
  • *.*.registry.io
  • registry.io:8080/path
defaultCacheDuration [Required]
meta/v1.Duration

defaultCacheDuration is the default duration the plugin will cache credentials in-memory if a cache duration is not provided in the plugin response. This field is required.

apiVersion [Required]
string

Required input version of the exec CredentialProviderRequest. The returned CredentialProviderResponse MUST use the same encoding version as the input. Current supported values are:

  • credentialprovider.kubelet.k8s.io/v1
args
[]string

Arguments to pass to the command when executing it.

env
[]ExecEnvVar

Env defines additional environment variables to expose to the process. These are unioned with the host's environment, as well as variables client-go uses to pass argument to the plugin.

ExecEnvVar

Appears in:

ExecEnvVar is used for setting environment variables when executing an exec-based credential plugin.

FieldDescription
name [Required]
string
No description provided.
value [Required]
string
No description provided.

17 - Kubelet Configuration (v1alpha1)

Resource Types

CredentialProviderConfig

CredentialProviderConfig is the configuration containing information about each exec credential provider. Kubelet reads this configuration from disk and enables each provider as specified by the CredentialProvider type.

FieldDescription
apiVersion
string
kubelet.config.k8s.io/v1alpha1
kind
string
CredentialProviderConfig
providers [Required]
[]CredentialProvider

providers is a list of credential provider plugins that will be enabled by the kubelet. Multiple providers may match against a single image, in which case credentials from all providers will be returned to the kubelet. If multiple providers are called for a single image, the results are combined. If providers return overlapping auth keys, the value from the provider earlier in this list is used.

CredentialProvider

Appears in:

CredentialProvider represents an exec plugin to be invoked by the kubelet. The plugin is only invoked when an image being pulled matches the images handled by the plugin (see matchImages).

FieldDescription
name [Required]
string

name is the required name of the credential provider. It must match the name of the provider executable as seen by the kubelet. The executable must be in the kubelet's bin directory (set by the --image-credential-provider-bin-dir flag).

matchImages [Required]
[]string

matchImages is a required list of strings used to match against images in order to determine if this provider should be invoked. If one of the strings matches the requested image from the kubelet, the plugin will be invoked and given a chance to provide credentials. Images are expected to contain the registry domain and URL path.

Each entry in matchImages is a pattern which can optionally contain a port and a path. Globs can be used in the domain, but not in the port or the path. Globs are supported as subdomains like *.k8s.io or k8s.*.io, and top-level-domains such as k8s.*. Matching partial subdomains like app*.k8s.io is also supported. Each glob can only match a single subdomain segment, so *.io does not match *.k8s.io.

A match exists between an image and a matchImage when all of the below are true:

  • Both contain the same number of domain parts and each part matches.
  • The URL path of an imageMatch must be a prefix of the target image URL path.
  • If the imageMatch contains a port, then the port must match in the image as well.

Example values of matchImages:

  • 123456789.dkr.ecr.us-east-1.amazonaws.com
  • *.azurecr.io
  • gcr.io
  • *.*.registry.io
  • registry.io:8080/path
defaultCacheDuration [Required]
meta/v1.Duration

defaultCacheDuration is the default duration the plugin will cache credentials in-memory if a cache duration is not provided in the plugin response. This field is required.

apiVersion [Required]
string

Required input version of the exec CredentialProviderRequest. The returned CredentialProviderResponse MUST use the same encoding version as the input. Current supported values are:

  • credentialprovider.kubelet.k8s.io/v1alpha1
args
[]string

Arguments to pass to the command when executing it.

env
[]ExecEnvVar

Env defines additional environment variables to expose to the process. These are unioned with the host's environment, as well as variables client-go uses to pass argument to the plugin.

ExecEnvVar

Appears in:

ExecEnvVar is used for setting environment variables when executing an exec-based credential plugin.

FieldDescription
name [Required]
string
No description provided.
value [Required]
string
No description provided.

18 - Kubelet Configuration (v1beta1)

Resource Types

FormatOptions

Appears in:

FormatOptions contains options for the different logging formats.

FieldDescription
text [Required]
TextOptions

[Alpha] Text contains options for logging format "text". Only available when the LoggingAlphaOptions feature gate is enabled.

json [Required]
JSONOptions

[Alpha] JSON contains options for logging format "json". Only available when the LoggingAlphaOptions feature gate is enabled.

JSONOptions

Appears in:

JSONOptions contains options for logging format "json".

FieldDescription
OutputRoutingOptions [Required]
OutputRoutingOptions
(Members of OutputRoutingOptions are embedded into this type.) No description provided.

LogFormatFactory

LogFormatFactory provides support for a certain additional, non-default log format.

LoggingConfiguration

Appears in:

LoggingConfiguration contains logging options.

FieldDescription
format [Required]
string

Format Flag specifies the structure of log messages. default value of format is text

flushFrequency [Required]
TimeOrMetaDuration

Maximum time between log flushes. If a string, parsed as a duration (i.e. "1s") If an int, the maximum number of nanoseconds (i.e. 1s = 1000000000). Ignored if the selected logging backend writes log messages without buffering.

verbosity [Required]
VerbosityLevel

Verbosity is the threshold that determines which log messages are logged. Default is zero which logs only the most important messages. Higher values enable additional messages. Error messages are always logged.

vmodule [Required]
VModuleConfiguration

VModule overrides the verbosity threshold for individual files. Only supported for "text" log format.

options [Required]
FormatOptions

[Alpha] Options holds additional parameters that are specific to the different logging formats. Only the options for the selected format get used, but all of them get validated. Only available when the LoggingAlphaOptions feature gate is enabled.

LoggingOptions

LoggingOptions can be used with ValidateAndApplyWithOptions to override certain global defaults.

FieldDescription
ErrorStream [Required]
io.Writer

ErrorStream can be used to override the os.Stderr default.

InfoStream [Required]
io.Writer

InfoStream can be used to override the os.Stdout default.

OutputRoutingOptions

Appears in:

OutputRoutingOptions contains options that are supported by both "text" and "json".

FieldDescription
splitStream [Required]
bool

[Alpha] SplitStream redirects error messages to stderr while info messages go to stdout, with buffering. The default is to write both to stdout, without buffering. Only available when the LoggingAlphaOptions feature gate is enabled.

infoBufferSize [Required]
k8s.io/apimachinery/pkg/api/resource.QuantityValue

[Alpha] InfoBufferSize sets the size of the info stream when using split streams. The default is zero, which disables buffering. Only available when the LoggingAlphaOptions feature gate is enabled.

TextOptions

Appears in:

TextOptions contains options for logging format "text".

FieldDescription
OutputRoutingOptions [Required]
OutputRoutingOptions
(Members of OutputRoutingOptions are embedded into this type.) No description provided.

TimeOrMetaDuration

Appears in:

TimeOrMetaDuration is present only for backwards compatibility for the flushFrequency field, and new fields should use metav1.Duration.

FieldDescription
Duration [Required]
meta/v1.Duration

Duration holds the duration

- [Required]
bool

SerializeAsString controls whether the value is serialized as a string or an integer

TracingConfiguration

Appears in:

TracingConfiguration provides versioned configuration for OpenTelemetry tracing clients.

FieldDescription
endpoint
string

Endpoint of the collector this component will report traces to. The connection is insecure, and does not currently support TLS. Recommended is unset, and endpoint is the otlp grpc default, localhost:4317.

samplingRatePerMillion
int32

SamplingRatePerMillion is the number of samples to collect per million spans. Recommended is unset. If unset, sampler respects its parent span's sampling rate, but otherwise never samples.

VModuleConfiguration

(Alias of []k8s.io/component-base/logs/api/v1.VModuleItem)

Appears in:

VModuleConfiguration is a collection of individual file names or patterns and the corresponding verbosity threshold.

VerbosityLevel

(Alias of uint32)

Appears in:

VerbosityLevel represents a klog or logr verbosity threshold.

CredentialProviderConfig

CredentialProviderConfig is the configuration containing information about each exec credential provider. Kubelet reads this configuration from disk and enables each provider as specified by the CredentialProvider type.

FieldDescription
apiVersion
string
kubelet.config.k8s.io/v1beta1
kind
string
CredentialProviderConfig
providers [Required]
[]CredentialProvider

providers is a list of credential provider plugins that will be enabled by the kubelet. Multiple providers may match against a single image, in which case credentials from all providers will be returned to the kubelet. If multiple providers are called for a single image, the results are combined. If providers return overlapping auth keys, the value from the provider earlier in this list is used.

KubeletConfiguration

KubeletConfiguration contains the configuration for the Kubelet

FieldDescription
apiVersion
string
kubelet.config.k8s.io/v1beta1
kind
string
KubeletConfiguration
enableServer [Required]
bool

enableServer enables Kubelet's secured server. Note: Kubelet's insecure port is controlled by the readOnlyPort option. Default: true

staticPodPath
string

staticPodPath is the path to the directory containing local (static) pods to run, or the path to a single static pod file. Default: ""

podLogsDir
string

podLogsDir is a custom root directory path kubelet will use to place pod's log files. Default: "/var/log/pods/" Note: it is not recommended to use the temp folder as a log directory as it may cause unexpected behavior in many places.

syncFrequency
meta/v1.Duration

syncFrequency is the max period between synchronizing running containers and config. Default: "1m"

fileCheckFrequency
meta/v1.Duration

fileCheckFrequency is the duration between checking config files for new data. Default: "20s"

httpCheckFrequency
meta/v1.Duration

httpCheckFrequency is the duration between checking http for new data. Default: "20s"

staticPodURL
string

staticPodURL is the URL for accessing static pods to run. Default: ""

staticPodURLHeader
map[string][]string

staticPodURLHeader is a map of slices with HTTP headers to use when accessing the podURL. Default: nil

address
string

address is the IP address for the Kubelet to serve on (set to 0.0.0.0 for all interfaces). Default: "0.0.0.0"

port
int32

port is the port for the Kubelet to serve on. The port number must be between 1 and 65535, inclusive. Default: 10250

readOnlyPort
int32

readOnlyPort is the read-only port for the Kubelet to serve on with no authentication/authorization. The port number must be between 1 and 65535, inclusive. Setting this field to 0 disables the read-only service. Default: 0 (disabled)

tlsCertFile
string

tlsCertFile is the file containing x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If tlsCertFile and tlsPrivateKeyFile are not provided, a self-signed certificate and key are generated for the public address and saved to the directory passed to the Kubelet's --cert-dir flag. Default: ""

tlsPrivateKeyFile
string

tlsPrivateKeyFile is the file containing x509 private key matching tlsCertFile. Default: ""

tlsCipherSuites
[]string

tlsCipherSuites is the list of allowed cipher suites for the server. Note that TLS 1.3 ciphersuites are not configurable. Values are from tls package constants (https://golang.org/pkg/crypto/tls/#pkg-constants). Default: nil

tlsMinVersion
string

tlsMinVersion is the minimum TLS version supported. Values are from tls package constants (https://golang.org/pkg/crypto/tls/#pkg-constants). Default: ""

rotateCertificates
bool

rotateCertificates enables client certificate rotation. The Kubelet will request a new certificate from the certificates.k8s.io API. This requires an approver to approve the certificate signing requests. Default: false

serverTLSBootstrap
bool

serverTLSBootstrap enables server certificate bootstrap. Instead of self signing a serving certificate, the Kubelet will request a certificate from the 'certificates.k8s.io' API. This requires an approver to approve the certificate signing requests (CSR). The RotateKubeletServerCertificate feature must be enabled when setting this field. Default: false

authentication
KubeletAuthentication

authentication specifies how requests to the Kubelet's server are authenticated. Defaults: anonymous: enabled: false webhook: enabled: true cacheTTL: "2m"

authorization
KubeletAuthorization

authorization specifies how requests to the Kubelet's server are authorized. Defaults: mode: Webhook webhook: cacheAuthorizedTTL: "5m" cacheUnauthorizedTTL: "30s"

registryPullQPS
int32

registryPullQPS is the limit of registry pulls per second. The value must not be a negative number. Setting it to 0 means no limit. Default: 5

registryBurst
int32

registryBurst is the maximum size of bursty pulls, temporarily allows pulls to burst to this number, while still not exceeding registryPullQPS. The value must not be a negative number. Only used if registryPullQPS is greater than 0. Default: 10

eventRecordQPS
int32

eventRecordQPS is the maximum event creations per second. If 0, there is no limit enforced. The value cannot be a negative number. Default: 50

eventBurst
int32

eventBurst is the maximum size of a burst of event creations, temporarily allows event creations to burst to this number, while still not exceeding eventRecordQPS. This field canot be a negative number and it is only used when eventRecordQPS > 0. Default: 100

enableDebuggingHandlers
bool

enableDebuggingHandlers enables server endpoints for log access and local running of containers and commands, including the exec, attach, logs, and portforward features. Default: true

enableContentionProfiling
bool

enableContentionProfiling enables block profiling, if enableDebuggingHandlers is true. Default: false

healthzPort
int32

healthzPort is the port of the localhost healthz endpoint (set to 0 to disable). A valid number is between 1 and 65535. Default: 10248

healthzBindAddress
string

healthzBindAddress is the IP address for the healthz server to serve on. Default: "127.0.0.1"

oomScoreAdj
int32

oomScoreAdj is The oom-score-adj value for kubelet process. Values must be within the range [-1000, 1000]. Default: -999

clusterDomain
string

clusterDomain is the DNS domain for this cluster. If set, kubelet will configure all containers to search this domain in addition to the host's search domains. Default: ""

clusterDNS
[]string

clusterDNS is a list of IP addresses for the cluster DNS server. If set, kubelet will configure all containers to use this for DNS resolution instead of the host's DNS servers. Default: nil

streamingConnectionIdleTimeout
meta/v1.Duration

streamingConnectionIdleTimeout is the maximum time a streaming connection can be idle before the connection is automatically closed. Default: "4h"

nodeStatusUpdateFrequency
meta/v1.Duration

nodeStatusUpdateFrequency is the frequency that kubelet computes node status. If node lease feature is not enabled, it is also the frequency that kubelet posts node status to master. Note: When node lease feature is not enabled, be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller. Default: "10s"

nodeStatusReportFrequency
meta/v1.Duration

nodeStatusReportFrequency is the frequency that kubelet posts node status to master if node status does not change. Kubelet will ignore this frequency and post node status immediately if any change is detected. It is only used when node lease feature is enabled. nodeStatusReportFrequency's default value is 5m. But if nodeStatusUpdateFrequency is set explicitly, nodeStatusReportFrequency's default value will be set to nodeStatusUpdateFrequency for backward compatibility. Default: "5m"

nodeLeaseDurationSeconds
int32

nodeLeaseDurationSeconds is the duration the Kubelet will set on its corresponding Lease. NodeLease provides an indicator of node health by having the Kubelet create and periodically renew a lease, named after the node, in the kube-node-lease namespace. If the lease expires, the node can be considered unhealthy. The lease is currently renewed every 10s, per KEP-0009. In the future, the lease renewal interval may be set based on the lease duration. The field value must be greater than 0. Default: 40

imageMinimumGCAge
meta/v1.Duration

imageMinimumGCAge is the minimum age for an unused image before it is garbage collected. Default: "2m"

imageMaximumGCAge
meta/v1.Duration

imageMaximumGCAge is the maximum age an image can be unused before it is garbage collected. The default of this field is "0s", which disables this field--meaning images won't be garbage collected based on being unused for too long. Default: "0s" (disabled)

imageGCHighThresholdPercent
int32

imageGCHighThresholdPercent is the percent of disk usage after which image garbage collection is always run. The percent is calculated by dividing this field value by 100, so this field must be between 0 and 100, inclusive. When specified, the value must be greater than imageGCLowThresholdPercent. Default: 85

imageGCLowThresholdPercent
int32

imageGCLowThresholdPercent is the percent of disk usage before which image garbage collection is never run. Lowest disk usage to garbage collect to. The percent is calculated by dividing this field value by 100, so the field value must be between 0 and 100, inclusive. When specified, the value must be less than imageGCHighThresholdPercent. Default: 80

volumeStatsAggPeriod
meta/v1.Duration

volumeStatsAggPeriod is the frequency for calculating and caching volume disk usage for all pods. Default: "1m"

kubeletCgroups
string

kubeletCgroups is the absolute name of cgroups to isolate the kubelet in Default: ""

systemCgroups
string

systemCgroups is absolute name of cgroups in which to place all non-kernel processes that are not already in a container. Empty for no container. Rolling back the flag requires a reboot. The cgroupRoot must be specified if this field is not empty. Default: ""

cgroupRoot
string

cgroupRoot is the root cgroup to use for pods. This is handled by the container runtime on a best effort basis.

cgroupsPerQOS
bool

cgroupsPerQOS enable QoS based CGroup hierarchy: top level CGroups for QoS classes and all Burstable and BestEffort Pods are brought up under their specific top level QoS CGroup. Default: true

cgroupDriver
string

cgroupDriver is the driver kubelet uses to manipulate CGroups on the host (cgroupfs or systemd). Default: "cgroupfs"

cpuManagerPolicy
string

cpuManagerPolicy is the name of the policy to use. Requires the CPUManager feature gate to be enabled. Default: "None"

cpuManagerPolicyOptions
map[string]string

cpuManagerPolicyOptions is a set of key=value which allows to set extra options to fine tune the behaviour of the cpu manager policies. Requires both the "CPUManager" and "CPUManagerPolicyOptions" feature gates to be enabled. Default: nil

cpuManagerReconcilePeriod
meta/v1.Duration

cpuManagerReconcilePeriod is the reconciliation period for the CPU Manager. Requires the CPUManager feature gate to be enabled. Default: "10s"

memoryManagerPolicy
string

memoryManagerPolicy is the name of the policy to use by memory manager. Requires the MemoryManager feature gate to be enabled. Default: "none"

topologyManagerPolicy
string

topologyManagerPolicy is the name of the topology manager policy to use. Valid values include:

  • restricted: kubelet only allows pods with optimal NUMA node alignment for requested resources;
  • best-effort: kubelet will favor pods with NUMA alignment of CPU and device resources;
  • none: kubelet has no knowledge of NUMA alignment of a pod's CPU and device resources.
  • single-numa-node: kubelet only allows pods with a single NUMA alignment of CPU and device resources.

Default: "none"

topologyManagerScope
string

topologyManagerScope represents the scope of topology hint generation that topology manager requests and hint providers generate. Valid values include:

  • container: topology policy is applied on a per-container basis.
  • pod: topology policy is applied on a per-pod basis.

Default: "container"

topologyManagerPolicyOptions
map[string]string

TopologyManagerPolicyOptions is a set of key=value which allows to set extra options to fine tune the behaviour of the topology manager policies. Requires both the "TopologyManager" and "TopologyManagerPolicyOptions" feature gates to be enabled. Default: nil

qosReserved
map[string]string

qosReserved is a set of resource name to percentage pairs that specify the minimum percentage of a resource reserved for exclusive use by the guaranteed QoS tier. Currently supported resources: "memory" Requires the QOSReserved feature gate to be enabled. Default: nil

runtimeRequestTimeout
meta/v1.Duration

runtimeRequestTimeout is the timeout for all runtime requests except long running requests - pull, logs, exec and attach. Default: "2m"

hairpinMode
string

hairpinMode specifies how the Kubelet should configure the container bridge for hairpin packets. Setting this flag allows endpoints in a Service to loadbalance back to themselves if they should try to access their own Service. Values:

  • "promiscuous-bridge": make the container bridge promiscuous.
  • "hairpin-veth": set the hairpin flag on container veth interfaces.
  • "none": do nothing.

Generally, one must set --hairpin-mode=hairpin-veth to achieve hairpin NAT, because promiscuous-bridge assumes the existence of a container bridge named cbr0. Default: "promiscuous-bridge"

maxPods
int32

maxPods is the maximum number of Pods that can run on this Kubelet. The value must be a non-negative integer. Default: 110

podCIDR
string

podCIDR is the CIDR to use for pod IP addresses, only used in standalone mode. In cluster mode, this is obtained from the control plane. Default: ""

podPidsLimit
int64

podPidsLimit is the maximum number of PIDs in any pod. Default: -1

resolvConf
string

resolvConf is the resolver configuration file used as the basis for the container DNS resolution configuration. If set to the empty string, will override the default and effectively disable DNS lookups. Default: "/etc/resolv.conf"

runOnce
bool

runOnce causes the Kubelet to check the API server once for pods, run those in addition to the pods specified by static pod files, and exit. Default: false

cpuCFSQuota
bool

cpuCFSQuota enables CPU CFS quota enforcement for containers that specify CPU limits. Default: true

cpuCFSQuotaPeriod
meta/v1.Duration

cpuCFSQuotaPeriod is the CPU CFS quota period value, cpu.cfs_period_us. The value must be between 1 ms and 1 second, inclusive. Requires the CustomCPUCFSQuotaPeriod feature gate to be enabled. Default: "100ms"

nodeStatusMaxImages
int32

nodeStatusMaxImages caps the number of images reported in Node.status.images. The value must be greater than -2. Note: If -1 is specified, no cap will be applied. If 0 is specified, no image is returned. Default: 50

maxOpenFiles
int64

maxOpenFiles is Number of files that can be opened by Kubelet process. The value must be a non-negative number. Default: 1000000

contentType
string

contentType is contentType of requests sent to apiserver. Default: "application/vnd.kubernetes.protobuf"

kubeAPIQPS
int32

kubeAPIQPS is the QPS to use while talking with kubernetes apiserver. Default: 50

kubeAPIBurst
int32

kubeAPIBurst is the burst to allow while talking with kubernetes API server. This field cannot be a negative number. Default: 100

serializeImagePulls
bool

serializeImagePulls when enabled, tells the Kubelet to pull images one at a time. We recommend not changing the default value on nodes that run docker daemon with version < 1.9 or an Aufs storage backend. Issue #10959 has more details. Default: true

maxParallelImagePulls
int32

MaxParallelImagePulls sets the maximum number of image pulls in parallel. This field cannot be set if SerializeImagePulls is true. Setting it to nil means no limit. Default: nil

evictionHard
map[string]string

evictionHard is a map of signal names to quantities that defines hard eviction thresholds. For example: {"memory.available": "300Mi"}. To explicitly disable, pass a 0% or 100% threshold on an arbitrary resource. Default: memory.available: "100Mi" nodefs.available: "10%" nodefs.inodesFree: "5%" imagefs.available: "15%"

evictionSoft
map[string]string

evictionSoft is a map of signal names to quantities that defines soft eviction thresholds. For example: {"memory.available": "300Mi"}. Default: nil

evictionSoftGracePeriod
map[string]string

evictionSoftGracePeriod is a map of signal names to quantities that defines grace periods for each soft eviction signal. For example: {"memory.available": "30s"}. Default: nil

evictionPressureTransitionPeriod
meta/v1.Duration

evictionPressureTransitionPeriod is the duration for which the kubelet has to wait before transitioning out of an eviction pressure condition. Default: "5m"

evictionMaxPodGracePeriod
int32

evictionMaxPodGracePeriod is the maximum allowed grace period (in seconds) to use when terminating pods in response to a soft eviction threshold being met. This value effectively caps the Pod's terminationGracePeriodSeconds value during soft evictions. Note: Due to issue #64530, the behavior has a bug where this value currently just overrides the grace period during soft eviction, which can increase the grace period from what is set on the Pod. This bug will be fixed in a future release. Default: 0

evictionMinimumReclaim
map[string]string

evictionMinimumReclaim is a map of signal names to quantities that defines minimum reclaims, which describe the minimum amount of a given resource the kubelet will reclaim when performing a pod eviction while that resource is under pressure. For example: {"imagefs.available": "2Gi"}. Default: nil

podsPerCore
int32

podsPerCore is the maximum number of pods per core. Cannot exceed maxPods. The value must be a non-negative integer. If 0, there is no limit on the number of Pods. Default: 0

enableControllerAttachDetach
bool

enableControllerAttachDetach enables the Attach/Detach controller to manage attachment/detachment of volumes scheduled to this node, and disables kubelet from executing any attach/detach operations. Note: attaching/detaching CSI volumes is not supported by the kubelet, so this option needs to be true for that use case. Default: true

protectKernelDefaults
bool

protectKernelDefaults, if true, causes the Kubelet to error if kernel flags are not as it expects. Otherwise the Kubelet will attempt to modify kernel flags to match its expectation. Default: false

makeIPTablesUtilChains
bool

makeIPTablesUtilChains, if true, causes the Kubelet to create the KUBE-IPTABLES-HINT chain in iptables as a hint to other components about the configuration of iptables on the system. Default: true

iptablesMasqueradeBit
int32

iptablesMasqueradeBit formerly controlled the creation of the KUBE-MARK-MASQ chain. Deprecated: no longer has any effect. Default: 14

iptablesDropBit
int32

iptablesDropBit formerly controlled the creation of the KUBE-MARK-DROP chain. Deprecated: no longer has any effect. Default: 15

featureGates
map[string]bool

featureGates is a map of feature names to bools that enable or disable experimental features. This field modifies piecemeal the built-in default values from "k8s.io/kubernetes/pkg/features/kube_features.go". Default: nil

failSwapOn
bool

failSwapOn tells the Kubelet to fail to start if swap is enabled on the node. Default: true

memorySwap
MemorySwapConfiguration

memorySwap configures swap memory available to container workloads.

containerLogMaxSize
string

containerLogMaxSize is a quantity defining the maximum size of the container log file before it is rotated. For example: "5Mi" or "256Ki". Default: "10Mi"

containerLogMaxFiles
int32

containerLogMaxFiles specifies the maximum number of container log files that can be present for a container. Default: 5

containerLogMaxWorkers
int32

ContainerLogMaxWorkers specifies the maximum number of concurrent workers to spawn for performing the log rotate operations. Set this count to 1 for disabling the concurrent log rotation workflows Default: 1

containerLogMonitorInterval
meta/v1.Duration

ContainerLogMonitorInterval specifies the duration at which the container logs are monitored for performing the log rotate operation. This defaults to 10 * time.Seconds. But can be customized to a smaller value based on the log generation rate and the size required to be rotated against Default: 10s

configMapAndSecretChangeDetectionStrategy
ResourceChangeDetectionStrategy

configMapAndSecretChangeDetectionStrategy is a mode in which ConfigMap and Secret managers are running. Valid values include:

  • Get: kubelet fetches necessary objects directly from the API server;
  • Cache: kubelet uses TTL cache for object fetched from the API server;
  • Watch: kubelet uses watches to observe changes to objects that are in its interest.

Default: "Watch"

systemReserved
map[string]string

systemReserved is a set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=150G) pairs that describe resources reserved for non-kubernetes components. Currently only cpu and memory are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. Default: nil

kubeReserved
map[string]string

kubeReserved is a set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=150G) pairs that describe resources reserved for kubernetes system components. Currently cpu, memory and local storage for root file system are supported. See https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ for more details. Default: nil

reservedSystemCPUs [Required]
string

The reservedSystemCPUs option specifies the CPU list reserved for the host level system threads and kubernetes related threads. This provide a "static" CPU list rather than the "dynamic" list by systemReserved and kubeReserved. This option does not support systemReservedCgroup or kubeReservedCgroup.

showHiddenMetricsForVersion
string

showHiddenMetricsForVersion is the previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <major>.<minor>, e.g.: 1.16. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that. Default: ""

systemReservedCgroup
string

systemReservedCgroup helps the kubelet identify absolute name of top level CGroup used to enforce systemReserved compute resource reservation for OS system daemons. Refer to Node Allocatable doc for more information. Default: ""

kubeReservedCgroup
string

kubeReservedCgroup helps the kubelet identify absolute name of top level CGroup used to enforce KubeReserved compute resource reservation for Kubernetes node system daemons. Refer to Node Allocatable doc for more information. Default: ""

enforceNodeAllocatable
[]string

This flag specifies the various Node Allocatable enforcements that Kubelet needs to perform. This flag accepts a list of options. Acceptable options are none, pods, system-reserved and kube-reserved. If none is specified, no other options may be specified. When system-reserved is in the list, systemReservedCgroup must be specified. When kube-reserved is in the list, kubeReservedCgroup must be specified. This field is supported only when cgroupsPerQOS is set to true. Refer to Node Allocatable for more information. Default: ["pods"]

allowedUnsafeSysctls
[]string

A comma separated whitelist of unsafe sysctls or sysctl patterns (ending in *). Unsafe sysctl groups are kernel.shm*, kernel.msg*, kernel.sem, fs.mqueue.*, and net.*. For example: "kernel.msg*,net.ipv4.route.min_pmtu" Default: []

volumePluginDir
string

volumePluginDir is the full path of the directory in which to search for additional third party volume plugins. Default: "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/"

providerID
string

providerID, if set, sets the unique ID of the instance that an external provider (i.e. cloudprovider) can use to identify a specific node. Default: ""

kernelMemcgNotification
bool

kernelMemcgNotification, if set, instructs the kubelet to integrate with the kernel memcg notification for determining if memory eviction thresholds are exceeded rather than polling. Default: false

logging [Required]
LoggingConfiguration

logging specifies the options of logging. Refer to Logs Options for more information. Default: Format: text

enableSystemLogHandler
bool

enableSystemLogHandler enables system logs via web interface host:port/logs/ Default: true

enableSystemLogQuery
bool

enableSystemLogQuery enables the node log query feature on the /logs endpoint. EnableSystemLogHandler has to be enabled in addition for this feature to work. Default: false

shutdownGracePeriod
meta/v1.Duration

shutdownGracePeriod specifies the total duration that the node should delay the shutdown and total grace period for pod termination during a node shutdown. Default: "0s"

shutdownGracePeriodCriticalPods
meta/v1.Duration

shutdownGracePeriodCriticalPods specifies the duration used to terminate critical pods during a node shutdown. This should be less than shutdownGracePeriod. For example, if shutdownGracePeriod=30s, and shutdownGracePeriodCriticalPods=10s, during a node shutdown the first 20 seconds would be reserved for gracefully terminating normal pods, and the last 10 seconds would be reserved for terminating critical pods. Default: "0s"

shutdownGracePeriodByPodPriority
[]ShutdownGracePeriodByPodPriority

shutdownGracePeriodByPodPriority specifies the shutdown grace period for Pods based on their associated priority class value. When a shutdown request is received, the Kubelet will initiate shutdown on all pods running on the node with a grace period that depends on the priority of the pod, and then wait for all pods to exit. Each entry in the array represents the graceful shutdown time a pod with a priority class value that lies in the range of that value and the next higher entry in the list when the node is shutting down. For example, to allow critical pods 10s to shutdown, priority>=10000 pods 20s to shutdown, and all remaining pods 30s to shutdown.

shutdownGracePeriodByPodPriority:

  • priority: 2000000000 shutdownGracePeriodSeconds: 10
  • priority: 10000 shutdownGracePeriodSeconds: 20
  • priority: 0 shutdownGracePeriodSeconds: 30

The time the Kubelet will wait before exiting will at most be the maximum of all shutdownGracePeriodSeconds for each priority class range represented on the node. When all pods have exited or reached their grace periods, the Kubelet will release the shutdown inhibit lock. Requires the GracefulNodeShutdown feature gate to be enabled. This configuration must be empty if either ShutdownGracePeriod or ShutdownGracePeriodCriticalPods is set. Default: nil

reservedMemory
[]MemoryReservation

reservedMemory specifies a comma-separated list of memory reservations for NUMA nodes. The parameter makes sense only in the context of the memory manager feature. The memory manager will not allocate reserved memory for container workloads. For example, if you have a NUMA0 with 10Gi of memory and the reservedMemory was specified to reserve 1Gi of memory at NUMA0, the memory manager will assume that only 9Gi is available for allocation. You can specify a different amount of NUMA node and memory types. You can omit this parameter at all, but you should be aware that the amount of reserved memory from all NUMA nodes should be equal to the amount of memory specified by the node allocatable. If at least one node allocatable parameter has a non-zero value, you will need to specify at least one NUMA node. Also, avoid specifying:

  1. Duplicates, the same NUMA node, and memory type, but with a different value.
  2. zero limits for any memory type.
  3. NUMAs nodes IDs that do not exist under the machine.
  4. memory types except for memory and hugepages-<size>

Default: nil

enableProfilingHandler
bool

enableProfilingHandler enables profiling via web interface host:port/debug/pprof/ Default: true

enableDebugFlagsHandler
bool

enableDebugFlagsHandler enables flags endpoint via web interface host:port/debug/flags/v Default: true

seccompDefault
bool

SeccompDefault enables the use of RuntimeDefault as the default seccomp profile for all workloads. Default: false

memoryThrottlingFactor
float64

MemoryThrottlingFactor specifies the factor multiplied by the memory limit or node allocatable memory when setting the cgroupv2 memory.high value to enforce MemoryQoS. Decreasing this factor will set lower high limit for container cgroups and put heavier reclaim pressure while increasing will put less reclaim pressure. See https://kep.k8s.io/2570 for more details. Default: 0.9

registerWithTaints
[]core/v1.Taint

registerWithTaints are an array of taints to add to a node object when the kubelet registers itself. This only takes effect when registerNode is true and upon the initial registration of the node. Default: nil

registerNode
bool

registerNode enables automatic registration with the apiserver. Default: true

tracing
TracingConfiguration

Tracing specifies the versioned configuration for OpenTelemetry tracing clients. See https://kep.k8s.io/2832 for more details. Default: nil

localStorageCapacityIsolation
bool

LocalStorageCapacityIsolation enables local ephemeral storage isolation feature. The default setting is true. This feature allows users to set request/limit for container's ephemeral storage and manage it in a similar way as cpu and memory. It also allows setting sizeLimit for emptyDir volume, which will trigger pod eviction if disk usage from the volume exceeds the limit. This feature depends on the capability of detecting correct root file system disk usage. For certain systems, such as kind rootless, if this capability cannot be supported, the feature LocalStorageCapacityIsolation should be disabled. Once disabled, user should not set request/limit for container's ephemeral storage, or sizeLimit for emptyDir. Default: true

containerRuntimeEndpoint [Required]
string

ContainerRuntimeEndpoint is the endpoint of container runtime. Unix Domain Sockets are supported on Linux, while npipe and tcp endpoints are supported on Windows. Examples:'unix:///path/to/runtime.sock', 'npipe:////./pipe/runtime'

imageServiceEndpoint
string

ImageServiceEndpoint is the endpoint of container image service. Unix Domain Socket are supported on Linux, while npipe and tcp endpoints are supported on Windows. Examples:'unix:///path/to/runtime.sock', 'npipe:////./pipe/runtime'. If not specified, the value in containerRuntimeEndpoint is used.

SerializedNodeConfigSource

SerializedNodeConfigSource allows us to serialize v1.NodeConfigSource. This type is used internally by the Kubelet for tracking checkpointed dynamic configs. It exists in the kubeletconfig API group because it is classified as a versioned input to the Kubelet.

FieldDescription
apiVersion
string
kubelet.config.k8s.io/v1beta1
kind
string
SerializedNodeConfigSource
source
core/v1.NodeConfigSource

source is the source that we are serializing.

CredentialProvider

Appears in:

CredentialProvider represents an exec plugin to be invoked by the kubelet. The plugin is only invoked when an image being pulled matches the images handled by the plugin (see matchImages).

FieldDescription
name [Required]
string

name is the required name of the credential provider. It must match the name of the provider executable as seen by the kubelet. The executable must be in the kubelet's bin directory (set by the --image-credential-provider-bin-dir flag).

matchImages [Required]
[]string

matchImages is a required list of strings used to match against images in order to determine if this provider should be invoked. If one of the strings matches the requested image from the kubelet, the plugin will be invoked and given a chance to provide credentials. Images are expected to contain the registry domain and URL path.

Each entry in matchImages is a pattern which can optionally contain a port and a path. Globs can be used in the domain, but not in the port or the path. Globs are supported as subdomains like '*.k8s.io' or 'k8s.*.io', and top-level-domains such as 'k8s.*'. Matching partial subdomains like 'app*.k8s.io' is also supported. Each glob can only match a single subdomain segment, so '*.io' does not match '*.k8s.io'.

A match exists between an image and a matchImage when all of the below are true:

  • Both contain the same number of domain parts and each part matches.
  • The URL path of an imageMatch must be a prefix of the target image URL path.
  • If the imageMatch contains a port, then the port must match in the image as well.

Example values of matchImages:

  • 123456789.dkr.ecr.us-east-1.amazonaws.com
  • *.azurecr.io
  • gcr.io
  • *.*.registry.io
  • registry.io:8080/path
defaultCacheDuration [Required]
meta/v1.Duration

defaultCacheDuration is the default duration the plugin will cache credentials in-memory if a cache duration is not provided in the plugin response. This field is required.

apiVersion [Required]
string

Required input version of the exec CredentialProviderRequest. The returned CredentialProviderResponse MUST use the same encoding version as the input. Current supported values are:

  • credentialprovider.kubelet.k8s.io/v1beta1
args
[]string

Arguments to pass to the command when executing it.

env
[]ExecEnvVar

Env defines additional environment variables to expose to the process. These are unioned with the host's environment, as well as variables client-go uses to pass argument to the plugin.

ExecEnvVar

Appears in:

ExecEnvVar is used for setting environment variables when executing an exec-based credential plugin.

FieldDescription
name [Required]
string
No description provided.
value [Required]
string
No description provided.

KubeletAnonymousAuthentication

Appears in:

FieldDescription
enabled
bool

enabled allows anonymous requests to the kubelet server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of system:anonymous, and a group name of system:unauthenticated.

KubeletAuthentication

Appears in:

FieldDescription
x509
KubeletX509Authentication

x509 contains settings related to x509 client certificate authentication.

webhook
KubeletWebhookAuthentication

webhook contains settings related to webhook bearer token authentication.

anonymous
KubeletAnonymousAuthentication

anonymous contains settings related to anonymous authentication.

KubeletAuthorization

Appears in:

FieldDescription
mode
KubeletAuthorizationMode

mode is the authorization mode to apply to requests to the kubelet server. Valid values are AlwaysAllow and Webhook. Webhook mode uses the SubjectAccessReview API to determine authorization.

webhook
KubeletWebhookAuthorization

webhook contains settings related to Webhook authorization.

KubeletAuthorizationMode

(Alias of string)

Appears in:

KubeletWebhookAuthentication

Appears in:

FieldDescription
enabled
bool

enabled allows bearer token authentication backed by the tokenreviews.authentication.k8s.io API.

cacheTTL
meta/v1.Duration

cacheTTL enables caching of authentication results

KubeletWebhookAuthorization

Appears in:

FieldDescription
cacheAuthorizedTTL
meta/v1.Duration

cacheAuthorizedTTL is the duration to cache 'authorized' responses from the webhook authorizer.

cacheUnauthorizedTTL
meta/v1.Duration

cacheUnauthorizedTTL is the duration to cache 'unauthorized' responses from the webhook authorizer.

KubeletX509Authentication

Appears in:

FieldDescription
clientCAFile
string

clientCAFile is the path to a PEM-encoded certificate bundle. If set, any request presenting a client certificate signed by one of the authorities in the bundle is authenticated with a username corresponding to the CommonName, and groups corresponding to the Organization in the client certificate.

MemoryReservation

Appears in:

MemoryReservation specifies the memory reservation of different types for each NUMA node

FieldDescription
numaNode [Required]
int32
No description provided.
limits [Required]
core/v1.ResourceList
No description provided.

MemorySwapConfiguration

Appears in:

FieldDescription
swapBehavior
string

swapBehavior configures swap memory available to container workloads. May be one of "", "NoSwap": workloads can not use swap, default option. "LimitedSwap": workload swap usage is limited. The swap limit is proportionate to the container's memory request.

ResourceChangeDetectionStrategy

(Alias of string)

Appears in:

ResourceChangeDetectionStrategy denotes a mode in which internal managers (secret, configmap) are discovering object changes.

ShutdownGracePeriodByPodPriority

Appears in:

ShutdownGracePeriodByPodPriority specifies the shutdown grace period for Pods based on their associated priority class value

FieldDescription
priority [Required]
int32

priority is the priority value associated with the shutdown grace period

shutdownGracePeriodSeconds [Required]
int64

shutdownGracePeriodSeconds is the shutdown grace period in seconds

19 - Kubelet CredentialProvider (v1)

Resource Types

CredentialProviderRequest

CredentialProviderRequest includes the image that the kubelet requires authentication for. Kubelet will pass this request object to the plugin via stdin. In general, plugins should prefer responding with the same apiVersion they were sent.

FieldDescription
apiVersion
string
credentialprovider.kubelet.k8s.io/v1
kind
string
CredentialProviderRequest
image [Required]
string

image is the container image that is being pulled as part of the credential provider plugin request. Plugins may optionally parse the image to extract any information required to fetch credentials.

CredentialProviderResponse

CredentialProviderResponse holds credentials that the kubelet should use for the specified image provided in the original request. Kubelet will read the response from the plugin via stdout. This response should be set to the same apiVersion as CredentialProviderRequest.

FieldDescription
apiVersion
string
credentialprovider.kubelet.k8s.io/v1
kind
string
CredentialProviderResponse
cacheKeyType [Required]
PluginCacheKeyType

cacheKeyType indiciates the type of caching key to use based on the image provided in the request. There are three valid values for the cache key type: Image, Registry, and Global. If an invalid value is specified, the response will NOT be used by the kubelet.

cacheDuration
meta/v1.Duration

cacheDuration indicates the duration the provided credentials should be cached for. The kubelet will use this field to set the in-memory cache duration for credentials in the AuthConfig. If null, the kubelet will use defaultCacheDuration provided in CredentialProviderConfig. If set to 0, the kubelet will not cache the provided AuthConfig.

auth
map[string]AuthConfig

auth is a map containing authentication information passed into the kubelet. Each key is a match image string (more on this below). The corresponding authConfig value should be valid for all images that match against this key. A plugin should set this field to null if no valid credentials can be returned for the requested image.

Each key in the map is a pattern which can optionally contain a port and a path. Globs can be used in the domain, but not in the port or the path. Globs are supported as subdomains like '*.k8s.io' or 'k8s.*.io', and top-level-domains such as 'k8s.*'. Matching partial subdomains like 'app*.k8s.io' is also supported. Each glob can only match a single subdomain segment, so '*.io' does not match '*.k8s.io'.

The kubelet will match images against the key when all of the below are true:

  • Both contain the same number of domain parts and each part matches.
  • The URL path of an imageMatch must be a prefix of the target image URL path.
  • If the imageMatch contains a port, then the port must match in the image as well.

When multiple keys are returned, the kubelet will traverse all keys in reverse order so that:

  • longer keys come before shorter keys with the same prefix
  • non-wildcard keys come before wildcard keys with the same prefix.

For any given match, the kubelet will attempt an image pull with the provided credentials, stopping after the first successfully authenticated pull.

Example keys:

  • 123456789.dkr.ecr.us-east-1.amazonaws.com
  • *.azurecr.io
  • gcr.io
  • *.*.registry.io
  • registry.io:8080/path

AuthConfig

Appears in:

AuthConfig contains authentication information for a container registry. Only username/password based authentication is supported today, but more authentication mechanisms may be added in the future.

FieldDescription
username [Required]
string

username is the username used for authenticating to the container registry An empty username is valid.

password [Required]
string

password is the password used for authenticating to the container registry An empty password is valid.

PluginCacheKeyType

(Alias of string)

Appears in:

20 - WebhookAdmission Configuration (v1)

Package v1 is the v1 version of the API.

Resource Types

WebhookAdmission

WebhookAdmission provides configuration for the webhook admission controller.

FieldDescription
apiVersion
string
apiserver.config.k8s.io/v1
kind
string
WebhookAdmission
kubeConfigFile [Required]
string

KubeConfigFile is the path to the kubeconfig file.