This document describes how to configure and use kernel parameters within a Kubernetes cluster using the sysctl interface.
You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using Minikube, or you can use one of these Kubernetes playgrounds:
To check the version, enter
In Linux, the sysctl interface allows an administrator to modify kernel
parameters at runtime. Parameters are available via the
process file system. The parameters cover various subsystems such as:
To get a list of all parameters, you can run
$ sudo sysctl -a
Sysctls are grouped into safe and unsafe sysctls. In addition to proper namespacing a safe sysctl must be properly isolated between pods on the same node. This means that setting a safe sysctl for one pod
By far, most of the namespaced sysctls are not necessarily considered safe. The following sysctls are supported in the safe set:
Note: The example
net.ipv4.tcp_syncookiesis not namespaced on Linux kernel version 4.4 or lower.
This list will be extended in future Kubernetes versions when the kubelet supports better isolation mechanisms.
All safe sysctls are enabled by default.
All unsafe sysctls are disabled by default and must be allowed manually by the cluster admin on a per-node basis. Pods with disabled unsafe sysctls will be scheduled, but will fail to launch.
With the warning above in mind, the cluster admin can allow certain unsafe sysctls for very special situations like e.g. high-performance or real-time application tuning. Unsafe sysctls are enabled on a node-by-node basis with a flag of the kubelet, e.g.:
$ kubelet --allowed-unsafe-sysctls \ 'kernel.msg*,net.ipv4.route.min_pmtu' ...
For minikube, this can be done via the
$ minikube start --extra-config="kubelet.AllowedUnsafeSysctls=kernel.msg*,net.ipv4.route.min_pmtu"...
Only namespaced sysctls can be enabled this way.
A number of sysctls are namespaced in today’s Linux kernels. This means that they can be set independently for each pod on a node. Only namespaced sysctls are configurable via the pod securityContext within Kubernetes.
The following sysctls are known to be namespaced. This list could change in future versions of the Linux kernel.
Sysctls with no namespace are called node-level sysctls. If you need to set them, you must manually configure them on each node’s operating system, or by using a DaemonSet with privileged containers.
Use the pod securityContext to configure namespaced sysctls. The securityContext applies to all containers in the same pod.
This example uses the pod securityContext to set a safe sysctl
kernel.shm_rmid_forced and two unsafe sysctls
kernel.msgmax There is no distinction between safe and unsafe sysctls in
Warning: Only modify sysctl parameters after you understand their effects, to avoid destabilizing your operating system.
apiVersion: v1 kind: Pod metadata: name: sysctl-example spec: securityContext: sysctls: - name: kernel.shm_rmid_forced value: "0" - name: net.ipv4.route.min_pmtu value: "552" - name: kernel.msgmax value: "65536" ...
Warning: Due to their nature of being unsafe, the use of unsafe sysctls is at-your-own-risk and can lead to severe problems like wrong behavior of containers, resource shortage or complete breakage of a node.
It is good practice to consider nodes with special sysctl settings as tainted within a cluster, and only schedule pods onto them which need those sysctl settings. It is suggested to use the Kubernetes taints and toleration feature to implement this.
A pod with the unsafe sysctls will fail to launch on any node which has not enabled those two unsafe sysctls explicitly. As with node-level sysctls it is recommended to use taints and toleration feature or taints on nodes to schedule those pods onto the right nodes.
You can further control which sysctls can be set in pods by specifying lists of
sysctls or sysctl patterns in the
allowedUnsafeSysctls fields of the PodSecurityPolicy. A sysctl pattern ends
* character, such as
* character on its own matches
By default, all safe sysctls are allowed.
allowedUnsafeSysctls are lists of plain sysctl names
or sysctl patterns (which end with
*). The string
* matches all sysctls.
forbiddenSysctls field excludes specific sysctls. You can forbid a
combination of safe and unsafe sysctls in the list. To forbid setting any
* on its own.
If you specify any unsafe sysctl in the
allowedUnsafeSysctls field and it is
not present in the
forbiddenSysctls field, that sysctl can be used in Pods
using this PodSecurityPolicy. To allow all unsafe sysctls in the
PodSecurityPolicy to be set, use
* on its own.
Do not configure these two fields such that there is overlap, meaning that a given sysctl is both allowed and forbidden.
Warning: If you whitelist unsafe sysctls via the
allowedUnsafeSysctlsfield in a PodSecurityPolicy, any pod using such a sysctl will fail to start if the sysctl is not whitelisted via the
--allowed-unsafe-sysctlskubelet flag as well on that node.
This example allows unsafe sysctls prefixed with
kernel.msg to be set and
disallows setting of the
apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: sysctl-psp spec: allowedUnsafeSysctls: - kernel.msg* forbiddenSysctls: - kernel.shm_rmid_forced ...