Windows debugging tips
My Pods are stuck at "Container Creating" or restarting over and over
Ensure that your pause image is compatible with your Windows OS version. See Pause container to see the latest / recommended pause image and/or get more information.Note: If using containerd as your container runtime the pause image is specified in the
plugins.plugins.cri.sandbox_imagefield of the of config.toml configration file.
My pods show status as
Ensure that your Pod is getting scheduled to a compatible Windows Node.
More information on how to specify a compatible node for your Pod can be found in this guide.
My Windows Pods do not have network connectivity
If you are using virtual machines, ensure that MAC spoofing is enabled on all the VM network adapter(s).
My Windows Pods cannot ping external resources
Windows Pods do not have outbound rules programmed for the ICMP protocol. However, TCP/UDP is supported. When trying to demonstrate connectivity to resources outside of the cluster, substitute
ping <IP>with corresponding
If you are still facing problems, most likely your network configuration in cni.conf deserves some extra attention. You can always edit this static file. The configuration update will apply to any new Kubernetes resources.
One of the Kubernetes networking requirements (see Kubernetes model) is for cluster communication to occur without NAT internally. To honor this requirement, there is an ExceptionList for all the communication where you do not want outbound NAT to occur. However, this also means that you need to exclude the external IP you are trying to query from the
ExceptionList. Only then will the traffic originating from your Windows pods be SNAT'ed correctly to receive a response from the outside world. In this regard, your
cni.confshould look as follows:
"ExceptionList": [ "10.244.0.0/16", # Cluster subnet "10.96.0.0/12", # Service subnet "10.127.130.0/24" # Management (host) subnet ]
My Windows node cannot access
Local NodePort access from the node itself fails. This is a known limitation. NodePort access works from other nodes or external clients.
vNICs and HNS endpoints of containers are being deleted
This issue can be caused when the
hostname-overrideparameter is not passed to kube-proxy. To resolve it, users need to pass the hostname to kube-proxy as follows:
My Windows node cannot access my services using the service IP
This is a known limitation of the networking stack on Windows. However, Windows Pods can access the Service IP.
No network adapter is found when starting the kubelet
The Windows networking stack needs a virtual adapter for Kubernetes networking to work. If the following commands return no results (in an admin shell), virtual network creation — a necessary prerequisite for the kubelet to work — has failed:
Get-HnsNetwork | ? Name -ieq "cbr0" Get-NetAdapter | ? Name -Like "vEthernet (Ethernet*"
Often it is worthwhile to modify the InterfaceName parameter of the
start.ps1script, in cases where the host's network adapter isn't "Ethernet". Otherwise, consult the output of the
start-kubelet.ps1script to see if there are errors during virtual network creation.
DNS resolution is not properly working
Check the DNS limitations for Windows in this section.
kubectl port-forwardfails with "unable to do port forwarding: wincat not found"
This was implemented in Kubernetes 1.15 by including
wincat.exein the pause infrastructure container
mcr.microsoft.com/oss/kubernetes/pause:3.6. Be sure to use a supported version of Kubernetes. If you would like to build your own pause infrastructure container be sure to include wincat.
My Kubernetes installation is failing because my Windows Server node is behind a proxy
If you are behind a proxy, the following PowerShell environment variables must be defined:
[Environment]::SetEnvironmentVariable("HTTP_PROXY", "http://proxy.example.com:80/", [EnvironmentVariableTarget]::Machine) [Environment]::SetEnvironmentVariable("HTTPS_PROXY", "http://proxy.example.com:443/", [EnvironmentVariableTarget]::Machine)
With Flannel, my nodes are having issues after rejoining a cluster
Whenever a previously deleted node is being re-joined to the cluster, flannelD tries to assign a new pod subnet to the node. Users should remove the old pod subnet configuration files in the following paths:
Remove-Item C:\k\SourceVip.json Remove-Item C:\k\SourceVipRequest.json
Flanneld is stuck in "Waiting for the Network to be created"
There are numerous reports of this issue; most likely it is a timing issue for when the management IP of the flannel network is set. A workaround is to relaunch
start.ps1or relaunch it manually as follows:
[Environment]::SetEnvironmentVariable("NODE_NAME", "<Windows_Worker_Hostname>") C:\flannel\flanneld.exe --kubeconfig-file=c:\k\config --iface=<Windows_Worker_Node_IP> --ip-masq=1 --kube-subnet-mgr=1
My Windows Pods cannot launch because of missing
This indicates that Flannel didn't launch correctly. You can either try to restart
flanneld.exeor you can copy the files over manually from
/run/flannel/subnet.envon the Kubernetes master to
C:\run\flannel\subnet.envon the Windows worker node and modify the
FLANNEL_SUBNETrow to a different number. For example, if node subnet 10.244.4.1/24 is desired:
FLANNEL_NETWORK=10.244.0.0/16 FLANNEL_SUBNET=10.244.4.1/24 FLANNEL_MTU=1500 FLANNEL_IPMASQ=true
If these steps don't resolve your problem, you can get help running Windows containers on Windows nodes in Kubernetes through: