Kubernetes Blog

Kubernetes on OpenStack

May 19 2015

Today, the OpenStack foundation made it even easier for you deploy and manage clusters of Docker containers on OpenStack clouds by including Kubernetes in its Community App Catalog.  At a keynote today at the OpenStack Summit in Vancouver, Mark Collier, COO of the OpenStack Foundation, and Craig Peters,  Mirantis product line manager, demonstrated the Community App Catalog workflow by launching a Kubernetes cluster in a matter of seconds by leveraging the compute, storage, networking and identity systems already present in an OpenStack cloud.

The entries in the catalog include not just the ability to start a Kubernetes cluster, but also a range of applications deployed in Docker containers managed by Kubernetes. These applications include:

- Apache web server - Nginx web server - Crate - The Distributed Database for Docker - GlassFish - Java EE 7 Application Server - Tomcat - An open-source web server and servlet container - InfluxDB - An open-source, distributed, time series database - Grafana - Metrics dashboard for InfluxDB - Jenkins - An extensible open source continuous integration server - MariaDB database - MySql database - Redis - Key-value cache and store - PostgreSQL database - MongoDB NoSQL database - Zend Server - The Complete PHP Application Platform

This list will grow, and is curated here. You can examine (and contribute to) the YAML file that tells Murano how to install and start the Kubernetes cluster here.

The Kubernetes open source project has continued to see fantastic community adoption and increasing momentum, with over 11,000 commits and 7,648 stars on GitHub. With supporters ranging from Red Hat and Intel to CoreOS and Box.net, it has come to represent a range of customer interests ranging from enterprise IT to cutting edge startups. We encourage you to give it a try, give us your feedback, and get involved in our growing community.

  • Martin Buhr, Product Manager, Kubernetes Open Source Project

Weekly Kubernetes Community Hangout Notes - May 15 2015

May 18 2015

Every week the Kubernetes contributing community meet virtually over Google Hangouts. We want anyone who’s interested to know what’s discussed in this forum.

  • v1 API - what’s in, what’s out
    • We’re trying to fix critical issues we discover with v1beta3
    • Would like to make a number of minor cleanups that will be expensive to do later
      • defaulting replication controller spec default to 1
      • deduplicating security context
      • change id field to name
      • rename host
      • inconsistent times
      • typo in container states terminated (termination vs. terminated)
      • flatten structure (requested by heavy API user)
      • pod templates - could be added after V1, field is not implemented, remove template ref field
      • in general remove any fields not implemented (can be added later)
      • if we want to change any of the identifier validation rules, should do it now
      • recently changed label validation rules to be more precise
    • Bigger changes
      • generalized label selectors
      • service - change the fields in a way that we can add features in a forward compatible manner if possible
      • public IPs - what to do from a security perspective
      • Support aci format - there is an image field - add properties to signify the image, or include it in a string
      • inconsistent on object use / cross reference - needs design discussion
    • Things to do later
      • volume source cleanup
      • multiple API prefixes
      • watch changes - watch client is not notified of progress
  • A few other proposals
    • swagger spec fixes - ongoing
    • additional field selectors - additive, backward compatible
    • additional status - additive, backward compatible
    • elimination of phase - won’t make it for v1
  • Service discussion - Public IPs
    • with public ips as it exists we can’t go to v1
    • Tim has been developing a mitigation if we can’t get Justin’s overhaul in (but hopefully we will)
    • Justin’s fix will describe public IPs in a much better way
    • The general problem is it’s too flexible and you can do things that are scary, the mitigation is to restrict public ip usage to specific use cases – validated public ips would be copied to status, which is what kube-proxy would use
    • public ips used for -
      • binding to nodes / node
      • request a specific load balancer IP (GCE only)
      • emulate multi-port services – now we support multi-port services, so no longer necessary
    • This is a large change, 70% code complete, Tim & Justin working together, parallel code review and updates, need to reconcile and test
    • Do we want to allow people to request host ports - is there any value in letting people ask for a public port? or should we assign you one?
      • Tim: we should assign one
    • discussion of what to do with status - if users set to empty then probably their intention
    • general answer to the pattern is binding
    • post v1: if we can make portal ip a non-user settable field, then we need to figure out the transition plan. need to have a fixed ip for dns.
    • we should be able to just randomly assign services a new port and everything should adjust, but this is not feasible for v1
    • next iteration of the proposal: PR is being iterated on, testing over the weekend, so PR hopefully ready early next week - gonna be a doozie!
  • API transition
    • actively removing all dependencies on v1beta1 and v1beta2, announced their going away
    • working on a script that will touch everything in the system and will force everything to flip to v1beta3
    • a release with both APIs supported and with this script can make sure clusters are moved over and we can move the API
    • Should be gone by 0.19
    • Help is welcome, especially for trivial things and will try to get as much done as possible in next few weeks
    • Release candidate targeting mid june
    • The new kubectl will not work for old APIs, will be a problem for GKE for clusters pinned to old version. Will be a problem for k8s users as well if they update kubectl
    • Since there’s no way to upgrade a GKE cluster, users are going to have to tear down and upgrade their cluster
    • we’re going to stop testing v1beta1 very soon, trying to streamline the testing paths in our CI pipelines
  • Did we decide we are not going to do namespace autoprovisioning?
    • Brian would like to turn it off - no objections
    • Documentation should include creating namepspaces
    • Would like to impose a default CPU for the default namespace
    • would cap the number of pods, would reduce the resource exhaustion issue
    • would eliminate need to explicitly cap the number of pods on a node due to IP exhaustion
    • could add resources as arguments to the porcelain commands
    • kubectl run is a simplified command, but it could include some common things (image, command, ports). but could add resources
  • Kubernetes 1.0 Launch Event
    • Save the date: July 21st in Portland, OR - a part of OSCON
    • Blog posts, whitepapers, etc. welcome to be published
    • Event will be live streamed, mostly demos & customer talks, keynote
    • Big launch party in the evening
    • Kit to send more info in next couple weeks

Docker and Kubernetes and AppC

May 18 2015

Recently we announced the intent in Kubernetes, our open source cluster manager, to support AppC and RKT, an alternative container format that has been driven by CoreOS with input from many companies (including Google).  This announcement has generated a surprising amount of buzz and has been construed as a move from Google to support Appc over Docker.  Many have taken it as signal that Google is moving away from supporting Docker.  I would like to take a moment to clarify Google’s position in this.

Google has consistently supported the Docker initiative and has invested heavily in Docker. In the early days of containers, we decided to de-emphasize our own open source offering (LMCTFY) and to instead focus on Docker.  As a result of that we have two engineers that are active maintainers of LibContainer, a critical piece of the Docker ecosystem and are working closely with Docker to add many additional features and capabilities.  Docker is currently the only supported runtime in GKE (Google Container Engine) our commercial containers product, and in GAE (Google App Engine), our Platform-as-a-Service product.  

While we may introduce AppC support at some point in the future to GKE based on our customers demand, we intend to continue to support the Docker project and product, and Docker the company indefinitely.  To date Docker is by far the most mature and widely used container offering in the market, with over 400 million downloads.  It has been production ready for almost a year and seen widespread use in industry, and also here inside Google.

Beyond the obvious traction Docker has in the market, we are heartened by many of Docker’s recent initiatives to open the project and support ‘batteries included, but swappable options across the stack and recognize that it offers a great developer experience for engineers new to the containers world.  We are encouraged, for example, by the separation of the Docker Machine and Swarm projects from the core runtime, and are glad to see support for Docker Machine emerging for Google Compute Engine.

Our intent with our announcement for AppC and RKT support was to establish Kubernetes (our open source project) as a neutral ground in the world of containers.  Customers should be able to pick their container runtime and format based solely on its technical merits, and we do see AppC as offering some legitimate potential merits as the technology matures.  Somehow this was misconstrued as an ‘a vs b’ selection which is simply untrue.  The world is almost always better for having choice, and it is perfectly natural that different tools should be available for different purposes.  

Stepping back a little, one must recognize that Docker has done remarkable work in democratizing container technologies and making them accessible to everyone.  We believe that Docker will continue to drive great experiences for developers looking to use containers and plan to support this technology and its burgeoning community indefinitely.  We, for one,  are looking forward to the upcoming Dockercon where Brendan Burns (a Kubernetes co-founder) will be talking about the role of Docker in modern distributed systems design.

– Craig McLuckie

Google Group Product Manager, and Kubernetes Project Co-Founder

Kubernetes Release: 0.17.0

May 15 2015

Release Notes:

  • Cleanups

    • Remove old salt configs #8065 (roberthbailey)
    • Kubelet: minor cleanups #8069 (yujuhong)
  • v1beta3

    • update example/walkthrough to v1beta3 #7940 (caesarxuchao)
    • update example/rethinkdb to v1beta3 #7946 (caesarxuchao)
    • verify the v1beta3 yaml files all work; merge the yaml files #7917 (caesarxuchao)
    • update examples/cassandra to api v1beta3 #7258 (caesarxuchao)
    • update service.json in persistent-volume example to v1beta3 #7899 (caesarxuchao)
    • update mysql-wordpress example to use v1beta3 API #7864 (caesarxuchao)
    • Update examples/meteor to use API v1beta3 #7848 (caesarxuchao)
    • update node-selector example to API v1beta3 #7872 (caesarxuchao)
    • update logging-demo to use API v1beta3; modify the way to access Elasticsearch and Kibana services #7824 (caesarxuchao)
    • Convert the skydns rc to use v1beta3 and add a health check to it #7619 (a-robinson)
    • update the hazelcast example to API version v1beta3 #7728 (caesarxuchao)
    • Fix YAML parsing for v1beta3 objects in the kubelet for file/http #7515 (brendandburns)
    • Updated kubectl cluster-info to show v1beta3 addresses #7502 (piosz)
  • Kubelet

    • kubelet: Fix racy kubelet tests. #7980 (yifan-gu)
    • kubelet/container: Move prober.ContainerCommandRunner to container. #8079 (yifan-gu)
    • Kubelet: set host field in the pending pod status #6127 (yujuhong)
    • Fix the kubelet node watch #6442 (yujuhong)
    • Kubelet: recreate mirror pod if the static pod changes #6607 (yujuhong)
    • Kubelet: record the timestamp correctly in the runtime cache #7749 (yujuhong)
    • Kubelet: wait until container runtime is up #7729 (yujuhong)
    • Kubelet: replace DockerManager with the Runtime interface #7674 (yujuhong)
    • Kubelet: filter out terminated pods in SyncPods #7301 (yujuhong)
    • Kubelet: parallelize cleaning up containers in unwanted pods #7048 (yujuhong)
    • kubelet: Add container runtime option for rkt. #7952 (yifan-gu)
    • kubelet/rkt: Remove build label. #7916 (yifan-gu)
    • kubelet/metrics: Move instrumented_docker.go to dockertools. #7327 (yifan-gu)
    • kubelet/rkt: Add GetPods() for rkt. #7599 (yifan-gu)
    • kubelet/rkt: Add KillPod() and GetPodStatus() for rkt. #7605 (yifan-gu)
    • pkg/kubelet: Fix logging. #4755 (yifan-gu)
    • kubelet: Refactor RunInContainer/ExecInContainer/PortForward. #6491 (yifan-gu)
    • kubelet/DockerManager: Fix returning empty error from GetPodStatus(). #6609 (yifan-gu)
    • kubelet: Move pod infra container image setting to dockertools. #6634 (yifan-gu)
    • kubelet/fake_docker_client: Use self’s PID instead of 42 in testing. #6653 (yifan-gu)
    • kubelet/dockertool: Move Getpods() to DockerManager. #6778 (yifan-gu)
    • kubelet/dockertools: Add puller interfaces in the containerManager. #6776 (yifan-gu)
    • kubelet: Introduce PodInfraContainerChanged(). #6608 (yifan-gu)
    • kubelet/container: Replace DockerCache with RuntimeCache. #6795 (yifan-gu)
    • kubelet: Clean up computePodContainerChanges. #6844 (yifan-gu)
    • kubelet: Refactor prober. #7009 (yifan-gu)
    • kubelet/container: Update the runtime interface. #7466 (yifan-gu)
    • kubelet: Refactor isPodRunning() in runonce.go #7477 (yifan-gu)
    • kubelet/rkt: Add basic rkt runtime routines. #7465 (yifan-gu)
    • kubelet/rkt: Add podInfo. #7555 (yifan-gu)
    • kubelet/container: Add GetContainerLogs to runtime interface. #7488 (yifan-gu)
    • kubelet/rkt: Add routines for converting kubelet pod to rkt pod. #7543 (yifan-gu)
    • kubelet/rkt: Add RunPod() for rkt. #7589 (yifan-gu)
    • kubelet/rkt: Add RunInContainer()/ExecInContainer()/PortForward(). #7553 (yifan-gu)
    • kubelet/container: Move ShouldContainerBeRestarted() to runtime. #7613 (yifan-gu)
    • kubelet/rkt: Add SyncPod() to rkt. #7611 (yifan-gu)
    • Kubelet: persist restart count of a container #6794 (yujuhong)
    • kubelet/container: Move pty*.go to container runtime package. #7951 (yifan-gu)
    • kubelet: Add container runtime option for rkt. #7900 (yifan-gu)
    • kubelet/rkt: Add docker prefix to image string. #7803 (yifan-gu)
    • kubelet/rkt: Inject dependencies to rkt. #7849 (yifan-gu)
    • kubelet/rkt: Remove dependencies on rkt.store #7859 (yifan-gu)
    • Kubelet talks securely to apiserver #2387 (erictune)
    • Rename EnvVarSource.FieldPath -> FieldRef and add example #7592 (pmorie)
    • Add containerized option to kubelet binary #7741 (pmorie)
    • Ease building kubelet image #7948 (pmorie)
    • Remove unnecessary bind-mount from dockerized kubelet run #7854 (pmorie)
    • Add ability to dockerize kubelet in local cluster #7798 (pmorie)
    • Create docker image for kubelet #7797 (pmorie)
    • Security context - types, kubelet, admission #7343 (pweil-)
    • Kubelet: Add rkt as a runtime option #7743 (vmarmol)
    • Fix kubelet’s docker RunInContainer implementation #7746 (vishh)
  • AWS

    • AWS: Don’t try to copy gce_keys in jenkins e2e job #8018 (justinsb)
    • AWS: Copy some new properties from config-default => config.test #7992 (justinsb)
    • AWS: make it possible to disable minion public ip assignment #7928 (manolitto)
    • update AWS CloudFormation template and cloud-configs #7667 (antoineco)
    • AWS: Fix variable naming that meant not all tokens were written #7736 (justinsb)
    • AWS: Change apiserver to listen on 443 directly, not through nginx #7678 (justinsb)
    • AWS: Improving getting existing VPC and subnet #6606 (gust1n)
    • AWS EBS volume support #5138 (justinsb)
  • Introduce an ‘svc’ segment for DNS search #8089 (thockin)
  • Adds ability to define a prefix for etcd paths #5707 (kbeecher)
  • Add kubectl log –previous support to view last terminated container log #7973 (dchen1107)
  • Add a flag to disable legacy APIs #8083 (brendandburns)
  • make the dockerkeyring handle mutiple matching credentials #7971 (deads2k)
  • Convert Fluentd to Cloud Logging pod specs to YAML #8078 (satnam6502)
  • Use etcd to allocate PortalIPs instead of in-mem #7704 (smarterclayton)
  • eliminate auth-path #8064 (deads2k)
  • Record failure reasons for image pulling #7981 (yujuhong)
  • Rate limit replica creation #7869 (bprashanth)
  • Upgrade to Kibana 4 for cluster logging #7995 (satnam6502)
  • Added name to kube-dns service #8049 (piosz)
  • Fix validation by moving it into the resource builder. #7919 (brendandburns)
  • Add cache with multiple shards to decrease lock contention #8050 (fgrzadkowski)
  • Delete status from displayable resources #8039 (nak3)
  • Refactor volume interfaces to receive pod instead of ObjectReference #8044 (pmorie)
  • fix kube-down for provider gke #7565 (jlowdermilk)
  • Service port names are required for multi-port #7786 (thockin)
  • Increase disk size for kubernetes master. #8051 (fgrzadkowski)
  • expose: Load input object for increased safety #7774 (kargakis)
  • Improments to conversion methods generator #7896 (wojtek-t)
  • Added displaying external IPs to kubectl cluster-info #7557 (piosz)
  • Add missing Errorf formatting directives #8037 (shawnps)
  • Add startup code to apiserver to migrate etcd keys #7567 (kbeecher)
  • Use error type from docker go-client instead of string #8021 (ddysher)
  • Accurately get hardware cpu count in Vagrantfile. #8024 (BenTheElder)
  • Stop setting a GKE specific version of the kubeconfig file #7921 (roberthbailey)
  • Make the API server deal with HEAD requests via the service proxy #7950 (satnam6502)
  • GlusterFS Critical Bug Resolved - Removing warning in README #7983 (wattsteve)
  • Don’t use the first token uname -n as the hostname #7967 (yujuhong)
  • Call kube-down in test-teardown for vagrant. #7982 (BenTheElder)
  • defaults_tests: verify defaults when converting to an API object #6235 (yujuhong)
  • Use the full hostname for mirror pod name. #7910 (yujuhong)
  • Removes RunPod in the Runtime interface #7657 (yujuhong)
  • Clean up dockertools/manager.go and add more unit tests #7533 (yujuhong)
  • Adapt pod killing and cleanup for generic container runtime #7525 (yujuhong)
  • Fix pod filtering in replication controller #7198 (yujuhong)
  • Print container statuses in kubectl get pods #7116 (yujuhong)
  • Prioritize deleting the non-running pods when reducing replicas #6992 (yujuhong)
  • Fix locking issue in pod manager #6872 (yujuhong)
  • Limit the number of concurrent tests in integration.go #6655 (yujuhong)
  • Fix typos in different config comments #7931 (pmorie)
  • Update cAdvisor dependency. #7929 (vmarmol)
  • Ubuntu-distro: deprecate & merge ubuntu single node work to ubuntu cluster node stuff#5498 (resouer)
  • Add control variables to Jenkins E2E script #7935 (saad-ali)
  • Check node status as part of validate-cluster.sh. #7932 (fabioy)
  • Add old endpoint cleanup function #7821 (lavalamp)
  • Support recovery from in the middle of a rename. #7620 (brendandburns)
  • Update Exec and Portforward client to use pod subresource #7715 (csrwng)
  • Added NFS to PV structs #7564 (markturansky)
  • Fix environment variable error in Vagrant docs #7904 (posita)
  • Adds a simple release-note builder that scrapes the Github API for recent PRs #7616(brendandburns)
  • Scheduler ignores nodes that are in a bad state #7668 (bprashanth)
  • Set GOMAXPROCS for etcd #7863 (fgrzadkowski)
  • Auto-generated conversion methods calling one another #7556 (wojtek-t)
  • Bring up a kuberenetes cluster using coreos image as worker nodes #7445 (dchen1107)
  • Godep: Add godep for rkt. #7410 (yifan-gu)
  • Add volumeGetter to rkt. #7870 (yifan-gu)
  • Update cAdvisor dependency. #7897 (vmarmol)
  • DNS: expose 53/TCP #7822 (thockin)
  • Set NodeReady=False when docker is dead #7763 (wojtek-t)
  • Ignore latency metrics for events #7857 (fgrzadkowski)
  • SecurityContext admission clean up #7792 (pweil-)
  • Support manually-created and generated conversion functions #7832 (wojtek-t)
  • Add latency metrics for etcd operations #7833 (fgrzadkowski)
  • Update errors_test.go #7885 (hurf)
  • Change signature of container runtime PullImage to allow pull w/ secret #7861 (pmorie)
  • Fix bug in Service documentation: incorrect location of “selector” in JSON #7873(bkeroackdsc)
  • Fix controller-manager manifest for providers that don’t specify CLUSTER_IP_RANGE#7876 (cjcullen)
  • Fix controller unittests #7867 (bprashanth)
  • Enable GCM and GCL instead of InfluxDB on GCE #7751 (saad-ali)
  • Remove restriction that cluster-cidr be a class-b #7862 (cjcullen)
  • Fix OpenShift example #7591 (derekwaynecarr)
  • API Server - pass path name in context of create request for subresource #7718 (csrwng)
  • Rolling Updates: Add support for –rollback. #7575 (brendandburns)
  • Update to container-vm-v20150505 (Also updates GCE to Docker 1.6) #7820 (zmerlynn)
  • Fix metric label #7830 (rhcarvalho)
  • Fix v1beta1 typos in v1beta2 conversions #7838 (pmorie)
  • skydns: use the etcd-2.x native syntax, enable IANA attributed ports. #7764(AntonioMeireles)
  • Added port 6443 to kube-proxy default IP address for api-server #7794 (markllama)
  • Added client header info for authentication doc. #7834 (ashcrow)
  • Clean up safe_format_and_mount spam in the startup logs #7827 (zmerlynn)
  • Set allocate_node_cidrs to be blank by default. #7829 (roberthbailey)
  • Fix sync problems in #5246 #7799 (cjcullen)
  • Fix event doc link #7823 (saad-ali)
  • Cobra update and bash completions fix #7776 (eparis)
  • Fix kube2sky flakes. Fix tools.GetEtcdVersion to work with etcd > 2.0.7 #7675 (cjcullen)
  • Change kube2sky to use token-system-dns secret, point at https endpoint … #7154(cjcullen)
  • replica: serialize created-by reference #7468 (simon3z)
  • Inject mounter into volume plugins #7702 (pmorie)
  • bringing CoreOS cloud-configs up-to-date (against 0.15.x and latest OS’ alpha) #6973(AntonioMeireles)
  • Update kubeconfig-file doc. #7787 (jlowdermilk)
  • Throw an API error when deleting namespace in termination #7780 (derekwaynecarr)
  • Fix command field PodExecOptions #7773 (csrwng)
  • Start ImageManager housekeeping in Run(). #7785 (vmarmol)
  • fix DeepCopy to properly support runtime.EmbeddedObject #7769 (deads2k)
  • fix master service endpoint system for multiple masters #7273 (lavalamp)
  • Add genbashcomp to KUBE_TEST_TARGETS #7757 (nak3)
  • Change the cloud provider TCPLoadBalancerExists function to GetTCPLoadBalancer…#7669 (a-robinson)
  • Add containerized option to kubelet binary #7772 (pmorie)
  • Fix swagger spec #7779 (pmorie)
  • FIX: Issue #7750 - Hyperkube docker image needs certificates to connect to cloud-providers#7755 (viklas)
  • Add build labels to rkt #7752 (vmarmol)
  • Check license boilerplate for python files #7672 (eparis)
  • Reliable updates in rollingupdate #7705 (bprashanth)
  • Don’t exit abruptly if there aren’t yet any minions right after the cluster is created. #7650(roberthbailey)
  • Make changes suggested in #7675 #7742 (cjcullen)
  • A guide to set up kubernetes multiple nodes cluster with flannel on fedora #7357(aveshagarwal)
  • Setup generators in factory #7760 (kargakis)
  • Reduce usage of time.After #7737 (lavalamp)
  • Remove node status from “componentstatuses” call. #7735 (fabioy)
  • React to failure by growing the remaining clusters #7614 (tamsky)
  • Fix typo in runtime_cache.go #7725 (pmorie)
  • Update non-GCE Salt distros to 1.6.0, fallback to ContainerVM Docker version on GCE#7740 (zmerlynn)
  • Skip SaltStack install if it’s already installed #7744 (zmerlynn)
  • Expose pod name as a label on containers. #7712 (rjnagal)
  • Log which SSH key is used in e2e SSH test #7732 (mbforbes)
  • Add a central simple getting started guide with kubernetes guide. #7649 (brendandburns)
  • Explicitly state the lack of support for ‘Requests’ for the purposes of scheduling #7443(vishh)
  • Select IPv4-only from host interfaces #7721 (smarterclayton)
  • Metrics tests can’t run on Mac #7723 (smarterclayton)
  • Add step to API changes doc for swagger regen #7727 (pmorie)
  • Add NsenterMounter mount implementation #7703 (pmorie)
  • add StringSet.HasAny #7509 (deads2k)
  • Add an integration test that checks for the metrics we expect to be exported from the master #6941 (a-robinson)
  • Minor bash update found by shellcheck.net #7722 (eparis)
  • Add –hostport to run-container. #7536 (rjnagal)
  • Have rkt implement the container Runtime interface #7659 (vmarmol)
  • Change the order the different versions of API are registered #7629 (caesarxuchao)
  • expose: Create objects in a generic way #7699 (kargakis)
  • Requeue rc if a single get/put retry on status.Replicas fails #7643 (bprashanth)
  • logs for master components #7316 (ArtfulCoder)
  • cloudproviders: add ovirt getting started guide #7522 (simon3z)
  • Make rkt-install a oneshot. #7671 (vmarmol)
  • Provide container_runtime flag to Kubelet in CoreOS. #7665 (vmarmol)
  • Boilerplate speedup #7654 (eparis)
  • Log host for failed pod in Density test #7700 (wojtek-t)
  • Removes spurious quotation mark #7655 (alindeman)
  • Add kubectl_label to custom functions in bash completion #7694 (nak3)
  • Enable profiling in kube-controller #7696 (wojtek-t)
  • Set vagrant test cluster default NUM_MINIONS=2 #7690 (BenTheElder)
  • Add metrics to measure cache hit ratio #7695 (fgrzadkowski)
  • Change IP to IP(S) in service columns for kubectl get #7662 (jlowdermilk)
  • annotate required flags for bash_completions #7076 (eparis)
  • (minor) Add pgrep debugging to etcd error #7685 (jayunit100)
  • Fixed nil pointer issue in describe when volume is unbound #7676 (markturansky)
  • Removed unnecessary closing bracket #7691 (piosz)
  • Added TerminationGracePeriod field to PodSpec and grace-period flag to kubectl stop#7432 (piosz)
  • Fix boilerplate in test/e2e/scale.go #7689 (wojtek-t)
  • Update expiration timeout based on observed latencies #7628 (bprashanth)
  • Output generated conversion functions/names #7644 (liggitt)
  • Moved the Scale tests into a scale file. #7645 #7646 (rrati)
  • Truncate GCE load balancer names to 63 chars #7609 (brendandburns)
  • Add SyncPod() and remove Kill/Run InContainer(). #7603 (vmarmol)
  • Merge release 0.16 to master #7663 (brendandburns)
  • Update license boilerplate for examples/rethinkdb #7637 (eparis)
  • First part of improved rolling update, allow dynamic next replication controller generation.#7268 (brendandburns)
  • Add license boilerplate to examples/phabricator #7638 (eparis)
  • Use generic copyright holder name in license boilerplate #7597 (eparis)
  • Retry incrementing quota if there is a conflict #7633 (derekwaynecarr)
  • Remove GetContainers from Runtime interface #7568 (yujuhong)
  • Add image-related methods to DockerManager #7578 (yujuhong)
  • Remove more docker references in kubelet #7586 (yujuhong)
  • Add KillContainerInPod in DockerManager #7601 (yujuhong)
  • Kubelet: Add container runtime option. #7652 (vmarmol)
  • bump heapster to v0.11.0 and grafana to v0.7.0 #7626 (idosh)
  • Build github.com/onsi/ginkgo/ginkgo as a part of the release #7593 (ixdy)
  • Do not automatically decode runtime.RawExtension #7490 (smarterclayton)
  • Update changelog. #7500 (brendandburns)
  • Add SyncPod() to DockerManager and use it in Kubelet #7610 (vmarmol)
  • Build: Push .md5 and .sha1 files for every file we push to GCS #7602 (zmerlynn)
  • Fix rolling update –image #7540 (bprashanth)
  • Update license boilerplate for docs/man/md2man-all.sh #7636 (eparis)
  • Include shell license boilerplate in examples/k8petstore #7632 (eparis)
  • Add –cgroup_parent flag to Kubelet to set the parent cgroup for pods #7277 (guenter)
  • change the current dir to the config dir #7209 (you-n-g)
  • Set Weave To 0.9.0 And Update Etcd Configuration For Azure #7158 (idosh)
  • Augment describe to search for matching things if it doesn’t match the original resource.#7467 (brendandburns)
  • Add a simple cache for objects stored in etcd. #7559 (fgrzadkowski)
  • Rkt gc #7549 (yifan-gu)
  • Rkt pull #7550 (yifan-gu)
  • Implement Mount interface using mount(8) and umount(8) #6400 (ddysher)
  • Trim Fleuntd tag for Cloud Logging #7588 (satnam6502)
  • GCE CoreOS cluster - set master name based on variable #7569 (bakins)
  • Capitalization of KubeProxyVersion wrong in JSON #7535 (smarterclayton)
  • Make nodes report their external IP rather than the master’s. #7530 (mbforbes)
  • Trim cluster log tags to pod name and container name #7539 (satnam6502)
  • Handle conversion of boolean query parameters with a value of “false” #7541 (csrwng)
  • Add image-related methods to Runtime interface. #7532 (vmarmol)
  • Test whether auto-generated conversions weren’t manually edited #7560 (wojtek-t)
  • Mention :latest behavior for image version tag #7484 (colemickens)
  • readinessProbe calls livenessProbe.Exec.Command which cause “invalid memory address or nil pointer dereference”. #7487 (njuicsgz)
  • Add RuntimeHooks to abstract Kubelet logic #7520 (vmarmol)
  • Expose URL() on Request to allow building URLs #7546 (smarterclayton)
  • Add a simple cache for objects stored in etcd #7288 (fgrzadkowski)
  • Prepare for chaining autogenerated conversion methods #7431 (wojtek-t)
  • Increase maxIdleConnection limit when creating etcd client in apiserver. #7353 (wojtek-t)
  • Improvements to generator of conversion methods. #7354 (wojtek-t)
  • Code to automatically generate conversion methods #7107 (wojtek-t)
  • Support recovery for anonymous roll outs #7407 (brendandburns)
  • Bump kube2sky to 1.2. Point it at https endpoint (3rd try). #7527 (cjcullen)
  • cluster/gce/coreos: Add metadata-service in node.yaml #7526 (yifan-gu)
  • Move ComputePodChanges to the Docker runtime #7480 (vmarmol)
  • Cobra rebase #7510 (eparis)
  • Adding system oom events from kubelet #6718 (vishh)
  • Move Prober to its own subpackage #7479 (vmarmol)
  • Fix parallel-e2e.sh to work on my macbook (bash v3.2) #7513 (cjcullen)
  • Move network plugin TearDown to DockerManager #7449 (vmarmol)
  • Fixes #7498 - CoreOS Getting Started Guide had invalid cloud config #7499 (elsonrodriguez)
  • Fix invalid character ‘”’ after object key:value pair #7504 (resouer)
  • Fixed kubelet deleting data from volumes on stop (#7317). #7503 (jsafrane)
  • Fixing hooks/description to catch API fields without description tags #7482 (nikhiljindal)
  • cadvisor is obsoleted so kubelet service does not require it. #7457 (aveshagarwal)
  • Set the default namespace for events to be “default” #7408 (vishh)
  • Fix typo in namespace conversion #7446 (liggitt)
  • Convert Secret registry to use update/create strategy, allow filtering by Type #7419 (liggitt)
  • Use pod namespace when looking for its GlusterFS endpoints. #7102 (jsafrane)
  • Fixed name of kube-proxy path in deployment scripts. #7427 (jsafrane)

To download, please visit https://github.com/GoogleCloudPlatform/kubernetes/releases/tag/v0.17.0

Simple theme. Powered by [Blogger][385].

[ ![][327] ][386]


Resource Usage Monitoring in Kubernetes

May 12 2015

Understanding how an application behaves when deployed is crucial to scaling the application and providing a reliable service. In a Kubernetes cluster, application performance can be examined at many different levels: containers, pods, services, and whole clusters. As part of Kubernetes we want to provide users with detailed resource usage information about their running applications at all these levels. This will give users deep insights into how their applications are performing and where possible application bottlenecks may be found. In comes Heapster, a project meant to provide a base monitoring platform on Kubernetes.


Heapster is a cluster-wide aggregator of monitoring and event data. It currently supports Kubernetes natively and works on all Kubernetes setups. Heapster runs as a pod in the cluster, similar to how any Kubernetes application would run. The Heapster pod discovers all nodes in the cluster and queries usage information from the nodes’ Kubelets, the on-machine Kubernetes agent. The Kubelet itself fetches the data from cAdvisor. Heapster groups the information by pod along with the relevant labels. This data is then pushed to a configurable backend for storage and visualization. Currently supported backends include InfluxDB (with Grafana for visualization), Google Cloud Monitoring and many others described in more details here. The overall architecture of the service can be seen below:

Let’s look at some of the other components in more detail.


cAdvisor is an open source container resource usage and performance analysis agent. It is purpose built for containers and supports Docker containers natively. In Kubernetes, cadvisor is integrated into the Kubelet binary. cAdvisor auto-discovers all containers in the machine and collects CPU, memory, filesystem, and network usage statistics. cAdvisor also provides the overall machine usage by analyzing the ‘root’? container on the machine.

On most Kubernetes clusters, cAdvisor exposes a simple UI for on-machine containers on port 4194. Here is a snapshot of part of cAdvisor’s UI that shows the overall machine usage:


The Kubelet acts as a bridge between the Kubernetes master and the nodes. It manages the pods and containers running on a machine. Kubelet translates each pod into its constituent containers and fetches individual container usage statistics from cAdvisor. It then exposes the aggregated pod resource usage statistics via a REST API.


InfluxDB and Grafana

A Grafana setup with InfluxDB is a very popular combination for monitoring in the open source world. InfluxDB exposes an easy to use API to write and fetch time series data. Heapster is setup to use this storage backend by default on most Kubernetes clusters. A detailed setup guide can be found here. InfluxDB and Grafana run in Pods. The pod exposes itself as a Kubernetes service which is how Heapster discovers it.

The Grafana container serves Grafana’s UI which provides an easy to configure dashboard interface. The default dashboard for Kubernetes contains an example dashboard that monitors resource usage of the cluster and the pods inside of it. This dashboard can easily be customized and expanded. Take a look at the storage schema for InfluxDB here.

Here is a video showing how to monitor a Kubernetes cluster using heapster, InfluxDB and Grafana:

Here is a snapshot of the default Kubernetes Grafana dashboard that shows the CPU and Memory usage of the entire cluster, individual pods and containers:

Google Cloud Monitoring

Google Cloud Monitoring is a hosted monitoring service that allows you to visualize and alert on important metrics in your application. Heapster can be setup to automatically push all collected metrics to Google Cloud Monitoring. These metrics are then available in the Cloud Monitoring Console. This storage backend is the easiest to setup and maintain. The monitoring console allows you to easily create and customize dashboards using the exported data.

Here is a video showing how to setup and run a Google Cloud Monitoring backed Heapster: “https://youtube.com/embed/xSMNR2fcoLs” Here is a snapshot of the a Google Cloud Monitoring dashboard showing cluster-wide resource usage.

Try it out!

Now that you’ve learned a bit about Heapster, feel free to try it out on your own clusters! The Heapster repository is available on GitHub. It contains detailed instructions to setup Heapster and its storage backends. Heapster runs by default on most Kubernetes clusters, so you may already have it! Feedback is always welcome. Please let us know if you run into any issues via the troubleshooting channels.

– Vishnu Kannan and Victor Marmol, Google Software Engineers

Weekly Kubernetes Community Hangout Notes - May 1 2015

May 11 2015

Every week the Kubernetes contributing community meet virtually over Google Hangouts. We want anyone who’s interested to know what’s discussed in this forum.

  • Simple rolling update - Brendan

    • Rolling update = nice example of why RCs and Pods are good.

    • …pause… (Brendan needs demo recovery tips from Kelsey)

    • Rolling update has recovery: Cancel update and restart, update continues from where it stopped.

    • New controller gets name of old controller, so appearance is pure update.

    • Can also name versions in update (won’t do rename at the end).

  • Rocket demo - CoreOS folks

    • 2 major differences between rocket & docker: Rocket is daemonless & pod-centric.

    • Rocket has AppContainer format as native, but also supports docker image format.

    • Can run AppContainer and docker containers in same pod.

    • Changes are close to merged.

  • demo service accounts and secrets being added to pods - Jordan

    • Problem: It’s hard to get a token to talk to the API.

    • New API object: “ServiceAccount”

    • ServiceAccount is namespaced, controller makes sure that at least 1 default service account exists in a namespace.

    • Typed secret “ServiceAccountToken”, controller makes sure there is at least 1 default token.

    • DEMO

      • Can create new service account with ServiceAccountToken. Controller will create token for it.
    • Can create a pod with service account, pods will have service account secret mounted at /var/run/secrets/kubernetes.io/…
  • Kubelet running in a container - Paul

    • Kubelet successfully ran pod w/ mounted secret.

Kubernetes Release: 0.16.0

May 11 2015

Release Notes:

  • Bring up a kuberenetes cluster using coreos image as worker nodes #7445 (dchen1107)
  • Cloning v1beta3 as v1 and exposing it in the apiserver #7454 (nikhiljindal)
  • API Conventions for Late-initializers #7366 (erictune)
  • Upgrade Elasticsearch to 1.5.2 for cluster logging #7455 (satnam6502)
  • Make delete actually stop resources by default. #7210 (brendandburns)
  • Change kube2sky to use token-system-dns secret, point at https endpoint … #7154(cjcullen)
  • Updated CoreOS bare metal docs for 0.15.0 #7364 (hvolkmer)
  • Print named ports in ‘describe service’ #7424 (thockin)
  • AWS
  • Return public & private addresses in GetNodeAddresses #7040 (justinsb)
  • Improving getting existing VPC and subnet #6606 (gust1n)
  • Set hostname_override for minions, back to fully-qualified name #7182 (justinsb)
  • Conversion to v1beta3
  • Convert node level logging agents to v1beta3 #7274 (satnam6502)
  • Removing more references to v1beta1 from pkg/ #7128 (nikhiljindal)
  • update examples/cassandra to api v1beta3 #7258 (caesarxuchao)
  • Convert Elasticsearch logging to v1beta3 and de-salt #7246 (satnam6502)
  • Update examples/storm for v1beta3 #7231 (bcbroussard)
  • Update examples/spark for v1beta3 #7230 (bcbroussard)
  • Update Kibana RC and service to v1beta3 #7240 (satnam6502)
  • Updating the guestbook example to v1beta3 #7194 (nikhiljindal)
  • Update Phabricator to v1beta3 example #7232 (bcbroussard)
  • Update Kibana pod to speak to Elasticsearch using v1beta3 #7206 (satnam6502)
  • Validate Node IPs; clean up validation code #7180 (ddysher)
  • Add PortForward to runtime API. #7391 (vmarmol)
  • kube-proxy uses token to access port 443 of apiserver #7303 (erictune)
  • Move the logging-related directories to where I think they belong #7014 (a-robinson)
  • Make client service requests use the default timeout now that external load balancers are created asynchronously #6870 (a-robinson)
  • Fix bug in kube-proxy of not updating iptables rules if a service’s public IPs change #6123(a-robinson)
  • PersistentVolumeClaimBinder #6105 (markturansky)
  • Fixed validation message when trying to submit incorrect secret #7356 (soltysh)
  • First step to supporting multiple k8s clusters #6006 (justinsb)
  • Parity for namespace handling in secrets E2E #7361 (pmorie)
  • Add cleanup policy to RollingUpdater #6996 (ironcladlou)
  • Use narrowly scoped interfaces for client access #6871 (ironcladlou)
  • Warning about Critical bug in the GlusterFS Volume Plugin #7319 (wattsteve)
  • Rolling update
  • First part of improved rolling update, allow dynamic next replication controller generation. #7268 (brendandburns)
  • Further implementation of rolling-update, add rename #7279 (brendandburns)
  • Added basic apiserver authz tests. #7293 (ashcrow)
  • Retry pod update on version conflict error in e2e test. #7297 (quinton-hoole)
  • Add “kubectl validate” command to do a cluster health check. #6597 (fabioy)
  • coreos/azure: Weave version bump, various other enhancements #7224 (errordeveloper)
  • Azure: Wait for salt completion on cluster initialization #6576 (jeffmendoza)
  • Tighten label parsing #6674 (kargakis)
  • fix watch of single object #7263 (lavalamp)
  • Upgrade go-dockerclient dependency to support CgroupParent #7247 (guenter)
  • Make secret volume plugin idempotent #7166 (pmorie)
  • Salt reconfiguration to get rid of nginx on GCE #6618 (roberthbailey)
  • Revert “Change kube2sky to use token-system-dns secret, point at https e… #7207 (fabioy)
  • Pod templates as their own type #5012 (smarterclayton)
  • iscsi Test: Add explicit check for attach and detach calls. #7110 (swagiaal)
  • Added field selector for listing pods #7067 (ravigadde)
  • Record an event on node schedulable changes #7138 (pravisankar)
  • Resolve #6812, limit length of load balancer names #7145 (caesarxuchao)
  • Convert error strings to proper validation errors. #7131 (rjnagal)
  • ResourceQuota add object count support for secret and volume claims #6593(derekwaynecarr)
  • Use Pod.Spec.Host instead of Pod.Status.HostIP for pod subresources #6985 (csrwng)
  • Prioritize deleting the non-running pods when reducing replicas #6992 (yujuhong)
  • Kubernetes UI with Dashboard component #7056 (preillyme)

To download, please visit https://github.com/GoogleCloudPlatform/kubernetes/releases/tag/v0.16.0

@Kubernetesio View on Github #kubernetes-users Stack Overflow Download Kubernetes