Kubernetes

This is a smal how-to to setup a kmaster and a knode.

With ubuntu 16.04

just update the OS

sudo su
apt-get update
apt-get upgrade

set hostname

sudo vi /etc/hostname

kmaster

disable swap

sudo swapoff -a

and permanently by editing fstab, only comment the line containing swap

/dev/mapper/surrogate--vg-root /                  ext4    errors=remount-ro 0       1
UUID=61f86b59-899d-4be7-b60f-233ee9158663 /boot   ext2    defaults          0       2
#/dev/mapper/surrogate--vg-swap_1 none            swap    sw                0       0

setup static ip address, first check status

ifconfig

enp0s8    Link encap:Ethernet  HWaddr 08:00:27:ae:17:31
          inet addr:192.168.56.201  Bcast:192.168.56.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:feae:1731/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:111852 errors:0 dropped:0 overruns:0 frame:0
          TX packets:138460 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:11077514 (11.0 MB)  TX bytes:130166521 (130.1 MB)

sudo vi /etc/network/interfaces

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto enp0s3
iface enp0s3 inet dhcp

auto enp0s8
iface enp0s8 inet static
address 192.168.56.201

restart network

sudo /etc/init.d/networking restart

update hosts

sudo vi /etc/hosts

127.0.0.1       localhost
127.0.1.1       kmaster
192.168.56.201 kmaster
192.168.56.202 knode

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

apt-get update && apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl

From here all operations should be done only on kmaster

check cgroup driver

sudo docker info | grep cgroup

if Cgroup Driver is cgroupfs we need to update 10-kubeadm.conf

sudo vi /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

add Environment=“cgroup-driver=systemd/cgroup-driver=cgroupfs”

# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
Environment="cgroup-driver=systemd/cgroup-driver=cgroupfs"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/default/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS

vi /etc/hosts
kubeadm init --apiserver-advertise-address=192.168.56.201 --pod-network-cidr=192.168.0.0/16

you should see the following:


  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join 192.168.56.201:6443 --token usymgb.c5s530zqqs89napy --discovery-token-ca-cert-hash sha256:dadb42d6961e21cba21265dc345c23e1bf33bf9dcd26c96b4dd7eb9b66522614

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

install flannel

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/k8s-manifests/kube-flannel-rbac.yml

check if everything is running

kubectl get pods -o wide --all-namespaces

should look like

NAMESPACE     NAME                                   READY   STATUS    RESTARTS   AGE    IP               NODE      NOMINATED NODE   READINESS GATES
kube-system   coredns-86c58d9df4-hpl5s               1/1     Running   0          2d2h   192.168.0.3      kmaster   <none>           <none>
kube-system   coredns-86c58d9df4-jbvxv               1/1     Running   0          2d2h   192.168.0.2      kmaster   <none>           <none>
kube-system   etcd-kmaster                           1/1     Running   1          2d2h   192.168.56.201   kmaster   <none>           <none>
kube-system   kube-apiserver-kmaster                 1/1     Running   1          2d2h   192.168.56.201   kmaster   <none>           <none>
kube-system   kube-controller-manager-kmaster        1/1     Running   1          2d2h   192.168.56.201   kmaster   <none>           <none>
kube-system   kube-flannel-ds-amd64-q5lgj            1/1     Running   0          44h    192.168.56.202   knode     <none>           <none>
kube-system   kube-flannel-ds-amd64-zjxh8            1/1     Running   0          46h    192.168.56.201   kmaster   <none>           <none>
kube-system   kube-proxy-dj8h7                       1/1     Running   1          2d2h   192.168.56.201   kmaster   <none>           <none>
kube-system   kube-proxy-qwh2x                       1/1     Running   0          44h    192.168.56.202   knode     <none>           <none>
kube-system   kube-scheduler-kmaster                 1/1     Running   1          2d2h   192.168.56.201   kmaster   <none>           <none>
kube-system   kubernetes-dashboard-57df4db6b-2r4th   1/1     Running   0          45h    192.168.0.4      kmaster   <none>           <none>

* install the kubernetes dashboard

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
# get token to login to dashboard
kubectl create serviceaccount dashboard -n default
kubectl create clusterrolebinding dashboard-admin -n default --clusterrole=cluster-admin --serviceaccount=default:dashboard
kubectl get secret $(kubectl get serviceaccount dashboard -o jsonpath="{.secrets[0].name}") -o jsonpath="{.data.token}" | base64 --decode

start dashboard

kubectl proxy

now you can reach the dashbard at

http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy

to reach it from the host os you can tunnel the port 8001 from kmaster to your host

So now the kmaster is running. For the knode you have to repeate all operations above until the bold message which says: From here all operations should be done only on kmaster. Just set a diffrent host knode and a new static IP 192.168.56.202

After all operations were executed successfully do the connection. To join type in the above join statement. This is exactly what you got as message after starting kubernetes on master: Refere to the output above, it is the same!

sudo kubeadm join 192.168.56.201:6443 --token usymgb.c5s530zqqs89napy --discovery-token-ca-cert-hash sha256:dadb42d6961e21cba21265dc345c23e1bf33bf9dcd26c96b4dd7eb9b66522614

Sources:

https://kubernetes.io/

permalink · 2019/01/04 15:30 · Alexej Sailer · 2575 Comments · Tags:

kubernetes

Release / History

License: Apache 2.0
Started by Solomon Hykes at dotCloud
- Jeff Lindsay independent collaborator
Released as open source in March 2013
In the years 2013 – 2016 many organizations started to support Docker
- Cisco, Google, Huawei, IBM, Microsoft, and Red Hat
October 2015, the project had over 25,600 GitHub stars

Source: https://de.wikipedia.org/wiki/Docker_(Software)
Picture Source:
https://commons.wikimedia.org/wiki/File:Docker_(container_engine)_logo.svg

LXC → libcontainer

Operating System Level Virtualization (no Hypervisor)
- based on Linux Containers (LXC)
- March 2014, with release version 0.9 Docker dropped LXC
Development of “libcontainer” started
- cross-system abstraction layer
- Docker execution environment
- builds up a new container specification
- wraps control-groups, namespaces and UnionFS
Similar implementations
- rkt, FreeBSD Jail, OpenVZ, system-nspawn

Picture License: Public Domain
https://commons.wikimedia.org/wiki/File:Docker-linux-interfaces.svg#/media/File:Docker-linux-interfaces.svg

OS manages everything

a container is an isolation by defining limits through kernel features
- Namespaces
  - process ids, hostnames, user ids and network access isolation
- cgroups
  - Resource Management/-Limiting, CPU, memory, disk I/O, network
Docker introduces copy-on-write storage to manage images and containers
Docker aims to simplify and abstract this kernel level operations
a container feels like a VM but exists only due to an isolation mechanism provided by the OS

Docker as an abstraction layer

Software
- Docker daemon
  - listens to requests sent via Docker-Engine-API (REST)
- Docker client
  - uses Docker-Engine-API
Objects
- Images
- Containers
- Services
Registries
- Hub
- Sharing Images
- Public: Docker Hub, Docker Cloud
Platform independent Container Specification

Docker Images and Containers

Images
- built up from a series of layers
- each represents an instruction in a dockerfile or may be a manual configuration
- layers are read only, except the last one
Container
- a read write layer
- to be able to modify data which was read in an image layer, a copy-on-write process must be performed
- copy-on-write copies a file from an image layer to the container layer before the system can modify it
- all modifications may only be done on container layer
Flexible and Fast
- it is possible to start multiple containers of the same kind quickly
- versioning is easy to implement

Docker Volumes

Volumes
- define a volume which will be available in a container as folder
- managed by Docker daemon on host
- enables a sharing mechanism between containers
- flexible, as directly managed by Docker and thus platform independent
- remotely available
- may be started pre-populated in a container, to avoid copy-on-write
Bind Mounts
- directly mount a host folder into a container
- not managed by Docker, not platform independent
Tmpfs mounts
- temporary and only in memory
- useful for sensitive data
- no sharing, only on linux

Registry and Services

Registries enable sharing of images
- push images to a registry to start sharing
- pull images, to start from
- thousands of apps available, each in an single image
- different application versions, pre-configured in images
Public Registries
- Docker Hub, Docker Cloud
Services
- Scale containers across Docker daemons, swarm

Used to interact with Docker daemon

Docker-Toolbox
- works on Windows and Mac
- needs a virtualization driver to load a boot2docker iso
  - Virtualbox
  - Hyper-V
- provides Docker-machine command to remotely interact with the Docker host, which is running in a VM
DockerCLI
- provides commands to interact with the daemon, interacts mostly over Docker-Engine-API
Docker-Compose
- runs multi-container Docker applications
- configure a landscape using a YAML file
- define ports, networks, volumes, environment variables, start order, dependencies
Kubernetes
- Container Orchestration
- automated deployment, scaling, and management