First off, if you have a few spare physical machines; or machines with enough resources to run a few well resourced VMs it is fairly simple to install Kubernetes itself. I have a fairly old OpenStack stack deployment yaml file that I still use to throw-up\/tear-down multicompute node environments under OpenStack if I need something more powerful than MiniKube, but MiniKube is all you really need for development\/testing on a regular basis.<\/p>\n
However this post is on minikube<\/b>. MiniKube is the best tool for testing out Kubernetes for a home lab if you are comfortable running everything on one machine if you have one powerful enough. Minikube provides a working environment including mutiple nodes (or course on the local machine) if required.<\/p>\n
What is a powerful enough machine is a matter of debate; for example to test istio<\/em> it is recomended to use 6 cpus and 8Gb of memory; I had no trouble with 2 cpus and a physical machine with only 6Gb of memory and only a wireless interface for downloads, running all examples and kiali (it was slow, bit everything worked).<\/p>\n As a general rule you should probably allocate as much resource as you can, especially as minikube can run multiple nodes if you wish to by simply passing a command line flag to the start command.<\/p>\n One important thing to note about this post. I run minikube on a machine running docker using the docker driver. I strongly recomend you do the same so you can use docker commands to manage images in the cluster as discussed in some of the tips and tricks later on<\/em>.<\/p>\n This post is about a few of the tips and tricks I have picked up using it.<\/p>\n For things I am testing (or even keeping) I prefer to keep them in their individual versioned directories where possible; for that reason I skip the steps some installers want of copying things to \/usr\/local\/bin as you would only do that if you wanted every user on your machine to use them plus do not want aliases in your global profile. One advantage is that you can easily have multiple versions and just update the aliases.<\/p>\n We will start off with configuring it to be useful. Note that I install everything under a directory ~\/installs\/kubernetes; you can place it in any directory of you choice.<\/p>\n For things I am testing (or even keeping) I prefer to keep them in their individual versioned directories where possible; for that reason I skip the steps some installers want of copying things to \/usr\/local\/bin as you would only do that if you wanted every user on your machine to use them plus do not want aliases in your global profile.<\/p>\n I just refer to the commands by aliases. So add the below lines to your ~\/,bashrc (if using bash) or the profile file of whatever shell you use. Note the alias entry for ‘kubectl’, most documentation will recomend you download the latest copy of kubectl but as minikube has its own copy built in which is at the correct version for minikube you should use that copy, an example is the last of the three aliases shown below allowing the command ‘kubectl’ to be used at te terminal so copy\/paste from websites you are interested in will work.<\/p>\n Right, we are ready to start things up. Remember to ‘source ~\/.bashrc’ (or start a new shell)<\/p>\n At this point just occasionally use the command ‘kubectl get pod -A’. Wait until all pods are running before continuing.<\/p>\n Then you want istio installed<\/p>\n At this point just occasionally use the command ‘kubectl get pod -A’. Wait until all pods are running before continuing.<\/p>\n Lets add some of the whizzy-bang tools you will want to play with to monitor\/visualize what you deploy now<\/p>\n For istio to be injected into pods you must set a label on each namespace you want istio used in, for playing about you will probably use the ‘default’ namespace so enter<\/p>\n At this point you will probably want to test some of your own deployments. One additional tool I would suggest is a very strict kubernetes yaml file checker. That can be installed into its own directory and aliased as were the other commands<\/p>\n Now, you may want to use a local docker registry for images; good luck with that !.<\/b><\/p>\n There probably is a way to tell minikube to lookup local dns, its internal dns is perfectly able to resolve the internet addresses needed to download the images it needs to run, but it ignores the local host \/etc\/hsosts file and dns settings by default. Even if it could be overridden most 'local' docker registries are insecure so could not be used easily anyway.<\/p>\n However this is where the benefits of running minikube on a machine running docker come into play. <\/p>\n MiniKube has a 'minikube load xxx.tar' command where you can load into the cluster images you can manually save from your local docker repository and copy across to the machine running minikube to load; as an example (same machine running docker and minikube using that docker as the driver).<\/p>\n Important: a image loaded with 'minikube load xxx.tar' will not be shown with a 'minikube image ls' command<\/b>. It is available and will be used by your containers, the pod logs will show 'image already present on local machine' when the pod starts; it seems to be invisible in cache until then.<\/p>\n However if your machine runs docker you can easily switch it from managing the machines docker instance to the kubernetes docker instance with the simple command 'eval $(minikube docker-env)'<\/b> which allows you to use normal docker commands directly against the image within the minikube cluster<\/em> as shown below where I switch the environment.<\/p>\n You can use ordinary docker commands against images within the minikube kubernetes cluster at this point; for example 'docker image rm 83e6a8464b84' will remove the image; although you should probably use 'minikube image rm' and just use docker to check.<\/p>\n Do not expect docker images you download from dockerhub to run under kubernetes without modification. There are design issues to take into consideration, personally all my containers get an environment variable passed to them to indicate which application startup login chain to take. You may be able to get them to run if you set the kubernetes parameters for the container runasuser\/runasgroup to 0 (if kubernetes allows such a thing) but that's obviously not ideal.<\/p>\n So create your own containers, or stick to kubernetes repositories not dockerhub ones until you know how to customise them.<\/p>\n To remove everything again, another benefit of keeping everything under its own directory structure is how easy it is to remove.<\/p>\n First off, if you have a few spare physical machines; or machines with enough resources to run a few well resourced VMs it is fairly simple to install Kubernetes itself. I have a fairly old OpenStack stack deployment yaml file … Continue reading Installing MiniKube and additional components<\/h2>\n
\r\n# --- I keep everything under one directory and use aliases to run them\r\nINST_DIR=\"~\/installs\/kuberernetes\"\r\nmkdir -p ${INST_DIR}\r\n# --- get minikube\r\nmkdir -p ${INST_DIR}\/kuberernetes\/minikube\r\ncd ${INST_DIR}\/kuberernetes\/minikube\r\ncurl -LO https:\/\/storage.googleapis.com\/minikube\/releases\/latest\/minikube-linux-amd64\r\n# --- get istio\r\nmkdir -p ${INST_DIR}\/kuberernetes\/istio\r\ncd ${INST_DIR}\/kuberernetes\/istio\r\n# check https:\/\/github.com\/istio\/istio\/releases for the latest release available ---\r\n# I am using 1.10.1 which is the latest at the time I write this\r\nwget https:\/\/github.com\/istio\/istio\/releases\/download\/1.10.1\/istio-1.10.1-linux-amd64.tar.gz\r\ntar -zxvf istio-1.10.1-linux-amd64.tar.gz\r\n\/bin\/rm istio-1.10.1-linux-amd64.tar.gz\r\n<\/pre>\n\r\nalias minikube=\"\/home\/mark\/installs\/kubernetes\/minikube\/minikube-linux-amd64\"\r\nalias istioctl=\"\/home\/mark\/installs\/kubernetes\/istio\/istio-1.10.1\/bin\/istioctl\"\r\nalias kubectl='minikube kubectl --'\r\n<\/pre>\n
\r\ncd ~\r\nminikube start --cpus 6 --memory 8192\r\n<\/pre>\n
\r\nistioctl install\r\n<\/pre>\n
\r\nkubectl apply -f istio\/istio-1.10.1\/samples\/addons\/grafana.yaml\r\nkubectl apply -f istio\/istio-1.10.1\/samples\/addons\/jaeger.yaml\r\nkubectl apply -f istio\/istio-1.10.1\/samples\/addons\/kiali.yaml\r\nkubectl apply -f istio\/istio-1.10.1\/samples\/addons\/prometheus.yaml\r\n<\/pre>\n
\r\nkubectl label namespace default istio-injection=enabled\r\n<\/pre>\n
\r\nmkdir -p ~\/installs\/kuberernetes\/kube-score\r\ncd ~\/installs\/kuberernetes\/kube-score\r\n# ---- check https:\/\/github.com\/zegl\/kube-score\/releases for the latest release available ---\r\nwget https:\/\/github.com\/zegl\/kube-score\/releases\/download\/v1.11.0\/kube-score_1.11.0_linux_amd64.tar.gz\r\ntar -zxvf kube-score_1.11.0_linux_amd64.tar.gz\r\nalias kube-score=\"\/home\/mark\/installs\/kubernetes\/kube-score\/kube-score\" # << and add to ~\/.bashrc with the other aliases\r\n# usage kube-score score xxxx.yaml\r\n<\/pre>\n
Loading images into MikiKube<\/h2>\n
\r\n[mark@hawk ~]$ docker image list\r\nREPOSITORY TAG IMAGE ID CREATED SIZE\r\ngcr.io\/k8s-minikube\/kicbase v0.0.23 9fce26cb202e 10 days ago 1.09GB\r\ndocker-local:5000\/portainer-ce latest 96a1c6cc3d15 4 months ago 209MB\r\nportainer\/portainer-ce latest 96a1c6cc3d15 4 months ago 209MB\r\nlocalhost\/mvs38j latest 1df77f61cbed 6 months ago 787MB\r\n[mark@hawk ~]$ docker image save localhost\/mvs38j > mvs38j.tar # <-- save from docker \r\n[mark@hawk ~]$ minikube image load mvs38j.tar # <-- load to minikube\r\n<\/pre>\n
\r\n[mark@hawk ~]$ docker image list # <--- local machine, not many\r\nREPOSITORY TAG IMAGE ID CREATED SIZE\r\ngcr.io\/k8s-minikube\/kicbase v0.0.23 9fce26cb202e 10 days ago 1.09GB\r\ndocker-local:5000\/portainer-ce latest 96a1c6cc3d15 4 months ago 209MB\r\nportainer\/portainer-ce latest 96a1c6cc3d15 4 months ago 209MB\r\nlocalhost\/mvs38j latest 1df77f61cbed 6 months ago 787MB\r\n[mark@hawk ~]$ \r\n[mark@hawk ~]$ eval $(minikube docker-env) # <--- switch to minikube environment\r\n[mark@hawk ~]$ docker image list # <--- and we see lots of images\r\nREPOSITORY TAG IMAGE ID CREATED SIZE\r\nistio\/proxyv2 1.10.1 5c66e8ac89a7 2 weeks ago 282MB\r\nistio\/pilot 1.10.1 07d6b563f74b 2 weeks ago 217MB\r\nquay.io\/kiali\/kiali v1.34 1d3ab1649f0b 5 weeks ago 194MB\r\nk8s.gcr.io\/kube-proxy v1.20.7 ff54c88b8ecf 5 weeks ago 118MB\r\nk8s.gcr.io\/kube-apiserver v1.20.7 034671b24f0f 5 weeks ago 122MB\r\nk8s.gcr.io\/kube-controller-manager v1.20.7 22d1a2072ec7 5 weeks ago 116MB\r\nk8s.gcr.io\/kube-scheduler v1.20.7 38f903b54010 5 weeks ago 47.3MB\r\ngcr.io\/k8s-minikube\/storage-provisioner v5 6e38f40d628d 2 months ago 31.5MB\r\ngrafana\/grafana 7.4.3 c9e576dccd68 3 months ago 198MB\r\njimmidyson\/configmap-reload v0.5.0 d771cc9785a1 4 months ago 9.99MB\r\nprom\/prometheus v2.24.0 53fd5ed1cd48 5 months ago 173MB\r\nlocalhost\/mvs38j latest 1df77f61cbed 6 months ago 787MB\r\nkubernetesui\/dashboard v2.1.0 9a07b5b4bfac 6 months ago 226MB\r\njaegertracing\/all-in-one 1.20 84b5c715abd0 8 months ago 45.7MB\r\nk8s.gcr.io\/etcd 3.4.13-0 0369cf4303ff 9 months ago 253MB\r\nk8s.gcr.io\/coredns 1.7.0 bfe3a36ebd25 12 months ago 45.2MB\r\nkubernetesui\/metrics-scraper v1.0.4 86262685d9ab 14 months ago 36.9MB\r\nk8s.gcr.io\/pause 3.2 80d28bedfe5d 16 months ago 683kB\r\n[mark@hawk ~]$ \r\n<\/pre>\n
Important notes<\/h2>\n
Cleaning it all up again<\/h2>\n
\n