Exploring Kubernetes with Minikube


Here at Hosco, we’ve been using Kubernetes in production for over a year. We started with Rancher and then moved to EKS from AWS. For us, it’s all about the Netflix mindset: “You build it, you run it.”

Some months ago, we decided to move away from using docker-compose for our local environments and migrate to Minikube, a small and lightweight Kubernetes cluster.

Why? To overcome a fear of the unknown. A lot of us were scared of Kubernetes. Our day-to-day lives didn’t involve a lot of manipulation of our Kubernetes cluster and there was a certain level of apprehension towards working with it.

We know that the best way to become comfortable with something is to use it every day… that’s why we do daily deployments and run our CI each time. And that’s why we decided to bring Kubernetes into our local development environments.

Why Minikube?

Kubernetes is like an elephant… it takes up a lot of space. Minikube allows you to have the functionality of Kubernetes without sacrificing enormous amounts of memory and CPU. It can be installed on your computer on just one node.

Minikube is also quick to get going. We shut down our development environments for the night and start them again in the morning. Regularly restarting regular Kubernetes clusters can be a headache but Minikube can be started and stopped whenever you want in seconds using the CLI.

Sounds great, right? Just don’t be tempted to use it for production.

How is it configured?

Minikube will use a driver to create a new virtual machine on your computer and then install Kubernetes inside this virtual instance. We needed to increase the memory limit (minikube config set memory 8192) in order to deploy both Elasticsearch and Postgres.

Then you can begin the configuration.

We use two docker registries: gitlab and AWS ECR. Again, it’s simple to do with CLI: minikube addons enable registry-creds.

-- Enter AWS Access Key ID: <your access key> (`more .aws/credentials`)
-- Enter AWS Secret Access Key: <your secret key> (`more .aws/credentials`)
-- (Optional) Enter AWS Session Token: <empty>
-- Enter AWS Region: <region>
-- Enter 12 digit AWS Account ID (Comma separated list): <aws_account_id>
-- (Optional) Enter ARN of AWS role to assume: <empty>
Do you want to enable Google Container Registry? [y/n]: n
Do you want to enable Docker Registry? [y/n]: y
-- Enter docker registry server url: registry.gitlab.com
-- Enter docker registry username: <your Gitlab username/email>
-- Enter docker registry password: <your Gitlab password or personal access token(if you have 2FA enabled)>

This will create a new pod inside the kube-system namespace called registry-creds. Here you’ll have two secrets awsecr-cred and dpr-secret. You’ll need to add the right one (awsecr-cred if you use ECR registry and dpr-secret if you use Gitlab) to your yaml file. Indeed, Minikube is like Kubernetes: to deploy or create something, you’ll need a yaml file and then: kubectl apply -f <your_yaml_file>.yml.

And that’s it! We can now start Minikube: minikube start.

Discover Minikube

Minikube comes with a great CLI:

  • minikube dashboard - access a UI dashboard
  • minikube stop to stop your Minikube instance
  • minikube delete to delete your Minikube instance and the virtual machine

You can now apply and create your pod, deployment, secret, … By default, Minikube doesn’t expose anything to your computer.

You can expose to a port from Minikube on your PC in two ways: Using a port forward. For example kubectl port-forward --address=0.0.0.0 service/web-hoscov2 80:80 will expose a service named web-hoscov2. Using Minikube service with minikube service <service_name>. When you run this command, Minikube will expose the service on a port on your computer.

(⎈ |minikube:default)➜  ~ minikube service web-hoscov2
|-----------|-------------|-----------------------------|
| NAMESPACE |    NAME     |             URL             |
|-----------|-------------|-----------------------------|
| default   | web-hoscov2 | http://192.168.99.105:31237 |
|-----------|-------------|-----------------------------|
🎉  Opening kubernetes service  default/web-hoscov2 in default browser...

We had one issue with Minikube to expose a service with the port-forward method : https://github.com/kubernetes/minikube/issues/1568

kind: Service
apiVersion: v1
metadata:
  ***
spec:
  clusterIP: None # https://github.com/kubernetes/minikube/issues/1568
  selector:
    ***
  ports:
    - ***

Conclusion

We’ve been using Minikube for a few months now and, despite some initial resistance, we’re converts. Using Minikube has allowed us to deepen our understanding of Kubernetes without fear of ‘breaking Hosco’. We’re confident that our Kubernetes cluster is now in even safer hands.

So what’s next? We want to continue to deploy more and more microservices locally and experiment more with Minikube. Next step… a service mesh?!