Kubernetes Security

Kubernetes Security Best Practices You Must Know

Ilai Bavati Developer Tips, Tricks & Resources

Kubernetes (k8s) enables you to efficiently orchestrate container management, in the cloud or on-premises. As a whole, k8s provides many benefits, including features for self-healing, automated rollouts and rollbacks, load distribution, and scalability. However, k8s is a highly complex platform and requires extensive configuration. To ensure your k8s operations are secure, you need to address key challenges related to architecture vulnerabilities and platform dependencies, and implement relevant Kubernetes security best practices.

What Is Kubernetes?

Kubernetes (k8s) is an open-source orchestration platform that you can use to automate the deployment and management of containers. It enables developers and IT professionals to maintain, monitor, and scale container deployments on-premises or in the cloud. 

K8s is based on a client-server architecture that uses clusters of nodes (servers or VMs), each hosting a number of pods (containers). Each server and client includes the following components:

  • Server—include the kube-apiserver (validates and configures API data), etcd (a key-value store for configuration information), kube-scheduler (schedules containers for deployment), and a DNS server.
  • Client—includes Docker containers, kubelet (an agent that defines pod configurations), and kube-proxy (a proxy for communications).

Kubernetes is operated through a REST API that enables programmatic management of the platform. It includes features for self-healing, automated rollouts and rollbacks, load distribution, and scalability.

If you are using or plan to use Kubernetes it’s important that you have monitoring in place that can scale with your projects. Stackify Retrace supports containers like Kubernetes and enables continuous application improvements. Learn more here. 

Kubernetes Security Challenges

When using Kubernetes to manage container deployments there are a few challenges you may face. 

Architecture vulnerabilities

The architecture of Kubernetes creates a large and dynamic surface area for attack. Each node and pod is an additional possible entry point for attackers and must be carefully configured. 

Additionally, pods are constantly being created and destroyed to meet workload demands and to maintain deployment health. This makes monitoring and detection of threats a significant challenge. Systems must incorporate service discovery features to ensure that all assets are known – otherwise components are missed. 

The dynamic nature of k8s deployments also create a challenge for maintaining regulatory compliance. Compliance requires logging and auditability that are difficult to achieve with ephemeral containers. For example, to maintain a history of events, you need to export logs from your pods to a persistent data store. However, IP addresses may be reused for pods so your logs may identify two different containers as the same because of a shared IP.


New call-to-action

Platform dependencies

Kubernetes is not a freestanding platform. It requires numerous dependencies and often teams incorporate a range of third-party tooling to help ease management. 

Some of these tools are designed specifically to increase Kubernetes security by centralizing monitoring and detection. However, others may introduce vulnerabilities. This means you need to carefully assess which components k8s relies on and ensure that those components are internally secure and externally monitored and protected. 

Container vulnerabilities

Although containers do have some protection from attacks due to being isolated instances, these assets are not invulnerable. The biggest threats to container security are infected images, improper access controls, and root privileges. These threats remain as long as containers are running.

Infected images may contain malware that is used to override container configurations or to spread infection to persistent systems. Improper access controls may enable services or users to either access restricted data in the container or to leverage containers to access system data. 

Meanwhile, root privileges enable successful attackers to perform just about any action they want with a container. This includes modifying system settings or deploying new containers.

Example of Kubernetes Security Threats to Watch Out For

Below are a few examples of how the above vulnerabilities can be used in an attack:

  • Container compromise—application misconfigurations or vulnerabilities can be leveraged by attackers to gain access to containers. From there, attackers can probe for weaknesses in the file system, process controls, or network.
  • Unauthorized connections between pods—attackers can use a compromised container to communicate with other pods, passing malicious code or requesting sensitive data. 
  • Data exfiltration from a pod—attackers may be able to infect pods with reverse shells, enabling control over command servers and allowing attackers to tunnel into your network. 

Top Kubernetes Security Best Practices

When deploying Kubernetes, there are several best practices that can help you ensure that your deployment is as secure as possible. Below are a few to start with. 

Enable Role-Based Access Controls (RBACs)

RBACs enable you to define permissions by role, making management simpler, and reducing the chance of misconfiguration. These controls are enabled by default in k8s v1.6 and up. However, if you are using an older version of Kubernetes, managed services, or have recently upgraded, you should double-check your settings. 

For RBAC to function correctly, you need to have it enabled and the legacy attribute-based access controls (ABACs) disabled. Once you are sure your controls are enabled correctly, you should prioritize defining namespace-specific permissions over cluster-wide permissions. This helps reduce the chance that attackers are able to move laterally through your clusters if credentials are compromised. 

Harden Kubernetes Nodes

Hardening your k8s nodes restricts attacker access to assets and reduces the chance of compromise. Generally, this involves tightening your configurations and limiting the traffic allowed to or between pods. 

To harden your nodes, you can start by comparing your configurations to the Center for Internet Security (CIS) Benchmarks. These benchmarks provide a step-by-step checklist of Kubernetes security best practices. 

Next, ensure that you are controlling network traffic to sensitive ports. For example, those used by kubelet. You should also consider limiting k8s API access to only trusted networks or requiring authentication and authorization before access is given. 

Turn on Audit Logging

Properly monitoring events in your deployment is vital. You should ensure that you have audit logging enabled and that you are analyzing these logs continuously. In particular, you should be looking for suspicious or unexpected API calls and authorization failures. 

If your deployment is hosted on a public cloud, your provider should make this data available through their console or API. They should also include the ability to alert on certain events. If you are self-hosting, you need to ingest this data into a third-party tool for analysis and alerting. 

Protect Your etcd Cluster

Your etcd cluster is one of the most sensitive assets in your k8s deployment. It contains all of the information on your cluster states, including configurations and secrets. This information makes it an appealing target for attackers. 

When protecting etcd, you need to be mindful of both read and write access permissions. Write access can provide attackers full control over your clusters and read access can be used for reconnaissance or privilege escalation. 

To protect your cluster, you should enable TLS for both client-to-server and server-to-server communications. You should also configure a firewall between your etcd cluster and API server. 

Conclusion

Kubernetes can be incredibly useful for managing containers, but you should properly understand Kubernetes security challenges before adopting Kubernetes. To ensure the continual health of your operations, you need to secure your architecture, platform dependencies, and containers. 

There is a wide range of techniques you can use to secure Kuberenetes workloads. You can start by enabling role-based access controls, to prevent exploitation of privileges. You should also harden Kubernetes nodes, to restrict attacker access to assets. To protect your etcd cluster, you can enable TLS and configure a firewall. Finally, you should turn on audit logging to ensure you have continuous visibility.

About Ilai Bavati

I'm a technology writer and editor based in Tel Aviv. I cover topics ranging from machine learning and cybersecurity to cloud computing and the Internet of Things. I'm interested in the real-world application of emerging technologies, and I see our increasingly connected reality as both disruptive and potentially life-saving.