Kubernetes 1.27 is finally here and it's giving.. chill vibes?

First 2023 release

First things first, let’s address the sloth in the room because, just like us, you are probably also wondering what that is about. According to the release team, this has been a very calm release and the first one with no exception requests after the enhancements freeze.The team worked hard to make this release a smooth process and it worked, hence the name and the sloth on the logo.

Now let’s get into the subject! This first release of 2023 consists of 60 enhancements: 

  • 18 of those enhancements are entering Alpha
  • 29 are graduating to Beta
  • 13 are graduating to Stable

Let’s take a look at its most interesting features!

1. Freeze k8s.gcr.io image registry [Stable]

The Kubernetes image registry has moved from k8s.gcr.io  to registry.k8s.io.

The previous registry was hosted in Google Cloud, who had been providing  its own Google Container Registry (GCR) for that purpose. But with the increased popularity of K8s, relying on a single cloud provider  became insufficient. Nowadays the world runs on multi-cloud, which creates the necessity for a registry with a distributed system that is closer to your datacenter, and that spreads the load across multiple Cloud Providers & Regions.

This new registry Kubernetes can provide higher availability regardless of the cloud provider you use.

What does this mean for you?

  • k8s.gcr.io will be frozen and will not be updated with new images after April 3rd, 2023
  • k8s.gcr.io will not get any new releases, patches, or security updates
  • It will remain available to help people migrate, but it WILL be phased out entirely in the future
  • From 1.27 onward, container images for K8s releases are published only to registry.k8s.io
  • If you run in a restricted environment and apply strict domain/IP address access policies limiting to k8s.gcr.io only, the image pulls will not function after the migration to the new registry. For these users, the recommended method is to mirror the release images to own private registry

Benefits:

  • The project’s reliance on a single entity will be reduced
  • You no longer have to depend on a legacy service 
  • Image pulls will be faster (for clients running inside AWS, image layers will usually be served from S3)
  • The download experience will be faster for a large number of users

You can check if your cluster have dependencies on the old image registry by running the following command:

kubectl get pods — all-namespaces -o

jsonpath=”.{items[*].spec.containers[*].image}” |\

tr -s ‘[[:space:]]’ ‘\n’ |\

sort|\

uniq-c

2. SeccompDefault [graduates to stable]

The kubelet now has an option to enable a default seccomp profile for workloads that do not specify a seccomp profile. By being able to execute Containers using a Seccomp profile by default, the security of your containers will be increased while preserving the functionality of the workload. It is possible that the default profiles differ between container runtimes and their release versions.

To use seccomp profile defaulting:

  1. Run the kubelet with the –seccomp-default command line flag enabled for each node where you want to use it.
  2. If enabled, the kubelet will use the RuntimeDefault seccomp profile by default, which is defined by the container runtime, instead of using the Unconfined (seccomp disabled) mode.

3. VolumeGroupSnapshot [graduating to alpha]

We already have a VolumeSnapshot API that allows us to take a snapshot of a persistent volume to protect against data loss or data corruption. However, VolumeGroupSnapshot proposes a Kubernetes API that allows users to take a crash consistent snapshot of multiple volumes together.

This is a major improvement for disaster recovery and security research. Volumes will be backed up more consistently so that the snapshots can represent the state of the Pod more accurately and prevent data loss.

CSI drivers must support the CREATE_DELETE_GET_VOLUME_GROUP_SNAPSHOT capability in order to perform a VolumeGroupSnapshot.

4. Mutable scheduling directives for suspended Jobs [Graduates to GA]

A Job creates one or more Pods and will continue to retry execution of the Pods until a specified number of them successfully terminate. You can also use a Job to run multiple Pods in parallel. 

This enhancement makes node affinity mutable for jobs, it was introduced in v1.22 at beta level and is now stable.

A parallel job will usually want the pods to run with constraints, like all in the same zone, or all either on GPU model x or y but not a mix of both. The suspend field is the first step towards achieving those semantics. suspend allows a custom queue controller to decide when a job should start. However, once a job is unsuspended, a custom queue controller has no influence on where the pods of a job will actually land.

You can update the node affinity, node selector, tolerations, labels, and annotations fields in a Job’s pod template before it starts. That allows you to influence where the Pods will run (all in the same zone vs. in nodes with the same GPU model) at the same time offloading actual pod-to-node assignment to kube-scheduler. This is allowed only for suspended Jobs that have never been unsuspended before.

5. In-Place Update of Pod Resources

This feature allows you to update Pod’s container resources without having to restart the Pod. It will mostly benefit operators dealing with workloads, such as database clusters, that do not deal well with restarts and might need resource tuning from time to time.

What are the implications?

Now Kubernetes schedulers and monitoring tools alike will also need to look at the new fields in the PodStatus to properly evaluate available resources in a node.

The current fields in the pod spec, Pod.Spec.Containers[i].Resources, become a declaration of the desired state of pod resources. Meanwhile, new fields are added to the Pod.Status sub-resource:

  • ContainerStatuses[i].ResourcesAllocated: Describe the resources allocated to the containers.
  • ContainerStatuses[i].Resources: For the actual resources held by the containers.
  • Resize: With details on resource resizing operations progress.

The containers spec in the Pod also can specify ResizePolicy for CPU and memory, with possible values Restart and RestartNotRequired, detailing whether or not a restart of the container is needed to properly apply new values.

6. Multiple Service CIDRs

When you have to deal with a big cluster with lots of services, it is best if you do not have to manage arbitrary limits on the internal IPs of the cluster.

Some limits have been removed in a way that will provide better insights when querying the cluster resources. This will benefit all cluster admins.

This enhancement comprises the work done to refactor how internal IP addresses are assigned to Services. This will remove the current limits on IP ranges (service-cluster-ip-range). It will also make it possible to inspect the IP addresses assigned to their Services using 

kubectl:

$ kubectl get services

NAME         TYPE        CLUSTER-IP        EXTERNAL-IP   PORT(S)   AGE

kubernetes   ClusterIP   2001:db8:1:2::1   <none>        443/TCP   3d1h

$ kubectl get ipaddresses

NAME              PARENTREF

2001:db8:1:2::1   services/default/kubernetes

2001:db8:1:2::a   services/kube-system/kube-dns

Code language: Bash (bash)

 

This has been implemented by replacing the current etcd allocation map with the use of IPAddress objects that will be referenced from each Service object.

6. Deprecated/ removed APIs and features + alternatives

A few beta APIs and features have been deprecated or removed in Kubernetes 1.27, including:

Deprecated Alternative
Kubeadm v1beta2
kubeadm config migrate (can be used to migrate to v1beta3 )
resource.k8s.io/v1alpha1.PodScheduling
resource.k8s.io/v1alpha2.PodSchedulingContext
DynamicResourceManagement v1alpha1
v1alpha2
CSIStorageCapacity:  Storage.k8s.io/v1beta1
v1
seccomp.security.alpha.kubernetes.io/pod and  container.seccomp.security.alpha.kubernetes.io annotations 
securityContext.seccompProfile field
service.kubernetes.io/topology-aware-hints
SecurityContextDeny admission plugin

 Removed:

  • storage.k8s.io/v1beta1 from CSIStorageCapacity
  • support for deprecated seccomp annotations
  • –master-service-namespace command line argument
  • ControllerManagerLeaderMigration feature gate
  • –enable-taint-manager command line argument
  • –pod-eviction-timeout command line argument
  • CSI Migration feature gate
  • CSIInlineVolume feature gate
  • EphemeralContainers feature gate
  • LocalStorageCapacityIsolation feature gate
  • NetworkPolicyEndPort feature gate
  • StatefulSetMinReadySeconds feature gate
  • IdentifyPodOS feature gate
  • DaemonSetUpdateSurge feature gate

Conclusion

With this release 1.27, the Kubernetes community continues to enhance and refine the platform, addressing user needs and advancing the state of container orchestration.

The featured enhancements in this release contribute to improved security, flexibility, disaster recovery, resource management, and cluster scalability.

We look forward to next Kubernetes releases and already planning to support our customers in day-2 operations involving K8s upgrades.

Stay up to date with the latest developments and relevant news in the world of cloud computing and Kubernetes by following our LinkedIn page. Join our growing community of technology enthusiasts and professionals here!