Kubernetes v1.35.0-beta.0: Gang Scheduling, Workload API, and Major Platform Enhancements

The kubernetes 1.35.0-beta.0 release notes detail the first beta phase of Kubernetes v1.35, released on November 19, 2025. Built with Golang 1.25.3, this beta introduces significant enhancements including the new Workload API for expressing workload-level scheduling requirements, Gang Scheduling plugin for “all-or-nothing” scheduling, and Node Declared Features for publishing node-specific capabilities. The release includes 21 commits to master since v1.35.0-alpha.3, featuring major improvements in Dynamic Resource Allocation (DRA), in-place pod resource resizing, and enhanced security features. This beta phase enables developers and cluster administrators to validate upcoming v1.35 functionality in testing environments before the stable release.

This beta release is recommended for DevOps engineers, Kubernetes administrators, and platform developers who need to validate upcoming features and ensure compatibility with existing infrastructure. Upgrade recommendation: Deploy only to non-production testing environments – this is an unstable development build not suitable for production workloads.

What Changed

Major New Features

Workload API (Alpha): New scheduling.k8s.io/v1alpha1 API for expressing workload-level scheduling requirements, allowing kube-scheduler to act on collective scheduling constraints
Gang Scheduling Plugin (Alpha): Enables “all-or-nothing” scheduling using the new Workload API to ensure entire pod groups are scheduled together or not at all
Node Declared Features (Alpha): Nodes can now publish node-specific features via Node.Status.DeclaredFeatures field, with corresponding scheduler and admission plugins for feature matching
In-Place Pod Resource Resize: Support for resizing pod-level resources with new Resources and AllocatedResources fields in PodStatus
Mutable PV Node Affinity: PersistentVolume node affinity can now be modified after creation
Numeric Toleration Operators: Extended core/v1 Toleration to support numeric comparison operators (Gt, Lt) for taint values

Graduated Features

Pod Certificates (Beta): Moved to beta with new UserAnnotations field in PodCertificateProjection API (feature gate still disabled by default)
EnvFiles (Beta): Promoted to beta and enabled by default with restricted POSIX shell syntax (all values must be wrapped in single quotes)
KubeletEnsureSecretPulledImages (Beta): Now beta and enabled by default for improved image security
ImageGCMaximumAge (Stable): Graduated to stable for improved image garbage collection control
Image Volume Source (Beta): Now beta and enabled by default
PodTopologyLabelsAdmission (Beta): Enabled by default – pods automatically receive topology.kubernetes.io/zone and topology.kubernetes.io/region labels
Fine-grained Supplemental Groups (GA): KEP-3619 graduated to general availability

Dynamic Resource Allocation (DRA) Enhancements

DRA Extended Resources: ResourceQuota now counts device class requests with new quota types for device classes and extended resources
DRA Device Taints: Separate DRADeviceTaintRules feature gate for controlling DeviceTaintRules support
Partitionable Devices Updates: Support for referencing counter sets across ResourceSlices with backwards incompatible changes (requires cleanup before upgrade)
DRA Health Monitoring: Configurable per-device health check timeouts
Extended Resource Scoring: Implement scoring for DRA-backed extended resources in scheduler

Scheduler & Workload Management

Opportunistic Batching (KEP-5598): Optimize scheduling for pods with identical scheduling requirements
Mutable Scheduling for Suspended Jobs (Alpha): Allow mutation of Job scheduling directives when suspended, with automatic status.startTime clearing
Job Resource Resizing (Alpha, KEP-5440): Allow resizing of resources while job is suspended
Restart All Containers (Alpha): Enable restarting all containers when source container exits with matching restart policy

Security & Authentication

Kuberc Configuration Schema: New optional fields credPluginPolicy and credPluginAllowlist for credential plugin management (KEP-3104)
Trusted Types API: Enabled for preventing cross-site scripting attacks

Platform & Runtime

User Namespaces with Host Network (Alpha): New UserNamespacesHostNetworkSupport feature gate allows hostNetwork pods to use user namespaces
WatchListClient (Beta): Feature gate now enabled by default
KubeletCrashLoopBackOffMax (Beta): Promoted to beta and enabled by default
CoreDNS: Upgraded to v1.12.4
Go Version: Built with Golang 1.25.4

Why It Matters

Gang Scheduling enables complex multi-pod workloads (AI/ML training, distributed databases) to schedule atomically, preventing resource deadlocks and partial deployments
Workload API provides a standard way to express collective scheduling requirements, replacing custom controller patterns
Node Declared Features allows hardware-specific capabilities (GPUs, specialized accelerators) to be advertised and matched without manual node labeling
In-Place Resource Resize reduces pod disruption by allowing resource adjustments without pod restarts
DRA Enhancements improve dynamic resource allocation for specialized hardware with better quota management and health monitoring
Beta Promotions signal production-readiness for features like EnvFiles, PodTopologyLabels, and ImageVolumes

Who Should Upgrade

This release is particularly relevant for:

Platform Engineers building Kubernetes distributions or internal platforms who need to validate v1.35 compatibility
AI/ML Infrastructure Teams requiring gang scheduling for distributed training workloads
Hardware Vendors implementing DRA drivers for specialized accelerators
Cluster Administrators testing migration paths for upcoming stable features
DevOps Teams evaluating new scheduling capabilities for complex workload patterns

How to Upgrade

Backup Critical Data: Export cluster state, etcd snapshots, and configuration files
Review Breaking Changes:
- Partitionable Devices users must remove existing ResourceSlices before upgrade
- EnvFiles syntax now restricted to single-quoted POSIX shell syntax
- Networking v1beta1 Ingress/IngressClass support dropped in kubectl
Deploy to Isolated Test Cluster: Use dedicated non-production environment

Verify Component Compatibility:

kubectl version --output=yaml
kubectl get nodes -o wide

Test Critical Workloads: Validate existing deployments, StatefulSets, and Jobs
Enable Feature Gates: Configure alpha features as needed in component configs
Monitor Metrics: Watch for new metrics like scheduler_resourceclaim_creates_total and kubelet_imagemanager_*
Review Logs: Check for deprecation warnings and API compatibility issues

Usage Examples

Gang Scheduling with Workload API

apiVersion: scheduling.k8s.io/v1alpha1
kind: Workload
metadata:
  name: distributed-training
spec:
  podSets:
  - count: 4
    name: workers
  schedulingPolicy:
    gangScheduling: {}

Node Declared Features

# Node publishes features
status:
  declaredFeatures:
  - name: nvidia.com/gpu-a100
  - name: custom.io/specialized-accelerator

# Pod requests features
spec:
  nodeSelector:
    feature.node.kubernetes.io/nvidia.com/gpu-a100: "true"

Numeric Taint Tolerations

spec:
  tolerations:
  - key: "gpu-memory-gb"
    operator: "Gt"
    value: "40"
    effect: "NoSchedule"

In-Place Pod Resource Resize

# Pod resources can now be updated without restart
spec:
  containers:
  - name: app
    resources:
      requests:
        cpu: "2"  # Can be changed in-place
        memory: "4Gi"

Breaking Changes

Partitionable Devices: Backwards incompatible changes require removal of ResourceSlices using the feature before upgrading between 1.34 and 1.35
EnvFiles Syntax: Environment variable values must now be wrapped in single quotes (POSIX shell subset)
kubectl Deprecations: Dropped support for networking/v1beta1 Ingress and IngressClass
AggregatedDiscoveryRemoveBetaType: Feature gate deprecated and locked to True

Known Issues

Beta Stability: This is a pre-release version not recommended for production environments
Alpha Features: Gang Scheduling, Node Declared Features, and In-Place Resize are alpha quality and may have undiscovered bugs
API Changes: Alpha APIs may change incompatibly in future releases
Testing Required: Comprehensive validation needed before considering for production use

Bug Fixes

Fixed Windows kube-proxy issue with stale RemoteEndpoints when Deployments referenced by multiple Services
Fixed nil pointer exception in ValidatingAdmissionPolicy with additionalProperties:true schemas
Fixed restart policy issues on static pods
Fixed concurrent map write error in kubelet with LocalStorageCapacityIsolationFSQuotaMonitoring
Fixed Device Manager health status affecting system watchdog
Fixed extended resource allocation for initContainers using automatic ResourceClaims

New Metrics

scheduler_resourceclaim_creates_total: Track DRAExtendedResource claim creation
resourceclaim_controller_resource_claims: New source label added
kubelet_imagemanager_ondisk_pullintents: Track pull intent records on disk
kubelet_imagemanager_ondisk_pulledrecords: Track image pulled records on disk
kubelet_imagemanager_image_mustpull_checks_total: Track image pull record checks

Official Release Notes

View full release notes on GitHub →

Read detailed changelog →

Release Information:

Version: v1.35.0-beta.0
Release Date: November 19, 2025
Golang Version: 1.25.4
Git Commit: 54900791c46f14fdd009102275e05e3510adc474
GPG Signature: Verified with key B5690EEEBB952194

Important: Always refer to the official CHANGELOG-1.35.md for complete technical details and upgrade considerations.

Kubernetes v1.35.0-beta.0: Beta Feature Testing