The workload active.spec field does not work properly for StatefulSet

[mimowo]

What happened:

When the spec.active field is switched to false the workload is getting deleted. As the result the workload is re-created with spec.active=true, and re-admitted. This makes the field useless for users, and will also confuse Kueue when trying to deactivate the Workload, for example due to waitForPodsReady.

What you expected to happen:

The workload should not be deleted after setting the field to false.

How to reproduce it (as minimally and precisely as possible):

Create the Stateful set, for example:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nginx-statefulset
  labels:
    app: nginx
    kueue.x-k8s.io/queue-name: user-queue
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: registry.k8s.io/nginx-slim:0.26
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: "100m"
  serviceName: "nginx"

2. Find the created workload and edit the field to false with k edit wl
3. Observe that the workload gets deleted, and the re-created workload is re-admitted.

Anything else we need to know?:

I believe the correct solution is to make sure the Workload is owned by the StatefulSet, not just the StatefulSet pods.

[mimowo]

/cc @varshaprasad96 @mbobrovskyi
Created the issue independent of the integration with Notebook. This is problematic for manual suspend, and automated by Kueue using waitForPodsReady (but also ProviRequest).

[mbobrovskyi]

/assign