[Enhancement] Expose sidecar resources in the add-on configuration

[NicholasRaymondiSpot]

Is your feature request related to a problem? Please describe.
In our current helm chart deployment, we provide overrides for some of our sidecar containers:

  sidecars:
    provisioner:
      resources:
        requests:
          cpu: 50m
          memory: 64Mi
        limits:
          memory: 512Mi
    attacher:
      resources:
        requests:
          cpu: 20m
    livenessProbe:
      resources:
        requests:
          cpu: 20m
          memory: 32Mi
        limits:
          memory: 256Mi

However the current EBS add-on configuration schema only allows for setting controller & node resource overrides. The others only expose additionalArgs:

"sidecars": {
    "additionalProperties": false,
    "properties": {
      "attacher": {
        "additionalProperties": false,
        "properties": {
          "additionalArgs": {
            "default": [],
            "description": "Additional arguments passed to the attacher container",
            "items": {
              "type": "string"
            },
            "type": "array"
          }
        },
        "type": "object"
      },
      "livenessProbe": {
        "additionalProperties": false,
        "properties": {
          "additionalArgs": {
            "default": [],
            "description": "Additional arguments passed to the livenessprobe container",
            "items": {
              "type": "string"
            },
            "type": "array"
          }
        },
        "type": "object"
      },
      "provisioner": {
        "additionalProperties": false,
        "properties": {
          "additionalArgs": {
            "default": [],
            "description": "Additional arguments passed to the provisioner container",
            "items": {
              "type": "string"
            },
            "type": "array"
          }
        },
        "type": "object"
      }

Describe the solution you'd like in detail
Please add resource properties to each of the sidecar containers so we can manage the resource requests & limits for these deployments.

Describe alternatives you've considered
We're sticking with Helm charts for now, removing these resources causes issues when our clusters take on heavy workloads.

Additional context
N/A

[torredil]

Hi there @NicholasRaymondiSpot

Thank you for the request. We plan to expose these parameters in the 1.41 release.

/retitle [Enhancement] Expose sidecar resources in the add-on configuration
/kind feature