Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs/design.md #2242

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions docs/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,66 @@ Blindly return:
- STAGE\_UNSTAGE\_VOLUME
```

## Coalescing ControllerExpandVolume & ControllerModifyVolume

### EC2 ModifyVolume and Request Coalescing

AWS exposes one unified ModifyVolume API to change the size, volume-type, IOPS, or throughput of your volume. AWS imposes a 6-hour cooldown after a successful volume modification.

However, the CSI Specification exposes two separate RPCs that rely on ebs-plugin calling this EC2 ModifyVolume API: ControllerExpandVolume, for increasing volume size, and ControllerModifyVolume, for all other volume modifications. To avoid the 6-hour cooldown, we coalesce these separate expansion and modification requests by waiting for up to two seconds, and then perform one merged EC2 ModifyVolume API Call.

Here is an overview of what may happen when you patch a PVC's size and VolumeAttributesClassName at the same time:

```mermaid
sequenceDiagram
participant o as Kubernetes
box ebs-csi-controller Pod
participant s as csi-resizer
participant p as ebs-plugin <br/> (controller service)
end
participant a as AWS API
participant n as ebs-plugin <br/> (node service)

o->>s: Updated PVC VACName + PVC capacity
activate o
activate s

s->>p: ControllerExpandVolume RPC
activate p
note over p: Wait up to 2s for other RPC
s->>p: ControllerModifyVolume RPC

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to remember to add a describevolumes call here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And that modification state could be either optimizing OR completed

p->>p: Merge Expand + Modify Requests
p->>a: EC2 CreateTags
a-->>p:
p->>a: EC2 ModifyVolume
activate a
a-->>p:
p->>a: Poll EC2 DescribeVolumeModifications
note over a: 1+ seconds
a-->>p: Volume state == 'optimizing'
deactivate a

p-->>s: EBS Volume Modified
s-->>o: Emit VolumeModify Success Event
p-->>s: EBS Volume Expanded
deactivate p

alt if Block Device
s-->>o: Emit ExpandVolume Success Event
else if Filesystem
s-->>o: Mark PVC as FSResizeRequired
deactivate s
o->>n: Kubelet triggers NodeExpandVolume RPC
activate n
n->>n: Online resize of FS
note over n: 1+ seconds
n-->>o: Resize Success
deactivate n
deactivate o
end
```

## Driver modes

Traditionally, you run the CSI controllers together with the EBS driver in the same Kubernetes cluster.
Expand Down