feat(api): Add variable-cost token bucket rate limiting algorithm

[rifont]

What change does this PR introduce?

• Adds a variable-cost token bucket rate limiting algorithm implemented in a Redis Lua script
  • Add benchmarking and throttle evaluation tests. These are skipped but can be used when making changes to the algorithm to verify expected performance is kept within bounds.
• Adds tests for varying cost endpoints

Why was this change needed?

A variable cost rate limiting algorithm is needed to provide the capability for different API controllers and methods to have a varying cost. It also lays a foundation for dynamic costing of resource consumption.

Other information (Screenshots)

Bulk Endpoint E2E Tests

Redis Benchmark Tests

Test output in text form

``` Redis EVAL script benchmarks Low Load - 0% Throttled - Sustained Single Window Params: Total Req: 5,000 Users: 2,500 Throttled: 0% High Cost: 0% Jitter: 800ms Stats: Total Time: 804ms Avg: 0.8ms Stdev: 1.2 p(95): 3 Throttled: 0 Request Time (ms) 0.80: ************************************************** 1.60: ****************************** 2.40: ********* 3.20: *** 4.00: 4.80: * 5.60: * 6.40: 7.20: 8.00: Script Performance ✔ should be able to process 5,000 evaluations in less than 1000ms ✔ should have average evaluation duration less than 10ms ✔ should have 95th percentile evaluation duration less than 30ms Script Throttle Evaluation ✔ should throttle between 0 and 0 requests Medium Load - 0% Throttled - Sustained Single Window Params: Total Req: 10,000 Users: 5,000 Throttled: 0% High Cost: 0% Jitter: 800ms Stats: Total Time: 799ms Avg: 1.5ms Stdev: 2.0 p(95): 6 Throttled: 0 Request Time (ms) 1.20: ************************************************** 2.40: ******* 3.60: ** 4.80: *** 6.00: * 7.20: * 8.40: 9.60: 10.80: 12.00: Script Performance ✔ should be able to process 10,000 evaluations in less than 1000ms ✔ should have average evaluation duration less than 20ms ✔ should have 95th percentile evaluation duration less than 50ms Script Throttle Evaluation ✔ should throttle between 0 and 0 requests High Load - 0% Throttled - Sustained Single Window Params: Total Req: 20,000 Users: 10,000 Throttled: 0% High Cost: 0% Jitter: 800ms Stats: Total Time: 799ms Avg: 7.4ms Stdev: 15.5 p(95): 53 Throttled: 0 Request Time (ms) 6.60: ************************************************** 13.20: * 19.80: * 26.40: * 33.00: * 39.60: 46.20: ** 52.80: 59.40: ** 66.00: Script Performance ✔ should be able to process 20,000 evaluations in less than 1000ms ✔ should have average evaluation duration less than 200ms ✔ should have 95th percentile evaluation duration less than 500ms Script Throttle Evaluation ✔ should throttle between 0 and 0 requests Extreme Load - 0% Throttled - Sustained Single Window Params: Total Req: 40,000 Users: 20,000 Throttled: 0% High Cost: 0% Jitter: 800ms Stats: Total Time: 799ms Avg: 46.6ms Stdev: 39.0 p(95): 107 Throttled: 0 Request Time (ms) 12.80: ************************************************** 25.60: ****************************** 38.40: ****** 51.20: ******* 64.00: ********** 76.80: ***** 89.60: ******************* 102.40: ********************* 115.20: ************** 128.00: * Script Performance ✔ should be able to process 40,000 evaluations in less than 2000ms ✔ should have average evaluation duration less than 500ms ✔ should have 95th percentile evaluation duration less than 2000ms Script Throttle Evaluation ✔ should throttle between 0 and 0 requests High Load - 0% Throttled - Burst Single Window Params: Total Req: 20,000 Users: 10,000 Throttled: 0% High Cost: 0% Jitter: 200ms Stats: Total Time: 582ms Avg: 274.3ms Stdev: 51.6 p(95): 322 Throttled: 0 Request Time (ms) 33.50: * 67.00: * 100.50: 134.00: * 167.50: 201.00: 234.50: ********* 268.00: *************************** 301.50: ************************************************** 335.00: **************************************** Script Performance ✔ should be able to process 20,000 evaluations in less than 1000ms ✔ should have average evaluation duration less than 500ms ✔ should have 95th percentile evaluation duration less than 1000ms Script Throttle Evaluation ✔ should throttle between 0 and 0 requests Extreme Load - 0% Throttled - Burst Single Window Params: Total Req: 40,000 Users: 20,000 Throttled: 0% High Cost: 0% Jitter: 200ms Stats: Total Time: 1,365ms Avg: 630.5ms Stdev: 176.8 p(95): 940 Throttled: 0 Request Time (ms) 98.00: * 196.00: 294.00: 392.00: 490.00: ************************ 588.00: ************* 686.00: ************************************************** 784.00: 882.00: ******** 980.00: *************** Script Performance ✔ should be able to process 40,000 evaluations in less than 3000ms ✔ should have average evaluation duration less than 1500ms ✔ should have 95th percentile evaluation duration less than 2000ms Script Throttle Evaluation ✔ should throttle between 0 and 0 requests High Load - 50% Throttled - Burst Single Window Params: Total Req: 20,000 Users: 10,000 Throttled: 50% High Cost: 0% Jitter: 200ms Stats: Total Time: 449ms Avg: 177.3ms Stdev: 46.7 p(95): 211 Throttled: 0 Request Time (ms) 21.90: ** 43.80: *** 65.70: ** 87.60: 109.50: * 131.40: ****** 153.30: 175.20: *********** 197.10: ************************************************** 219.00: ******************************************** Script Performance ✔ should be able to process 20,000 evaluations in less than 1000ms ✔ should have average evaluation duration less than 500ms ✔ should have 95th percentile evaluation duration less than 1000ms Script Throttle Evaluation ✔ should throttle between 0 and 0 requests High Load - 50% Throttled - Sustained Single Window Params: Total Req: 20,000 Users: 10,000 Throttled: 50% High Cost: 0% Jitter: 800ms Stats: Total Time: 800ms Avg: 4.7ms Stdev: 10.2 p(95): 33 Throttled: 0 Request Time (ms) 4.30: ************************************************** 8.60: * 12.90: 17.20: 21.50: 25.80: 30.10: * 34.40: * 38.70: * 43.00: * Script Performance ✔ should be able to process 20,000 evaluations in less than 1000ms ✔ should have average evaluation duration less than 500ms ✔ should have 95th percentile evaluation duration less than 500ms Script Throttle Evaluation ✔ should throttle between 0 and 0 requests High Load - 50% Throttled & 50% High-Cost - Sustained Multiple Windows Params: Total Req: 40,000 Users: 20,000 Throttled: 50% High Cost: 50% Jitter: 2200ms Stats: Total Time: 2,200ms Avg: 1.6ms Stdev: 3.2 p(95): 7 Throttled: 9,405 Request Time (ms) 2.70: ************************************************** 5.40: **** 8.10: * 10.80: * 13.50: 16.20: 18.90: 21.60: 24.30: 27.00: Script Performance ✔ should be able to process 40,000 evaluations in less than 3000ms ✔ should have average evaluation duration less than 30ms ✔ should have 95th percentile evaluation duration less than 100ms Script Throttle Evaluation ✔ should throttle between 8064 and 12096 requests Extreme Load - 50% Throttled & 50% High-Cost - Sustained Multiple Windows Params: Total Req: 80,000 Users: 40,000 Throttled: 50% High Cost: 50% Jitter: 2200ms Stats: Total Time: 2,320ms Avg: 13.6ms Stdev: 27.2 p(95): 79 Throttled: 18,671 Request Time (ms) 19.50: ************************************************** 39.00: ***** 58.50: *** 78.00: ** 97.50: * 117.00: 136.50: 156.00: 175.50: 195.00: Script Performance ✔ should be able to process 80,000 evaluations in less than 4000ms ✔ should have average evaluation duration less than 1000ms ✔ should have 95th percentile evaluation duration less than 1500ms Script Throttle Evaluation ✔ should throttle between 16130 and 24196 requests High Load - 50% Throttled & 90% High-Cost - Sustained Multiple Windows Params: Total Req: 40,000 Users: 20,000 Throttled: 50% High Cost: 90% Jitter: 2200ms Stats: Total Time: 2,194ms Avg: 1.1ms Stdev: 1.9 p(95): 4 Throttled: 14,582 Request Time (ms) 2.00: ************************************************** 4.00: ******* 6.00: * 8.00: 10.00: 12.00: 14.00: 16.00: 18.00: 20.00: Script Performance ✔ should be able to process 40,000 evaluations in less than 3000ms ✔ should have average evaluation duration less than 50ms ✔ should have 95th percentile evaluation duration less than 200ms Script Throttle Evaluation ✔ should throttle between 10754 and 16131 requests High Load - 50% Throttled & 0% Unique - Sustained Multiple Windows Params: Total Req: 40,000 Users: 1 Throttled: 50% High Cost: 0% Jitter: 2200ms Stats: Total Time: 2,200ms Avg: 0.9ms Stdev: 1.4 p(95): 4 Throttled: 0 Request Time (ms) 1.10: ************************************************** 2.20: *** 3.30: * 4.40: * 5.50: 6.60: 7.70: 8.80: 9.90: 11.00: Script Performance ✔ should be able to process 40,000 evaluations in less than 3000ms ✔ should have average evaluation duration less than 30ms ✔ should have 95th percentile evaluation duration less than 200ms Script Throttle Evaluation ✔ should throttle between 0 and 0 requests High Load - 50% Throttled & 100% Unique - Sustained Multiple Windows Params: Total Req: 40,000 Users: 40,000 Throttled: 50% High Cost: 0% Jitter: 2200ms Stats: Total Time: 2,210ms Avg: 1.8ms Stdev: 3.6 p(95): 7 Throttled: 0 Request Time (ms) 4.10: ************************************************** 8.20: ** 12.30: 16.40: 20.50: 24.60: 28.70: 32.80: 36.90: 41.00: Script Performance ✔ should be able to process 40,000 evaluations in less than 3000ms ✔ should have average evaluation duration less than 30ms ✔ should have 95th percentile evaluation duration less than 100ms Script Throttle Evaluation ✔ should throttle between 0 and 0 requests

[Nest] 23992 - 11/28/2023, 5:48:36 PM ERROR [InMemoryProviderService] Error: Connection is closed.
[Provider: Elasticache] In-memory provider service shutdown has failed

57 passing (27s)

</p>
</details> 

[Cliftonz]

The load testing needs to be moved out into its own k6 scripts in enterpise repo but other then that it looks good

[rifont]

The load testing needs to be moved out into its own k6 scripts in enterpise repo but other then that it looks good

My intention here is for these Redis-only benchmark tests to be used if anyone (team or community) makes further changes to the algorithm, ensuring that the current baseline is still met. Without these tests being public, changes could be made to the algorithm without proper testing.

I still intend to perform E2E load testing via API requests in K6, as it's important for us to gauge how the system performs as a whole, especially given shared use of the In-Memory Redis cluster. The User/time variation strategy in these tests will be useful here, and probably quite similar. These tests will live in the enterprise repo.

[rifont]

This one shouldn't have been put here. It's moved to the describe block for the custom algorithm.