Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(redis): added endpoints #1263

Merged
merged 47 commits into from
Jun 14, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
69d77c8
feat(redis): cluster resource
Mia-Cross Apr 25, 2022
b72de89
Merge branch 'master' into redis_resource
Monitob Apr 25, 2022
c577339
fix(redis): wait for cluster in create and update instead of read
Codelax Apr 25, 2022
13e7e68
fix(redis): set forcenew to node_type and redis version
Codelax Apr 25, 2022
aff273f
fix(redis): remove id on read if resource does not exist
Codelax Apr 25, 2022
54fe631
fix(redis): remove unused fields
Codelax Apr 25, 2022
dbdfa30
feat(redis): add redis to github workflow
Codelax Apr 25, 2022
39737b7
update cassette
Codelax Apr 25, 2022
c4f61a5
Merge branch 'master' into redis_resource
remyleone Apr 26, 2022
ff5aa89
Update scaleway/resource_redis_cluster.go
remyleone Apr 26, 2022
dc9dd91
Update scaleway/helpers_redis.go
remyleone Apr 26, 2022
ac909ba
feat(redis): add tags to cluster
Codelax Apr 27, 2022
8119917
feat(redis): add cluster_size to cluster schema
Codelax Apr 27, 2022
e812411
feat(redis): add tls_enabled to cluster schema
Codelax Apr 27, 2022
0d0c924
feat(redis): improved tests for basic schema
Mia-Cross Apr 27, 2022
8486b90
feat(redis): added cluster migration
Mia-Cross Apr 27, 2022
4c7b522
feat(redis): update cassettes
Mia-Cross Apr 27, 2022
13958f4
Merge branch 'master' into redis_resource
Codelax Apr 27, 2022
63ab291
fix(redis): remove forcenew for fields used in migration
Codelax Apr 27, 2022
f6f0451
update cassettes
Codelax Apr 27, 2022
286d7d7
add support for created_at, updated_at and cluster_size
remyleone Apr 27, 2022
b4bacfb
fix(redis): skip tags if empty in read
Codelax Apr 27, 2022
c8d1488
doc(redis): added first draft of documentation with supported features
Mia-Cross Apr 27, 2022
b1d4054
feat(redis): added overriding of zone in the tests
Mia-Cross Apr 27, 2022
e2c6d85
feat(redis): debugging endpoints in progress
Mia-Cross May 6, 2022
83e2b7d
Merge scaleway/t/master into fork/t/redis_resource
Mia-Cross May 6, 2022
9a587f4
fixed merge errors
Mia-Cross May 6, 2022
043c2d0
feat(redis): fixed "x_network.attribute not found" error in tests
Mia-Cross May 6, 2022
3659c3c
feat(redis): fixed all tests and added some
Mia-Cross May 6, 2022
facd4af
feat(redis): private_network: can't detach pvn + added doc
Mia-Cross Jun 1, 2022
fbf49c8
Merge branch 'master' into redis_resource
remyleone Jun 1, 2022
942c093
removed unused lb cassette
Mia-Cross Jun 2, 2022
3ed1028
Merge branch 'scaleway:master' into redis_resource
Mia-Cross Jun 8, 2022
fdeb3d0
bug(redis): hasChanges bug + PN: typelist->typeset + moved PN updates…
Mia-Cross Jun 8, 2022
2e7ddf4
fix(redis): fixed tests services_ips so they all end with 0 instead o…
Mia-Cross Jun 8, 2022
f14adac
test(redis): changed back typeSet -> typeList to see if it fixes test…
Mia-Cross Jun 9, 2022
5a606d3
test(redis): added debug prints for another round of CI
Mia-Cross Jun 9, 2022
61e9dcb
test(redis): added cassette for another round of CI
Mia-Cross Jun 9, 2022
d53ff14
feat(redis): removed CI debugging prints + WIP : orderPrivateNetworks…
Mia-Cross Jun 10, 2022
3244f7a
Merge branch 'scaleway:master' into redis_resource
Mia-Cross Jun 10, 2022
e59b829
feat(redis): orderPrivateNetworks functions work (+ tests) + cluster …
Mia-Cross Jun 10, 2022
1a7f163
feat(redis): standalone mode cassette updated
Mia-Cross Jun 10, 2022
674260a
feat(redis): removed depends_on and it still works + removed debug pr…
Mia-Cross Jun 10, 2022
59a36aa
Merge branch 'master' into redis_resource
remyleone Jun 13, 2022
13af290
feat(redis): added custom test function to avoid order pbs + removed …
Mia-Cross Jun 13, 2022
daf80f1
feat(redis): update cassettes after a lot of runs
Mia-Cross Jun 13, 2022
d0c0dca
feat(redis): removed redundant return in helpers_redis
Mia-Cross Jun 14, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 47 additions & 4 deletions docs/resources/redis_cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,30 @@ resource "scaleway_redis_cluster" "main" {
}
}
```
### With a private network
```hcl
resource "scaleway_vpc_private_network" "pn" {
name = "private-network"
}

resource "scaleway_redis_cluster" "main" {
name = "test_redis_endpoints"
version = "6.2.6"
node_type = "MDB-BETA-M"
user_name = "my_initial_user"
password = "thiZ_is_v&ry_s3cret"
cluster_size = 1
private_network {
id = "${scaleway_vpc_private_network.pn.id}"
service_ips = [
"10.12.1.1/20",
]
}
depends_on = [
scaleway_vpc_private_network.pn
]
}
```

## Arguments Reference

Expand All @@ -72,21 +96,40 @@ The following arguments are supported:

- `cluster_size` - (Optional) The number of nodes in the Redis Cluster.

~> **Important:** You can set a bigger `cluster_size`, it will migrate the Redis Cluster, but keep in mind that you cannot downgrade a Redis Cluster so setting a smaller `cluster_size` will not have any effect.
~> **Important:** You cannot set `cluster_size` to 2, you either have to choose Standalone mode (1 node) or Cluster mode which is minimum 3 (1 main node + 2 secondary nodes)

~> **Important:** You can set a bigger `cluster_size` than you initially did, it will migrate the Redis Cluster, but keep in mind that you cannot downgrade a Redis Cluster so setting a smaller `cluster_size` will not have any effect.

- `tls_enabled` - (Defaults to false) Whether TLS is enabled or not.

- `project_id` - (Defaults to [provider](../index.md) `project_id`) The ID of the project the Redis Cluster is associated with.

- `acl` - (Optional) List of acl rules, this is cluster's authorized IPs.

The `acl` block supports:
~> The `acl` block supports:

- `ip` - (Required) The ip range to whitelist in [CIDR notation](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_notation)
- `description` - (Optional) A text describing this rule. Default description: `Allow IP`
- `ip` - (Required) The ip range to whitelist in [CIDR notation](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_notation)
- `description` - (Optional) A text describing this rule. Default description: `Allow IP`

- `settings` - (Optional) Map of settings for redis cluster. Available settings can be found by listing redis versions with scaleway API or CLI

- `private_network` - (Optional) Describes the private network you want to connect to your cluster. If not set, a public network will be provided.

~> The `private_network` block supports :
- `id` - (Required) The UUID of the private network resource.
- `service_ips` - (Required) Endpoint IPv4 addresses in [CIDR notation](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_notation). You must provide at least one IP per node.

~> **Important:** The way to use private networks differs whether you are using redis in standalone or cluster mode.
- Standalone mode (`cluster_size` = 1) : you can attach as many private networks as you want (each must be a separate block). If you detach your only private network, your cluster won't be reachable until you define a new private or public network. You can modify your private_network and its specs, you can have both a private and public network side by side.
- Cluster mode (`cluster_size` > 1) : you can define a single private network as you create your cluster, you won't be able to edit or detach it afterwards, unless you create another cluster. Your `service_ips` must be listed as follows:
```hcl
service_ips = [
"10.12.1.10/20",
"10.12.1.11/20",
"10.12.1.12/20",
]
```

## Attributes Reference

In addition to all arguments above, the following attributes are exported:
Expand Down
69 changes: 69 additions & 0 deletions scaleway/helpers_redis.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,33 @@ func waitForRedisCluster(ctx context.Context, api *redis.API, zone scw.Zone, id
}, scw.WithContext(ctx))
}

func expandRedisPrivateNetwork(data []interface{}) ([]*redis.EndpointSpec, error) {
if data == nil {
return nil, nil
}
var epSpecs []*redis.EndpointSpec

for _, rawPN := range data {
pn := rawPN.(map[string]interface{})
pnID := expandID(pn["id"].(string))
rawIPs := pn["service_ips"].([]interface{})
ips := []scw.IPNet(nil)
for _, rawIP := range rawIPs {
ip, err := expandIPNet(rawIP.(string))
if err != nil {
return epSpecs, err
}
ips = append(ips, ip)
}
spec := &redis.EndpointSpecPrivateNetworkSpec{
ID: pnID,
ServiceIPs: ips,
}
epSpecs = append(epSpecs, &redis.EndpointSpec{PrivateNetwork: spec})
}
return epSpecs, nil
}

func expandRedisACLSpecs(i interface{}) ([]*redis.ACLRuleSpec, error) {
rules := []*redis.ACLRuleSpec(nil)

Expand Down Expand Up @@ -106,3 +133,45 @@ func flattenRedisSettings(settings []*redis.ClusterSetting) interface{} {
}
return rawSettings
}

func flattenRedisPrivateNetwork(endpoints []*redis.Endpoint) (interface{}, bool) {
pnFlat := []map[string]interface{}(nil)
for _, endpoint := range endpoints {
if endpoint.PrivateNetwork == nil {
continue
}
pn := endpoint.PrivateNetwork
pnZonedID := newZonedIDString(pn.Zone, pn.ID)
serviceIps := []interface{}(nil)
for _, ip := range pn.ServiceIPs {
serviceIps = append(serviceIps, ip.String())
}
pnFlat = append(pnFlat, map[string]interface{}{
"endpoint_id": endpoint.ID,
"zone": pn.Zone,
"id": pnZonedID,
"service_ips": serviceIps,
})
}
return pnFlat, len(pnFlat) != 0
}

func flattenRedisPublicNetwork(endpoints []*redis.Endpoint) interface{} {
pnFlat := []map[string]interface{}(nil)
for _, endpoint := range endpoints {
if endpoint.PublicNetwork == nil {
continue
}
ipsFlat := []interface{}(nil)
for _, ip := range endpoint.IPs {
ipsFlat = append(ipsFlat, ip.String())
}
pnFlat = append(pnFlat, map[string]interface{}{
"id": endpoint.ID,
"port": int(endpoint.Port),
"ips": ipsFlat,
})
break
}
return pnFlat
}
138 changes: 128 additions & 10 deletions scaleway/resource_redis_cluster.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (

"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/validation"
redis "github.com/scaleway/scaleway-sdk-go/api/redis/v1alpha1"
"github.com/scaleway/scaleway-sdk-go/scw"
)
Expand Down Expand Up @@ -74,16 +75,6 @@ func resourceScalewayRedisCluster() *schema.Resource {
Description: "Whether or not TLS is enabled.",
ForceNew: true,
},
"created_at": {
Type: schema.TypeString,
Computed: true,
Description: "The date and time of the creation of the Redis cluster",
},
"updated_at": {
Type: schema.TypeString,
Computed: true,
Description: "The date and time of the last update of the Redis cluster",
},
"acl": {
Type: schema.TypeList,
Description: "List of acl rules.",
Expand Down Expand Up @@ -117,6 +108,74 @@ func resourceScalewayRedisCluster() *schema.Resource {
Type: schema.TypeString,
},
},
"private_network": {
Type: schema.TypeSet,
Optional: true,
Description: "Private network specs details",
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"endpoint_id": {
Type: schema.TypeString,
Computed: true,
Description: "UUID of the endpoint to be connected to the cluster",
},
"id": {
Type: schema.TypeString,
Required: true,
ValidateFunc: validationUUIDorUUIDWithLocality(),
Description: "UUID of the private network to be connected to the cluster",
},
"service_ips": {
Type: schema.TypeList,
Required: true,
Elem: &schema.Schema{
Type: schema.TypeString,
ValidateFunc: validation.IsCIDR,
},
Description: "List of IPv4 addresses of the private network with a CIDR notation",
},
"zone": zoneSchema(),
},
},
},
// Computed
"public_network": {
Type: schema.TypeList,
Optional: true,
Computed: true,
MaxItems: 1,
Description: "Public network specs details",
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"id": {
Type: schema.TypeString,
Computed: true,
},
"port": {
Type: schema.TypeInt,
Computed: true,
Description: "TCP port of the endpoint",
},
"ips": {
Type: schema.TypeList,
Elem: &schema.Schema{
Type: schema.TypeString,
},
Computed: true,
},
},
},
},
"created_at": {
Type: schema.TypeString,
Computed: true,
Description: "The date and time of the creation of the Redis cluster",
},
"updated_at": {
Type: schema.TypeString,
Computed: true,
Description: "The date and time of the last update of the Redis cluster",
},
// Common
"zone": zoneSchema(),
"project_id": projectIDSchema(),
Expand Down Expand Up @@ -165,6 +224,15 @@ func resourceScalewayRedisClusterCreate(ctx context.Context, d *schema.ResourceD
createReq.ClusterSettings = expandRedisSettings(settings)
}

privN, privNExists := d.GetOk("private_network")
if privNExists {
pnSpecs, err := expandRedisPrivateNetwork(privN.(*schema.Set).List())
if err != nil {
return diag.FromErr(err)
}
createReq.Endpoints = pnSpecs
}

res, err := redisAPI.CreateCluster(createReq, scw.WithContext(ctx))
if err != nil {
return diag.FromErr(err)
Expand Down Expand Up @@ -216,6 +284,13 @@ func resourceScalewayRedisClusterRead(ctx context.Context, d *schema.ResourceDat
_ = d.Set("tags", cluster.Tags)
}

// set endpoints
pnI, pnExists := flattenRedisPrivateNetwork(cluster.Endpoints)
if pnExists {
_ = d.Set("private_network", pnI)
}
_ = d.Set("public_network", flattenRedisPublicNetwork(cluster.Endpoints))

return nil
}

Expand Down Expand Up @@ -303,6 +378,13 @@ func resourceScalewayRedisClusterUpdate(ctx context.Context, d *schema.ResourceD
}
}

if d.HasChanges("private_network") {
diagnostics := resourceScalewayRedisClusterUpdateEndpoints(ctx, d, redisAPI, zone, ID)
if diagnostics != nil {
return diagnostics
}
}

_, err = waitForRedisCluster(ctx, redisAPI, zone, ID, d.Timeout(schema.TimeoutUpdate))
if err != nil {
return diag.FromErr(err)
Expand Down Expand Up @@ -344,6 +426,42 @@ func resourceScalewayRedisClusterUpdateSettings(ctx context.Context, d *schema.R
return nil
}

func resourceScalewayRedisClusterUpdateEndpoints(ctx context.Context, d *schema.ResourceData, redisAPI *redis.API, zone scw.Zone, clusterID string) diag.Diagnostics {
// retrieve state
cluster, err := waitForRedisCluster(ctx, redisAPI, zone, clusterID, d.Timeout(schema.TimeoutUpdate))
if err != nil {
return diag.FromErr(err)
}

// get new desired state of endpoints
rawNewEndpoints := d.Get("private_network")
newEndpoints, err := expandRedisPrivateNetwork(rawNewEndpoints.(*schema.Set).List())
if err != nil {
return diag.FromErr(err)
}
if len(newEndpoints) == 0 {
newEndpoints = append(newEndpoints, &redis.EndpointSpec{
PublicNetwork: &redis.EndpointSpecPublicNetworkSpec{},
})
}
// send request
_, err = redisAPI.SetEndpoints(&redis.SetEndpointsRequest{
Zone: cluster.Zone,
ClusterID: cluster.ID,
Endpoints: newEndpoints,
}, scw.WithContext(ctx))
if err != nil {
return diag.FromErr(err)
}

_, err = waitForRedisCluster(ctx, redisAPI, zone, clusterID, d.Timeout(schema.TimeoutUpdate))
if err != nil {
return diag.FromErr(err)
}

return nil
}

func resourceScalewayRedisClusterDelete(ctx context.Context, d *schema.ResourceData, meta interface{}) diag.Diagnostics {
redisAPI, zone, ID, err := redisAPIWithZoneAndID(meta, d.Id())
if err != nil {
Expand Down
Loading