Lack of support for nest groups

[robotica-labs]

Describe the bug
When running SSO Sync receive error message:
Exception has occurred: panic
"runtime error: index out of range [0] with length 0"
Stack:
3 0x0000000000eca5d3 in github.com/awslabs/ssosync/internal.(*syncGSuite).getGoogleGroupsAndUsers
at C:/Users/matth/Desktop/Stuff/AWS SSO/SSOSYNC/ssosync/internal/sync.go:511
4 0x0000000000ec6ae7 in github.com/awslabs/ssosync/internal.(*syncGSuite).SyncGroupsUsers
at C:/Users/matth/Desktop/Stuff/AWS SSO/SSOSYNC/ssosync/internal/sync.go:291
5 0x0000000000ecceec in github.com/awslabs/ssosync/internal.DoSync
at C:/Users/matth/Desktop/Stuff/AWS SSO/SSOSYNC/ssosync/internal/sync.go:706
6 0x0000000000fee6ab in github.com/awslabs/ssosync/cmd.glob..func1
at C:/Users/matth/Desktop/Stuff/AWS SSO/SSOSYNC/ssosync/cmd/root.go:55
7 0x0000000000fb2f70 in github.com/spf13/cobra.(*Command).execute
at C:/Users/matth/go/pkg/mod/github.com/spf13/[email protected]/command.go:852
8 0x0000000000fb3ce6 in github.com/spf13/cobra.(*Command).ExecuteC
at C:/Users/matth/go/pkg/mod/github.com/spf13/[email protected]/command.go:960
9 0x0000000000fb33ef in github.com/spf13/cobra.(*Command).Execute
at C:/Users/matth/go/pkg/mod/github.com/spf13/[email protected]/command.go:897
10 0x0000000000fecec5 in github.com/awslabs/ssosync/cmd.Execute
at C:/Users/matth/Desktop/Stuff/AWS SSO/SSOSYNC/ssosync/cmd/root.go:72
11 0x0000000000fee877 in main.main
at C:/Users/matth/Desktop/Stuff/AWS SSO/SSOSYNC/ssosync/main.go:29

Error hits sync.go line 511: membersUsers = append(membersUsers, u[0])

u evaluates to "[]*google.golang.org/api/admin/directory/v1.User len: 0, cap: 0, []"

I am trying to sync a small subset of groups and about 40-80 users from Gsuite to AWS SSO. I cannot sync all user accounts and groups, it has to be a subset.

To Reproduce
Steps to reproduce the behavior:

1. Debug mode with VSCode
2. Ran with args using launch.json (sanitized here) to pass args.
   "args": [
   ...
   "--google-admin",
   "[email protected]",
   "--sync-method",
   "groups",
   "--ignore-users",
   "email:*",
   "--include-groups",
   "name:[email protected]"
   ]
3. See error

Expected behavior
Expect it to continue and finish process.

Additional context
https://alegiovanardi.medium.com/go-slices-explained-part-1-1f1143b568fc seems to indicate that the array (or slice) is too small. I'm not sure why because it appears to be dynamically assigned. I've spent several hours trying to fix it, but I have no experience with GOLang and can't get over that hurdle.
This gsuite domain has thousands of user accounts.

Thanks for your attention

[mmb-davidsmith]

I found that this is being caused by nested groups. If you have a google group that included another group as a member, this script can't handle it.

[mmb-davidsmith]

It could also be if you have groups that contain members from outside your domain.

[gleidin]

I found that this is being caused by nested groups. If you have a google group that included another group as a member, this script can't handle it.

is there a workaround for that? I'm having the same issue here.

[mmb-davidsmith]

In internal/google/client.go I adjusted GetGroupMembers to:

func (c *client) GetGroupMembers(g *admin.Group) ([]*admin.Member, error) {
	m := make([]*admin.Member, 0)
	err := c.service.Members.List(g.Id).IncludeDerivedMembership(true).Pages(context.TODO(), func(members *admin.Members) error {
		m = append(m, members.Members...)
		return nil
	})

	return m, err
}

and in internal/sync.go I updated getGoogleGroupsAndUsers with the addition of

			if m.Type == "GROUP" {
				log.WithField("id", m.Email).Debug("ignoring group address")
				continue
			}

after

			if s.ignoreUser(m.Email) {
				log.WithField("id", m.Email).Debug("ignoring user")
				continue
			}

as well as

				if len(u) == 0 {
					log.WithField("email", m.Email).Debug("Ignoring Unknown User")
					continue
				}

before

				membersUsers = append(membersUsers, u[0])

[christiangda]

@robotica-labs and @mmb-davidsmith my PR #47 fix this problem and others important ones, if you want to have a more stable version, over the main branch apply this PR and the other impr¡ortan one is #45

[mmb-davidsmith]

@christiangda - Does that PR also take into account the 50 user limit when querying the AWS SCIM endpoint. That's caused a big issue for me, and I really don't know how to fix it properly from what I read on their documentation. For now I go through the list of google users and if they don't exist in the AWS user list I query for the user by email to determine if they exist in AWS.

[christiangda]

@mmb-davidsmithI understand, unfortunately, AWS SSO API doesn't support pagination and for that reason and because AWS stoped this project I'm working in my free time on a new approach to this functionality, I hope I can deliver this project soon.

If you want to take a look-see the "develop" branch here https://github.com/slashdevops/idp-scim-sync/tree/develop
I'm implementing an intermediate "state" with the information of the sync, I was inspired by Terraform states. until now this is not even an alpha, but also I'm implementing a cli to check the google filters and AWS queries.

[robotica-labs]

This is helping me get further, but I'm not sure if what I want to do can be done in current codebase. What I need is to pass a list of groups (as email addresses) and have SSSO populate those groups in AWS as well as the members of those groups, but not any other users from outside those groups. That way only a half dozen groups and about 50 users need to be imported, not the entire enterprise of users, which is as far as I get. The membership of the groups will change overtime so it needs to be managed within GSuite. When I run it to sync our entire enterprise it takes 30 minutes and then AWS forcibly disconnects my session.

[christiangda]

Hi guys, I finished the first version of the new alternative project to this one.

https://github.com/slashdevops/idp-scim-sync is a fresh implementation, it is not a ssosync fork, and the main idea is to keep a "state file" after the sync to avoid unnecessary requests to AWS SSO SCIM API.

this project is composed of different artefacts repository but the most convenient one is the AWS Serverless Application https://serverlessrepo.aws.amazon.com/applications/us-east-1/889836709304/idp-scim-sync

[omerls-pw]

when will this fix be merged to main?

[ChrisPates]

v2.0.2 no longer panics but simply ignores group members that are themselves groups. I'll be looking to implement a 'group flattening' approach in a future feature branch.

[naharoo]

Hi. Any ETA on implementing group flattening?

[omerls-pw]

@naharoo
they published another way to sync users only.
https://docs.aws.amazon.com/singlesignon/latest/userguide/gs-gwp.html

[ChrisPates]

I’m hoping to get to this in v2.2, there is an official integration although the feedback I’ve received is that the lack of support for groups means folk want to continue with ssosync for now. Chris On 14 Jan 2024, at 12:28, Omer Livne ***@***.***> wrote:  @naharoo they published another way to sync users only. https://docs.aws.amazon.com/singlesignon/latest/userguide/gs-gwp.html <https://docs.aws.amazon.com/singlesignon/latest/userguide/gs-gwp.html> — Reply to this email directly, view it on GitHub <#66 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVULYKLAJF3NLJMHAHHYHLYOPFOXAVCNFSM5GHDHEHKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBZGA4TGOBZGEZQ> . You are receiving this because you were assigned. <https://github.com/notifications/beacon/ABVULYJSXW43FEPVZL22PLDYOPFOXA5CNFSM5GHDHEHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOOC2XAII.gif> Message ID: ***@***.***>