* Full compatibility with oldest behavior previous to , and flags

* Fixed issue#37 when and/or are empty or not defined * fixed flag to ignore group when is staff everyone localaccounts _appserverusr admin _appserveradm _lpadmin com.apple.sharepoint.group.1 _appstore _lpoperator _developer _analyticsusers com.apple.access_ftp com.apple.access_screensharing com.apple.access_ssh com.apple.access_remote_ae mentioned in PR#43 * Dependencies updated * Improved debug messages * Improved to describe the flag * Added function to the Makefile to build locally * Fixed some typos
fac · May 8, 2021 · 209ad2d · 209ad2d
1 parent cb8a8aa
commit 209ad2d
Show file tree

Hide file tree

Showing 9 changed files with 168 additions and 93 deletions.
diff --git a/.gitignore b/.gitignore
@@ -26,4 +26,7 @@ packaged.yaml
 .vscode/
 
 # SAM
-.aws-sam/
+.aws-sam/
+
+# Local binary
+ssosync
diff --git a/Makefile b/Makefile
@@ -4,11 +4,17 @@ PACKAGED_TEMPLATE = packaged.yaml
 STACK_NAME := $(STACK_NAME)
 S3_BUCKET := $(S3_BUCKET)
 TEMPLATE = template.yaml
+APP_NAME 	 ?= ssosync
+
 
 .PHONY: test
 test:
 	go test ./...
 
+.PHONY: go-build
+go-build:
+	go build -o $(APP_NAME) main.go
+
 .PHONY: clean
 clean:
 	rm -f $(OUTPUT) $(PACKAGED_TEMPLATE)

diff --git a/README.md b/README.md
@@ -63,13 +63,15 @@ as locally running the ssosync tool.
 
 First, you have to setup your API. In the project you want to use go to the [Console](https://console.developers.google.com/apis) and select *API & Services* > *Enable APIs and Services*. Search for *Admin SDK* and *Enable* the API.
 
-You have to perform this [tutorial](https://developers.google.com/admin-sdk/directory/v1/guides/delegation) to create a service account that you use to sync your users. Save the JSON file you create during the process and rename it to `credentials.json`.
+You have to perform this [tutorial](https://developers.google.com/admin-sdk/directory/v1/guides/delegation) to create a service account that you use to sync your users. Save the `JSON file` you create during the process and rename it to `credentials.json`.
 
 > you can also use the `--google-credentials` parameter to explicitly specify the file with the service credentials. Please, keep this file safe, or store it in the AWS Secrets Manager
 
 In the domain-wide delegation for the Admin API, you have to specify the following scopes for the user.
 
-`https://www.googleapis.com/auth/admin.directory.group.readonly,https://www.googleapis.com/auth/admin.directory.group.member.readonly,https://www.googleapis.com/auth/admin.directory.user.readonly`
+* https://www.googleapis.com/auth/admin.directory.group.readonly
+* https://www.googleapis.com/auth/admin.directory.group.member.readonly
+* https://www.googleapis.com/auth/admin.directory.user.readonly
 
 Back in the Console go to the Dashboard for the API & Services and select "Enable API and Services".
 In the Search box type `Admin` and select the `Admin SDK` option. Click the `Enable` button.
@@ -86,54 +88,66 @@ at this stage. You want to copy both of these as a parameter to the `ssosync` co
 
 Or you specific these as environment variables.
 
-```
+```bash
 SSOSYNC_SCIM_ACCESS_TOKEN=<YOUR_TOKEN>
 SSOSYNC_SCIM_ENDPOINT=<YOUR_ENDPOINT>
 ```
 
 ## Local Usage
 
-Usage:
+```bash
+git clone https://github.com/awslabs/ssosync.git
+cd ssosync/
+make go-build
+```
 
-The default for ssosync is to run through the sync.
+```bash
+./ssosync --help
+```
 
-```text
-A command line tool to enable you to synchronise your GoogleApps (G-Suite) users to AWS Single Sign-on (AWS SSO). Complete documentation is available at https://github.com/awslabs/ssosync
+```bash
+A command line tool to enable you to synchronise your Google
+Apps (Google Workspace) users to AWS Single Sign-on (AWS SSO)
+Complete documentation is available at https://github.com/awslabs/ssosync
 
 Usage:
   ssosync [flags]
 
 Flags:
-  -t, --access-token string         AWS SCIM Access Token
+  -t, --access-token string         AWS SSO SCIM API Access Token
   -d, --debug                       enable verbose / debug logging
-  -e, --endpoint string             AWS SCIM Endpoint
-  -u, --google-admin string         Google admin user email
-  -c, --google-credentials string   path to find credentials file for Google (default "credentials.json")
-  -g, --group-match string          Google groups query parameter, example: 'name:Admin* email:aws-*', see: https://developers.google.com/admin-sdk/directory/v1/guides/search-groups
+  -e, --endpoint string             AWS SSO SCIM API Endpoint
+  -u, --google-admin string         Google Workspace admin user email
+  -c, --google-credentials string   path to Google Workspace credentials file (default "credentials.json")
+  -g, --group-match string          Google Workspace Groups filter query parameter, example: 'name:Admin* email:aws-*', see: https://developers.google.com/admin-sdk/directory/v1/guides/search-groups
   -h, --help                        help for ssosync
-      --ignore-groups strings       ignores these Google groups
-      --ignore-users strings        ignores these Google users
-      --include-groups strings      include only these Google groups
+      --ignore-groups strings       ignores these Google Workspace groups
+      --ignore-users strings        ignores these Google Workspace users
+      --include-groups strings      include only these Google Workspace groups, NOTE: only works when --sync-method 'users_groups'
       --log-format string           log format (default "text")
       --log-level string            log level (default "info")
-  -s, --sync-method string          Select the sync method to use (users_groups|groups) (default "groups")
-  -m, --user-match string           Google users query parameter, example: 'name:John* email:admin*', see: https://developers.google.com/admin-sdk/directory/v1/guides/search-users
+  -s, --sync-method string          Sync method to use (users_groups|groups) (default "groups")
+  -m, --user-match string           Google Workspace Users filter query parameter, example: 'name:John* email:admin*', see: https://developers.google.com/admin-sdk/directory/v1/guides/search-users
   -v, --version                     version for ssosync
 ```
 
-The output of the command when run without 'debug' turned on looks like this:
+The function has `two behaviour` and these are controlled by the `--sync-method` flag, this behavior could be
 
-```
-2020-05-26T12:08:14.083+0100	INFO	cmd/root.go:43	Creating the Google and AWS Clients needed
-2020-05-26T12:08:14.084+0100	INFO	internal/sync.go:38	Start user sync
-2020-05-26T12:08:14.979+0100	INFO	internal/sync.go:73	Clean up AWS Users
-2020-05-26T12:08:14.979+0100	INFO	internal/sync.go:89	Start group sync
-2020-05-26T12:08:15.578+0100	INFO	internal/sync.go:135	Start group user sync	{"group": "AWS Administrators"}
-2020-05-26T12:08:15.703+0100	INFO	internal/sync.go:172	Clean up AWS groups
-2020-05-26T12:08:15.703+0100	INFO	internal/sync.go:183	Done sync groups
-```
+1. `groups`: __(default)__ The sync procedure work base on Groups, gets the Google Workspace groups and their members, then creates in AWS SSO the users (members of the Google Workspace groups), then the groups and at the end assign the users to their respective groups.
+2. `users_groups`: __(original behavior, previous versions)__ The sync procedure is simple, gets the Google Workspace users and creates these in AWS SSO Users; then gets Google Workspace groups and creates these in AWS SSO Groups and assigns users to belong to the AWS SSO Groups.
+
+Flags Notes:
+
+* `--include-groups` only works when `--sync-method` is `users_groups`
+* `--ignore-users` works for both `--sync-method` values.  Example: `--ignore-users user1@example.com,user2@example.com` or `SSOSYNC_IGNORE_USERS=user1@example.com,user2@example.com`
+* `--ignore-groups` works for both `--sync-method` values. Example: --ignore-groups group1@example.com,group1@example.com` or `SSOSYNC_IGNORE_GROUPS=group1@example.com,group1@example.com`
+* `--group-match` works for both `--sync-method` values and also in combination with `--ignore-groups` and `--ignore-users`.  This is the filter query passed to the [Google Workspace Directory API when search Groups](https://developers.google.com/admin-sdk/directory/v1/guides/search-groups), if the flag is not used, groups are not filtered.
+* `--user-match` works for both `--sync-method` values and also in combination with `--ignore-groups` and `--ignore-users`.  This is the filter query passed to the [Google Workspace Directory API when search Users](https://developers.google.com/admin-sdk/directory/v1/guides/search-users), if the flag is not used, users are not filtered.
+
+NOTES:
 
-You can ignore users to be synced by setting `--ignore-users user1@example.com,user2@example.com` or `SSOSYNC_IGNORE_USERS=user1@example.com,user2@example.com`. Groups are ignored by setting `--ignore-groups group1@example.com,group1@example.com` or `SSOSYNC_IGNORE_GROUPS=group1@example.com,group1@example.com`.
+1. Depending on the number of users and groups you have, maybe you can get `AWS SSO SCIM API rate limits errors`, and more frequently happens if you execute the sync many times in a short time.
+2. Depending on the number of users and groups you have, `--debug` flag generate too much logs lines in your AWS Lambda function.  So test it in locally with the `--debug` flag enabled and disable it when you use a AWS Lambda function.
 
 ## AWS Lambda Usage
 

diff --git a/cmd/root.go b/cmd/root.go
@@ -46,7 +46,7 @@ var rootCmd = &cobra.Command{
 	Use:     "ssosync",
 	Short:   "SSO Sync, making AWS SSO be populated automagically",
 	Long: `A command line tool to enable you to synchronise your Google
-Apps (G-Suite) users to AWS Single Sign-on (AWS SSO)
+Apps (Google Workspace) users to AWS Single Sign-on (AWS SSO)
 Complete documentation is available at https://github.com/awslabs/ssosync`,
 	RunE: func(cmd *cobra.Command, args []string) error {
 		ctx, cancel := context.WithCancel(context.Background())
@@ -159,20 +159,20 @@ func configLambda() {
 }
 
 func addFlags(cmd *cobra.Command, cfg *config.Config) {
-	rootCmd.PersistentFlags().StringVarP(&cfg.GoogleCredentials, "google-admin", "a", config.DefaultGoogleCredentials, "path to find credentials file for Google")
+	rootCmd.PersistentFlags().StringVarP(&cfg.GoogleCredentials, "google-admin", "a", config.DefaultGoogleCredentials, "path to find credentials file for Google Workspace")
 	rootCmd.PersistentFlags().BoolVarP(&cfg.Debug, "debug", "d", config.DefaultDebug, "enable verbose / debug logging")
 	rootCmd.PersistentFlags().StringVarP(&cfg.LogFormat, "log-format", "", config.DefaultLogFormat, "log format")
 	rootCmd.PersistentFlags().StringVarP(&cfg.LogLevel, "log-level", "", config.DefaultLogLevel, "log level")
-	rootCmd.Flags().StringVarP(&cfg.SCIMAccessToken, "access-token", "t", "", "AWS SCIM Access Token")
-	rootCmd.Flags().StringVarP(&cfg.SCIMEndpoint, "endpoint", "e", "", "AWS SCIM Endpoint")
-	rootCmd.Flags().StringVarP(&cfg.GoogleCredentials, "google-credentials", "c", config.DefaultGoogleCredentials, "path to find credentials file for Google")
-	rootCmd.Flags().StringVarP(&cfg.GoogleAdmin, "google-admin", "u", "", "Google admin user email")
-	rootCmd.Flags().StringSliceVar(&cfg.IgnoreUsers, "ignore-users", []string{}, "ignores these Google users")
-	rootCmd.Flags().StringSliceVar(&cfg.IgnoreGroups, "ignore-groups", []string{}, "ignores these Google groups")
-  rootCmd.Flags().StringSliceVar(&cfg.IncludeGroups, "include-groups", []string{}, "include only these Google groups")
-	rootCmd.Flags().StringVarP(&cfg.UserMatch, "user-match", "m", "", "Google users query parameter, example: 'name:John* email:admin*', see: https://developers.google.com/admin-sdk/directory/v1/guides/search-users")
-	rootCmd.Flags().StringVarP(&cfg.GroupMatch, "group-match", "g", "", "Google groups query parameter, example: 'name:Admin* email:aws-*', see: https://developers.google.com/admin-sdk/directory/v1/guides/search-groups")
-	rootCmd.Flags().StringVarP(&cfg.SyncMethod, "sync-method", "s", config.DefaultSyncMethod, "Select the sync method to use (users_groups|groups)")
+	rootCmd.Flags().StringVarP(&cfg.SCIMAccessToken, "access-token", "t", "", "AWS SSO SCIM API Access Token")
+	rootCmd.Flags().StringVarP(&cfg.SCIMEndpoint, "endpoint", "e", "", "AWS SSO SCIM API Endpoint")
+	rootCmd.Flags().StringVarP(&cfg.GoogleCredentials, "google-credentials", "c", config.DefaultGoogleCredentials, "path to Google Workspace credentials file")
+	rootCmd.Flags().StringVarP(&cfg.GoogleAdmin, "google-admin", "u", "", "Google Workspace admin user email")
+	rootCmd.Flags().StringSliceVar(&cfg.IgnoreUsers, "ignore-users", []string{}, "ignores these Google Workspace users")
+	rootCmd.Flags().StringSliceVar(&cfg.IgnoreGroups, "ignore-groups", []string{}, "ignores these Google Workspace groups")
+	rootCmd.Flags().StringSliceVar(&cfg.IncludeGroups, "include-groups", []string{}, "include only these Google Workspace groups, NOTE: only works when --sync-method 'users_groups'")
+	rootCmd.Flags().StringVarP(&cfg.UserMatch, "user-match", "m", "", "Google Workspace Users filter query parameter, example: 'name:John* email:admin*', see: https://developers.google.com/admin-sdk/directory/v1/guides/search-users")
+	rootCmd.Flags().StringVarP(&cfg.GroupMatch, "group-match", "g", "", "Google Workspace Groups filter query parameter, example: 'name:Admin* email:aws-*', see: https://developers.google.com/admin-sdk/directory/v1/guides/search-groups")
+	rootCmd.Flags().StringVarP(&cfg.SyncMethod, "sync-method", "s", config.DefaultSyncMethod, "Sync method to use (users_groups|groups)")
 }
 
 func logConfig(cfg *config.Config) {

diff --git a/go.mod b/go.mod
@@ -5,25 +5,25 @@ go 1.16
 require (
 	github.com/BurntSushi/toml v0.3.1
 	github.com/aws/aws-lambda-go v1.23.0
-	github.com/aws/aws-sdk-go v1.38.6
+	github.com/aws/aws-sdk-go v1.38.36
 	github.com/fsnotify/fsnotify v1.4.9 // indirect
 	github.com/golang/mock v1.5.0
 	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
-	github.com/hashicorp/go-retryablehttp v0.6.8
+	github.com/hashicorp/go-retryablehttp v0.7.0
 	github.com/magiconair/properties v1.8.5 // indirect
 	github.com/mitchellh/mapstructure v1.4.1 // indirect
-	github.com/pelletier/go-toml v1.8.1 // indirect
+	github.com/pelletier/go-toml v1.9.0 // indirect
 	github.com/pkg/errors v0.9.1
 	github.com/sirupsen/logrus v1.8.1
 	github.com/spf13/afero v1.6.0 // indirect
 	github.com/spf13/cast v1.3.1 // indirect
 	github.com/spf13/cobra v1.1.3
 	github.com/spf13/jwalterweatherman v1.1.0 // indirect
 	github.com/spf13/viper v1.7.1
-	github.com/stretchr/testify v1.6.1
-	golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4 // indirect
-	golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84
-	golang.org/x/sys v0.0.0-20210324051608-47abb6519492 // indirect
-	google.golang.org/api v0.43.0
+	github.com/stretchr/testify v1.7.0
+	golang.org/x/net v0.0.0-20210508051633-16afe75a6701 // indirect
+	golang.org/x/oauth2 v0.0.0-20210427180440-81ed05c6b58c
+	golang.org/x/sys v0.0.0-20210507161434-a76c4d0a0096 // indirect
+	google.golang.org/api v0.46.0
 	gopkg.in/ini.v1 v1.62.0 // indirect
 )