Installing Google Apps Manager (GAM) on Google Cloud Shell

GAM

Google Apps Manager, known as GAM, is a command line tool created by jay0lee that makes it easier to manage G Suite domains.

Advanced GAM, known as GAMADV-X, is a re-write of GAM by [taers232c}(https://github.com/taers232c). It adds several benefits over the original version including improved error handling, better performance and a number of new features.

GAMADV-XTD is an extended version of GAMADV-X which adds Team Drive support and uses the newer Google Drive Rest API v3.

Cloud Shell

Google Cloud Shell is an interactive shell environment used to manage Google Cloud Platform resources through your browser.

When you start Cloud Shell, it provisions a g1-small Google Compute Engine virtual machine running a Debian-based Linux operating system. Cloud Shell provisions 5 GB of free persistent disk storage mounted as your $HOME directory on the virtual machine instance. This storage is on a per-user basis and is available across projects.

GAM can be installed locally or on a Google Cloud Shell virtual machine.

Installing GAM in your local environment may be an issue if your organization uses a web proxy or SSL filtering. GAM does not officially support proxy servers, though setting your environment variables may allow you to get around this limitation. Additional configuration is also required if you need to set a custom Root CA.

As Google Cloud Shell runs in the cloud, it’s unaffected by any local web filtering policy that your organization may use, with the added benefit of being accessible on any device with a web browser.

One downside of running GAM in Google Cloud Shell is that you won’t be able to combine it with Windows scripting languages like PowerShell.

Installing GAMADV-X / GAMADV-XTD on Google Cloud Shell

Before beginning, ensure you are signed in with a super administrator account. Then launch Google Cloud Shell.

https://console.cloud.google.com/cloudshell/editor?shellonly=true

Once it connects, you’ll be presented with a bash shell.

Welcome to Cloud Shell! Type "help" to get started.
To set your Cloud Platform project in this session use “gcloud config set project [PROJECT_ID]”
basuta@cloudshell:~$

Download and install GAMADV-X or GAMADV-XTD. In this example, we will download GAMADV-XTD and install it to the default path. The default installation path is $HOME/bin but you can change this if required.

bash <(curl -s -S -L https://git.io/vAAj9)

If you want to install GAMADV-X instead, simply change the download URL.

GAMADV-XTD will download and the installation will begin. You’ll be asked if you have a full browser and whether you’re ready to set up a Google API project. Google Cloud Shell has no browser and we’ll set up the project later, so answer no to both of these questions.

Checking GitHub URL https://api.github.com/repos/taers232c/GAMADV-XTD/releases for latest GAM release...

Getting file and download URL...

Downloading file gamadv-xtd-4.60.18-linux-x86_64.tar.xz from https://github.com/taers232c/GAMADV-XTD/releases/download/v4.60.18/gamadv-xtd-4.60.18-linux-x86_64.tar.xz to /tmp/tmp.vzPoTEDmfz.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   628    0   628    0     0   1498      0 --:--:-- --:--:-- --:--:--  1498
100 7630k  100 7630k    0     0   585k      0  0:00:13  0:00:13 --:--:-- 1210k
Extracting archive to /home/basuta/bin

Finished extracting GAM archive.

Adding gam alias to profile file /home/basuta/.bashrc.

Can you run a full browser on this machine? (usually Y for MacOS, N for Linux if you SSH into this machine) n

GAM is now installed. Are you ready to set up a Google API project for GAM? (yes or no) n

You can create an API project later by running:

gam create project

Here's information about your new GAM installation:


Created: /home/basuta/.gam
Created: /home/basuta/.gam/gamcache
Created: /home/basuta/Downloads
Config File: /home/basuta/.gam/gam.cfg, Initialized
WARNING: Config File: /home/basuta/.gam/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /home/basuta/.gam/oauth2.txt, Not Found
WARNING: Config File: /home/basuta/.gam/gam.cfg, Section: DEFAULT, Item: oauth2service_json, Value: /home/basuta/.gam/oauth2service.json, Not Found
WARNING: Config File: /home/basuta/.gam/gam.cfg, Section: DEFAULT, Item: client_secrets_json, Value: /home/basuta/.gam/client_secrets.json, Not Found
GAM 4.60.18 - https://github.com/taers232c/GAMADV-XTD
Ross Scroggs <ross.scroggs@gmail.com>
Python 2.7.14 64-bit final
google-api-python-client 1.7.3
oauth2client 4.1.2
Linux-4.14.33+-x86_64-with-debian-9.5 x86_64
Path: /home/basuta/bin/gamadv-xtd
GAM installation and setup complete!

Please restart your terminal shell or to get started right away run:

alias gam="/home/basuta/bin/gamadv-xtd/gam"

Use the above alias command to set the gam alias. If you’re copying the below command, update it with your own username.

alias gam="/home/username/bin/gamadv-xtd/gam"

You may notice the warnings about the client_secrets.json, oauth2service.json and oauth2.txt files being missing. These can be ignored as the GAM installation process will create all of these files once complete.


A note on existing GAM installations

This guide assumes that there is no existing installation of GAM already set up. If there is, a GAM project will already exist and you can copy the client_secrets.json, oauth2service.json and oauth2.txt files from where you have installed GAM to the Google Cloud Shell virtual machine.

Follow the below steps if you have an existing installation, or skip to the next section to set up GAM for the first time.

Upload your client_secrets.json, oauth2service.json and oauth2.txt files. By default, files uploaded to the Cloud Shell are placed in your home directory, so move them to the GAM directory.

mv client_secrets.json oauth2service.json oauth2.txt ~/.gam

Verify that the files have been moved.

ls ~/.gam

Check that GAM is working.

gam info domain

You should see some information about your domain, and GAM will be ready to use on Google Cloud Shell.


Setting up GAM for the first time

It’s a good idea to read the documentation for an overview of how GAM is configured. Use nano or another text editor to read it.

nano ~/bin/gamadv-xtd/GamConfig.txt

Once you’ve read the documentation, press Ctrl-X to exit nano.

As Google Cloud Shell has no browser, we need to edit the value of no_browser in the gam configuration file located in ~/.gam/gam.cfg. To check the current value, we can use grep.

grep -i 'no_browser' ~/.gam/gam.cfg

This shows the following output.

no_browser = false

If the value is false, use sed to change it to true.

sed -i 's/^\(no_browser\s*=\s*\).*$/\1true/' ~/.gam/gam.cfg

Use the same grep command again to confirm it’s changed.

grep -i 'no_browser' ~/.gam/gam.cfg

The value of no_browser should now be true.

no_browser = true

You can also edit the configuration file with a text editor like nano.

nano ~/.gam/gam.cfg

Any of the values here can be changed depending on your needs. We will keep the default values for everything else so press Ctrl-X to exit.

Next we will build the gam configuration file.

gam config verify

All the values in the file will be displayed.

Section: DEFAULT
  activity_max_results = 100
  auto_batch_min = 0
  batch_size = 50
  cacerts_pem = ''
  cache_dir = /home/basuta/.gam/gamcache
  cache_discovery_only = true
  charset = utf-8
  classroom_max_results = 0
  client_secrets_json = client_secrets.json ; /home/basuta/.gam/client_secrets.json
  config_dir = /home/basuta/.gam
  contact_max_results = 100
  csv_input_column_delimiter = ,
  csv_input_quote_char = '"'
  csv_output_column_delimiter = ,
  csv_output_convert_cr_nl = false
  csv_output_field_delimiter = ' '
  csv_output_line_terminator = lf
  csv_output_quote_char = '"'
  csv_output_users_audit = false
  customer_id = my_customer
  debug_level = 0
  device_max_results = 500
  domain = ''
  drive_dir = /home/basuta/Downloads
  drive_max_results = 1000
  drive_v3_native_names = true
  email_batch_size = 100
  event_max_results = 250
  extra_args = ''
  inter_batch_wait = 0
  member_max_results = 200
  message_batch_size = 1000
  message_max_results = 1000
  never_time = Never
  no_browser = true
  no_cache = false
  no_update_check = false
  no_verify_ssl = false
  num_tbatch_threads = 2
  num_threads = 5
  oauth2_txt = oauth2.txt ; /home/basuta/.gam/oauth2.txt
  oauth2service_json = oauth2service.json ; /home/basuta/.gam/oauth2service.json
  print_native_names = true
  section = ''
  show_convert_cr_nl = false
  show_counts_min = 1
  show_gettings = true
  show_multiprocess_info = false
  timezone = utc
  todrive_conversion = true
  todrive_localcopy = false
  todrive_parent = root
  todrive_timestamp = false
  todrive_user = ''
  user_max_results = 500

We can now build the GAM project by running the following command.

gam create project

Follow the instructions and authorize access. You should see something like the below.

WARNING: Config File: /home/basuta/.gam/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /home/basuta/.gam/oauth2.txt, Not Found
WARNING: Config File: /home/basuta/.gam/gam.cfg, Section: DEFAULT, Item: oauth2service_json, Value: /home/basuta/.gam/oauth2service.json, Not Found
WARNING: Config File: /home/basuta/.gam/gam.cfg, Section: DEFAULT, Item: client_secrets_json, Value: /home/basuta/.gam/client_secrets.json, Not Found

What is your G Suite admin email address? admin@example.com

Go to the following link in your browser:

https://accounts.google.com/o/oauth2/v2/auth?redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&client_id=[removed].apps.googleusercontent.com&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform&login_hint=admin%40example.com&access_type=online

Enter verification code: 4/TgDCoUxQcYTCTGQeHBXFp3QwV9Cgyh6ApjJ1w2EMB72mZgHF8O_J6k4
Authentication successful.
Creating project "GAM Project"...
Checking project status...
Project still being created. Sleeping 1 seconds
Checking project status...
Enable 17 APIs
  API: admin.googleapis.com, Enabled (1/17)
  API: appsactivity.googleapis.com, Enabled (2/17)
  API: audit.googleapis.com, Enabled (3/17)
  API: calendar-json.googleapis.com, Enabled (4/17)
  API: classroom.googleapis.com, Enabled (5/17)
  API: contacts.googleapis.com, Enabled (6/17)
  API: drive.googleapis.com, Enabled (7/17)
  API: gmail.googleapis.com, Enabled (8/17)
  API: groupsmigration.googleapis.com, Enabled (9/17)
  API: groupssettings.googleapis.com, Enabled (10/17)
  API: licensing.googleapis.com, Enabled (11/17)
  API: plus.googleapis.com, Enabled (12/17)
  API: reseller.googleapis.com, Enabled (13/17)
  API: sheets.googleapis.com, Enabled (14/17)
  API: siteverification.googleapis.com, Enabled (15/17)
  API: storage-api.googleapis.com, Enabled (16/17)
  API: vault.googleapis.com, Enabled (17/17)
Creating Service Account
Please go to:

https://console.developers.google.com/apis/credentials?project=gam-project

1. Click the blue "Create credentials" button. Choose "OAuth client ID".
2. Click the blue "Configure consent screen" button. Enter "GAM" for "Product name to show to users".
3. Leave other fields blank. Click "Save" button.
3. Choose "Other" and click the blue "Create" button.
4. Copy your "client ID" value.

Enter your Client ID: [removed].apps.googleusercontent.com

Now go back to your browser and copy your client secret.
Enter your Client Secret: rxt50epwVcZ5ECsRzsVmnmOT
Almost there! Now please switch back to your browser and:

1. Click OK to close "OAuth client" popup if it's still open.
2. Click "Manage service accounts" on the right of the screen.
3. Click the 3 dots to the right of your service account.
4. Choose Edit.
5. Check the "Enable G Suite Domain-wide Delegation" box and click Save.

Press Enter when done...
That's it! Your GAM Project is created and ready to use.

Now we need to authorize the GAM client.

gam oauth create

Change the scope if needed or press c to continue, then follow the instructions.

Select the authorized scopes by entering a number.
Append an 'r' to grant read-only access or an 'a' to grant action-only access.

[*]  0)  Admin Settings API
[*]  1)  Calendar API (supports readonly)
[*]  2)  Classroom API - Courses (supports readonly)
[*]  3)  Classroom API - Course Announcements (supports readonly)
[*]  4)  Classroom API - Course Work/Submissions (supports readonly)
[*]  5)  Classroom API - Student Guardians (supports readonly)
[*]  6)  Classroom API - Profile Emails
[*]  7)  Classroom API - Profile Photos
[*]  8)  Classroom API - Rosters (supports readonly)
[*]  9)  Cloudprint API
[*] 10)  Cloud Storage (Vault Export - read only)
[*] 11)  Contacts API - Domain Shared and Users and GAL
[*] 12)  Data Transfer API (supports readonly)
[*] 13)  Directory API - Chrome OS Devices (supports readonly)
[*] 14)  Directory API - Customers (supports readonly)
[*] 15)  Directory API - Domains (supports readonly)
[*] 16)  Directory API - Groups (supports readonly)
[*] 17)  Directory API - Mobile Devices Directory (supports readonly and action)
[*] 18)  Directory API - Notifications
[*] 19)  Directory API - Organizational Units (supports readonly)
[*] 20)  Directory API - Resource Calendars (supports readonly)
[*] 21)  Directory API - Roles (supports readonly)
[*] 22)  Directory API - User Schemas (supports readonly)
[*] 23)  Directory API - User Security
[*] 24)  Directory API - Users (supports readonly)
[*] 25)  Email Audit API
[*] 26)  Email Settings API - Users
[*] 27)  Groups Migration API
[*] 28)  Groups Settings API
[*] 29)  License Manager API
[*] 30)  Reports API - Audit Reports
[*] 31)  Reports API - Usage Reports
[ ] 32)  Reseller API
[*] 33)  Site Verification API
[*] 34)  Sites API
[*] 35)  Vault API (supports readonly)

     s)  Select all scopes
     u)  Unselect all scopes
     e)  Exit without changes
     c)  Continue to authorization
Please enter 0-35[a|r] or s|u|e|c: c

What is your G Suite admin email address? admin@example.com

Go to the following link in your browser:

    https://accounts.google.com/o/oauth2/v2/auth?redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&client_id=[removed]&scope=email+profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcalendar+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.datatransfer+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.device.chromeos+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.customer+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.domain+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.group+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.device.mobile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.notifications+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.orgunit+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.resource.calendar+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.rolemanagement+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.userschema+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.user.security+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.user+https%3A%2F%2Fapps-apis.google.com%2Fa%2Ffeeds%2Femailsettings%2F2.0%2F+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fapps.licensing&login_hint=admin%40example.com&access_type=offline

Enter verification code: 4/TgAHgxsJHrpxCHGktFSqmxdVN9GpzGqHsv-1tl3A-I36mDEfWCR1uVc
Authentication successful.

Go to the following link in your browser:

    https://accounts.google.com/o/oauth2/v2/auth?redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&client_id=[removed]&scope=email+profile+https%3A%2F%2Fapps-apis.google.com%2Fa%2Ffeeds%2Fdomain%2F+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fclassroom.courses+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fclassroom.announcements+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fclassroom.coursework.students+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fclassroom.guardianlinks.students+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fclassroom.profile.emails+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fclassroom.profile.photos+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fclassroom.rosters+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloudprint+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdevstorage.read_only+https%3A%2F%2Fwww.google.com%2Fm8%2Ffeeds+https%3A%2F%2Fapps-apis.google.com%2Fa%2Ffeeds%2Fcompliance%2Faudit%2F+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fapps.groups.migration+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fapps.groups.settings+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.reports.audit.readonly+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.reports.usage.readonly+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsiteverification+https%3A%2F%2Fsites.google.com%2Ffeeds+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fediscovery&login_hint=admin%40example.com&access_type=offline

Enter verification code: 4/TgBU6BhAYOahaGnAVfWOkMKNuzExX41KAk5SE6lFqbo4_7bnfMWbVPA
Authentication successful.
Client OAuth2 File: /home/basuta/.gam/oauth2.txt, Created

Authorize the GAM service account, replacing the email address below with that of a non-admin user in your domain.

gam user nonadminuser@example.com check serviceaccount

It’s expected that the checks will fail the first time the command is run.

User: nonadminuser@example.com, Check 14 Scopes
  Scope: https://mail.google.com/                                    , Checked: FAIL (1/14)
  Scope: https://sites.google.com/feeds                              , Checked: FAIL (2/14)
  Scope: https://www.google.com/m8/feeds                             , Checked: FAIL (3/14)
  Scope: https://www.googleapis.com/auth/activity                    , Checked: FAIL (4/14)
  Scope: https://www.googleapis.com/auth/calendar                    , Checked: FAIL (5/14)
  Scope: https://www.googleapis.com/auth/drive                       , Checked: FAIL (6/14)
  Scope: https://www.googleapis.com/auth/gmail.settings.basic        , Checked: FAIL (7/14)
  Scope: https://www.googleapis.com/auth/gmail.settings.sharing      , Checked: FAIL (8/14)
  Scope: https://www.googleapis.com/auth/plus.me                     , Checked: FAIL (9/14)
  Scope: https://www.googleapis.com/auth/plus.profile.agerange.read  , Checked: FAIL (10/14)
  Scope: https://www.googleapis.com/auth/plus.profile.language.read  , Checked: FAIL (11/14)
  Scope: https://www.googleapis.com/auth/spreadsheets                , Checked: FAIL (12/14)
  Scope: https://www.googleapis.com/auth/userinfo.email              , Checked: FAIL (13/14)
  Scope: https://www.googleapis.com/auth/userinfo.profile            , Checked: FAIL (14/14)
ERROR: Some scopes failed! Please go to:

https://admin.google.com/example.com/AdminHome?#OGX:ManageOauthClients

and grant Service Account Client name: 123456789012345678901 access to scopes:

https://mail.google.com/,
https://sites.google.com/feeds,
https://www.google.com/m8/feeds,
https://www.googleapis.com/auth/activity,
https://www.googleapis.com/auth/calendar,
https://www.googleapis.com/auth/drive,
https://www.googleapis.com/auth/gmail.settings.basic,
https://www.googleapis.com/auth/gmail.settings.sharing,
https://www.googleapis.com/auth/plus.me,
https://www.googleapis.com/auth/plus.profile.agerange.read,
https://www.googleapis.com/auth/plus.profile.language.read,
https://www.googleapis.com/auth/spreadsheets,
https://www.googleapis.com/auth/userinfo.email,
https://www.googleapis.com/auth/userinfo.profile

This is because we need to grant the service account access to the scopes. Follow the instructions and re-run the command.

gam user nonadminuser@example.com check serviceaccount

This time, everything will pass.

User: nonadminuser@example.com, Check 14 Scopes
  Scope: https://mail.google.com/                                    , Checked: PASS (1/14)
  Scope: https://sites.google.com/feeds                              , Checked: PASS (2/14)
  Scope: https://www.google.com/m8/feeds                             , Checked: PASS (3/14)
  Scope: https://www.googleapis.com/auth/activity                    , Checked: PASS (4/14)
  Scope: https://www.googleapis.com/auth/calendar                    , Checked: PASS (5/14)
  Scope: https://www.googleapis.com/auth/drive                       , Checked: PASS (6/14)
  Scope: https://www.googleapis.com/auth/gmail.settings.basic        , Checked: PASS (7/14)
  Scope: https://www.googleapis.com/auth/gmail.settings.sharing      , Checked: PASS (8/14)
  Scope: https://www.googleapis.com/auth/plus.me                     , Checked: PASS (9/14)
  Scope: https://www.googleapis.com/auth/plus.profile.agerange.read  , Checked: PASS (10/14)
  Scope: https://www.googleapis.com/auth/plus.profile.language.read  , Checked: PASS (11/14)
  Scope: https://www.googleapis.com/auth/spreadsheets                , Checked: PASS (12/14)
  Scope: https://www.googleapis.com/auth/userinfo.email              , Checked: PASS (13/14)
  Scope: https://www.googleapis.com/auth/userinfo.profile            , Checked: PASS (14/14)
All scopes passed!
Service Account Client name: 123456789012345678901 is fully authorized.

GAM is now set up and ready to be used. We can verify it’s working correctly by running:

gam info domain

This will display information about your domain.

Customer ID: A00a0a0aa
Primary Domain: example.com
Customer Creation Time: 2018-07-21T21:40:15.041Z
Primary Domain Verified: True
Default Language: en
Address:
  contactName: Admin
  organizationName: example.com
  countryCode: GB
Admin Secondary Email: admin@example.com
User counts as of 2018-08-31:
  G Suite Business Licenses: 1000
  G Suite Business Users: 1000
  Total Users: 1000
Maximum Users: 5000
Current Users: 1000
Domain Edition: premier
Customer PIN: 12345678
SSO Enabled: false
SSO Signon Page:
SSO Logout Page:
SSO Password Page:
SSO Whitelist IPs:
SSO Use Domain Specific Issuer: false

We have successfully installed GAM!

If you want to learn more about some of the things you can do with GAM, visit the Wiki or Google Group.