Gcloud - How to Automate Installation of Gcloud on a Server

Google Cloud SDK - Automate gcloud auth login web flow

Here's a quick check to see if you have the right --key-file .json file: it must contain a client_email field/attribute.

Automate gcloud components update

You're looking for the --quiet flag.

From gcloud --help:

 --quiet, -q
    Disable all interactive prompts when running gcloud commands. If input
    is required, defaults will be used, or an error will be raised.

This is generally a flag you'll want for non-interactive contexts.

You may also set the CLOUDSDK_CORE_DISABLE_PROMPTS environment variable to a non-empty value:

export CLOUDSDK_CORE_DISABLE_PROMPTS=1
gcloud components update # This works for all gcloud commands

Can I automate `gcloud auth login`?

I'm going to answer my own question here.

My Solution

Instead of using gsutil, I decided to use the Google Cloud Client Libraries.

What I did:

gsutil cp my_file.tgz gs://my_bucket

What I am doing now:

from gcloud import storage

# key file is located in my current directory
os.environ.get('GOOGLE_APPLICATION_CREDENTIALS', 'gcloud-auth.json')

client = storage.Client()
bucket = client.get_bucket("my_bucket")
blob = bucket.blob("my_file.tgz")
blob.upload_from_filename("my_file.tgz")

Hindsight 20/20

After getting the above solution working, it seems if I also set the environment variable, GOOGLE_APPLICATION_CREDENTIALS, my gsutil should've worked too. (untested)

Automating gsutil commands

You should be able to do this without diving in too deep to the implementation of authentication for gsutil.

If you're using standalone gsutil (if you installed via this method), the instructions in the linked question are still valid (as Travis points out).

If you'd like to continue using the gsutil supplied via the Cloud SDK, you should use service accounts. Service accounts are the preferred method of authenticating on headless machines or in non-interactive contexts.

Your flow would look something like the following:

1. Create a service account via the Google Cloud Developers Console.
2. On the remote machine, install the Cloud SDK and gsutil. If you're not installing interactively, it's better to skip the curl ... | bash method. Instead, download this install archive, extract it, and run the install.sh script. This script has options (visible with --help); if you specify choices to all of these options, it won't prompt you.
3. Copy the service account to the remote machine. Run gcloud auth activate-service-account --key-file=/path/to/service-account.json.
4. Run gsutil. You should be appropriately authenticated.

Authenticate gcloud service account in automated fashion

The command to configure the Google Cloud SDK CLI from service account Json file:

gcloud auth activate-service-account test@development-123456.iam.gserviceaccount.com --key-file=service_account.json

Replace the email address test@... with your service account email address.
Replace the Json filename service_account.json with the filename that you are using for your service account credentials.

The service account email address is inside the service account json file as client_email.

If you would like more information I wrote a number of articles on Google Credentials and OAuth:

Google Cloud – Setting up Gcloud with Service Account Credentials

---

Related Topics