How to Get a Gcp Bearer Token Programmatically with Python

How to get a GCP Bearer token programmatically with python

The answer depends on your environment and how you want to create / obtain credentials.

What are Google Cloud Credentials?

Google Cloud credentials are an OAuth 2.0 token. This token has at a minimum an Access Token and optionally a Refresh Token, Client ID Token, and supporting parameters such as expiration, Service Account Email or Client Email, etc.

The important item in Google Cloud APIs is the Access Token. This token is what authorizes access to the cloud. This token can be used in programs such as curl, software such as python, etc and does not require an SDK. The Access Token is used in the HTTP Authorization header.

What is an Access Token?

An access token is an opaque value generated by Google that is derived from a Signed JWT, more correctly called JWS. A JWT consists of a header and claims (the payload) Json structures. These two Json structures are signed with the Service Account's Private Key. These values are base64 encoded and concatenated to create the Access Key.

The format of an Access Token is: base64(header) + '.' + base64(payload) + '.' + base64(signature).

Here is an example JWT:

Header:

{
  "alg": "RS256",
  "typ": "JWT",
  "kid": "42ba1e234ac91ffca687a5b5b3d0ca2d7ce0fc0a"
}

Payload:

{
  "iss": "myservice@myproject.iam.gserviceaccount.com",
  "iat": 1493833746,
  "aud": "myservice.appspot.com",
  "exp": 1493837346,
  "sub": "myservice@myproject.iam.gserviceaccount.com"
}

Using an Access Token:

Example that will start a VM instance. Replace PROJECT_ID, ZONE and INSTANCE_NAME. This example is for Windows.

curl -v -X GET -H "Authorization: Bearer <access_token_here>" ^
https://www.googleapis.com/compute/v1/projects/%PROJECT_ID%/zones/%ZONE%/instances/%INSTANCE_NAME%/start

Compute Engine Service Account:

Dustin's answer is correct for this case, but I will include for completeness with some additional information.

These credentials are automatically created for you by GCP and are obtained from the VM Instance metadata. Permissions are controlled by Cloud API access scopes in the Google Console.

However, these credentials have some limitations. To modify the credentials you must stop the VM Instance first. Additionally, not all permissions (roles) are supported.

from google.auth import compute_engine

cred = compute_engine.Credentials()

Service Account Credentials:

Until you understand all of the types of credentials and their use cases, these are the credentials that you will use for everything except for gcloud and gsutil. Understanding these credentials will make working with Google Cloud much simpler when writing programs. Obtaining credentials from a Google Service Account Json file is easy. The only item to make note of is that credentials expire (typically 60 minutes) and either need to be refreshed or recreated.

gcloud auth print-access-token is NOT recommended. Service Account Credentials are the recommended method by Google.

These credentials are created by the Console, gcloud or via programs / APIs. Permissions are assigned to the creditials by IAM and function inside Compute Engine, App Engine, Firestore, Kubernetes, etc. as well as other environments outside of Google Cloud. These credentials are downloaded from Google Cloud and stored in a Json file. Notice the scopes parameter. This defines permissions that are granted to the resulting credentials object.

SCOPES = ['https://www.googleapis.com/auth/sqlservice.admin']
SERVICE_ACCOUNT_FILE = 'service-account-credentials.json'

from google.oauth2 import service_account

cred = service_account.Credentials.from_service_account_file(
            SERVICE_ACCOUNT_FILE, scopes=SCOPES)

Google OAuth 2.0 Credentials:

These credentials are derived from a full OAuth 2.0 flow. These credentials are generated when your browser is launched to access Google Accounts for authorizing access. This process is much more complicated and requires a fair amount of code to implement and requires a built-in web server for the callback for authorization.

This method provides additional features such as being able to run everything in a browser, example you can create a Cloud Storage File Browser, but be careful that you understand the security implications. This method is the technique used to support Google Sign-In, etc. I like to use this method to authenticate users before allowing posting on websites, etc. The possibilities are endless with correctly authorized OAuth 2.0 identities and scopes.

Example code using google_auth_oauthlib:

from google_auth_oauthlib.flow import InstalledAppFlow

flow = InstalledAppFlow.from_client_secrets_file(
    'client_secrets.json',
    scopes=scope)

cred = flow.run_local_server(
    host='localhost',
    port=8088,
    authorization_prompt_message='Please visit this URL: {url}',
    success_message='The auth flow is complete; you may close this window.',
    open_browser=True)

Example code using the requests_oauthlib library:

from requests_oauthlib import OAuth2Session

gcp = OAuth2Session(
        app.config['gcp_client_id'],
        scope=scope,
        redirect_uri=redirect_uri)

# print('Requesting authorization url:', authorization_base_url)

authorization_url, state = gcp.authorization_url(
                        authorization_base_url,
                        access_type="offline",
                        prompt="consent",
                        include_granted_scopes='true')

session['oauth_state'] = state

return redirect(authorization_url)

# Next section of code after the browser approves the request

token = gcp.fetch_token(
            token_url,
            client_secret=app.config['gcp_client_secret'],
            authorization_response=request.url)

How to get a GCP identity-token programmatically with python

Great topic! And it's a long long way, and months of tests and discussion with Google.

TL;DR: you can't generate an identity token with your user credential, you need to have a service account (or to impersonate a service) to generate an identity token.

If you have a service account key file, I can share a piece of code to generate an identity token, but generating and having a service account key file is globally a bad practice.

---

I released an article on this and 2 merge requests to implement an evolution in the Java Google auth library (I'm more Java developer that python developer even if I also contribute to python OSS project) here and here. You can read them if you want to understand what is missing and how works the gcloud command today.

On the latest merge request, I understood that something is coming from google, internally, but up to now, I didn't see anything...

How to get Access Token from Google Service Account key file?

The best way is to use the AI Platform Python client library. Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path to your service account file and you are good to go.

Better to not deal with access token and refresh tokens yourself if there is a ready-made library available.

How I can get access to GCP cloud function from python code using service account?

Your example code is generating an access token. Below is a real example that generates an identity token and uses that token to call a Cloud Functions endpoint. The Function needs to have the Cloud Function Invoker role for the service account being used for authorization.

import json
import base64
import requests

import google.auth.transport.requests
from google.oauth2.service_account import IDTokenCredentials

# The service account JSON key file to use to create the Identity Token
sa_filename = 'service-account.json'

# Endpoint to call
endpoint = 'https://us-east1-replace_with_project_id.cloudfunctions.net/main'

# The audience that this ID token is intended for (example Google Cloud Functions service URL)
aud = 'https://us-east1-replace_with_project_id.cloudfunctions.net/main'

def invoke_endpoint(url, id_token):
    headers = {'Authorization': 'Bearer ' + id_token}

    r = requests.get(url, headers=headers)

    if r.status_code != 200:
        print('Calling endpoint failed')
        print('HTTP Status Code:', r.status_code)
        print(r.content)
        return None

    return r.content.decode('utf-8')

if __name__ == '__main__':
    credentials = IDTokenCredentials.from_service_account_file(
            sa_filename,
            target_audience=aud)

    request = google.auth.transport.requests.Request()

    credentials.refresh(request)

    # This is debug code to show how to decode Identity Token
    # print('Decoded Identity Token:')
    # print_jwt(credentials.token.encode())

    response = invoke_endpoint(endpoint, credentials.token)

    if response is not None:
        print(response)

Google Cloud python autheticated request to cloud run using local application default identity token

Today it's impossible. I talked about it in an article. You need a service account to generate a valid JWT token with the Google Auth library. It's the same problem with all the library and all the languages.

Last week, I pushed a merge request in the Java auth library to solve this. I don't know why Google don't implement it by itself.

On your local environment, if you want to use the same code locally and in the cloud, you have to generate a service account key file and use it with ADC. And it's sadly a security issue...

How to retrieve access-token from GCP Golang SDK?

Have a look at oauth2/google.

You may (!?) be able to use workload identity federation (using this library too and thereby avoid using a Google Service Account key) but, using a Service Account (with roles/storage.objectAdmin see GCR: Granting IAM roles) and Application Default Credentials (export GOOGLE_APPLICATION_CREDENTIALS=/path/to/your/key.json) look at DefaultTokenSource.

The Token returned by Token() from TokenSource gives you an access_token (and refresh_token and expiry).

Update

You can use the Service Account key file directly to authenticate a Docker client to GCR. This would simplify your code (and avoid refreshing). The document reiterates the caution in using Service Account keys.

---

Related Topics