Open a File Directly from a Gitlab Private Repository

Open a file directly from a GitLab private repository

With Chris's valuable help, here is how you can run a script (drupal .make file in my case) from a GitLab server. (Probably it works for GitHub but I didn't test it. Maybe the syntax will be a bit different). (Of course this works for any type of script)

It can be done using the authentication tokens. Here is the documentation of the GitLab's API and here is the GitHub's API

For convenient I will use the https://gitlab.com as the example server.

• Go to https://gitlab.com/profile/account and find your "Private token"

• Then print the list of the projects and find the id of your project you are looking for

  curl https://gitlab.com/api/v3/projects?private_token=<your_private_token>

  or go there with your browser (a json viewer will help a lot)

• Then print the list of the files that are on this project and find the id of your file you are looking for

  curl https://gitlab.com/api/v3/projects/<project_id>/repository/tree?private_token=<your_private_token>

• Finally get / run the file!

  curl https://gitlab.com/api/v3/projects/<project_id>/repository/raw_blobs/<file_id>?private_token=<your_private_token>

In case you want to run the script (drupal .make)

drush make https://gitlab.com/api/v3/projects/<project_id>/repository/raw_blobs/<file_id>?private_token=<your_private_token> <drupal_folder>

(If you are here looking for a workflow to integrate GitLab with Aegir .make platforms without using tokens (maybe SSH?) please make a thread and paste here the link.)

EDIT

You can get the file without the project_id by using the encoded project name. For example the my-user-name/my-project will become: my-user-name%2Fmy-project

Making a single file of a private repo on Gitlab to be publicly accessible

If you can use curl, you can use the GitLab API to get a raw file from repository. You'd need to add your private-token as well to get this file.

For example:

curl --request GET --header 'PRIVATE-TOKEN: YOUR_PRIVATE_TOKEN' 'https://gitlab.example.com/api/v4/projects/PROJECT_ID/repository/files/FILE_NAME/raw?ref=BRANCH' --output FILE_NAME

How to download a single file from GitLab?

Gitlab has a rest API for that.

You can GET a file from repository with curl:

curl https://gitlab.com/api/v4/projects/:id/repository/files/:filename\?ref\=:ref

For example:

curl https://gitlab.com/api/v4/projects/12949323/repository/files/.gitignore\?ref\=master

If your repository isn't public you also need to provide an access token by adding --header 'Private-Token: <your_access_token>'.

---

Links:

You can check how to find repository api id here.

Api documentation

More on tokens

There is also a python library that uses this api.

Note that this is GitLab specific solution and won't work for other hostings.

How to use Go with a private GitLab repo

This issue is now resolved in Gitlab 8.* but is still unintuitive. The most difficult challenge indeed is go get and the following steps will allow you to overcome those:

1. Create an SSH key pair. Be sure to not overwrite an existing pair that is by default saved in ~/.ssh/.

   ssh-keygen -t rsa -b 4096
   

2. Create a new Secret Variable in your Gitlab project. Use SSH_PRIVATE_KEY as Key and the content of your private key as Value.

3. Modify your .gitlab-ci.yml with a before_script.

   before_script:
     # install ssh-agent if not already installed
     - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
     # run ssh-agent
     - eval $(ssh-agent -s)
     # add the SSH key stored in SSH_PRIVATE_KEY
     - ssh-add <(echo "$SSH_PRIVATE_KEY")
     # for Docker builds disable host key checking
     - mkdir -p ~/.ssh
     - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
   

4. Add the public key from the key pair created in step 1 as a Deploy Key in the project that you need to go get.

How to get the raw content of a file through GitLab REST API?

First, just in case, don't percent encode the ".":

.../files/pom.xml/raw?...
            ^^

Second, you can check how the files endpoint was affected from v3 to v4 in the merge request 9637 and this comparison

v3:
GET /projects/:id/repository/raw_blobs/:sha
v4:
GET /projects/:id/repository/blobs/:sha/raw

You can see the examples (in v3) did not percent encode the dot.

curl --request GET --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' \
  'https://gitlab.example.com/api/v3/projects/13083/repository/files?file_path=app/models/key.rb&ref=master'

However, the v3 API only allows to get raw blobs, not a raw file.

See merge request 16834:

• Modify /projects/:id/repository/files to /projects/:id/repository/files/:filepath (:filepath should be URL-encoded)
• Move /projects/:id/repository/blobs/:sha to /projects/:id/repository/files/:filepath/raw

Only v4 API allows for a :filepath parameters.

See "Git objects SHA-1 are file contents or file names?" to decode the raw blob you get from API v3.

Get raw file from repository

The documentation mentions:

Url encoded full path to new file. Ex. lib%2Fclass%2Erb

That means you need to URL encode test.js: test%2Ejs.

(See Percent encoding: character data)

http://gitadress/api/v4/projects/id/repository/files/test%2Ejs/raw?private_token=PRIVATE_TOKEN

You can add ?ref=master to make sure to get the content from the master branch for instance.

http://gitadress/api/v4/projects/id/repository/files/test%2Ejs/raw?ref=master&private_token=PRIVATE_TOKEN

That being said, you have the gitlab-ce issue 31470 which is still open:

API to "Get raw file from repository" fails for files with dots

A fix is in progress: gitlab-ce merge_request 13370, and will be delivered for GitLab 9.5.

---

Related Topics