When a GitLab CI/CD pipeline needs to persist job output or a rendered report, it will typically save it as an artifact on the job, or perhaps write it to an external storage service or as a GitLab Release archive. But it is also capable of pushing this file to its own git repository, stored alongside your source code.

In this article, I will take you through the steps needed to add, commit, and push a file generated in a GitLab pipeline into its GitLab source repository.

Overview

We are going to use git commit and push statements in the script section of a pipeline job in order to add a file to its source code repository. However, we need to change the git credentials, because the one used by the default pipeline does not have adequate permission to write to the repository.

To workaround this limitation we will create a project-level Access Token that can write to the repository and then add it as a CI/CD variable that is accessible from the pipeline.

Project-level Access Token

The first thing you need to do is generate a GitLab project-level Access Token that is capable of reading and writing to the repository.

From your GitLab project, navigate to Settings > Access Tokens and press “Add New Token”. Use

Token name: for-ci-cd
Expiration date: one year from today
Role: Developer
Scopes: read_repository, write_repository

The Access Token value will only be presented to you once, copy-paste this value and use it in the next step.

Note that project-level Access Tokens are not available if your git repository is part of a Group (unless you are on a paid subscription).

Project-level CI/CD Variable

From your GitLab project, navigate to Settings > CI/CD and then scroll down and press “Variables”. Press “Add Variables”. Use

Key: PROJECT_ACCESS_TOKEN
Value: <paste Access Token>
Check “Mask variable”

This makes the Access Token value available in the pipeline as the variable PROJECT_ACCESS_TOKEN.

Pipeline job that adds file to git repo

The full .gitlab-ci.yml is posted here, but let me go through the critical sections.

Job that creates artifact file

First, we create a very simple Artifact file “generated/file.txt”.

# create file to be added to git repo
create_generated_file:
  stage: build
  script:
    - mkdir generated
    - dtime=$(date +%Y%m%dT%H%M%S)
    - echo "$dtime file created in ${CI_PROJECT_NAME}-${CI_JOB_NAME}-${CI_COMMIT_REF_SLUG}" | tee generated/file.txt
  artifacts:
    name: generated
    paths:
      - "generated/*.txt"

Job that adds file to git repo

Using the “generated/file.txt”, we then add it to the git repository as “reports/file-<timestamp>.txt” and use git to commit and push it to the current branch.

add_generated_file_to_git_repo:
  # makes artifact from previous job available
  needs:
    - create_generated_file
  script:
    # configure git with email and name of entity doing commit, pulled from std pipeline vars
    - git --version
    - git config --global user.email "$GITLAB_USER_EMAIL"
    - git config --global user.name "$GITLAB_USER_NAME"
    - echo "email/login/name $GITLAB_USER_EMAIL/$GITLAB_USER_LOGIN/$GITLAB_USER_NAME"
    - git config --global -l

    # transform pipeline git url to use our project access token name and credentials
    - git remote -v
    - url_host=`git remote get-url origin | sed -e "s/https:\/\/gitlab-ci-token:.*@//g"`
    - echo https://for-ci-cd:${PROJECT_ACCESS_TOKEN}@${url_host}
    - git remote set-url origin "https://for-ci-cd:${PROJECT_ACCESS_TOKEN}@${url_host}"
    - git remote -v

    # get latest from branch
    - echo git checkout of branch ${WHICH_BRANCH}
    - git fetch --all && git checkout ${WHICH_BRANCH}
    - git pull -r origin ${WHICH_BRANCH}

    # take artifact from previous job, create timestamped file in reports directory
    - mkdir -p reports
    - dtime=$(date +%Y%m%dT%H%M%S)
    - echo "dtime = $dtime"
    - cp generated/file.txt reports/file-$dtime.txt

    # add, commit, and push new file (skip any workflow)
    - git status
    - ls -l reports/
    - git add reports/file-$dtime.txt
    - git commit -a -m "added report for branch ${WHICH_BRANCH} at $dtime"
    - git push -o ci.skip

In order to use git we first have to setup a .gitconfig, which is done using the predefined variables from the GitLab pipeline.

Then we must change the default git credentials used by the pipeline:

https://gitlab-ci-token:MASKEDVALUE@gitlab.com/fabianlee/gitlab-pipeline-add-files-to-repo.git

Modify to the credentials of the PROJECT_ACCESS_TOKEN created earlier:

https://for-ci-cd:$PROJECT_ACCESS_TOKEN@gitlab.com/fabianlee/gitlab-pipeline-add-files-to-repo.git

This transformation is done with the sed command and our new values applied with ‘git remote set-url’.

The gitlab pipeline runner only does a shallow fetch, so we checkout our branch and pull the latest before moving on to add, commit, and then finally push the new file into the git repository.