GitLab API with Python: python-gitlab Past the Getting Started Page

Q: Why does gl.projects.list() only return 20 items?

That is the default page size. Use iterator=True to page lazily through all results, or all=True to load everything into memory. Set per_page=100 either way to reduce the number of requests.

Q: What token scope do I need for the GitLab API?

read_api for read-only work, which cannot modify anything. api only when you need writes. In CI, prefer CI_JOB_TOKEN, which is injected automatically and expires with the job.

Q: Why is my python-gitlab script making so many requests?

Almost certainly the lazy-object trap. Objects from list() are partial, and accessing an attribute absent from the list response triggers a full fetch per object. Request the extra data in the list call, or use lazy=True when you only need a handle.

Q: How do I handle GitLab rate limits?

Pass retry_transient_errors=True when constructing the Gitlab object. It honours Retry-After on 429 responses and retries 5xx errors instead of raising, which matters for any long-running script.

Q: How do I commit multiple files in one commit?

Use project.commits.create() with an actions list containing one entry per file. All changes land atomically in a single commit, rather than one commit per file.

python-gitlab wraps the v4 REST API in objects, and the two things worth learning early are how pagination works and how to avoid making a request for data you already have.

The GitLab API is large and well documented, and python-gitlab covers most of it with a consistent object interface. Getting started is genuinely easy.

What separates a script that works from one that works against a real instance is handling pagination without loading everything into memory, choosing token scopes that do not grant more than you need, and recognising when the library is silently making an extra HTTP request on your behalf.

Authentication and token scopes
Pagination, done properly
The lazy-object trap
Rate limits and retries
The operations you will actually write
How this fits the rest of the stack
FAQ

Authentication and token scopes

pip install python-gitlab

import gitlab

# Hosted GitLab with a personal access token
gl = gitlab.Gitlab("https://gitlab.com", private_token="glpat-xxxxxxxxxxxx")

# Self-managed instance
gl = gitlab.Gitlab("https://gitlab.example.com", private_token=TOKEN)

# Inside CI, using the job token -- scoped to that job, no secret to manage
import os
gl = gitlab.Gitlab(
    os.environ["CI_SERVER_URL"],
    job_token=os.environ["CI_JOB_TOKEN"],
)

# Verify credentials before doing anything else
gl.auth()
print(gl.user.username)

Choose the narrowest token scope that works. read_api is enough for anything that only reads, and it cannot modify a thing. Reach for api only when you genuinely need writes.

In CI, prefer CI_JOB_TOKEN over a personal access token. It is injected automatically, scoped to the job, and expires when the job ends — so there is no long-lived credential in your variables to leak or rotate.

Never put a token in source. Read it from the environment, and if you use .env files locally make sure they are in .gitignore. A leaked personal access token with api scope is equivalent to your account.

Pagination, done properly

This is where most scripts go wrong. The default returns only the first page — twenty items — and a script that looks correct silently processes a fraction of the data.

# Only the first 20. A very common silent bug.
projects = gl.projects.list()

# Everything, loaded into a list in memory
projects = gl.projects.list(all=True)

# Everything, as a generator -- the right default for large results
for project in gl.projects.list(iterator=True):
    print(project.path_with_namespace)

# Larger pages mean fewer round trips; 100 is the maximum
for issue in gl.issues.list(iterator=True, per_page=100, state="opened"):
    print(issue.title)

iterator=True is the one to reach for by default. It fetches pages lazily as you consume them, so memory stays flat regardless of result size and you can break out early without downloading the rest.

all=True fetches everything up front. On an instance with thousands of projects that is a long wait and a large amount of memory for data you may only partly need.

Always set per_page=100. The default of 20 means five times as many HTTP requests for the same data, and every request counts against rate limits.

The lazy-object trap

This one is subtle and costs real time on large scripts. Objects returned from a list() call are partial — the API’s list endpoints return a summary, not the full representation.

# The list endpoint returns a summary of each project
for project in gl.projects.list(iterator=True, per_page=100):
    print(project.name)              # present in the list response, free
    print(project.statistics)        # NOT in the list response
    # -> triggers a separate GET for this project, on every iteration

Accessing an attribute that was not in the list response makes python-gitlab fetch the full object. Do that inside a loop over a thousand projects and you have made a thousand extra requests without writing a single explicit call.

# Ask the API to include what you need in the list response
for project in gl.projects.list(iterator=True, per_page=100, statistics=True):
    print(project.name, project.statistics["repository_size"])

# Or fetch the full object deliberately, so the cost is visible
full = gl.projects.get(project.id)

# get_all=False with lazy=True skips the fetch when you only need the ID
project = gl.projects.get(project_id, lazy=True)
project.files.get(file_path="README.md", ref="main")

lazy=True is the inverse optimisation: it builds an object from an ID without fetching anything, which is right when you only need it as a handle for a subsequent call. No wasted request for data you will not read.

Rate limits and retries

GitLab.com enforces rate limits per user and per endpoint, and self-managed instances often set their own. A script iterating thousands of objects will hit them.

import gitlab

gl = gitlab.Gitlab(
    "https://gitlab.com",
    private_token=TOKEN,
    # Honour Retry-After on 429 responses instead of failing
    retry_transient_errors=True,
    timeout=30,
)

retry_transient_errors=True handles 429 and 5xx responses by waiting and retrying rather than raising. For any long-running script this is the difference between finishing and dying two thirds of the way through.

Handle the errors that are not transient explicitly, because they mean something you should not retry.

from gitlab.exceptions import GitlabGetError, GitlabAuthenticationError

try:
    project = gl.projects.get("group/does-not-exist")
except GitlabGetError as exc:
    if exc.response_code == 404:
        print("project not found")
    elif exc.response_code == 403:
        print("token lacks permission for this project")
    else:
        raise
except GitlabAuthenticationError:
    print("token is invalid or expired")
    raise

Distinguishing 404 from 403 matters more than it looks on GitLab, because it deliberately returns 404 for projects you cannot see. A 404 may mean the project does not exist, or that your token cannot see it — and those need different responses.

The operations you will actually write

A handful of patterns cover most real automation.

project = gl.projects.get("mygroup/myapp")

# Read a file at a ref
f = project.files.get(file_path="config/settings.yml", ref="main")
content = f.decode().decode("utf-8")

# Commit several changes in one API call -- atomic, one commit
project.commits.create({
    "branch": "automation/bump-version",
    "start_branch": "main",
    "commit_message": "chore: bump version to 1.4.2",
    "actions": [
        {"action": "update", "file_path": "VERSION", "content": "1.4.2\n"},
        {"action": "update", "file_path": "CHANGELOG.md", "content": changelog},
    ],
})

# Open a merge request
mr = project.mergerequests.create({
    "source_branch": "automation/bump-version",
    "target_branch": "main",
    "title": "Bump version to 1.4.2",
    "remove_source_branch": True,
})

# Trigger a pipeline and watch it
pipeline = project.pipelines.create({"ref": "main"})
pipeline.refresh()
print(pipeline.status)

# Manage CI/CD variables
project.variables.create({
    "key": "DEPLOY_TARGET",
    "value": "production",
    "masked": True,
    "protected": True,
})

The multi-action commit is the one worth remembering. Creating files one at a time produces one commit each and a messy history; the actions array applies them all in a single atomic commit.

When you need an endpoint the library does not wrap, use the low-level HTTP methods rather than reaching for requests separately — you keep the authentication, retry, and pagination handling.

data = gl.http_get("/projects/123/repository/contributors")
gl.http_post("/projects/123/hooks", post_data={"url": "https://example.com/hook"})

How this fits the rest of the stack

Most GitLab API scripts exist to move something from a repository into a running environment — bumping a version, triggering a pipeline, promoting a build. The API work is usually the easy half; the deploy on the other side is where the complexity sits. RunxBuild connects to a repository and builds on push, with logs and rollback attached, so the pipeline you are scripting has somewhere predictable to land. The RunxBuild hosting calculator shows what the resulting services cost before you wire the automation up.

Useful related references:

FAQ

Why does gl.projects.list() only return 20 items?

That is the default page size. Use iterator=True to page lazily through all results, or all=True to load everything into memory. Set per_page=100 either way to reduce the number of requests.

What token scope do I need for the GitLab API?

read_api for read-only work, which cannot modify anything. api only when you need writes. In CI, prefer CI_JOB_TOKEN, which is injected automatically and expires with the job.

Why is my python-gitlab script making so many requests?

Almost certainly the lazy-object trap. Objects from list() are partial, and accessing an attribute absent from the list response triggers a full fetch per object. Request the extra data in the list call, or use lazy=True when you only need a handle.

How do I handle GitLab rate limits?

Pass retry_transient_errors=True when constructing the Gitlab object. It honours Retry-After on 429 responses and retries 5xx errors instead of raising, which matters for any long-running script.

How do I commit multiple files in one commit?

Use project.commits.create() with an actions list containing one entry per file. All changes land atomically in a single commit, rather than one commit per file.