I am migrating away from the GAFAM and co. Part 1/n
It’s been a long time that I want to leave the GAFAM. I will not speak much here about the ethical reasons, not that they are not interesting, but because I first want to document the technical part. Mainly for my future self, but it could help someone else.
I have two objectives: 1) be ready for the day when github.com and gitlab.com will delete my accounts and all the code there (it may happen) and 2) have a local backup. Apart from my engagement in the FLOSS mentality, I am not (yet) a targeted activist, but it may happen on day. So until point 1) is reached, I can consider that the code hosted on github.com is more or less a backup. Yes, a backup should be safe, but I am not sure that a hard drive on my desk is more secured or safe. I am not a system engineer, my home server itself is only partly backuped, and I can live with this risk. Unfortunately, my days are only 24 hours long and I have to sleep at night. I will thus focus on point 2).
Before I explain what I did, here is a little summary of my usage of the online git providers:
- github.com: my historic online git platform. I have 62 repos, public or private, with personal code, work, forks, etc. Inspired by Ploum (https://ploum.net/2023-02-22-leaving-github.html) I decided to leave github, but to go where? Cf next items.
- gitlab.com: I moved my work-related projects there
- framagit.org: I moved my personnal projects there
Self host forgejo
I decided to self host forgejo, I already have a raspberry pi running yunohost so it was very easy using the web-ui (https://apps.yunohost.org/app/forgejo).
Migrate / backup github
Forgejo offers the option to mirror github repos. Perfect! However, after trying two repos by hand I decided to write a small script to do the work for me.
Let’s use the github API to get all urls of my repositories (Github API). It seems that I get also some other repositories (that I starred? followed? No idea), but it is not a problem as they are repos I am interested in.
# edit your github token
GITHUB_TOKEN=XXXX
all_repos=$(curl -L \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer $GITHUB_TOKEN" \
-H "X-GitHub-Api-Version: 2026-03-10" \
https://api.github.com/user/repos?per_page=100 \
)
# I can use per_page=100 as I have less than 100 repos
Now I need to parse the json to get all urls, and remove everything I don’t need (yes it could probably be done is a smarter way, but this solution works).
# we want to filter out items like the following and extract url
# "html_url": "https://github.com/user_XXX/repo_XXX",
all_urls=$(echo "$all_repos" | grep "html_url.*//.*/.*/.*" | sed 's/ "html_url": "//' | sed 's/",//')
And now, use the forgejo API to mirror these urls (Forgejo API):
FORGEJO_URL=https://XXX
FORGEJO_USR=XXX
FORGEJO_TOKEN=XXX
for url in $all_urls
do
echo "migrate $url"
curl -s \
-H "Authorization: Bearer $FORGEJO_TOKEN" \
-X POST \
-H "Content-Type: application/json" \
--data "{\"auth_token\":\"$GITHUB_TOKEN\", \"clone_addr\": \"$url\", \"repo_name\": \"GITHUB.COM_$(basename $url)\", \"mirror\": true}" \
$FORGEJO_URL/api/v1/repos/migrate
sleep 5 # to not saturate my small server
done
And voilà! Not very delicate, but it worked for me. If a repository already exists in
the forgejo instance, an error will be returned, so it seems safe to rerun the script
multiple times: it will only add new repositories.
For a regular use, Gitea mirror may be a better option https://gitea-mirror.raylabs.io/.
Migrate / backup gitlab
The forgejo api part is already written, what is left is to get all gitlab repositories
(see Gitlab API),
I only need personal repositories (but you could replace GITLAB_USR by a gitlab group
if you are part of groups, and replace users by groups in the API call).
# edit your gitlab token and username
GITLAB_TOKEN=XXX
GITLAB_USR=XXX
GITLAB_URL=https://XXX
all_repos=$(curl \
--header "Content-Type: application/json" \
--header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
"${GITLAB_URL}/api/v4/users/$GITLAB_USR/projects?per_page=100"
)
all_urls=$(printf "${all_repos//,/\\n}" | grep "web_url.*//.*/.*/.*" | sed 's/.*"web_url":"//' | sed 's/"//')
And now, mirror into forgejo:
for url in $all_urls
do
echo "migrate $url"
curl -s \
-H "Authorization: Bearer $FORGEJO_TOKEN" \
-X POST \
-H "Content-Type: application/json" \
--data "{\"auth_token\":\"$GITLAB_TOKEN\", \"clone_addr\": \"${url}.git\", \"repo_name\": \"$(basename $GITLAB_URL | tr '[:lower:]' '[:upper:]')_$(basename $url)\", \"mirror\": true}" \
$FORGEJO_URL/api/v1/repos/migrate
sleep 5 # to not saturate my small server
done