Preface#
A few days ago, I switched the blog's Shiro to the sponsored version of the theme Shiroi (closed source) with more features. Since the original repository is private and does not allow direct forking, in this case, I need to keep my repository in sync with the original repository regularly. I didn't want to push the command line myself, so I turned my attention to GitHub Actions and wrote this article while stepping on the pit.
Using Existing Solutions#
In general, for simple requirements like this, there are usually existing solutions and experiences from previous users. It's just a matter of how to use them.
There are many similar apps and actions templates available, such as Pull, GitHub Sync, etc. Finally, I chose aormsby/Fork-Sync-With-Upstream-action as the solution for writing the synchronization configuration. It supports pulling from private repositories using an access token, allows you to choose the branches of the upstream repository and the target repository, and provides examples, which is very user-friendly.
Now that I have found the solution, the next step is to create a separate sync branch for syncing with the upstream repository, and make sure that all modifications are made on the main branch.
Okay, let's create a workflow file to configure it. Here, I created a sync branch to ensure consistency with the upstream repository, and the next step is to sync it to the main branch.
name: 'Upstream Sync'
on:
schedule:
- cron: '0 0 * * *'
# Run every day at 8:00 (UTC+8)
workflow_dispatch: # click the button on Github repo!
inputs:
sync_test_mode: # Adds a boolean option that appears during manual workflow run for easy test mode config
description: 'Fork Sync Test Mode'
type: boolean
default: false
jobs:
sync_latest_from_upstream:
runs-on: ubuntu-latest
name: Sync latest commits from upstream repo
steps:
# REQUIRED step
# Step 1: run a standard checkout action, provided by github
- name: Checkout target repo
uses: actions/checkout@v3
with:
# optional: set the branch to checkout,
# sync action checks out your 'target_sync_branch' anyway
# Switch to the sync branch to be synced
ref: sync
# REQUIRED if your upstream repo is private (see wiki)
persist-credentials: false
# REQUIRED step
# Step 2: run the sync action
- name: Sync upstream changes
id: sync
uses: aormsby/Fork-Sync-With-Upstream-action@v3.4.1
with:
target_sync_branch: sync
# REQUIRED 'target_repo_token' exactly like this!
target_repo_token: ${{ secrets.GITHUB_TOKEN }}
upstream_sync_branch: main
upstream_sync_repo: upstream/Shiroi
upstream_repo_access_token: ${{ secrets.UPSTREAM_REPO_SECRET }}
# Set test_mode true during manual dispatch to run tests instead of the true action!!
test_mode: ${{ inputs.sync_test_mode }}
# Step 3: Display a sample message based on the sync output var 'has_new_commits'
- name: New commits found
if: steps.sync.outputs.has_new_commits == 'true'
run: echo "New commits were found to sync."
- name: No new commits
if: steps.sync.outputs.has_new_commits == 'false'
run: echo "There were no new commits."
- name: Show value of 'has_new_commits'
run: echo ${{ steps.sync.outputs.has_new_commits }}
The only things that need to be modified are the scheduled trigger time (cron), the ref in Step 1 and the target_sync_branch in Step 2, which should be consistent and point to the sync branch of your own project (in this case, it is "sync"), and the information of the upstream repository and the source branch (upstream_sync_repo, upstream_sync_branch).
After writing the workflow, configure the relevant variables. First, prepare an access token that can read the private upstream repository. Here, I chose Classic because Fine-grained tokens seem to be unable to read private projects that do not belong to me (not sure). Copy the generated token and go to Repository settings -> Secrets and Variables -> Actions to add a Repository secret. The variable name is UPSTREAM_REPO_SECRET, and the value is the token generated just now. Save it. Then go to Actions Setting and change the Workflow permissions to Read and write, and save it.
Testing#
Go to the Workflow page you configured and check the Test Mode to see if there are any issues with the configuration (at this time, only the configuration is tested, and no operations are performed on the repository). If all configurations show PASSED, you can uncheck the box and run it formally.
Pitfall Diary#
- CI log shows
fatal: could not read Username for 'https://github.com': No such device or address
After looking at the original issues, this is a known bug that does not affect usage, so I left it as it is.
fatal: refusing to merge unrelated histories
This problem occurred on the second day after manually resolving conflicts and merging the PR. It is speculated that the sync branch contains private changes (self-modified commits), which prevents synchronization. There is no other way but to manually pull the repository source code and force-push it.
- Sync failure caused by changes to the workflow in the upstream repository, error example:
! [remote rejected] sync -> sync (refusing to allow a GitHub App to create or update workflow `.github/workflows/build.yml` without `workflows` permission)
error: failed to push some refs to 'https://github.com/wuhang2003/Shiroi.git'
Before discussing how to solve this problem, let's first curse the Token generated by GitHub Actions. Why do I say that?
This is the permission of the Token generated by Actions:
As you can see, it has write permissions for Actions, but the log says it doesn't have the workflow permission. I believe you should notice that something is wrong. This is what I want to complain about. I only found one related discussion in the official discussion area, and there is no reply yet...
Of course, the solution to the permission issue is also simple and straightforward. Create a new Personal Access Token (PAT) with workflow and repo permissions, and add it to the secrets as UPSTREAM_REPO_SECRET to replace the existing ${{ secrets.GITHUB_TOKEN }}.
After This...#
Initially, I only used the above workflow to sync and manually merge it into the main branch. However, after a few days of tinkering, I felt that it was too cumbersome and I was too lazy. Moreover, if there were conflicts, I had to resolve them myself, which was troublesome. So I also explored how to automatically create a PR/automatically merge it into the main branch and push it. Later, based on the actual situation, I chose the method of rebasing to merge the changes (Introduction to Rebase), the advantage of which is that it does not mix my changes with the entire commit history, but instead puts them after the upstream commits. The workflow used here is tiacsys/git-rebase. Simply add the rebase step to the original CI configuration sync.yml. The configuration is as follows:
- name: git-rebase
uses: tiacsys/git-rebase@v1
with:
repo: "git@github.com:wuhang2003/Shiroi.git"
source_branch: "main" # The main branch to be modified
destination_branch: "sync" # The upstream sync branch
ssh_private_key: ${{ secrets.SSH_PRIVATE_KEY }}
After that, generate an SSH Key for the commit, give the public key write permission in the repository's deploy ssh, and set the private key as the value of the Actions secret SSH_PRIVATE_KEY. Done! You can click the button to trigger the Workflow and test the effect.
No issues, you can leave now.
After going through all this, a workflow that can ensure regular synchronization (and support manual triggering) of a private upstream repository and sync it to the main branch after rebasing has been completed. Of course, the additional rebase workflow can also be modified according to your needs to merge changes using the Merge method or automatically create a PR and merge it after evaluating it. There are many ways to do it. I can only say that being lazy/tinkering is the most productive thing (I'm sure)
This article is synchronized and updated to xLog by Mix Space.
The original link is https://zwh.moe/posts/technology/actions-sync