Preface#
A few days ago, I switched the blog's Shiro to the sponsored version of the theme Shiroi with more features (closed source). Since the original repository is a private repository and does not allow direct forking, in this case, I need to keep my repository in sync with the original repository on a regular basis. I didn't want to push the changes myself using the command line, so I turned to GitHub Actions while stumbling upon this topic.
Using Existing Solutions#
In general, for simple requirements like this, there are usually existing solutions and experiences from others, so it's just a matter of finding the right one to use. Similar apps and actions templates are available, such as Pull and GitHub Sync, but I ultimately chose aormsby/Fork-Sync-With-Upstream-action as the solution for my synchronization configuration. It supports pulling from private repositories using an access token, allows you to choose the branches of the upstream and target repositories, and provides examples, which is very user-friendly.
With the solution in place, the next step is to create a separate sync branch dedicated to syncing with the upstream repository, while keeping the modifications on the main branch.
Okay, let's create a workflow file to configure the synchronization. Here, I created a sync branch to ensure consistency with the upstream repository, and the next step is to sync it with the main branch.
name: 'Upstream Sync'
on:
schedule:
- cron: '0 0 * * *'
# Run every day at 8 AM (UTC+8)
workflow_dispatch: # Click the button on the Github repo!
inputs:
sync_test_mode: # Adds a boolean option that appears during manual workflow run for easy test mode config
description: 'Fork Sync Test Mode'
type: boolean
default: false
jobs:
sync_latest_from_upstream:
runs-on: ubuntu-latest
name: Sync latest commits from upstream repo
steps:
# REQUIRED step
# Step 1: run a standard checkout action, provided by github
- name: Checkout target repo
uses: actions/checkout@v3
with:
# optional: set the branch to checkout,
# sync action checks out your 'target_sync_branch' anyway
# Switch to the sync branch for synchronization
ref: sync
# REQUIRED if your upstream repo is private (see wiki)
persist-credentials: false
# REQUIRED step
# Step 2: run the sync action
- name: Sync upstream changes
id: sync
uses: aormsby/Fork-Sync-With-Upstream-action@v3.4.1
with:
target_sync_branch: sync
# REQUIRED 'target_repo_token' exactly like this!
target_repo_token: ${{ secrets.GITHUB_TOKEN }}
upstream_sync_branch: main
upstream_sync_repo: upstream/Shiroi
upstream_repo_access_token: ${{ secrets.UPSTREAM_REPO_SECRET }}
# Set test_mode true during manual dispatch to run tests instead of the true action!!
test_mode: ${{ inputs.sync_test_mode }}
# Step 3: Display a sample message based on the sync output var 'has_new_commits'
- name: New commits found
if: steps.sync.outputs.has_new_commits == 'true'
run: echo "New commits were found to sync."
- name: No new commits
if: steps.sync.outputs.has_new_commits == 'false'
run: echo "There were no new commits."
- name: Show value of 'has_new_commits'
run: echo ${{ steps.sync.outputs.has_new_commits }}
There are only a few things that need to be modified. First, the scheduled trigger time (cron), the ref in Step 1 and the target_sync_branch in Step 2 need to be consistent and point to the sync branch of your own project (in this case, "sync"). Also, modify the information of the upstream repository and the source branch (upstream_sync_repo, upstream_sync_branch).
After writing the workflow, configure the relevant variables. First, prepare an access token that can access the private upstream repository. In this case, I chose the Classic option because Fine-grained tokens seem to be unable to access private projects that I don't own (not sure about this). Copy the generated token and go to Repository settings -> Secrets and Variables -> Actions to add a Repository secret. The variable name should be UPSTREAM_REPO_SECRET, and the value should be the token generated earlier. Then go to Actions Setting and change the Workflow permissions to Read and write, and save.
Testing#
Go to the Workflow page you configured and enable Test Mode to test the configuration for any issues (this only tests the configuration and does not involve any operations on the repository). If all configurations show as PASSED, you can uncheck Test Mode and run the workflow for real.
Diary of Pitfalls#
- CI log shows
fatal: could not read Username for 'https://github.com': No such device or address
After looking at the original issues, this is a known bug that does not affect usage, so I left it as it is.
fatal: refusing to merge unrelated histories
This issue occurred on the second day after manually resolving conflicts and merging the PR. I suspect that the sync branch contains private changes (my own modified commits), which prevents synchronization. There was no other way, so I had to manually pull the source code of the repository and force-push it.
- Sync failure due to changes in the upstream repository's workflow, with the following error message:
! [remote rejected] sync -> sync (refusing to allow a GitHub App to create or update workflow `.github/workflows/build.yml` without `workflows` permission)
error: failed to push some refs to 'https://github.com/wuhang2003/Shiroi.git'
Before discussing how to solve this problem, let me first complain about the Token generated by GitHub Actions. Why do I say that?
This is the permission of the one-time Token generated by Actions:
As you can see, it has write permissions for Actions, but the error message says it doesn't have the workflow permission. I'm sure you can see that something is not right. This is what I want to complain about. I only found one related discussion in the official discussion area, and there hasn't been a reply yet...
Of course, the solution to the permission issue is simple and straightforward. Just create a Personal Access Token (PAT) with workflow and repo permissions, and add it to the secrets as UPSTREAM_REPO_SECRET to replace the existing ${{ secrets.GITHUB_TOKEN }}.
After This...#
Initially, I only used the above workflow to sync the changes and manually merge them into the main branch. However, after a few days of tinkering, I found it too cumbersome and not lazy enough. Additionally, if there were conflicts, I had to resolve them manually, which was troublesome. So I also explored how to automatically create a PR/automatically merge into the main branch after syncing. Based on the actual situation, I chose the method of rebasing the changes (rebase introduction), which ensures that my changes are not mixed with the entire commit history, but are unified after the upstream commits. The workflow I used for this is tiacsys/git-rebase. I added the rebase step to the original CI configuration file sync.yml. The configuration is as follows:
- name: git-rebase
uses: tiacsys/git-rebase@v1
with:
repo: "git@github.com:wuhang2003/Shiroi.git"
source_branch: "main" # The main branch to be modified
destination_branch: "sync" # The upstream sync branch
ssh_private_key: ${{ secrets.SSH_PRIVATE_KEY }}
After that, generate an SSH Key for the commit, give the public key write permission in the repository's deploy ssh, and set the private key as the value of the Actions secret SSH_PRIVATE_KEY. Done! You can now click the button to trigger the workflow and test the effect.
No issues, you can leave now.
By doing this, I have created a workflow that can ensure regular synchronization (with manual triggering support) of a private upstream repository and sync the changes to the main branch after rebasing. Of course, the additional rebase workflow can be modified according to your needs to merge changes using the Merge method or automatically create a PR and merge it after evaluating it. There are many possibilities. I can only say that being lazy/tinkering is the first productivity (I'm sure).
This article is synchronized and updated to xLog by Mix Space.
The original link is https://zwh.moe/posts/technology/actions-sync