~renerocksai/GH-to-SH

Migrates issues from GH to SH

52fbdfb removed unused import

~renerocksai pushed to ~renerocksai/gh-sh-issues git

1 year, 11 months ago

9e73979 default status of closed is now CLOSED

~renerocksai pushed to ~renerocksai/gh-sh-issues git

1 year, 11 months ago
View project feed  RSS 

#GitHub -> SourceHut Issue Migration

This converts:

  • Issues
    • title, body, author, issue date, labels, status
    • Issue comments
  • Pull-requests
    • title, body, author, issue date, labels, status
    • PR comments
  • Labels
    • text, background-color
  • Short-refs pointing issues/PRs/discussions
    • short-refs pointing to issues/pull-requests are translated to their new SourceHut ticket IDs.
    • short-refs pointing to discussions will be replaced by inline links like this: https://github.com/user/repo/discussions/#62

#DISCLAIMER

This is both a WIP and a hack. Use it at your own risk and hack it to suit your individual needs. --> See LICENSE.

Having disclaimed all that, I can state that I've successfully migrated telekasten.nvim and am happy with the result.

However, I don't have the time to make this "the canonical non-plus-ultra" GitHub to Sourcehut migration tool. If you want to take over, feel free to use this code as inspiration - and maybe even refer to it to give credit.

#CONTRIBUTION

Contributions are welcome! Please send them to the mailing list.

A good and welcome area for improvement would be to allow for "continuous migration": checking for existing labels, tickets on SourceHut and only migrating the current delta. I might get the chance to work on this eventually, but it's not a pressing issue for me.

Quick-win contribution ideas (I might tackle and remove from here):

  • supporting comments in the .env file.
  • migrating the legacy (REST) SH API client code to the new GraphQL API so that only a GraphQL token is needed for SourceHut.
  • more useful error reporting, from a user-perspective

#Tickets, PRs, Comments

All ticket text bodies and comments get a link to the original issue/PR/comment on GitHub in the footer:

---
Imported comment. See [the original on GitHub](https://github.com/renerocksai/telekasten.nvim/pull/152#issuecomment-1183528501)

As described above, short-refs (like #27) are translated to the new SourceHut ticket IDs. Since we don't migrate discussions, short-refs pointing to discussions get translated into links to the respective discussion on GitHub.

#Tickets

In the generated SourceHut tickets, authors are provided in the format domain:username (e.g. github.com:renerocksai), so SourceHut can display a link to the user's page on the origin platform.

The issue's submission date is preserved in the created ticket.

#Comments

Due to what seems to be an API-bug on SourceHut, comment submission date and author cannot be set. Hence, all comments are prepended with a markdown block like this:

[renerocksai](https://github.com/renerocksai) wrote on 2022-07-14 00:10:59:

...

#Status

GitHub only knows status open and closed. SourceHut has more options, and requires a resolution on closing a ticket. The translation in this tool is as follows:

  • GitHub open: SourceHut status: REPORTED (default), no resolution.
  • GitHub closed: SourceHut status: CLOSED, resolution: FIXED.

#Pull Requests

GitHub pull-requests are converted to tickets. Their subject gets prepended with [PULLREQUEST], for easy text-searching. In addition, they also receive the label PULL REQUEST (see below at labels).

#Labels

All GitHub labels are imported. Since GitHub only specifies a color but SourceHut requires both a foreground and a background color, this tool uses the color for the background-color, and calculates the foreground color the same way SourceHut does it when you change a label's color in the web interface.

In addition, a label PULL REQUEST is created for labeling pull-request tickets, if it doesn't exist in your GitHub repo already. This label is subsequently applied to all pull-request tickets on SourceHut.

#Using it

First, you need to clone the repository:

# using https here; feel free to use ssh if you have a SH account
git clone https://git.sr.ht/~renerocksai/gh-sh-issues
cd gh-sh-issues

#Prerequisites

This tool is written in python. A nix shell is provided for NixOS and Nix Package Manager users. Enter it via nix-shell [ENTER].

For non-Nix users: you need the following packages installed:

#Python packages via poetry

To install all python package dependencies of this tool, run in your (nix) shell:

poetry install

From then on, enter the created environment via:

poetry shell

#Configuration

This tool is configured via an environment file called .env. See an example here:

GH_TOKEN=abcdef
GH_REPO=renerocksai/telekasten.nvim
SH_TOKEN=1234
SH_GRAPHQL_TOKEN=ABDCEF
SH_USER=~renerocksai
SH_TRACKER=testtracker

Comments are not allowed at the moment.

  • GH_TOKEN: you need to create a personal access token on GitHub
  • GH_REPO: name of your GitHub repo in the format username/repository
  • SH_TOKEN: legacy oauth token for sourcehut
    • I know... But I was too lazy to convert my initial legacy API code to GraphQL
  • SH_GRAPHQL_TOKEN: GraphQL oauth 2.0 token for sourcehut
  • SH_USER: your SourceHut username, including the ~
  • SH_TRACKER: the name of the SourceHut tracker you want to import into.

#Running it

With the configuration all set, you can start the migration. It is a 2-step process:

  1. Export from GitHub
  2. Import to SourceHut

#Exporting from GitHub

python gh-sh-issues.py

This will first download all labels, then all issues and their comments from GitHub. It wil produce the files

  • username_reponame.issues.json: contains all issues, PRs, comments of the repo
  • username_reponame.labels.json: contains all labels of the repo

#Importing into SourceHut

python sh.py

Watch it uploading labels and creating tickets. Your e-mail inbox will get flooded. Luckily, with aerc featuring vim-visual-mode style message selection, moving or deleting the messages was a breeze during the test runs.

If anything goes wrong, error messages are provided. They might not be that useful, though. They usually contain the issue and the operation that went wrong.