Academy: Set up workflow to import issues from participating projects

[evilnick]

While this is a central repository for the Open Documentation Academy, documentation issues are likely (and properly) recorded directly in other repositories. However, for ease of use and visibility, these issues should also exist in the ODA.

There should be an established, automated and obvious way (e.g. by adding a label) to get issues included here

Possible solution: an action in each participating repo which picks up on a unique label, similar to
https://github.com/marketplace/actions/github-issue-copier

[alexvonme]

Can I please be assigned for this task? I wanted to learn more about Github actions.

[minaelee]

I can give this a whirl, though I would need someone to confirm/set up token access on a separate repository also owned by Canonical to test it.

I've tested the following workflow using two existing personal repositories, one in which I created an issue and labeled it with ODA, and one in which the GH action makes a copy of the ODA-labeled issue. The former (the repository being copied FROM) contains a GH_TOKEN repository secret with repo access.

I've tested to make sure that it only copies the issue when the ODA label is added, not when any others are.

I've added a link to the original issue in the body of the copied issue.

This is the workflow I tested, with the repo name and repo owner name changed to 'open-documentation-academy' and 'canonical'.

name: Copy Issue on ODA Label Added

on:
  issues:
    types:
      - labeled

jobs:
  copy-issue:
    if: github.event.label.name == 'ODA'
    runs-on: ubuntu-latest
    steps:
      - name: Copy issue to Open Documentation Academy repository
        uses: actions/github-script@v7
        with:
          github-token: ${{ secrets.GH_TOKEN }}
          script: |
            console.log('An issue has been labeled ODA. Copying the issue to the Open Documentation Academy repository.');
            const issue = context.payload.issue;
            const originRepo = context.payload.repository;

            const originalIssueUrl = originRepo.html_url + '/issues/' + issue.number;
            const copiedIssueBody = issue.body + '\n\n---\n\nCopied from: [' + originRepo.full_name + '#' + issue.number + '](' + originalIssueUrl + ')';

            await github.rest.issues.create({
              owner: 'canonical',
              repo: 'open-documentation-academy',
              title: issue.title,
              body: copiedIssueBody
            });

[degville]

Hello! Sorry for the long delay getting back to you, @minaelee and @alexvonme. I'll assign the task to both of you for now, but I'll also give @minaelee's suggestion a go and see if this works. This is actually one of the most important tasks in the repository because we really want to incorporate external projects. Thanks for taking the time to consider this!

[minaelee]

You're welcome, good luck with the action setup and let me know if you have questions or run into issues!

[evilnick]

@minaelee Thanks very much for this work. When I initially looked into it, the primary problem was that without a GH token issued from the ODA repo there weren't sufficient permissions to create an issue even on a public repo. Does this get around that limitation or does it still need a token issued from this repo?

I don't consider that a complete blocker, it just makes it slightly more work for projects to use as we will have to issue tokens to participants, but maybe that could be considered useful gatekeeping to prevent spam anyhow.

[minaelee]

That limitation does still exist. AFAIK a workflow in repo1 can't create an issue on repo2 without token-based authentication, even if repo2 is public. I can't think of any workaround that wouldn't be more complicated than setting a token. You should only need to issue one token from the ODA repo, then use that same token in all the other repos.

[tillkamppeter]

Hi, I am Till Kamppeter, lead of OpenPrinting. I want to participate in the Documentation Academy as an organization in need of documentation.

This issue is all about coupling documentation-related issues of participating organizations with the Academy and so it is essential for external organizations to participate. Therefore I have met @degville last week on Canonical's Engineering Sprint and talked with him about that and he agreed that we need to implement this feature request with high priority.

@degville could you mark this as high priority and assure that it gets quickly done, as you suggested in our conversation on the Sprint?

[evilnick]

@tillkamppeter @degville I have requested a PAT token specifically for this repo with issue-only permissions (which I think should be enough). Will update you when it gets generated and leave issue open for now

Thanks @minaelee

[tillkamppeter]

I can give this a whirl, though I would need someone to confirm/set up token access on a separate repository also owned by Canonical to test it.

I've tested the following workflow using two existing personal repositories, one in which I created an issue and labeled it with ODA, and one in which the GH action makes a copy of the ODA-labeled issue. The former (the repository being copied FROM) contains a GH_TOKEN repository secret with repo access.

I've tested to make sure that it only copies the issue when the ODA label is added, not when any others are.

I've added a link to the original issue in the body of the copied issue.

This is the workflow I tested, with the repo name and repo owner name changed to 'open-documentation-academy' and 'canonical'.

@minaelee Thanks a lot for your workflow!

I have now tried to do this in real life.

First, I have copied the exact workflow script (with "canonical" and "open-documentation-academy") into the file .github/workflows/open-document-academy.yml of the libcupsfilters repository of OpenPrinting and committed/pushed it. Then I have reported an issue telling that I need API documentation for libcupsfilters and labeled this one with "GOA".

In the libcupsfilters repo I see on the "Actions" tab that the workflow actually gets executed, triggered by my addition of the "GOA" tag.

Only it seems that I do not have a ${{ secrets.GH_TOKEN }} as I get an error message which looks for me like that the github-token parameter was empty (or the ${{ secrets.GH_TOKEN }} variable undefined):

Unhandled error: Error: Input required and not supplied: github-token

So I commented out the github-token: line as, according to the documentation of the "script" GitHub action then the default token is used. Then I get:

Unhandled error: HttpError: Resource not accessible by integration

The log shows that this is a 403. Canonical's repo seems to not accept reporting an issue via the API.

So I tried now the token ${{ secrets.GITHUB_TOKEN }} as this is the one which works for me with the Snap automation GitHub action and also get a 403:

Unhandled error: HttpError: Resource not accessible by integration

@degville for me it seems that the Open Documentation Academy GitHub repo needs some change in the settings for permission/security to allow issue reporting via GitHub API, so that actions and workflows can copy over/sync issues reports from external repos.

[tillkamppeter]

@tillkamppeter @degville I have requested a PAT token specifically for this repo with issue-only permissions (which I think should be enough). Will update you when it gets generated and leave issue open for now

Thanks @minaelee

How do you request such tokens and who issues them?

Do not close this issue when the token makes @minaelee 's workflow working. Seemy earlier comment about the test character of this workflow. We will need an action which is developed and maintained centrally here and has more advanced functionality.

[evilnick]

@tillkamppeter the token needs to be approved by the owner of the repo, in this case IS.
The lack of a token with adequate permissions is why you were getting the failures you mention, it can only be issued by the owner of the repo and is the only way an external repo can automatically create issues here

[userMaximilian]

Hello!

I saw this issue and wondered if anyone had any updates about importing issues from other repositories.

If the need for Canonical-issued personal access tokens is going to pose problems, then I have an idea for an alternative approach that might be worth considering. In short, it should be possible to run a GitHub Action on just this repository that uses the GitHub REST API to:

• periodically query each of the participating repositories for issues with the 'ODA' label (say 1-4 times a day); and
• copy those issues over if they don't already exist in this repository

This isn't an elegant solution at all, particularly when compared to the previously discussed 'run-when-labelled' approach. In fact, it's a bit inefficient. Nevertheless, it comes with some few advantages, for example:

• there shouldn't be any need to issue any access tokens to external projects; and
• the project 'on-boarding' process should be slightly simpler

All an external project would need to do is:

• register their repositories with the ODA;
• mark relevant issues with the 'ODA' label; and
• wait for the Action to run

We would need to define the list of participating repositories somewhere. This could be done in a text file in this repository, which the Action could parse whenever it runs. The registration process could involve someone from the external project filing a pull request against this file. The file also provides an easy way to remove a project if needed - perhaps temporarily in the case of spam, or permanently if there's a more serious issue (or if the project simply no longer wishes to participate) - that doesn't involve revoking a token.

The REST API provides a way to read issues from public repositories without needing to be authenticated to GitHub. That said, I think it would be worth creating a 'bot' account that can own the copied issues. The Action could also use that account to comment on the original issue, for example to confirm that the issue has been copied successfully.

To avoid duplicating issues, we could store the original issue's unique node_id as returned by the REST API somewhere in the body of the ODA copy, and then check for that each time the script is run.

As GitLab also offers a REST API, it may be possible to take a similar approach to copy issues from projects hosted on a GitLab instance.

What does everyone think? I haven't actually tested this out, but I'm happy to have a go if there's any interest - unless someone else wants to!

[userMaximilian]

Hello - I just wondered whether anyone had any thoughts about this, and/or if any additional work has been done behind the scenes (e.g. in relation to a Canonical-issued PAT).