Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update prune-prereleases.js pruning rules #6806

Merged
merged 4 commits into from
Jan 16, 2024
Merged

Conversation

hellwolf
Copy link
Contributor

@hellwolf hellwolf commented Jan 15, 2024

Pruning rules:

  1. only keep the earliest release of the month
  2. to keep the newest 3 nightlies

Fixes #6732

Here is my local testing:

import { Octokit, App } from "octokit";

// Create a personal access token at https://github.com/settings/tokens/new?scopes=repo
const octokit = new Octokit({ });

// In case node 21 is not used.
function groupBy(array, keyOrIterator) {
    var iterator;

    // use the function passed in, or create one
    if(typeof keyOrIterator !== 'function') {
        const key = String(keyOrIterator);
        iterator = function (item) { return item[key]; };
    } else {
        iterator = keyOrIterator;
    }

    return array.reduce(function (memo, item) {
        const key = iterator(item);
        memo[key] = memo[key] || [];
        memo[key].push(item);
        return memo;
    }, {});
}

async function separateReleases({ github, context }) {
    console.log("Pruning old prereleases");

    // doc: https://docs.github.com/en/rest/releases/releases
    const { data: releases } = await github.rest.repos.listReleases({
        owner: context.repo.owner,
        repo: context.repo.repo,
    });

    let nightlies = releases.filter(
        release =>
        // Only consider releases tagged `nightly-${SHA}` for deletion
        release.tag_name.includes("nightly") &&
        release.tag_name !== "nightly"
    );

    // group releases by months
    const groups = groupBy(nightlies, i => i.created_at.slice(0, 7));

    // Pruning rules:
    //   1. only keep the earliest release of the month
    //   2. to keep the newest 3 nightlies
    const toPrune = Object.values(groups)
        .reduce((acc, cur) => acc.concat(cur.slice(0, -1)), [])
        .slice(3);

    const toKeep = Object.values(groups).reduce((acc, cur) => acc.concat(cur.slice(-1)), []);

    return {
        toPrune,
        toKeep,
    };
};

(async() => {
    const releases = await separateReleases({
        github : octokit,
        context : {
            repo : { owner: "foundry-rs", repo: "foundry" }
        },
    });
    console.log("To prune:", releases.toPrune.map(i => i.name));
    console.log("To keep:", releases.toKeep.map(i => i.name));
})();
$ node index.mjs 
Pruning old prereleases
To prune: [ 'Nightly (2023-11-01)' ]
To keep: [
  'Nightly (2024-01-12)',
  'Nightly (2023-12-02)',
  'Nightly (2023-11-02)',
  'Nightly (2023-10-02)',
  'Nightly (2023-08-02)',
  'Nightly (2023-07-02)',
  'Nightly (2023-06-02)',
  'Nightly (2023-05-02)',
  'Nightly (2023-04-02)',
  'Nightly (2023-03-02)',
  'Nightly (2023-01-03)'
]

Motivation

Solution

    // Pruning rules:
    //   1. only keep the earliest release of the month
    //   2. to keep the newest 3 nightlies

```
import { Octokit, App } from "octokit";

// Create a personal access token at https://github.com/settings/tokens/new?scopes=repo
const octokit = new Octokit({ });

// In case node 21 is not used.
function groupBy(array, keyOrIterator) {
    var iterator;

    // use the function passed in, or create one
    if(typeof keyOrIterator !== 'function') {
        const key = String(keyOrIterator);
        iterator = function (item) { return item[key]; };
    } else {
        iterator = keyOrIterator;
    }

    return array.reduce(function (memo, item) {
        const key = iterator(item);
        memo[key] = memo[key] || [];
        memo[key].push(item);
        return memo;
    }, {});
}

async function separateReleases({ github, context }) {
    console.log("Pruning old prereleases");

    // doc: https://docs.github.com/en/rest/releases/releases
    const { data: releases } = await github.rest.repos.listReleases({
        owner: context.repo.owner,
        repo: context.repo.repo,
    });

    let nightlies = releases.filter(
        release =>
        // Only consider releases tagged `nightly-${SHA}` for deletion
        release.tag_name.includes("nightly") &&
        release.tag_name !== "nightly"
    );

    // group releases by months
    const groups = groupBy(nightlies, i => i.created_at.slice(0, 7));

    // Pruning rules:
    //   1. only keep the earliest release of the month
    //   2. to keep the newest 3 nightlies
    const toPrune = Object.values(groups)
        .reduce((acc, cur) => acc.concat(cur.slice(0, -1)), [])
        .slice(3);

    const toKeep = Object.values(groups).reduce((acc, cur) => acc.concat(cur.slice(-1)), []);

    return {
        toPrune,
        toKeep,
    };
};

(async() => {
    const releases = await separateReleases({
        github : octokit,
        context : {
            repo : { owner: "foundry-rs", repo: "foundry" }
        },
    });
    console.log("To prune:", releases.toPrune.map(i => i.name));
    console.log("To keep:", releases.toKeep.map(i => i.name));
})();
```

```
$ node index.mjs 
Pruning old prereleases
To prune: [ 'Nightly (2023-11-01)' ]
To keep: [
  'Nightly (2024-01-12)',
  'Nightly (2023-12-02)',
  'Nightly (2023-11-02)',
  'Nightly (2023-10-02)',
  'Nightly (2023-08-02)',
  'Nightly (2023-07-02)',
  'Nightly (2023-06-02)',
  'Nightly (2023-05-02)',
  'Nightly (2023-04-02)',
  'Nightly (2023-03-02)',
  'Nightly (2023-01-03)'
]
```
@onbjerg
Copy link
Member

onbjerg commented Jan 15, 2024

Is the test result not incorrect? If it is supposed to keep the first release of the month it should be pruning 2023-11-02, not 2023-11-01

@hellwolf
Copy link
Contributor Author

Is the test result not incorrect? If it is supposed to keep the first release of the month it should be pruning 2023-11-02, not 2023-11-01

I think there could be some mismatch between the "name" and created_time or published_time field, perhaps due to timezones.

By default the github releases seem to be sorted by created_time. That should make it consistent. To be precise, the comment could be improved to clarify this.

@onbjerg
Copy link
Member

onbjerg commented Jan 15, 2024

Let's include the clarification and this overall LGTM

@hellwolf
Copy link
Contributor Author

hellwolf commented Jan 16, 2024

Let's include the clarification and this overall LGTM

Yep. I find a quote from the doc:

| The latest release is the most recent non-prerelease, non-draft release, sorted by the created_at attribute.

Made a commit to the patch.

@shazow
Copy link

shazow commented Jan 16, 2024

@hellwolf Thanks for doing this!

Copy link
Member

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sgtm!
pending @onbjerg

@onbjerg onbjerg merged commit f180a13 into foundry-rs:master Jan 16, 2024
19 checks passed
@hellwolf hellwolf deleted the patch-1 branch January 16, 2024 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

foundry binaries that the nix overlay relies on are getting rugged
4 participants