Skip to content

Conversation

@thomashoneyman
Copy link
Member

Fixes #527 by moving the ref field. The ref is used for two things:

  1. Historical record of where a package came from, e.g. a Location + reference as provided in the publish payload
  2. Reconstruction of URLs for legacy package sets

Way back in the day we put it in metadata but that didn't really make sense; in the case of a transfer, the location and refs may be out of sync. Now's a good time to finally make the change because we're on the verge of a reupload that will require updating all the manifests anyway.

Copy link
Member

@f-f f-f left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!!

@f-f f-f merged commit fe141f1 into trh/scenarios Jan 7, 2026
20 checks passed
@f-f f-f deleted the trh/drop-ref branch January 7, 2026 19:11
f-f pushed a commit that referenced this pull request Jan 7, 2026
* refactor e2e tests with wiremock scenarios

also adds a number of new e2e tests for various scenarios

* format, etc.

* move out fixtures

* relax cache deletion

* strengthen assertions, fix discovered bugs

* drop ref, move to manifest (#714)

* review feedback

* more feedback
thomashoneyman added a commit that referenced this pull request Jan 8, 2026
* Update database schemas and add job executor loop

* Split Server module into Env, Router, JobExecutor, and Main

* Fix up build

* Run job executor

* Fix integration tests

* WIP matrix builds

* add missing version to publish fixtures

the publishCodec requires a version file but the test fixtures weren't
updated to include it

* Add missing packageName and packageVersion to InsertMatrixJob

The JS insertMatrixJobImpl expects columns [jobId, packageName,
packageVersion, compilerVersion, payload] but the PureScript types were
missing packageName and packageVersion

* Fix finishedAt timestamp to capture time after job execution

* Implement matrix jobs, and the recursive enqueuing of new ones

* Reset incomplete jobs so they can be picked up again

* Run matrix jobs for the whole registry when finding a new compiler version

* resolve build issues

* fix smoke test

* Split package jobs into separate tables, return all data from the job endpoint

* implement thin client for github issues

replaces the old GitHubIssue which ran registry jobs directly with
one that hits the registry api instead. also added integration
tests that ensure various jobs can be kicked off as github issue
events and we get the resulting comments, issue close events, etc.

* clean up test failures

* reinstate missing comments

* Remove COMMENT effect, add NOTIFY log

* Implement endpoint for returning jobs

* Check for existing jobs before enqueueing new ones

* Add E2E test: publishing a package enqueues matrix jobs

* Add E2E test: run a whole-registry upgrade when detecting a new compiler

* Don't fail job fetch on unreadable logs

* Fix archive seeder build

* remove effect-4.0.0 from storage in unit tests

* avoid race condition in initial jobs test

The "can list jobs" test was asserting that initial matrix jobs have
success: true, but the job executor runs asynchronously and jobs may
not have completed by the time the test queries the API.

Fixed by normalizing the 'success' field to a constant before
comparison.

* format

* second test

* Refactor e2e tests with wiremock scenarios (#713)

* refactor e2e tests with wiremock scenarios

also adds a number of new e2e tests for various scenarios

* format, etc.

* move out fixtures

* relax cache deletion

* strengthen assertions, fix discovered bugs

* drop ref, move to manifest (#714)

* review feedback

* more feedback

* trim tests down a bit to optimize speed to ~60s

* Add endpoint for package set jobs + e2e tests for it

* tweak unpublish test to verify matrix jobs fail gracefully

* tweak agents to refer to scratch logs

* remove slow archive seeder test

* fix tests by bumping compiler

---------

Co-authored-by: Thomas Honeyman <hello@thomashoneyman.com>
Co-authored-by: Fyodor Soikin <name.fa@gmail.com>
Co-authored-by: pacchettibotti <pacchettibotti@purescript.org>
Co-authored-by: Thomas Honeyman <admin@thomashoneyman.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants