Refactor e2e tests with wiremock scenarios

AGENTS.md

@@ -10,6 +10,10 @@ This project uses Nix with direnv. You should already be in the Nix shell automa
 nix develop
 ```
 
+Watch out for these Nix quirks:
+- If Nix tries to fetch from git during a build, it is likely that spago.yaml files were changed but the lock file was not updated; if so, update the lockfile with `spago build`
+- If a Nix build appears to be stale, then it is likely files were modified but are untracked by Git; if so, add modified files with `git add` and retry.
+
 ### Build and Test
 
 The registry is implemented in PureScript. Use spago to build it and run PureScript tests. These are cheap and fast and should be used when working on the registry packages.
@@ -19,17 +23,27 @@ spago build  # Build all PureScript code
 spago test   # Run unit tests
 ```
 
-Integration tests require two terminals (or the use of test-env in detached mode). The integration tests are only necessary to run if working on the server (app).
+#### End-to-End Tests
+
+The end-to-end (integration) tests are in `app-e2e`. They can be run via Nix on Linux:
+
+```
+nix build .#checks.x86_64-linux.integration
+```
+
+Alternately, they can be run on macOS or for more iterative development of tests using two terminals: one to start the test env, and one to execute the tests.
 
 ```sh
 # Terminal 1: Start test environment (wiremock mocks + registry server on port 9000)
 nix run .#test-env
 
 # Terminal 2: Run E2E tests once server is ready
-spago run -p registry-app-e2e
+spago-test-e2e
 ```
 
-Options: `nix run .#test-env -- --tui` for interactive TUI, `-- --detached` for background mode.
+Options: `nix run .#test-env -- --tui` for interactive TUI, `-- --detached` for background mode to use a single terminal.
+
+State is stored in `/tmp/registry-test-env` and cleaned up on each `nix run .#test-env`. To examine state after a test run (for debugging), stop the test-env but don't restart it.
 
 #### Smoke Test (Linux only)
 

CONTRIBUTING.md

@@ -72,20 +72,29 @@ nix build .#checks.x86_64-linux.smoke -L
 
 ### Integration Test
 
+You can run the integration tests with the following on Linux:
+
+```sh
+nix build .#checks.x86_64-linux.integration -L
+```
+
+On macOS or for iterative development, you can instead start the test environment and run the tests separately.
+
 ```sh
 # Terminal 1: Start the test environment (wiremock mocks + registry server)
 nix run .#test-env
 
-# Terminal 2: Once the server is ready, run the E2E tests
-spago run -p registry-app-e2e
+# Terminal 2: Run E2E tests once server is ready
+spago-test-e2e
 ```
 
 The test environment:
 - Starts wiremock services mocking GitHub, S3, Pursuit, etc.
-- Starts the registry server on port 9000 with a temporary SQLite database
+- Starts the registry server with a temporary SQLite database
 - Uses fixture data from `app/fixtures/`
+- State is stored in `/tmp/registry-test-env` and cleaned up on each `nix run .#test-env`
 
-Press `Ctrl+C` in Terminal 1 to stop all services. State is cleaned up automatically.
+Press `Ctrl+C` in Terminal 1 to stop all services.
 
 All arguments after `--` are passed directly to process-compose:
 
@@ -101,7 +110,11 @@ process-compose attach           # Attach TUI
 process-compose down             # Stop all services
 ```
 
-You can also set `STATE_DIR` to use a persistent state directory instead of a temp dir.
+To examine state after a test run (e.g., for debugging), stop the test-env but don't restart it. The state remains in `/tmp/registry-test-env`:
+- `db/registry.sqlite3` — SQLite database
+- `scratch/registry/` — Local registry clone with metadata
+- `scratch/registry-index/` — Local manifest index clone
+- `repo-fixtures/` — Git fixture repositories
 
 ## Available Nix Commands
 

SPEC.md

@@ -197,6 +197,7 @@ All packages in the registry contain a `purs.json` manifest file in their root d
 - `version`: a valid [`Version`](#version)
 - `license`: a valid [`License`](#license)
 - `location`: a valid [`Location`](#location)
+- `ref`: a `string` representing the reference (e.g., a Git commit or Git tag) at the `location` that was used to fetch this version's source code
 - `owners` (optional): a non-empty array of [`Owner`](#owner)
 - `description` (optional): a description of your library as a plain text string, not markdown, up to 300 characters
 - `includeFiles` (optional): a non-empty array of globs, where globs are used to match file paths (in addition to the `src` directory and other [always-included files](#always-included-files)) that you want included in your package tarball
@@ -221,6 +222,7 @@ For example:
     "githubOwner": "purescript",
     "githubRepo": "purescript-control"
   },
+  "ref": "v4.2.0",
   "include": ["test/**/*.purs"],
   "exclude": ["test/graphs"],
   "dependencies": { "newtype": ">=3.0.0 <4.0.0", "prelude": ">=4.0.0 <5.0.0" }

app-e2e/spago.yaml

@@ -8,21 +8,24 @@ package:
     - codec-json
     - console
     - datetime
-    - effect
-    - either
-    - foldable-traversable
+    - exceptions
+    - fetch
+    - integers
     - json
-    - maybe
+    - node-child-process
+    - node-execa
     - node-fs
     - node-path
     - node-process
-    - prelude
+    - ordered-collections
     - registry-app
     - registry-foreign
     - registry-lib
     - registry-test-utils
+    - routing-duplex
     - spec
     - spec-node
     - strings
+    - transformers
   run:
     main: Test.E2E.Main

app-e2e/src/Test/E2E/Endpoint/Jobs.purs

@@ -0,0 +1,76 @@
+module Test.E2E.Endpoint.Jobs (spec) where
+
+import Registry.App.Prelude
+
+import Data.Array as Array
+import Registry.API.V1 (JobId(..))
+import Registry.API.V1 as V1
+import Registry.Test.Assert as Assert
+import Test.E2E.Support.Client as Client
+import Test.E2E.Support.Env (E2ESpec)
+import Test.E2E.Support.Env as Env
+import Test.E2E.Support.Fixtures as Fixtures
+import Test.Spec as Spec
+
+spec :: E2ESpec
+spec = do
+  Spec.describe "Status endpoint" do
+    Spec.it "can reach the status endpoint" do
+      Client.getStatus
+
+  Spec.describe "Jobs list" do
+    Spec.it "excludes completed jobs when include_completed is false" do
+      -- Create a job and wait for it to complete
+      { jobId } <- Client.publish Fixtures.effectPublishData
+      _ <- Env.pollJobOrFail jobId
+
+      -- Now we have at least one completed job
+      recentJobs <- Client.getJobsWith Client.ActiveOnly
+      allJobs <- Client.getJobsWith Client.IncludeCompleted
+
+      -- All jobs should include the completed publish job
+      let allCount = Array.length allJobs
+      Assert.shouldSatisfy allCount (_ > 0)
+
+      -- Active-only should return fewer or equal jobs
+      let recentCount = Array.length recentJobs
+      Assert.shouldSatisfy recentCount (_ <= allCount)
+
+      -- Verify completed jobs are excluded from active-only results
+      let completedJob = Array.find (\job -> isJust (V1.jobInfo job).finishedAt) allJobs
+      case completedJob of
+        Just completed -> do
+          let
+            completedId = (V1.jobInfo completed).jobId
+            inRecent = Array.any (\job -> (V1.jobInfo job).jobId == completedId) recentJobs
+          when inRecent do
+            Assert.fail $ "Completed job " <> unwrap completedId <> " should be excluded from include_completed=false results"
+        Nothing -> pure unit
+
+  Spec.describe "Job query parameters" do
+    Spec.it "accepts level and since parameters" do
+      { jobId } <- Client.publish Fixtures.effectPublishData
+      job <- Env.pollJobOrFail jobId
+      let info = V1.jobInfo job
+
+      baseJob <- Client.getJob jobId Nothing Nothing
+      Assert.shouldEqual (V1.jobInfo baseJob).jobId info.jobId
+
+      debugJob <- Client.getJob jobId (Just V1.Debug) Nothing
+      Assert.shouldEqual (V1.jobInfo debugJob).jobId info.jobId
+
+      let sinceTime = fromMaybe info.createdAt info.finishedAt
+      sinceJob <- Client.getJob jobId Nothing (Just sinceTime)
+      Assert.shouldEqual (V1.jobInfo sinceJob).jobId info.jobId
+
+  Spec.describe "Jobs API error handling" do
+    Spec.it "returns HTTP 404 for non-existent job ID" do
+      let fakeJobId = JobId "nonexistent-job-id-12345"
+      result <- Client.tryGetJob fakeJobId Nothing Nothing
+      case result of
+        Right _ ->
+          Assert.fail "Expected HTTP 404 for non-existent job"
+        Left err ->
+          case Client.clientErrorStatus err of
+            Just 404 -> pure unit
+            _ -> Assert.fail $ "Expected HTTP 404, got: " <> Client.printClientError err

app-e2e/src/Test/E2E/Endpoint/Publish.purs

@@ -0,0 +1,76 @@
+module Test.E2E.Endpoint.Publish (spec) where
+
+import Registry.App.Prelude
+
+import Data.Array as Array
+import Data.Array.NonEmpty as NEA
+import Data.Map as Map
+import Data.Set as Set
+import Data.String as String
+import Registry.API.V1 (Job(..))
+import Registry.API.V1 as V1
+import Registry.Manifest (Manifest(..))
+import Registry.Metadata (Metadata(..))
+import Registry.Sha256 as Sha256
+import Registry.Test.Assert as Assert
+import Registry.Version as Version
+import Test.E2E.Support.Client as Client
+import Test.E2E.Support.Env (E2ESpec)
+import Test.E2E.Support.Env as Env
+import Test.E2E.Support.Fixtures as Fixtures
+import Test.E2E.Support.WireMock as WireMock
+import Test.Spec as Spec
+
+spec :: E2ESpec
+spec = do
+  Spec.describe "Publish workflow" do
+    Spec.it "can publish effect@4.0.0 and verify all state changes" do
+      { jobId } <- Client.publish Fixtures.effectPublishData
+      job <- Env.pollJobOrFail jobId
+      Assert.shouldSatisfy (V1.jobInfo job).finishedAt isJust
+
+      uploadOccurred <- Env.hasStorageUpload Fixtures.effect
+      unless uploadOccurred do
+        storageRequests <- WireMock.getStorageRequests
+        WireMock.failWithRequests "Expected S3 PUT for effect/4.0.0.tar.gz" storageRequests
+
+      Metadata metadata <- Env.readMetadata Fixtures.effect.name
+      case Map.lookup Fixtures.effect.version metadata.published of
+        Nothing -> Assert.fail $ "Expected version " <> Version.print Fixtures.effect.version <> " in metadata published versions"
+        Just publishedMeta -> do
+          Assert.shouldSatisfy (Sha256.print publishedMeta.hash) (not <<< String.null)
+
+      manifestEntries <- Env.readManifestIndexEntry Fixtures.effect.name
+      let hasVersion = Array.any (\(Manifest m) -> m.version == Fixtures.effect.version) manifestEntries
+      unless hasVersion do
+        Assert.fail $ "Expected version " <> Version.print Fixtures.effect.version <> " in manifest index"
+
+      Env.waitForAllMatrixJobs Fixtures.effect
+
+      -- Collect the compilers from the matrix jobs that ran for this package
+      allJobs <- Client.getJobsWith Client.IncludeCompleted
+      let
+        matrixCompilers = Array.mapMaybe
+          ( case _ of
+              MatrixJob { packageName, packageVersion, compilerVersion } ->
+                if packageName == Fixtures.effect.name && packageVersion == Fixtures.effect.version then Just compilerVersion
+                else Nothing
+              _ -> Nothing
+          )
+          allJobs
+        -- The expected compilers are: the publish compiler + all matrix job compilers
+        expectedCompilers = Set.fromFoldable $ Array.cons Fixtures.effectPublishData.compiler matrixCompilers
+
+      Metadata metadataAfter <- Env.readMetadata Fixtures.effect.name
+      case Map.lookup Fixtures.effect.version metadataAfter.published of
+        Nothing -> Assert.fail "Version missing after matrix jobs"
+        Just publishedMetaAfter -> do
+          let actualCompilers = Set.fromFoldable $ NEA.toArray publishedMetaAfter.compilers
+          Assert.shouldEqual actualCompilers expectedCompilers
+
+  Spec.describe "Publish state machine" do
+    Spec.it "returns same jobId for duplicate publish requests" do
+      { jobId: id1 } <- Client.publish Fixtures.effectPublishData
+      _ <- Env.pollJobOrFail id1
+      { jobId: id2 } <- Client.publish Fixtures.effectPublishData
+      Assert.shouldEqual id1 id2

app-e2e/src/Test/E2E/Endpoint/Transfer.purs

@@ -0,0 +1,51 @@
+module Test.E2E.Endpoint.Transfer (spec) where
+
+import Registry.App.Prelude
+
+import Data.Array as Array
+import Registry.API.V1 as V1
+import Registry.Location (Location(..))
+import Registry.Metadata (Metadata(..))
+import Registry.PackageName as PackageName
+import Registry.Test.Assert as Assert
+import Test.E2E.Support.Client as Client
+import Test.E2E.Support.Env (E2ESpec)
+import Test.E2E.Support.Env as Env
+import Test.E2E.Support.Fixtures as Fixtures
+import Test.E2E.Support.WireMock as WireMock
+import Test.Spec as Spec
+
+spec :: E2ESpec
+spec = do
+  Spec.describe "Transfer workflow" do
+    Spec.it "can transfer effect to a new location with full state verification" do
+      { jobId: publishJobId } <- Client.publish Fixtures.effectPublishData
+      _ <- Env.pollJobOrFail publishJobId
+      Env.waitForAllMatrixJobs Fixtures.effect
+
+      Metadata originalMetadata <- Env.readMetadata Fixtures.effect.name
+      case originalMetadata.location of
+        GitHub { owner } -> Assert.shouldEqual owner "purescript"
+        Git _ -> Assert.fail "Expected GitHub location, got Git"
+
+      -- clear the publish PUT so we can verify transfers leave storage unaffected
+      WireMock.clearStorageRequests
+
+      authData <- Env.signTransferOrFail Fixtures.effectTransferData
+      { jobId: transferJobId } <- Client.transfer authData
+      transferJob <- Env.pollJobOrFail transferJobId
+      Assert.shouldSatisfy (V1.jobInfo transferJob).finishedAt isJust
+
+      Metadata newMetadata <- Env.readMetadata Fixtures.effect.name
+      case newMetadata.location of
+        GitHub { owner } -> Assert.shouldEqual owner "new-owner"
+        Git _ -> Assert.fail "Expected GitHub location after transfer, got Git"
+
+      storageRequests <- WireMock.getStorageRequests
+      let
+        packagePath = PackageName.print Fixtures.effect.name
+        putOrDeleteRequests = Array.filter
+          (\r -> (r.method == "PUT" || r.method == "DELETE") && WireMock.filterByUrlContaining packagePath [ r ] /= [])
+          storageRequests
+      unless (Array.null putOrDeleteRequests) do
+        WireMock.failWithRequests "Transfer should not PUT or DELETE to storage" putOrDeleteRequests

app-e2e/src/Test/E2E/Endpoint/Unpublish.purs

@@ -0,0 +1,52 @@
+module Test.E2E.Endpoint.Unpublish (spec) where
+
+import Registry.App.Prelude
+
+import Data.Map as Map
+import Data.String as String
+import Registry.API.V1 as V1
+import Registry.Metadata (Metadata(..))
+import Registry.Test.Assert as Assert
+import Test.E2E.Support.Client as Client
+import Test.E2E.Support.Env (E2ESpec)
+import Test.E2E.Support.Env as Env
+import Test.E2E.Support.Fixtures as Fixtures
+import Test.E2E.Support.WireMock as WireMock
+import Test.Spec as Spec
+
+spec :: E2ESpec
+spec = do
+  Spec.describe "Publish-Unpublish workflow" do
+    Spec.it "can publish effect@4.0.0 then unpublish it with full state verification" do
+      { jobId: publishJobId } <- Client.publish Fixtures.effectPublishData
+      _ <- Env.pollJobOrFail publishJobId
+      Env.waitForAllMatrixJobs Fixtures.effect
+
+      existsBefore <- Env.manifestIndexEntryExists Fixtures.effect
+      unless existsBefore do
+        Assert.fail "Expected version to exist in manifest index before unpublish"
+
+      authData <- Env.signUnpublishOrFail Fixtures.effectUnpublishData
+      { jobId: unpublishJobId } <- Client.unpublish authData
+      unpublishJob <- Env.pollJobOrFail unpublishJobId
+      Assert.shouldSatisfy (V1.jobInfo unpublishJob).finishedAt isJust
+
+      Metadata metadata <- Env.readMetadata Fixtures.effect.name
+
+      case Map.lookup Fixtures.effect.version metadata.unpublished of
+        Nothing ->
+          Assert.fail "Expected version 4.0.0 to be in 'unpublished' metadata"
+        Just unpublishedInfo ->
+          Assert.shouldSatisfy unpublishedInfo.reason (not <<< String.null)
+
+      when (Map.member Fixtures.effect.version metadata.published) do
+        Assert.fail "Version 4.0.0 should not be in 'published' metadata after unpublish"
+
+      deleteOccurred <- Env.hasStorageDelete Fixtures.effect
+      unless deleteOccurred do
+        storageRequests <- WireMock.getStorageRequests
+        WireMock.failWithRequests "Expected S3 DELETE for effect/4.0.0.tar.gz" storageRequests
+
+      existsAfter <- Env.manifestIndexEntryExists Fixtures.effect
+      when existsAfter do
+        Assert.fail "Expected version to be removed from manifest index after unpublish"