Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mock service's set of interactions is not cleared out when the verification fails. #108

Closed
pmacik opened this issue Jan 21, 2019 · 5 comments
Closed

Mock service's set of interactions is not cleared out when the verification fails. #108

pmacik opened this issue Jan 21, 2019 · 5 comments
Labels
bug Indicates an unexpected problem or unintended behavior

Comments

@pmacik
Copy link

pmacik commented Jan 21, 2019

Software versions

  • OS: Fedora 28 x86_64
  • Consumer Pact library: e.g. Pact go v1.0.0-beta.3
  • Golang Version: go1.11.1 linux/amd64
  • Golang environment:
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/pmacik/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/pmacik/.go"
GOPROXY=""
GORACE=""
GOROOT="/opt/go"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build024070287=/tmp/go-build -gno-record-gcc-switches"

Expected behaviour

When verification of one endpoint interaction fails I want other tests to be run anyway (not to fail fast).
When I do so I'd expect that I only see failed interactions (expected/unexpected) and that the correct ones pass.

Actual behaviour

It appears, that the set of interactions at mock server is not cleared out, when an interaction verification fails api-1 in the example.
This issues causes to duplicate/multiply "Missing requests" records and fails all the rest tests (even the correct ones api-2 in the example:

➜   go test -v
=== RUN   TestAPIConsumer
=== RUN   TestAPIConsumer/api-1
2019/01/21 17:47:31 Invoking API1 now
2019/01/21 17:47:33 Error on Verify: Actual interactions do not match expected interactions for mock MockService.

Missing requests:
        GET /api/one

Unexpected requests:
        GET /api/one1

See logs/pact.log for details.
=== RUN   TestAPIConsumer/api-2
2019/01/21 17:47:33 Invoking API2 now
2019/01/21 17:47:33 Error on Verify: Actual interactions do not match expected interactions for mock MockService.

Missing requests:
        GET /api/one
        GET /api/one

Unexpected requests:
        GET /api/one1

See logs/pact.log for details.
--- FAIL: TestAPIConsumer (2.65s)
    --- FAIL: TestAPIConsumer/api-1 (1.97s)
    --- FAIL: TestAPIConsumer/api-2 (0.13s)
FAIL
exit status 1
FAIL    _/home/pmacik/work/pact-go-issue        2.652s

Steps to reproduce

Run: go test -v with https://gist.github.com/pmacik/6de15aa2a64937419fe54159863fd930

Relevent log files

➜  go test -v
=== RUN   TestAPIConsumer
=== RUN   TestAPIConsumer/api-1
2019/01/21 17:59:38 Invoking API1 now
2019/01/21 17:59:38 [DEBUG] pact setup logging
2019/01/21 17:59:38 [DEBUG] pact setup
2019/01/21 17:59:38 [DEBUG] checking CLI compatability
2019/01/21 17:59:38 [INFO] checking pact-broker within range >= 1.16.0, < 2.0.0
2019/01/21 17:59:38 [DEBUG] running binary pact-broker
2019/01/21 17:59:38 [DEBUG] checking version for binary pact-broker version 1.16.1
2019/01/21 17:59:38 [DEBUG] checking if version 1.16.1 within semver range >= 1.16.0, < 2.0.0
2019/01/21 17:59:38 [DEBUG] 1.16.1 satisfies constraints 1.16.1 >= 1.16.0, < 2.0.0
2019/01/21 17:59:38 [INFO] checking pact-mock-service within range >= 2.9.8, < 3.0.0
2019/01/21 17:59:38 [DEBUG] running binary pact-mock-service
2019/01/21 17:59:38 [DEBUG] checking version for binary pact-mock-service version 2.11.0
2019/01/21 17:59:38 [DEBUG] checking if version 2.11.0 within semver range >= 2.9.8, < 3.0.0
2019/01/21 17:59:38 [DEBUG] 2.11.0 satisfies constraints 2.11.0 >= 2.9.8, < 3.0.0
2019/01/21 17:59:38 [INFO] checking pact-provider-verifier within range >= 1.15.0, < 3.0.0
2019/01/21 17:59:38 [DEBUG] running binary pact-provider-verifier
2019/01/21 17:59:39 [DEBUG] checking version for binary pact-provider-verifier version 1.19.0
2019/01/21 17:59:39 [DEBUG] checking if version 1.19.0 within semver range >= 1.15.0, < 3.0.0
2019/01/21 17:59:39 [DEBUG] 1.19.0 satisfies constraints 1.19.0 >= 1.15.0, < 3.0.0
2019/01/21 17:59:39 [DEBUG] setting up a service manager
2019/01/21 17:59:39 [DEBUG] setting up a service manager
2019/01/21 17:59:39 [DEBUG] setting up a service manager
2019/01/21 17:59:39 [DEBUG] starting service creation monitor
2019/01/21 17:59:39 [DEBUG] starting service creation monitor
2019/01/21 17:59:39 [DEBUG] starting service removal monitor
2019/01/21 17:59:39 [DEBUG] starting service removal monitor
2019/01/21 17:59:39 [DEBUG] starting service removal monitor
2019/01/21 17:59:39 [DEBUG] starting service creation monitor
2019/01/21 17:59:39 [DEBUG] starting mock service on port: 33201
2019/01/21 17:59:39 [DEBUG] client: starting a server with args: [--pact-specification-version 2 --pact-dir /home/pmacik/work/pact-go-issue/pacts --log /home/pmacik/work/pact-go-issue/logs/pact.log --consumer test-consumer --provider test-provider --pact-file-write-mode overwrite] port: 33201
2019/01/21 17:59:39 [DEBUG] starting service
2019/01/21 17:59:39 [DEBUG] waiting for port 33201 to become available
2019/01/21 17:59:39 [INFO] INFO  WEBrick 1.4.2
2019/01/21 17:59:39 [INFO] INFO  ruby 2.5.3 (2018-10-18) [x86_64-linux]
2019/01/21 17:59:39 [INFO] INFO  WEBrick::HTTPServer#start: pid=5268 port=33201
2019/01/21 17:59:39 [DEBUG] pact add interaction
2019/01/21 17:59:39 [DEBUG] pact setup logging
2019/01/21 17:59:39 [DEBUG] pact setup
2019/01/21 17:59:39 [DEBUG] pact verify
2019/01/21 17:59:39 [DEBUG] mock service add interaction
2019/01/21 17:59:40 [DEBUG] mock service verify
2019/01/21 17:59:40 Error on Verify: Actual interactions do not match expected interactions for mock MockService.

Missing requests:
        GET /api/one

Unexpected requests:
        GET /api/one1

See logs/pact.log for details.
=== RUN   TestAPIConsumer/api-2
2019/01/21 17:59:40 Invoking API2 now
2019/01/21 17:59:40 [DEBUG] pact setup logging
2019/01/21 17:59:40 [DEBUG] pact setup
2019/01/21 17:59:40 [DEBUG] pact add interaction
2019/01/21 17:59:40 [DEBUG] pact setup logging
2019/01/21 17:59:40 [DEBUG] pact setup
2019/01/21 17:59:40 [DEBUG] pact verify
2019/01/21 17:59:40 [DEBUG] mock service add interaction
2019/01/21 17:59:40 [DEBUG] mock service add interaction
2019/01/21 17:59:40 [DEBUG] mock service verify
2019/01/21 17:59:40 Error on Verify: Actual interactions do not match expected interactions for mock MockService.

Missing requests:
        GET /api/one
        GET /api/one

Unexpected requests:
        GET /api/one1

See logs/pact.log for details.
2019/01/21 17:59:40 [DEBUG] pact setup logging
2019/01/21 17:59:40 [DEBUG] pact setup
2019/01/21 17:59:40 [DEBUG] pact write Pact file
2019/01/21 17:59:40 [DEBUG] mock service write pact
2019/01/21 17:59:40 [DEBUG] teardown
2019/01/21 17:59:40 [DEBUG] client: stop server
2019/01/21 17:59:40 [DEBUG] stopping service with pid 5268
2019/01/21 17:59:40 [INFO] INFO  going to shutdown ...
2019/01/21 17:59:41 [INFO] INFO  WEBrick::HTTPServer#start done.
--- FAIL: TestAPIConsumer (2.81s)
    --- FAIL: TestAPIConsumer/api-1 (2.16s)
    --- FAIL: TestAPIConsumer/api-2 (0.13s)
FAIL
exit status 1
FAIL    _/home/pmacik/work/pact-go-issue        2.808s
@mefellows mefellows added triage bug Indicates an unexpected problem or unintended behavior labels Jan 21, 2019
@mefellows
Copy link
Member

Thanks @pmacik. I'll take a look a bit later.

@pmacik
Copy link
Author

pmacik commented Jan 22, 2019

Thanks.

Found a workaround: that is to manually clear the mock server when verification failed:
https://gist.github.com/pmacik/c012015d94cc2cbb97c6bf43e2c157a5#file-consumer_workaround_test-go-L53

pmacik added a commit to pmacik/fabric8-wit that referenced this issue Jan 22, 2019
@pmacik
contract-tests: Update package names and workaround pact-foundation/p…
f9d4840 
…act-go#108 issue.
@mefellows
Copy link
Member

That'll work - clever!

pmacik added a commit to pmacik/fabric8-wit that referenced this issue Jan 22, 2019
@pmacik
contract-tests: Update package names and workaround pact-foundation/p…
4242601 
…act-go#108 issue.
pmacik added a commit to pmacik/fabric8-wit that referenced this issue Jan 23, 2019
@pmacik
contract-tests: Update package names and workaround pact-foundation/p…
0136f82 
…act-go#108 issue.
@frederic-gendebien
Copy link

frederic-gendebien commented Jan 18, 2022 •
edited
Loading

I just started to learn Pact, and got the same problem, even nastier. Even when the tests are ok, subsequent tests are failing because previously matched interactions are not removed... So had to put manually (very dirty and hope to find a better solution from pact team)

t.Run("Find An Existing User By Id", func(t *testing.T) {
		pact.Interactions = nil //this is the dirty dirty dirty thing that got me out of that hell
		pact.AddInteraction().
		...

@mefellows
Copy link
Member

If you're new to Pact, I'd look to get onto the 2.x.x which we'd like to release in Q1 this year. It's stable, and has new features (and this probably should be gone).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Indicates an unexpected problem or unintended behavior
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mefellows @pmacik @frederic-gendebien