No way to verify with a specific pact

[ims-swilkinson]

I'm using pact-python version 1.7.0.
There doesn't seem to be a way to verify with a specific pact by passing in a pact URL.

I tried passing a pact_url kwarg into MessageProvider.verify_with_broker but this always gives me the error pact.verify_wrapper.PactException: Pact urls or Pact broker required.

[YOU54F]

Hey @ims-swilkinson

can you confirm if the request was to verify a pact by URL, to be fetched from a broker (with a view to verify, and optionally publish the results back) as a result of webhook triggered provider jobs

https://docs.pact.io/pact_broker/webhooks#events

or was there also a request to be able to read files from a local file system (in which case, using a named function verify_with_broker - it wouldn't make sense to publish results of local files verification to a broker, unless it was retrieved from the broker.

Hopefully we can clarify and then get this feature out to you!

[YOU54F]

So there are at least three use cases for verifying

• verify with a file source (local pact file or directory of pacts)
• verify with a pact url (traditionally provided by a webhook triggered by a contract_content_changed or contract_requiring_verification_published event)
  • https://docs.pact.io/provider/recommended_configuration#verification-triggered-by-a-contract-requiring-verification-published
  • https://docs.pact.io/pact_broker/webhooks#events
• verify with a pact broker
  • https://docs.pact.io/provider/recommended_configuration#verification-triggered-by-provider-change

and 4 file sources

• https://docs.pact.io/implementation_guides/rust/pact_verifier_cli#pact-file-sources
• https://github.com/pact-foundation/pact-provider-verifier#usage

Rust distinguishes between file and dir, I would need to test if the ruby provider verifier will also accept a dir as as a arg (assume it will)

• file
• url

  • To verify a pact at a known URL (eg. when a verification is triggered by a 'contract content changed'
    webhook), pass in the pact URL(s) as the first argument(s) to the command, and do NOT set any of the
    other parameters apart from the base URL and credentials.

• dir
• broker url
  • --pact-broker-base-url=PACT_BROKER_BASE_URL

Verify with a file source

Verify a local file, provided by a file or dir, verification results should never be published and therefore this step does not require a broker

Verify by pact url (consumer change)

Verify a pact from a remote url, which will be retrieve from a pact broker, for a contract that has changed and requires verification.

• This step requires a broker to retrieve the pact from
• This step may require broker credentials if the broker is authenticated (either basic auth or token based)
• This step may publish verification results back to a broker, but this should really only be done in CI (it is traditionally user toggleable, and hooked up to an IS_CI or similar env var in their CI flow

note - this step does not require additional pact broker options

Verify by selectors (provider change)

Verify remote pacts, provided by querying the Pact Broker via consumer version selectors, verification results may be published

• This step requires a broker to retrieve the pact from
• This step may require broker credentials if the broker is authenticated (either basic auth or token based)
• This step may publish verification results back to a broker, but this should really only be done in CI (it is traditionally user toggleable, and hooked up to an IS_CI or similar env var in their CI flow
• This step accepts consumer version selectors
• This step accepts enable pending pacts
• This step accepts include work in progress pacts

State of Pact Python

Pact Python appears to

• allow for verify with a file source

  • pact-python/pact/verifier.py

    Line 36 in fc6ced8

    def verify_pacts(self, *pacts, enable_pending=False, include_wip_pacts_since=None, **kwargs):

• allow for verify by selectors

  • pact-python/pact/verifier.py

    Line 62 in fc6ced8

    def verify_with_broker(self, enable_pending=False, include_wip_pacts_since=None, **kwargs):

Smallest step forward

• allow for verify_with_pact_url or verify_with_pact_uri or similar naming, which is similar to verify_with_broker
• verify_with_pact_url can then only accept parameters relevant for that purpose
• verify_with_broker signature remains unchanged

Real world examples - Pact JS

Verify by pact url (consumer change)

https://github.com/pactflow/example-provider/blob/master/src/product/product.consumerChange.pact.test.js

uses these options

    const opts = {
      ...baseOpts,
      pactUrls: [process.env.PACT_URL],
      stateHandlers: stateHandlers,
      requestFilter: requestFilter
    };

Verify by selectors (provider change)

https://github.com/pactflow/example-provider/blob/master/src/product/product.providerChange.pact.test.js

    const fetchPactsDynamicallyOpts = {
      provider: 'pactflow-example-provider',
      consumerVersionSelectors: [
        { mainBranch: true },
        { deployed: true },
        { matchingBranch: true }
      ],
      pactBrokerUrl: process.env.PACT_BROKER_BASE_URL,
      // https://docs.pact.io/pact_broker/advanced_topics/pending_pacts
      enablePending: true,
      // https://docs.pact.io/pact_broker/advanced_topics/wip_pacts
      includeWipPactsSince: '2020-01-01'
    };

    const opts = {
      ...baseOpts,
      ...fetchPactsDynamicallyOpts,
      stateHandlers: stateHandlers,
      requestFilter: requestFilter
    };

There is some commonality between the modes the verifying with a broker

https://github.com/pactflow/example-provider/blob/master/src/product/pact.setup.js

const baseOpts = {
  logLevel: "INFO",
  providerBaseUrl: "http://localhost:8080",
  providerVersion: process.env.GIT_COMMIT,
  providerVersionBranch: process.env.GIT_BRANCH, // the recommended way of publishing verification results with the branch property
  verbose: process.env.VERBOSE === "true",
};

Real world examples - .NET

Verify by pact local file

https://github.com/pactflow/example-provider-dotnet/blob/34c615db8b828dcca1947e3f6d1c81a84c5e4b2c/tests/ProviderApiTests.cs#L72

Verify by pact url (consumer change)

https://github.com/pactflow/example-provider-dotnet/blob/34c615db8b828dcca1947e3f6d1c81a84c5e4b2c/tests/ProviderApiTests.cs#LL81C1-L99C27

Verify by selectors (provider change)

https://github.com/pactflow/example-provider-dotnet/blob/34c615db8b828dcca1947e3f6d1c81a84c5e4b2c/tests/ProviderApiTests.cs#L105

[YOU54F]

so this func

pact-python/pact/verifier.py

Line 36 in fc6ced8

def verify_pacts(self, *pacts, enable_pending=False, include_wip_pacts_since=None, **kwargs):

should probably have never allowed for setting wip pacts, or enable pending if its not for verifying with a broker.

There is a function called verify_with_broker, one would assume for that reason.

I've given this a go in #356

I am not a pythonista at all, so happy for reviews, feedback, additional commits 👍🏾

[ims-swilkinson]

Hey @ims-swilkinson

can you confirm if the request was to verify a pact by URL, to be fetched from a broker (with a view to verify, and optionally publish the results back) as a result of webhook triggered provider jobs

https://docs.pact.io/pact_broker/webhooks#events

or was there also a request to be able to read files from a local file system (in which case, using a named function verify_with_broker - it wouldn't make sense to publish results of local files verification to a broker, unless it was retrieved from the broker.

Hopefully we can clarify and then get this feature out to you!

Sorry for the late reply. Yes this is for verifying using the URL passed in a webhook request, not for local files.

[mefellows]

Thanks. Would you be up for a PR as per Yousaf's comments?