Verification fails with non-ASCII strings

[cehbz]

Hi I'm trying to verify a create call that has a "name" field in which I want to pass "Jean-Marie de La Beaujardière"†

pact-go is blowing up in:
/usr/bin/pact-provider-verifier/lib/vendor/ruby/2.2.0/gems/json-1.8.3/lib/json/pure/parser.rb:171:in `encode': "\xC3" on US-ASCII (Encoding::InvalidByteSequenceError)

è in utf-8 is 0xc3 0xa8

If I change the è to e the pact verifies.

† a former co-worker whose name is great for finding bugs in name parsers, First name contains a hyphen, last name begins with a non-capital letter, has spaces in it, and has accented characters in it, whole name is longer than most.

[mefellows]

Thanks @charles-haynes, as the stack trace implies this looks like a classic encoding problem! We'll get onto this.

[bethesque]

Do you have the rest of the stack trace there @charles-haynes?

[cehbz]

You bet - I'll repro it tomorrow at work and send you the full stack trace.

[cehbz]

Here's the stack trace:

/usr/bin/pact-provider-verifier/lib/vendor/ruby/2.2.0/gems/json-1.8.3/lib/json/pure/parser.rb:171:in `encode': "\xC3" on US-ASCII (Encoding::InvalidByteSequenceError)
	from /usr/bin/pact-provider-verifier/lib/vendor/ruby/2.2.0/gems/json-1.8.3/lib/json/pure/parser.rb:171:in `convert_encoding'
	from /usr/bin/pact-provider-verifier/lib/vendor/ruby/2.2.0/gems/json-1.8.3/lib/json/pure/parser.rb:76:in `initialize'
	from /usr/bin/pact-provider-verifier/lib/vendor/ruby/2.2.0/gems/json-1.8.3/lib/json/common.rb:155:in `new'
	from /usr/bin/pact-provider-verifier/lib/vendor/ruby/2.2.0/gems/json-1.8.3/lib/json/common.rb:155:in `parse'
	from /usr/bin/pact-provider-verifier/lib/app/lib/pact/provider_verifier/app.rb:105:in `get_json_from_local_file'
	from /usr/bin/pact-provider-verifier/lib/app/lib/pact/provider_verifier/app.rb:85:in `get_json'
	from /usr/bin/pact-provider-verifier/lib/app/lib/pact/provider_verifier/app.rb:31:in `get_pact_consumer_name'
	from /usr/bin/pact-provider-verifier/lib/app/lib/pact/provider_verifier/app.rb:56:in `block in verify_pacts'
	from /usr/bin/pact-provider-verifier/lib/app/lib/pact/provider_verifier/app.rb:55:in `collect'
	from /usr/bin/pact-provider-verifier/lib/app/lib/pact/provider_verifier/app.rb:55:in `verify_pacts'
	from /usr/bin/pact-provider-verifier/lib/app/lib/pact/provider_verifier/cli.rb:19:in `verify'
	from /usr/bin/pact-provider-verifier/lib/vendor/ruby/2.2.0/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
	from /usr/bin/pact-provider-verifier/lib/vendor/ruby/2.2.0/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
	from /usr/bin/pact-provider-verifier/lib/vendor/ruby/2.2.0/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
	from /usr/bin/pact-provider-verifier/lib/vendor/ruby/2.2.0/gems/thor-0.19.4/lib/thor/base.rb:444:in `start'
	from /usr/bin/pact-provider-verifier/lib/app/pact-provider-verifier.rb:3:in `<main>'

And here's the .json file

{
  "consumer": {
    "name": "TestApp"
  },
  "provider": {
    "name": "Contracts"
  },
  "interactions": [
    {
      "description": "a request to create a contract",
      "providerState": "i do not have a contract",
      "request": {
        "method": "POST",
        "path": "/v1/contracts/created",
        "headers": {
          "Accept": "application/json, text/plain, */*",
          "Authorization": "Bearer redacted"
        },
        "body": {
          "startDate": "2017-03-16T04:47:09Z",
          "endDate": "2019-03-16T04:47:09Z",
          "kilowatts": 2,
          "dispatchType": "reduceLoadTo",
          "resourceID": "66a0b67d-c5ad-4427-bccd-f0aa5242404b",
          "verificationURL": "http://example.com/verification",
          "name": "Jean-Marie de La Beaujardière",
          "address": "redacted",
          "phone": "05 9999 9999",
          "email": "itsatest@example.com"
        }
      },
      "response": {
        "status": 200,
        "headers": {
          "Content-Type": "application/json"
        },
        "body": {
          "id": "a1fcecdc-481c-48fe-bc17-fe150c182f10"
        },
        "matchingRules": {
          "$.body.id": {
            "match": "regex",
            "regex": "\\h{8}-\\h{4}-\\h{4}-\\h{4}-\\h{12}"
          }
        }
      }
    },
    {
      "description": "a request to validate acceptable readings",
      "providerState": "i have a contract",
      "request": {
        "method": "POST",
        "path": "/v1/verify",
        "headers": {
          "Accept": "application/json, text/plain, */*",
          "Authorization": "Bearer redacted"
        },
        "body": {
          "contract": {
            "startDate": "2017-03-16T04:47:09Z",
            "endDate": "2019-03-16T04:47:09Z",
            "meterID": "5907e22e-7442-4432-b5e6-bf5567b73b9d",
            "dispatchType": "reduceLoadTo",
            "kilowatts": 2,
            "valuePerDispatch": 20,
            "verificationURL": "http://example.com/verification",
            "name": "Jean-Marie de La Beaujardière",
            "address": "redacted",
            "phone": "05 9999 9999",
            "email": "itsatest@example.com"
          },
          "readings": [
            {
              "id": "abcd",
              "time": "2017-05-04T16:23:08Z",
              "value": 1
            },
            {
              "id": "abcd",
              "time": "2017-05-04T16:53:09Z",
              "value": 2
            }
          ]
        }
      },
      "response": {
        "status": 200,
        "headers": {
          "Content-Type": "application/json"
        },
        "body": {
          "result": "fulfilled",
          "value": 20
        }
      }
    },
    {
      "description": "a request to validate unacceptable readings",
      "providerState": "i have a contract",
      "request": {
        "method": "POST",
        "path": "/v1/verify",
        "headers": {
          "Accept": "application/json, text/plain, */*",
          "Authorization": "Bearer redacted"
        },
        "body": {
          "contract": {
            "startDate": "2017-03-16T04:47:09Z",
            "endDate": "2019-03-16T04:47:09Z",
            "meterID": "5907e22e-7442-4432-b5e6-bf5567b73b9d",
            "dispatchType": "reduceLoadTo",
            "kilowatts": 2,
            "valuePerDispatch": 20,
            "verificationURL": "http://example.com/verification",
            "name": "Jean-Marie de La Beaujardière",
            "address": "redacted",
            "phone": "05 9999 9999",
            "email": "itsatest@example.com"
          },
          "readings": [
            {
              "id": "abcd",
              "time": "2017-05-04T16:23:09Z",
              "value": 3
            }
          ]
        }
      },
      "response": {
        "status": 200,
        "headers": {
          "Content-Type": "application/json"
        },
        "body": {
          "result": "not fulfilled",
          "value": 0
        }
      }
    }
  ],
  "metadata": {
    "pactSpecification": {
      "version": "2.0.0"
    }
  }
}

[bethesque]

Ok, I've tried to reproduce it in the e2e ruby example here with the build here and the verification doesn't blow up. I'm not sure if it's relevant but you are using an older version of the verifier that has some extra code in it that this e2e example doesn't use.

@mefellows when you have a moment (sorry, I know you're very busy at the moment!) could you upgrade the go lib to use the latest standalone?

[mefellows]

Thanks Beth, will do (probably this weekend after Agile Aus).

[cehbz]

Thanks for getting on it so quickly!

…

-- Charles

On Wed, 21 Jun 2017 at 22:13 Matt Fellows ***@***.***> wrote: Thanks Beth, will do (probably this weekend after Agile Aus). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#36 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABBTnWDs577_lei_L9vxnnOwGsJKuaBeks5sGQjfgaJpZM4N4Cgi> .

[bethesque]

NP. We've got a lot going on at the moment, so nudge us if we don't get back to it within a week.

[mefellows]

I've fixed our Wercker builds, and pushed an update to v0.0.8 (beta) which runs the verifier v1.1.3-1. I ran a local test this morning briefly and noticed better output error messaging and it didn't scream at the funny characters.

@charles-haynes could you please give it a try with this version? No hurries.

I'll leave this open until we integrate the standalone package.

[mefellows]

Just tested both consumer and provider and added to the rudimentary E2E integration suite, I think the latest dependencies have resolved the issue. Let us know how you get on @charles-haynes.

(v0.0.8 should contain the fix)

[cehbz]

v0.0.8 seems to have fixed the issue, thanks!

[mefellows]

Excellent - no worries Charles, all the best!