Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: implement the cucumber conformance tests #238

Merged
merged 1 commit into from
Jul 13, 2020

Conversation

lance
Copy link
Member

@lance lance commented Jul 2, 2020

Fixes: #71

Items remaining

  • use git submodule to pull in the features from cloudevents/spec
  • investigate why time stamps are not identical
  • investigate why datacontenttype is not identical
  • validate event data

@lance lance self-assigned this Jul 3, 2020
@lance lance added module/test Issues and changes related to the tests spec/1.0 Issues related to the CE specification type/enhancement New feature or request labels Jul 3, 2020
@lance lance marked this pull request as ready for review July 3, 2020 15:23
@lance lance requested review from a team, danbev, lholmquist and helio-frota and removed request for a team July 3, 2020 15:23
Copy link
Contributor

@helio-frota helio-frota left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conformance tests are passing which is great!
But I'm not following why conformance submodule directory needs to have a golang project inside:

13:50 $ tree conformance/
conformance/
├── cmd
│   └── cloudevents
│       └── main.go
├── features
│   ├── http-protocol-binding.feature
│   └── kafka-protocol-binding.feature
├── go.mod
├── go.sum
├── hack
│   └── presubmit-test.sh
├── LICENSE
├── pkg
│   ├── commands
│   │   ├── commands.go

I'm guess that is because the (updated / common used) conformance spec is there.
If yes, maybe it can be downloaded and / or use the online version ?

It works fine (package.json):

"conformance": "cucumber-js https://raw.githubusercontent.com/cloudevents/conformance/master/features/http-protocol-binding.feature -p default",
13:51 $ npm run conformance

> cloudevents-sdk@2.0.2 conformance /home/hf/code/cloudevents/sdk-javascript
> cucumber-js https://raw.githubusercontent.com/cloudevents/conformance/master/features/http-protocol-binding.feature -p default

@http
Feature: HTTP protocol binding

  @http
  Scenario Outline: Binary content mode (application/json)
    Given HTTP Protocol Binding is supported
    Given an HTTP request
      """
      POST /someresource HTTP/1.1
...

@lance
Copy link
Member Author

lance commented Jul 3, 2020

@helio-frota that's a good suggestion. I didn't realize you could provide a URL. Definitely better than pulling in a submodule

@lance
Copy link
Member Author

lance commented Jul 6, 2020

@helio-frota I have not been able to make your suggestion work. Unclear why. 🤷‍♂️

@helio-frota
Copy link
Contributor

@lance that was a false positive : ( the submodule was here, and when running pointing to a remote file, the cucumber falls back to the local file in the disk. Sorry.

But to try to remedy that I have created a small package called downtotemp and you can try using this way:

"conformance": "npx downtotemp https://raw.githubusercontent.com/cloudevents/conformance/master/features/http-protocol-binding.feature && cucumber-js /tmp/http-protocol-binding.feature -p default",

It is working least.

This commit adds cucumber-js conformance steps and includes the cucumber
tests in 'npm test'.

Signed-off-by: Lance Ball <lball@redhat.com>
@lance lance force-pushed the 71-add-cucumber-conformance-tests branch from 3307be8 to f639fb6 Compare July 6, 2020 21:49
@lance
Copy link
Member Author

lance commented Jul 7, 2020

@cloudevents/sdk-javascript-maintainers PTAL

@helio-frota thanks for the suggestions - done.

@lance lance changed the title test: initial pass at implementing the cucumber conformance tests test: implement the cucumber conformance tests Jul 7, 2020
@lance
Copy link
Member Author

lance commented Jul 8, 2020

Ping @cloudevents/sdk-javascript-maintainers ptal

@helio-frota helio-frota self-requested a review July 8, 2020 18:52
Copy link
Contributor

@lholmquist lholmquist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reasoning for moving the tests to an integration directory? Will the move mess with the merging of #233 since those tests are being updated there?

@lance
Copy link
Member Author

lance commented Jul 8, 2020

@lholmquist it's mainly to keep the integration tests separate from the conformance steps that are now in test/conformance. I'm not sure how much trouble that merge will be. I imagine it won't be a problem, but 🤷

Copy link
Contributor

@lholmquist lholmquist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welp, LGTM then

@lance lance merged commit dca2811 into cloudevents:master Jul 13, 2020
@lance lance deleted the 71-add-cucumber-conformance-tests branch July 13, 2020 13:47
@lance lance mentioned this pull request Jul 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module/test Issues and changes related to the tests spec/1.0 Issues related to the CE specification type/enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate with conformance cucumber tests
4 participants