E2E tests for Python app failing when spawning child process

[Roblinde]

The repository with this issue will soon be open sourced, I will get back with the repo url then.

• Operating System: Ubuntu on CircleCI
• Cypress Version: 1.1.4
• Browser Version:

Is this a Feature or Bug?

Bug

Current behavior:

We have a set of e2e tests that we run for an application that is duplicated in several different languages. The e2e tests run fine for the app in .NET, PHP and Node. When trying to run them for the Python application they fail with the following error:

Opening Cypress...
  cypress:cli processing run options +3ms
  cypress:cli --key is not set, looking up environment variable CYPRESS_RECORD_KEY +0ms
  cypress:cli run to spawn.start args ["--run-project","/home/ubuntu/the-app.py","--spec","./test/e2e/specs/the-app-spec.js"] +0ms
  cypress:cli needs XVFB? false +1ms
  cypress:cli spawning Cypress /home/ubuntu/the-app.py/test/e2e/node_modules/cypress/dist/Cypress/Cypress +0ms
  cypress:cli spawn args ["--run-project","/home/ubuntu/the-app.py","--spec","./test/e2e/specs/the-app-spec.js"] +0ms

Started video recording: /home/ubuntu/the-app.py/cypress/videos/qk4pw.mp4

  (Tests Starting)
Error: /home/ubuntu/the-app.py/test/e2e/node_modules/cypress/dist/Cypress/resources/app/packages/runner/dist/index.html: Parse error on line 4:
...     {{ 'draftLabel'|trans }}    </div>
-----------------------^
Expecting 'CLOSE_RAW_BLOCK', 'CLOSE', 'CLOSE_UNESCAPED', 'OPEN_SEXPR', 'CLOSE_SEXPR', 'ID', 'OPEN_BLOCK_PARAMS', 'STRING', 'NUMBER', 'BOOLEAN', 'UNDEFINED', 'NULL', 'DATA', got 'CLOSE_BLOCK_PARAMS'
    at Object.parseError (/home/ubuntu/the-app.py/test/e2e/node_modules/cypress/dist/Cypress/resources/app/packages/server/node_modules/handlebars/dist/cjs/handlebars/compiler/parser.js:267:19)
    at Object.parse (/home/ubuntu/the-app.py/test/e2e/node_modules/cypress/dist/Cypress/resources/app/packages/server/node_modules/handlebars/dist/cjs/handlebars/compiler/parser.js:336:30)
    at HandlebarsEnvironment.parse (/home/ubuntu/the-app.py/test/e2e/node_modules/cypress/dist/Cypress/resources/app/packages/server/node_modules/handlebars/dist/cjs/handlebars/compiler/base.js:46:43)
    at compileInput (/home/ubuntu/the-app.py/test/e2e/node_modules/cypress/dist/Cypress/resources/app/packages/server/node_modules/handlebars/dist/cjs/handlebars/compiler/compiler.js:514:19)
    at ret (/home/ubuntu/the-app.py/test/e2e/node_modules/cypress/dist/Cypress/resources/app/packages/server/node_modules/handlebars/dist/cjs/handlebars/compiler/compiler.js:523:18)
    at /home/ubuntu/the-app.py/test/e2e/node_modules/cypress/dist/Cypress/resources/app/packages/server/node_modules/hbs/lib/hbs.js:87:17
    at /home/ubuntu/the-app.py/test/e2e/node_modules/cypress/dist/Cypress/resources/app/packages/server/node_modules/hbs/lib/hbs.js:69:11
    at undefined.done (/home/ubuntu/the-app.py/test/e2e/node_modules/cypress/dist/Cypress/resources/app/packages/server/node_modules/hbs/lib/async.js:74:20)
    at /home/ubuntu/the-app.py/test/e2e/node_modules/cypress/dist/Cypress/resources/app/packages/server/node_modules/hbs/lib/hbs.js:64:20
    at /home/ubuntu/the-app.py/test/e2e/node_modules/cypress/dist/Cypress/resources/app/packages/server/node_modules/graceful-fs/graceful-fs.js:78:16
    at tryToString (fs.js:455:3)
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:442:12)

What is extra strange here is that it fails on this {{ 'draftLabel'|trans }} </div> which is actually part of the layout.html of the python application.

Desired behavior:

The e2e tests run without trying to parse the application templates.

[bahmutov]

Can you restrict Python from going into the node_modules and parsing source there?!!! That does not sound like something we can control.

[Roblinde]

This isn't python parsing the node_modules from what I can tell. It seems to be the other way around. This is Cypress parsing the python files.

Could you please have a look again?

Thanks!

[brian-mann]

@Roblinde this is such an incredibly unusual error that none of us can make any sense as to how its happening.

When I first looked at the stack I thought - it looks as if you've modified the Cypress source itself. I cannot imagine a scenario where this happens. However I'm assuming you haven't actually modified any source and that this is in fact reproducible.

The thing is though - from the stack (because its async) there is nothing useful. It's coming from handlebars parsing a template. But the only time Cypress does this is when serving the initial .html page which loads up the Cypress app. All of this is Cypress source code - it has nothing to do with your project.

Can you provide us your test code - after reducing it down to the bare minimum number of commands you can before this bug is triggered? It may only be a single cy.visit. Remove anything that does not contribute to this error.

That way we can see what you're actually doing. Are you using Cypress as your webserver or are your spinning up your own? Your test code would answer all these questions.

You'll need to provide us with as much information - the exact steps you take, how you run Cypress, etc for us to be of any help.

[brian-mann]

Actually I have an idea. When Cypress sends the runner/index.html it parses it in handlebars and it includes all of the Cypress configuration including environment variables.

If you have an environment variable that uses the special characters that handlebars interprets... it would cause this error. We'd need to see exactly what you're setting though.

[Roblinde]

Thanks for looking into this Brian. I will get back with as much information as possible. I am as baffled as you are by this.

[Roblinde]

Ok @brian-mann here's a step by step guide of exactly what we're doing to run the tests.

We have a python repo (I will post the project url here once it's open sourced which should be tomorrow hopefully). It's a very simple flask app. It has a number of .html files that are used as templates. The layout.html for example contains this snippet:

{% macro entry_state(entry) -%}
  {% if entry.draft %}
    <div class="pill pill--draft">
      {{ 'draftLabel'|trans }}
    </div>
  {% endif %}

Which you might recognize from the error above.

We start this python app and even do a warm up request to it, which works and returns HTML. We then cloneour e2e test repo into a sub-directory (but I have tried putting it in the root directory side by side with the python app as well) and run NPM install there. Installing Cypress.

We then run DEBUG=cypress:* ./test/e2e/node_modules/.bin/cypress run --spec ./test/e2e/specs/the-app-spec.js

Now I have tried removing the spec file or pointing it to a non-existant file and the same error occurs. Cypress seems to never get to the part of actually loading the spec, but fails somewhere before it.

This is the cypress.json:

{
    "baseUrl": "http://localhost:3000",
    "fixturesFolder": false,
    "integrationFolder": "/home/ubuntu/test/e2e/specs/",
    "supportFile": false,
    "env": {
        "CONTENTFUL_SPACE_ID": "XXX",
        "CONTENTFUL_DELIVERY_TOKEN": "XXX",
        "CONTENTFUL_PREVIEW_TOKEN": "XXX"
    } 
  }

If I change the baseUrl it fails with a more normal error saying it can't find a server. But this, of course, being the correct one fails with the above error.

The only thing I can think of is that the runner/index.html you mention above searches for html files greedily and parses anything it thinks is a handlebars file. Even though that sounds sort of absurd it looks like it's trying to parse my layout.html.

Just let me know if I'm being unclear or you want any other information.

Thanks
Robert

[Roblinde]

The repo is now public: https://github.com/contentful/the-example-app.py

[jennifer-shehane]

@Roblinde Could you try removing the env json from the cypress.json to see if this error still occurs?

So remove this:

    "env": {
        "CONTENTFUL_SPACE_ID": "XXX",
        "CONTENTFUL_DELIVERY_TOKEN": "XXX",
        "CONTENTFUL_PREVIEW_TOKEN": "XXX"
    } 

[Roblinde]

Already tried without any environment variables, same result unfortunately.

It might be worth mentioning again that this exact spec runs fine for .NET, PHP, Node and Ruby, it's just python that's behaving funnily. All of them of course also uses the same env-variables.

[dlitvakb]

I can confirm that this issue is still happening for our Python application.

I've tried running it on macOS, just to see if not running it on Linux would make a difference.

I'm using Cypress 2.1.0

[dlitvakb]

I was able to run the test suite after changing all my template extensions from .html to .dhtml.

Which meant I had to make some other tweaks to the code base, but nothing serious.

Looks like indeed Cypress is trying to load my templates.

[jennifer-shehane]

Since this issue hasn't had activity in a while, we'll close the issue until we can confirm this is still happening. Please comment if there is new information to provide concerning the original issue and we'd be happy to reopen.