Docker fails with Cannot set property 'tempCompareConfigFileName' of undefined

[n1313]

My backstop.config.json:

{
  "id": "core",
  "viewports": [
    {
      "label": "phone",
      "width": 414,
      "height": 736
    },
    {
      "label": "desktop",
      "width": 1366,
      "height": 768
    }
  ],
  "onReadyScript": "onReady.js",
  "scenarios": [
    {
      "label": "Tests",
      "url": /* my url */,
      "delay": 0,
      "selectors": [
        /* a bunch of selectors */
      ],
      "selectorExpansion": true,
      "misMatchThreshold": 0,
      "requireSameDimensions": true,
      "readySelector": "#root"
    }
  ],
  "paths": {
    "bitmaps_reference": "./backstop/reference",
    "bitmaps_test": "./backstop/test",
    "engine_scripts": "./backstop/scripts",
    "html_report": "./reports/backstop",
    "ci_report": "./reports"
  },
  "fileNameTemplate": "{selectorLabel}-{viewportLabel}",
  "report": [
    "browser",
    "CI"
  ],
  "ci": {
    "testReportFileName": "backstop.xunit.xml"
  },
  "engine": "chromy",
  "engineOptions": {
    "waitTimeout": 1000,
    "chromeFlags": [
      "--force-device-scale-factor=1"
    ]
  },
  "asyncCaptureLimit": 1,
  "asyncCompareLimit": 10,
  "debug": false,
  "debugWindow": true,
  "resembleOutputOptions": {
    "ignoreAntialiasing": true
  }
}

Console output:

$ backstop test --docker --config=backstop.config.json
BackstopJS v3.5.10
Loading config:  /Users/[path]/backstop.config.json

COMMAND | Executing core for "test"
Delegating command to Docker... docker run --rm -it --mount type=bind,source="/Users/[path]",target=/src backstopjs/backstopjs test "--_=test" "--h=false" "--help=false" "--v=false" "--version=false" "--i=false" "--moby=true" "--config=backstop.config.json" "--moby" "--config=backstop.config.json"
BackstopJS v3.5.10
User config detected.
COMMAND | Executing core for "test"
/usr/local/lib/node_modules/backstopjs/core/util/createBitmaps.js:53
  configJSON.paths.tempCompareConfigFileName = config.tempCompareConfigFileName;
                                             ^

TypeError: Cannot set property 'tempCompareConfigFileName' of undefined
    at decorateConfigForCapture (/usr/local/lib/node_modules/backstopjs/core/util/createBitmaps.js:53:46)
    at writeReferenceCreateConfig (/usr/local/lib/node_modules/backstopjs/core/util/createBitmaps.js:81:68)
    at module.exports (/usr/local/lib/node_modules/backstopjs/core/util/createBitmaps.js:213:10)
    at Object.execute (/usr/local/lib/node_modules/backstopjs/core/command/test.js:13:14)
    at Object.execute [as test] (/usr/local/lib/node_modules/backstopjs/core/command/index.js:52:49)
    at execute (/usr/local/lib/node_modules/backstopjs/core/command/index.js:102:31)
    at module.exports (/usr/local/lib/node_modules/backstopjs/core/runner.js:6:10)
    at Object.<anonymous> (/usr/local/lib/node_modules/backstopjs/cli/index.js:45:3)
    at Module._compile (module.js:624:30)
    at Object.Module._extensions..js (module.js:635:10)

Backstop installed globally, v3.5.10, docker image v3.5.10, on OSX. This config works as expected when I run it without --docker. Could you please help me debug this?

[n1313]

#679 references the same error message, but I don't think it is relevant as I'm using a plain JSON file as my config.

[garris]

@n1313 Thanks for the bug report! Just to be sure there are no environmental issues, could you run this sanity check? (You can run from your Desktop directory or wherever you like.)

mkdir backstopSanityTest; cd backstopSanityTest; npm install backstopjs; node ./node_modules/backstopjs/cli/ init; node ./node_modules/backstopjs/cli/ test --docker

This will create a directory called backstopSanityTest, do a fresh install of backstopjs, initialize the test directory and then run a simple config using docker.

The log will be somewhat long -- but you should find these lines in there somewhere...

...
BackstopJS v3.5.10
COMMAND | Executing core for "genConfig"
genConfig | Copying '/Users/gshipon/Desktop/test3/backstopSanityTest/node_modules/backstopjs/capture/engine_scripts' to '/Users/gshipon/Desktop/test3/backstopSanityTest/backstop_data/engine_scripts'
genConfig | Configuration file written at '/Users/gshipon/Desktop/test3/backstopSanityTest/backstop.json'
  COMMAND | Command "genConfig" successfully executed in [0.028s]
BackstopJS v3.5.10
Loading config:  /Users/gshipon/Desktop/test3/backstopSanityTest/backstop.json

COMMAND | Executing core for "test"
Delegating command to Docker... docker run --rm -it --mount type=bind,source="/Users/gshipon/Desktop/test3/backstopSanityTest",target=/src backstopjs/backstopjs test "--_=test" "--h=false" "--help=false" "--v=false" "--version=false" "--i=false" "--moby=true" "--config=backstop.json" "--moby"
BackstopJS v3.5.10
Loading config:  /src/backstop.json

COMMAND | Executing core for "test"
createBitmaps | Selected 1 of 1 scenarios.
...

Please let me know if this works for you. If it runs ok then work backwards from there to try to figure out what the issue may be.

Please let me know how that sanity check goes!

[n1313]

@garris Thank you for your response! The sanity check works, here's the output (I ran reference first to generate test images):

$ node ./node_modules/backstopjs/cli/ init; node ./node_modules/backstopjs/cli/ test --docker
BackstopJS v3.5.10
COMMAND | Executing core for "genConfig"
genConfig | Copying '/[path]/backstopSanityTest/node_modules/backstopjs/capture/engine_scripts' to '/[path]/backstopSanityTest/backstop_data/engine_scripts'
genConfig | Configuration file written at '/[path]/backstopSanityTest/backstop.json'
  COMMAND | Command "genConfig" successfully executed in [0.019s]
BackstopJS v3.5.10
Loading config:  /[path]/backstopSanityTest/backstop.json

COMMAND | Executing core for "test"
Delegating command to Docker... docker run --rm -it --mount type=bind,source="/[path]/backstopSanityTest",target=/src backstopjs/backstopjs test "--_=test" "--h=false" "--help=false" "--v=false" "--version=false" "--i=false" "--moby=true" "--config=backstop.json" "--moby"
BackstopJS v3.5.10
Loading config:  /src/backstop.json

COMMAND | Executing core for "test"
createBitmaps | Selected 1 of 1 scenarios.
Cookie state restored with: [
  {
    "path": "/",
    "name": "yourCookieName",
    "value": "yourCookieValue",
    "expirationDate": 1798790400,
    "hostOnly": false,
    "httpOnly": false,
    "secure": false,
    "session": false,
    "sameSite": "no_restriction",
    "url": "https://.www.yourdomain.com"
  }
]
Cookie state restored with: [
  {
    "path": "/",
    "name": "yourCookieName",
    "value": "yourCookieValue",
    "expirationDate": 1798790400,
    "hostOnly": false,
    "httpOnly": false,
    "secure": false,
    "session": false,
    "sameSite": "no_restriction",
    "url": "https://.www.yourdomain.com"
  }
]
Browser Console Log 0: JSHandle:BackstopTools have been installed.
SCENARIO > BackstopJS Homepage
Browser Console Log 0: JSHandle:BackstopTools have been installed.
SCENARIO > BackstopJS Homepage
x Close Browser
x Close Browser
      COMMAND | Executing core for "report"
      compare | OK: BackstopJS Homepage backstop_default_BackstopJS_Homepage_0_document_0_phone.png
      compare | OK: BackstopJS Homepage backstop_default_BackstopJS_Homepage_0_document_1_tablet.png
       report | Test completed...
       report | 2 Passed
       report | 0 Failed
       report | Writing browser report
       report | Resources copied
       report | Copied configuration to: /src/backstop_data/html_report/config.js
      COMMAND | Executing core for "openReport"
   openReport | Opening report.
      COMMAND | Command "report" successfully executed in [0.143s]
      COMMAND | Command "test" successfully executed in [2.12s]
COMMAND | Executing core for "openReport"
openReport | Opening report.
   COMMAND | Command "test" successfully executed in [3.701s]

$

Does this mean the problem is in my config?

[n1313]

After comparing the outputs I have noticed that docker is called differently in my code and in sanity check repo:

Sanity check:

Delegating command to Docker... docker run --rm -it --mount type=bind,source="/[path]",target=/src backstopjs/backstopjs test "--_=test" "--h=false" "--help=false" "--v=false" "--version=false" "--i=false" "--moby=true" "--config=backstop.json" "--moby"

Mine (note the second "--config=backstop.config.json" in the end of arguments list):

Delegating command to Docker... docker run --rm -it --mount type=bind,source="/[path]",target=/src backstopjs/backstopjs test "--_=test" "--h=false" "--help=false" "--v=false" "--version=false" "--i=false" "--moby=true" "--config=backstop.config.json" "--moby" "--config=backstop.config.json"

This led me to the source of the problem: --config together with --docker fails. If I run sanity check with --config=backstop.json, it fails with the same Cannot set property 'tempCompareConfigFileName' of undefined error.

[garris]

Thank you for catching this issue!

A known issue also exists where config parameters are duplicated in the docker command.

I would like to address this as soon as possible. Until then are you able to simply change your config name to the default backstop.json As a workaround?

[n1313]

@garris yep, I'll do that. Thanks for confirming the issue!

[garris]

fixed with v3.5.14

[spacecowboyian]

My config file is generated by a node script (backstop.config.js) and I'm still seeing this error with 3.7.0

[garris]

can you please post a gist of your terminal output?

[spacecowboyian]

https://gist.github.com/spacecowboyian/e8dac9c6371e0d8c0239a4d2e8709893

[spacecowboyian]

The sanity test worked for me btw.

[garris]

Woah, stop the presses.

I just tested the current version and had no problem running it...

$ node ../../cli test --docker --config=backstop_alt.js
BackstopJS v3.7.0
Loading config:  /Users/gshipon/Development/BackstopJS/test/configs/backstop_alt.js

COMMAND | Executing core for "test"
Delegating command to Docker... docker run --rm -it --mount type=bind,source="/Users/gshipon/Development/BackstopJS/test/configs",target=/src backstopjs/backstopjs:3.7.0 test "--moby" "--config=backstop_alt.js"
BackstopJS v3.7.0
Loading config:  /src/backstop_alt.js

COMMAND | Executing core for "test"
createBitmaps | Selected 1 of 1 scenarios.
Cookie state restored with: []
Cookie state restored with: []
Browser Console Log 0: JSHandle:BackstopTools have been installed.
SCENARIO > Simple
Browser Console Log 0: JSHandle:BackstopTools have been installed.
SCENARIO > Simple
x Close Browser
x Close Browser
      COMMAND | Executing core for "report"
      compare | Reference image not found backstop_alt_Simple_0_document_0_phone.png
      compare | Reference image not found backstop_alt_Simple_0_document_1_tablet.png
       report | Test completed...
       report | 0 Passed
       report | 2 Failed
       report | Writing browser report
       report | Resources copied
       report | Copied configuration to: /src/backstop_data/html_report/config.js
      COMMAND | Executing core for "openReport"
   openReport | Attempting to ping
   openReport | Remote not found. Opening backstop_data/html_report/index.html
       report | *** Mismatch errors found ***
      COMMAND | Command "report" ended with an error after [0.163s]
      COMMAND | Error: Mismatch errors found.

[garris]

Hey! Before running, please cd to the root where your config is (or move your config to your current root. Does that fix it?

[spacecowboyian]

I was in the folder where the config lives. It ran with the alt config. I'll have to figure out what it doesn't like about my config.

[spacecowboyian]

Can you tell me what's wrong with this config? I've compared it to the alt and see no difference except values.

https://gist.github.com/spacecowboyian/b8c21f0a3e966fb31b61027f9e59de76

[garris]

Hmmmm, Does it work if you copy the alt file with a different name? If so, maybe just use that and change props in place until it breaks or works.

[spacecowboyian]

I figured it out. My export was a promise. I assume once I resolve that all will be well :) It would be nice if the config param, when used with docker, could accept an object like it does otherwise but I think I understand why it needs a file for docker.

[garris]

Actually, I have wanted the config interface to accept promises for a long time. (If implemented I'd expect it to work in docker as well). Would love it if someone added that feature. But until that time -- you are absolutely correct. Accepts json or pojo only. If you have a complex and or async operation (other than a require(xyz)) you could consider the node module option https://github.com/garris/BackstopJS#pass-a-config-object-to-the-command

cheers!

[spacecowboyian]

I ended up just having it write out the backstop.json first with every run and pointing to that. It'll be good to have that as a record of the last run anyway. We're using it in a UI Component monorepo and we wanted the ability to test a single package or the whole repo. Thanks so much for your help.