Deployment fails #30

kevinkuszyk · 2016-04-28T20:58:04Z

I just tried a couple of deploys, but both times it failed:

Are the any logs so I can try to help find a fix?

These are the screenshots from the Azure Portal:

mtaanquist · 2016-05-01T21:02:33Z

I am experiencing the same.

bricin · 2016-05-03T21:27:26Z

Seems like it's getting worse... the primary dialog gets stuck on Step 1.

kevinkuszyk · 2016-05-03T22:13:46Z

@davidebbo any chance you could have a look at this?

davidebbo · 2016-05-03T22:18:39Z

Try looking at the Microsoft.Resources/deployments for the resource group in https://resources.azure.com/

davidebbo · 2016-05-03T22:26:11Z

Other thing to try: after the failure, go to Portal under Deployment Source for the site, and look at the deployment log

simoneriksson100 · 2016-05-04T00:13:26Z

I am getting the same failure error attempting to deploy. When looking at the deploy logs in Publishing -> Deploy source it shows the failure reason as "REASON 🔧 Update to Node v4". The log is attached.
azure_deploy_failure_04052016.txt

davidebbo · 2016-05-04T00:52:24Z

Seems the errors all start with:

Selected npm version 3.5.1
npm ERR! fetch failed https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-2.0.1.tgz
npm WARN retry will retry, error on last attempt: Error: connect EACCES 23.235.47.162:443 - Local (undefined:undefined)

I wonder if something changed on npm that messes it up. I'm guessing that trying to run npm install manually in site\wwwroot from Kudu console would fail in the same way?

simoneriksson100 · 2016-05-04T01:14:50Z

I will do a bit more debugging in a little while, but it seems changing to http instead of https for the registry.npmjs.org fetches resolves the issue.

felixrieseberg · 2016-05-04T07:16:18Z

@davidebbo: Any reason why Web Apps wouldn't be able to establish a connection to npm? I don't think there's anything we can do from the repo here, is there?

davidebbo · 2016-05-04T14:16:50Z

I can't think of any. We need to isolate this further. e.g.

when going to Kudu Console, does it completely fail to talk to the npm server even when doing a simple curl to those uri?
does it happen everywhere or just in some locations?

kevinkuszyk · 2016-05-04T14:55:26Z

curl works for me in the Kudo Console:

but not npm install

Let me know if there's any more I can do my end to help identify the issue.

davidebbo · 2016-05-04T15:02:00Z

What if you directly try to install those same packages in Kudu Console? e.g. npm install -g wrappy. If that works, could there be anything special in the way npm runs in the context of the Ghost repo?

kevinkuszyk · 2016-05-04T15:17:25Z

Well, that's odd:

I don't know enough about npm or how the Azure Web App machines are setup to know why that worked but npm install doesn't. Any ideas?

davidebbo · 2016-05-04T18:04:50Z

I think it has to do with exceeding some connection quota in the sandbox. I need to dig deeper. Are you all seeing this on Free/Shared sites? In theory, this should not happen in Basic/Standard sites.

mtaanquist · 2016-05-04T18:06:11Z

I attempted to set up the Ghost install on a Free-tier, but I can try it out on a higher tier later.

mtaanquist · 2016-05-04T20:26:15Z

Just tried to spin it up on a Basic tier, and that seemed to work correctly.

davidebbo · 2016-05-04T22:56:43Z

Yes, here is the deal:

in Free/Shared mode, the sandbox caps the number of concurrent socket connections to 250, which normally is a lot. No cap in Basic/Standard.
but npm can go really crazy with opened connections, and in some cases can blow that cap, leading to connection failures

This appears to be a known issue with npm. e.g. read the more recent comments from npm/npm#7200, with people complaining about it (not on Azure).

I don't know why this only started to affect Ghost now, but I'd guess the number of dependencies might have increased and put it over the edge :(

kevinkuszyk · 2016-05-05T20:03:02Z

Thanks @davidebbo - thats fixed it for me too.

A brand new deploy to a basic plan worked, and scaling my free plan up to basic for the deploy and then back down to free also worked.

cicorias · 2016-05-11T18:28:52Z

Not sure what is going on, but I just did a Standard w/ Medium size worker and got this.. Sits on building for quite some time until it finally failed.

Omitting next output lines...
The iisnode.yml file explicitly sets nodeProcessCommandLine. Automatic node.js version selection is turned off.
Selected npm version 3.5.1
npm ERR! Windows_NT 6.2.9200
npm ERR! argv "D:\\Program Files (x86)\\nodejs\\4.2.3\\node.exe" "D:\\Program Files (x86)\\npm\\3.5.1\\node_modules\\npm\\bin\\npm-cli.js" "install" "--production"
npm ERR! node v4.2.3
npm ERR! npm  v3.5.1
npm ERR! code ECONNRESET
npm ERR! errno ECONNRESET
npm ERR! syscall read

npm ERR! network read ECONNRESET
npm ERR! network This is most likely not a problem with npm itself
npm ERR! network and is related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! network 
npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! network 'proxy' config is set properly.  See: 'npm help config'
npm ERR! Windows_NT 6.2.9200
npm ERR! argv "D:\\Program Files (x86)\\nodejs\\4.2.3\\node.exe" "D:\\Program Files (x86)\\npm\\3.5.1\\node_modules\\npm\\bin\\npm-cli.js" "install" "--production"
npm ERR! node v4.2.3
npm ERR! npm  v3.5.1
npm ERR! code ECONNRESET
npm ERR! errno ECONNRESET
npm ERR! syscall read

npm ERR! network read ECONNRESET
npm ERR! network This is most likely not a problem with npm itself
npm ERR! network and is related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! network 
npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! network 'proxy' config is set properly.  See: 'npm help config'
npm ERR! Windows_NT 6.2.9200
npm ERR! argv "D:\\Program Files (x86)\\nodejs\\4.2.3\\node.exe" "D:\\Program Files (x86)\\npm\\3.5.1\\node_modules\\npm\\bin\\npm-cli.js" "install" "--production"
npm ERR! node v4.2.3
npm ERR! npm  v3.5.1
npm ERR! code ECONNRESET
npm ERR! errno ECONNRESET
npm ERR! syscall read

npm ERR! network read ECONNRESET

...
ailed exitCode=1, command="D:\Program Files (x86)\nodejs\4.2.3\node.exe" "D:\Program Files (x86)\npm\3.5.1\node_modules\npm\bin\npm-cli.js" install --production
An error has occurred during web site deployment.
npm ERR! Windows_NT 6.2.9200\r\nnpm ERR! argv "D:\\Program Files (x86)\\nodejs\\4.2.3\\node.exe" "D:\\Program Files (x86)\\npm\\3.5.1\\node_modules\\npm\\bin\\npm-cli.js" "install" "--production"\r\nnpm ERR! node v4.2.3\r\nnpm ERR! npm  v3.5.1\r\nnpm ERR! code ECONNRESET\r\nnpm ERR! errno ECONNRESET\r\nnpm ERR! syscall read\r\n\r\nnpm ERR! network read ECONNRESET\r\nnpm ERR! network This is most likely not a problem with npm itself\r\nnpm ERR! network and is related to network connectivity.\r\nnpm ERR! network In most cases you are behind a proxy or have bad network settings.\r\nnpm ERR! network \r\nnpm ERR! network If you are behind a proxy, please make sure that the\r\nnpm ERR! network 'proxy' config is set properly.  See: 'npm help config'\r\nnpm ERR! Windows_NT 6.2.9200\r\nnpm ERR! argv "D:\\Program Files (x86)\\nodejs\\4.2.3\\node.exe" "D:\\Program Files (x86)\\npm\\3.5.1\\node_modules\\npm\\bin\\npm-cli.js" "install" "--production"\r\nnpm ERR! node v4.2.3\r\nnpm ERR! npm  v3.5.1\r\nnpm ERR! code ECONNRESET\r\nnpm ERR! errno ECONNRESET\r\nnpm ERR! syscall read\r\n\r\nnpm ERR! network read ECONNRESET\r\nnpm ERR! network This is most likely not a problem with npm itself\r\nnpm ERR! network and is related to network connectivity.\r\nnpm ERR! network In most cases you are behind a proxy or have bad network settings.\r\nnpm ERR! network \r\nnpm ERR! network If you are behind a proxy, please make sure that the\r\nnpm ERR! network 'proxy' config is set properly.  See: 'npm help config'\r\nnpm ERR! Windows_NT 6.2.9200\r\nnpm ERR! argv "D:\\Program Files (x86)\\nodejs\\4.2.3\\node.exe" "D:\\Program Files (x86)\\npm\\3.5.1\\node_modules\\npm\\bin\\npm-cli.js" "install" "--production"\r\nnpm ERR! node v4.2.3\r\nnpm ERR! npm  v3.5.1\r\nnpm ERR! code ECONNRESET\r\nnpm ERR! errno ECONNRESET\r\nnpm ERR! syscall read\r\n\r\nnpm ERR! network read ECONNRESET\r\nnpm ERR! network This is most likely not a problem with npm itself\r\nnpm ERR! network and is related to network connectivity.\r\nnpm ERR! network In most cases you are behind a proxy or have bad network settings.\r\nnpm ERR! network \r\nnpm ERR! network If you are behind a proxy, please make sure that the\r\nnpm ERR! network 'proxy' config is set properly.  See: 'npm help config'\r\nnpm ERR! Windows_NT 6.2.9200\r\nnpm ERR! argv "D:\\Program Files (x86)\\nodejs\\4.2.3\\node.exe" "D:\\Program Files (x86)\\npm\\3.5.1\\node_modules\\npm\\bin\\npm-cli.js" "install" "--production"\r\nnpm ERR! node v4.2.3\r\nnpm ERR! npm  v3.5.1\r\n\r\nnpm ERR! Callback called more than once.\r\nnpm ERR! \r\nnpm ERR! If you need help, you may report this error at:\r\nnpm ERR!     <https://github.com/npm/npm/issues>\r\n\r\nnpm ERR! Please include the following file with any support request:\r\nnpm ERR!     D:\home\site\wwwroot\npm-debug.log\r\nC:\Program Files (x86)\SiteExtensions\Kudu\54.50506.2209\bin\scripts\starter.cmd "D:\home\site\deployments\tools\deploy.cmd"

davidebbo · 2016-05-12T01:56:40Z

@cicorias That looks like a very separate issue, discussed on npm/registry#10. People are seeing this on EC2 and other places. It seems for that one upgrading npm is the answer. Also, I'm not sure those errors are actually harmful.

Back to the EACCES issue, we are looking at raising the limit, which should help. Look for a change around the end of the month.

cicorias · 2016-05-12T02:09:06Z

@davidebbo - ok - seems that others have been able to get passed that issue with a different version of node - 4.4.x - i'll try that with the IIS yaml and see how it goes, not sure what versions are on the box buy I'll poke around.. Then i'll try an update to npm as well per that npm/registry#10

Just so you know this was failing direct from the "Deploy to Azure" button integration. But, once that failed via Kudu console an npm install went just "fine" - however there were already some packages there it seemed.

If it's still an issue i'll open another issue.

davidebbo · 2016-05-14T16:30:42Z

Someone else reporting this here. @felixrieseberg any chance to upgrade so it uses newer npm?

felixrieseberg · 2016-05-14T17:44:51Z

Ghost supports the Node LTS release (so we're stuck with the latest LTS for now), but I set the npm version manually to the latest available on Azure (3.8.6).

@davidebbo: That too is a few versions behind the latest available (3.8.9 is the latest patch, 3.9.1 the latest minor). This is a total off-topic, but let me know if we should build you a little script that just shovels all available versions into the image!

cicorias · 2016-05-14T18:02:13Z

there are npm versions in "d:\program files (x86)\npm

03/06/2016  01:54 PM    <DIR>          2.1.17
03/06/2016  01:58 PM    <DIR>          2.11.2
03/06/2016  01:55 PM    <DIR>          2.14.12
03/06/2016  01:58 PM    <DIR>          2.14.2
03/10/2016  07:55 PM    <DIR>          2.14.20
03/06/2016  01:58 PM    <DIR>          2.14.3
03/06/2016  01:58 PM    <DIR>          2.14.4
04/01/2016  04:27 PM    <DIR>          2.15.1
03/06/2016  01:59 PM    <DIR>          2.5.1
03/06/2016  01:59 PM    <DIR>          2.7.4
03/06/2016  01:54 PM    <DIR>          2.7.6
03/06/2016  01:58 PM    <DIR>          2.9.1
03/06/2016  01:53 PM    <DIR>          3.1.0
03/06/2016  01:56 PM    <DIR>          3.3.12
03/06/2016  01:53 PM    <DIR>          3.3.6
03/06/2016  01:57 PM    <DIR>          3.3.9
03/06/2016  01:55 PM    <DIR>          3.5.1
03/06/2016  01:53 PM    <DIR>          3.5.3
03/06/2016  01:54 PM    <DIR>          3.6.0
03/10/2016  07:55 PM    <DIR>          3.7.3
04/01/2016  04:27 PM    <DIR>          3.8.3
04/29/2016  08:27 PM    <DIR>          3.8.6

felixrieseberg · 2016-05-14T18:03:07Z

Yes, exactly, please notice how 3.8.6 is the last available one ;-)

cicorias · 2016-05-14T18:07:47Z

also, instead of updating or specifying the node version in the iisnode.yaml file, why isn't the package.json updated to reflect preference but with the higher versions first?

  "engines": {
    "node": " ^4.2.0 || ~0.12.0  || ~0.10.0",
    "npm": "3.8.6"
  },

felixrieseberg · 2016-05-14T18:11:04Z

I'd like to change the original Ghost files as little as possible - the only reason I'm locking down the Node version at all is to ensure that pre-gyp (on the free tier) is able to find binaries for sqlite3 (which usually takes a while after a Node release).

This is basically a long flowchart of "if we could [...] then we wouldn't have to [...]" 🏄

davidebbo · 2016-05-14T20:26:26Z

One problem I found is that Kudu doesn't support the caret syntax (e.g. ^4.2.0), because it's using an ancient semver.js from days where the syntax didn't exist. I think that's at least partially the root of some issues. I'll look into fixing that.

davidebbo · 2016-05-16T21:51:51Z

This should be solved by this PR: #34. You can try it before it gets merged on my fork: https://github.com/davidebbo-test/Ghost-Azure

kevinkuszyk closed this as completed May 5, 2016

kevinkuszyk mentioned this issue May 5, 2016

Add readme note about deployment failure in free and shared app service plans #31

Merged

davidebbo mentioned this issue May 11, 2016

Error in deployment connect EACCES Azure/azure-sdk-for-node#1740

Closed

davidebbo mentioned this issue May 16, 2016

Don't specify nodeProcessCommandLine in yml file #34

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deployment fails #30

Deployment fails #30

kevinkuszyk commented Apr 28, 2016 •

edited

Loading

mtaanquist commented May 1, 2016

bricin commented May 3, 2016

kevinkuszyk commented May 3, 2016

davidebbo commented May 3, 2016

davidebbo commented May 3, 2016

simoneriksson100 commented May 4, 2016

davidebbo commented May 4, 2016

simoneriksson100 commented May 4, 2016

felixrieseberg commented May 4, 2016

davidebbo commented May 4, 2016

kevinkuszyk commented May 4, 2016

davidebbo commented May 4, 2016 •

edited

Loading

kevinkuszyk commented May 4, 2016

davidebbo commented May 4, 2016

mtaanquist commented May 4, 2016 •

edited

Loading

mtaanquist commented May 4, 2016

davidebbo commented May 4, 2016

kevinkuszyk commented May 5, 2016

cicorias commented May 11, 2016

davidebbo commented May 12, 2016

cicorias commented May 12, 2016

davidebbo commented May 14, 2016

felixrieseberg commented May 14, 2016

cicorias commented May 14, 2016

felixrieseberg commented May 14, 2016

cicorias commented May 14, 2016

felixrieseberg commented May 14, 2016

davidebbo commented May 14, 2016

davidebbo commented May 16, 2016

Deployment fails #30

Deployment fails #30

Comments

kevinkuszyk commented Apr 28, 2016 • edited Loading

mtaanquist commented May 1, 2016

bricin commented May 3, 2016

kevinkuszyk commented May 3, 2016

davidebbo commented May 3, 2016

davidebbo commented May 3, 2016

simoneriksson100 commented May 4, 2016

davidebbo commented May 4, 2016

simoneriksson100 commented May 4, 2016

felixrieseberg commented May 4, 2016

davidebbo commented May 4, 2016

kevinkuszyk commented May 4, 2016

davidebbo commented May 4, 2016 • edited Loading

kevinkuszyk commented May 4, 2016

davidebbo commented May 4, 2016

mtaanquist commented May 4, 2016 • edited Loading

mtaanquist commented May 4, 2016

davidebbo commented May 4, 2016

kevinkuszyk commented May 5, 2016

cicorias commented May 11, 2016

davidebbo commented May 12, 2016

cicorias commented May 12, 2016

davidebbo commented May 14, 2016

felixrieseberg commented May 14, 2016

cicorias commented May 14, 2016

felixrieseberg commented May 14, 2016

cicorias commented May 14, 2016

felixrieseberg commented May 14, 2016

davidebbo commented May 14, 2016

davidebbo commented May 16, 2016

kevinkuszyk commented Apr 28, 2016 •

edited

Loading

davidebbo commented May 4, 2016 •

edited

Loading

mtaanquist commented May 4, 2016 •

edited

Loading