[extension-upload] - unable to upload a brackets extension to the registry

[timoweiss]

Hey, tried to upload an extension to the registry but i always get the following error:
"The extension package is invalid: The uploaded content is not a valid zip file."

My zip file has a size of about 3.1Mb.
It has a main.js and a valid package.json.
I have the suspicion that the large amount of files cause the error. Is it possible that there is any limit?

I created the zip via terminal 'zip requireRelease.zip . -r'

See also: https://groups.google.com/forum/#!topic/brackets-dev/srV4w5O0r94

[dangoor]

I was able to upload the file on localhost.

@ingorichter is there something in the log on the server perhaps?

[timoweiss]

@ingorichter any news?

[pthiess]

Doesn't seem to impact many people, hence marking it low priority to investigate @ingorichter.

[peterflynn]

Note: @timoweiss mentioned in the forum that removing "font-awesome" from his set of NPM modules fixes this. I wonder if there's anything unusual inside that folder structure that we're choking on? (Or does it just reduce the number of files / depth of nesting dramatically?)

[timoweiss]

i was just about to write here :)

here's the file-tree of font-awesomes node_modules dir:

-node_modules
|---less
|-----benchmark
|-----bin
|-----build
|-----dist
|-----lib
|-------less
|---------tree
|-----node_modules
|-------mime
|---------types
|-------mkdirp
|---------examples
|---------test
|-------request
|---------lib
|---------node_modules
|-----------aws-sign2
|-----------forever-agent
|-----------form-data
|-------------lib
|-------------node_modules
|---------------async
|-----------------lib
|---------------combined-stream
|-----------------lib
|-----------------node_modules
|-------------------delayed-stream
|---------------------lib
|---------------------test
|-----------------------integration

[ingorichter]

@timoweiss @dangoor I'm sorry, but there was nothing in the logfiles that provides more information to resolve this issue.

[peterflynn]

@ingorichter Now that we know the "font-awesome" NPM package was the culprit, should we try reproducing it ourselves using a fake extension that includes that package?

[ingorichter]

@peterflynn Yes, I'm going to do this.

[timoweiss]

@ingorichter sorry, i couldn't wait, i tried it. And the upload was successful. But i had to remove some files from the node_module less to achieve the 5Mb. So, could you please remove my test-extension called font-awful from the registry.
Its maybe not font-awesome which caused the problem, but it might be the mass of files.
Thank you.

[peterflynn]

@timoweiss You should be able to remove it yourself now -- we've just updated the registry, and you should now see a Delete button next to any extensions you own.

[timoweiss]

@peterflynn perfect, it worked! Thanks!

[timoweiss]

@ingorichter @peterflynn i just ran again in this issue. I can't figure it out. The biggest change i made to the previous version was adding socket.io to the extension. First i thought that it was the huge amount of files/folders.
So i removed the socket.io folder out of the extension. But the uploader tells me the same error The extension package is invalid: The uploaded content is not a valid zip file.

Do you guys have any new knowledge to this issue?

[ingorichter]

@timoweiss I was working on the registry recently. I can have a look at it to figure out what happened. Where can I find the latest version of your extension?

[jdiehl]

I just ran into the same issue trying to update less-autocompile. The weird thing is, I haven't really changed anything in terms of files since my last update on April 5th.

You can download the pack here: https://dl.dropboxusercontent.com/u/1213482/less-autocompile.zip

[equiet]

I experience this too with my extension: https://github.com/equiet/recognizer

It's about 1.7 MB and was failing to upload. In the end, I had to remove all "larger" files from node_modules/ directory, usually some tests and a quite big icons/ folder.

[timoweiss]

@ingorichter this is my new version of the extension:
quickrequire-0.5.0
And this is the latest version which was in the registry.
quickrequire-0.4.0

[dangoor]

Given the multiple recent reports, I think we should raise this to medium priority. There are still new extensions showing up, so it's not some sort of total breakage, but it's something to be aware of.

I did update some libraries when I added the admin features to the registry, but I don't think I updated the file upload handler. Something to watch for, though.

[ingorichter]

Quick update: I was unable to reproduce this with any of the failing extensions. :-(
What I'm going to do next, is to get my Linux VM ready with Ubuntu 12.04.4 LTS and node version 0.10.12 that is used to run the registry. Perhaps I can reproduce it this way.
One other option would be to update the node version and the decompress-zip dependency of the extension registry.

[WebsiteDeveloper]

I also stumbled upon this problem when i tried to upload a locally made zip of my extension.
I got it to work when i downloaded the source code zip from my github repo and uploaded that.

[timoweiss]

@ingorichter @jdiehl @WebsiteDeveloper @equiet
After the downtime today, the registry is now accepting our extension again.

[ingorichter]

@timoweiss That is an interesting observation (IMHO). Unfortunately I had to restart the registry app and I don't have any evidence for previous problems that might have caused the misbehavior of the app this morning.
If this kind of failure happens only after the registry ran for a couple of days/weeks, this would explain why we were unable to reproduce this locally.

[equiet]

I can confirm, upload now works as expected. Thanks a lot.

[redmunds]

When you zip up your repo locally, it includes the .git folder. The Brackets Extension Registry should ignore that folder (which has caused problems in the past), but it adds a lot of unnecessary size to the zip file, so try to avoid including it.

[jdiehl]

less-autocompile also worked now!

[albertinad]

Hi @ingorichter , after a lot of failure intents uploading brackets-bower extension to the registry, I was able to reproduce the same error in my local environment. Below the steps I follow:

1. set up my local brackets registry
2. upload my package
   2.1) everything was right
3. clean up to start again
4. download 70 random extensions from the public brackets registry
5. start the local brackets registry
6. drag and drop all of the 70 extensions, wait until all of them where successfully uploaded
7. upload my package
   7.1) it fails, with the code that indicates that is not a vaild zip (which it is)

After digging a little and doing some debugging, I found the error code of it: EMFILE from the decompress-zip module/node. Related to bower/decompress-zip#26

Then, I updated the decompress-zip module to latest stable version, from 0.0.2 to 0.1.0, repeat all the steps above multiple times, and I was able to upload to my local brackets registry my extension.
Also updated it on brackets and the tests passed.

We are using an outdated version of decompress-zip module, maybe it is time to update it.

[zaggino]

Good job @albertinad 👍

[ingorichter]

Great job @albertinad. I suspected that decompress-zip might be the issue. I did some investigation some time ago but never finished the work (#7387). Sometimes the error disappeared after restarting the registry (I have not really an idea why).

[smiclea]

@ingorichter You can download it here: https://drive.google.com/file/d/0BxNY66eFHRb5YXI5UmlteVo2N0U/view?usp=sharing

This is the github repository: https://github.com/smiclea/brackets-http-server
Needs 'npm install' in node folder if you want to take it from there

[smiclea]

So it still doesn't work for me

[smiclea]

It works now. Thank you!

[kooohei]

does not work now, that was possible up until now.
I could not registered my extension "brackets-synapse" since yeasterday.

[kooohei]

sorry I can't want, I have been required npm 's lodash and moment.
it works when canceled that.

thx.

[acbarbosa1964]

Dear friends,

I have several extensions I can upload, including previous version of the one I'm facing this same problem. I don't now what happened, I just change the project version and a string in my project and now I can't upload :(

It falls wit message:

acb-tartan-background-master.zip
1.5 MB
✘
The extension package is invalid: The uploaded content is not a valid zip file.

[swmitra]

@acbarbosa1964 if you have the source code for your extension uploaded to github, use 'download zip' utility from github project page and upload the same to registry.

One more thing to validate is the package.json structure. Make sure it's not broken after the last update and still a valid json.

[acbarbosa1964]

Dear Mitra,

Thank you, but I have done that several time and I'm very sure JSON is
correct once I have used same structure for all my extensions.

Attached you have the zip file if you want to take a look.

Best Regards,

Thank you.

Atenciosamente,

Antonio Carlos Barbosa

*Fone: *+55 15 2102-6500

http://www.alphatour.com.br http://www.alphatour.com.br
http://www.alphatour.info http://www.alphatour.info

Um pensamento na madrugada...

"Muitas vezes, as melhores coisas que temos a dizer para quem amamos,
temos como amigo ou nos relacionamos profissionalmente, são tão boas que
não conseguimos coloca-las para fora, mas as coisas ruins como críticas,
descontentamentos, raiva ..., que não queremos dentro de nós, são mais
fáceis de externalizar." Ainda hoje aprendi que dividindo estes sentimentos
bons que ficam guardados dentro do nosso peito, estamos na realidade
criando um efeito multiplicador, pois eles voltam para nós em dobro.

Antonio Carlos Barbosa

​
A thought in the early hours ...

"Often, the best things we have to say to those we love, we have as friend
or relate professionally, are so good that we couldn't put them out, but
the bad things like criticism, discontent, anger ... that we don't want
within us, are easier to speak out. *
*"
Today I've learned that sharing these good feelings we have stored within
our chest, we are actually creating a multiplier effect, because they come
back to us twice.

• Antonio Carlos Barbosa

2015-10-19 15:22 GMT-02:00 Swagatam Mitra notifications@github.com:

@acbarbosa1964 https://github.com/acbarbosa1964 if you have the source
code for your extension uploaded to github, use 'download zip' utility from
github project page and upload the same to registry.

One more thing to validate is the package.json structure. Make sure it's
not broken after the last update and still a valid json.

—
Reply to this email directly or view it on GitHub
#7387 (comment).

[petetnt]

I am running into this issue with my new extension https://github.com/petetnt/brackets-sass-lint

I am getting {"errors":["The extension package is invalid:","The uploaded content is not a valid zip file."]} despite being able to install it from URL or .zip with no issues. My package is 4.7 megabytes. None of the workarounds listed didn't help. Tried with Chrome and Firefox.

[swmitra]

@petetnt I have faced the same problem while updating my extension last year. Although I am not sure about the actual cause but the following steps got rid of the problem.

1. Logout from extension registry
2. Clear browser cache ( Not sure if mandatory but I did )
3. Close your browser
4. Open registry again and then log in
5. Try to upload the extension ( It worked for me !)

[mitchellsimoens]

I've also started using the brackets-npm-registry. It did take a full day for it to show up but publish to npm with brackets-extension in the keywords array of package.json and it picked it up. To update, just issue a npm publish and the registry picks up the update.

Would be great to migrate to this instead of the builtin one that I've had issues with it going down (has been a while for that tho).

[petetnt]

@swmitra Good idea, but sadly those steps wouldn't work for me. 😿

[petetnt]

Confirmed that it was the node_modules that are the culprit: there's a lot of them (3387 Files, 547 Folders) but the combined size is still less than ~4 megabytes. Removing them from the zip file made the process a success.

I tried to pack the modules with npm@v2 but that didn't work either: only thing it did was was get the package size up to 7 megabytes.

The thing is that I can't even lower my amount of files: I only have one external dependency, sass-lint that just happens to use tons of external dependencies 😿

[Intronet]

The extensions registry upload is not working again, and I see it is nearly a full year that this topic was opened - I need to upload v0.0.5
Can you restart the server please
[Brackets Website Admin]

[alemonteiro]

I'm unable to upload an new version also. None of the work around suggestions worked for me.
The new package: http://alemonteiro.com.br/brackets-sql-connector.zip

[alemonteiro]

Still can't upload the update for sql-connector but was able to upload a new smaller extension.
The connector one tried from linux using file roller, zip and rar, and from windows using winrar e 7zip, none worked.

[alemonteiro]

Maybe this issue is related to file size? I couldn't upload anything larger than 2mb but never had problem with small packages.

So added auto install of node dependencies on the extension so the package would be small and only then was able to upload and update the version on registry.

[dnbard]

@alemonteiro I cannot upload 99kb extension right now so it's not related to size, I think.
edited: I was able to upload extension with no compression option enabled for zip archive.

[ingorichter]

I always had the impression that it was related to the program that created the zip file. I don't have 100% proof for it, but it seems that the zip library used to decompress the uploaded zip chokes on some of the files.

[alemonteiro]

@dnbard , i don't think it's only size related, but it may play a role...

@ingorichter , i had this impression also, but it got mixed up with the size thing...

Using wirar over windows I never had a problem with upload, did various uploads until 1.5mb.
But some weeks ago I migrated to Debian and the first package that I tried to upload, zipped with archiver, wasn't accepted. So I tried using xarchiver, rar and zip directly, but with no luck.
Then I got this same package, about 2mb, and tried from windows (with Winrar and 7zip) but also didn't get accepted.

I could only get it to be accepted again after removing the dependecies from the package, so it would be like 200kb, and with that size any zipper that i tried worked.

I think this won't help much because it feels inconsistent. I'll try to run a version locally to debug it once I get some free time.

[swmitra]

Closing this issue now. Registry server has been upgraded. Please reopen this issue if you still have problem in uploading extension.