Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

failure: Updating C/C++ dependencies in offline mode #298

Closed
RyanSung opened this issue Oct 19, 2016 · 56 comments
Closed

failure: Updating C/C++ dependencies in offline mode #298

RyanSung opened this issue Oct 19, 2016 · 56 comments
Labels
Feature Request fixed Check the Milestone for the release in which the fix is or will be available. Language Service
Milestone

Comments

@RyanSung
Copy link

Updating C/C++ dependencies...

Downloading package 'Mono Framework Assemblies' (4751 KB) .................... Done!
Downloading package 'Mono Runtime (OS X)' (1549 KB) .................... Done!
Downloading package 'LLDB 3.8.0 (OS X)' (18486 KB) .................... Done!
Downloading package 'C/C++ language components (OS X)' Failed at stage: downloadPackages
Error: write EPROTO 140735231705088:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:../../vendor/node/deps/openssl/openssl/ssl/s23_clnt.c:769:

Finished

@sean-mcmanus
Copy link
Collaborator

sean-mcmanus commented Oct 19, 2016

Sounds like the Node.js OpenSSL library we depend on is failing. I haven't seen that before. What happens if you restart VS Code and try again? What are your VS Code "http." settings? Are you able to download these?
https://go.microsoft.com/fwlink/?LinkID=826082
https://go.microsoft.com/fwlink/?LinkID=826083

@Jevyn
Copy link

Jevyn commented Oct 20, 2016

On the topic of updating C/C++ dependencies, can we please have an offline install option that has all the dependencies bundled together? I work in a development environment intentionally disconnected from the internet. Right now I'm having to hack a solution by reading the package.json files to determine what I need to download and then reading the source for how the package manager works. It's slightly annoying to do for each release.

I realize this is a special case, but anything you can do to make this a bit easier would be much appreciated. Example: if the "Download dependencies" vs "Install dependencies" functionality of the extension were split apart as separate functions (instead of a bunch of chained lines of code), a standalone script could be easily implemented to call the install functionality and point it at the downloaded zip files.

Thanks.

@delmyers
Copy link
Contributor

@Jevyn, @RyanSung , I'm sorry that you are experiencing these difficulties. Unfortunately, we depend on a number of native components that have to be compiled for each platform. If we shipped them all in the same extension, then a large portion of the extension would not be useable to you, and you would have to wait a long time for the initial download.

@Jevyn, do you have a suggestion for the "offline" install? How would you forsee getting the files in the first place?

@Jevyn
Copy link

Jevyn commented Oct 20, 2016

@delmyers I get the files the same way I get vscode and extensions. I download them on an internet connected machine. Virus scan them, burn to CD and physically move them to the disconnected network. Don't get me wrong - it's not an ideal solution, but is the environment I have to work in. I'd need all of the dependencies for all platforms anyway - we have multiple developers using vscode who work on Windows, Mac and Linux platforms). I'm not concerned about the download size of the files.

As for a suggestion. Have a look at the extensions main.js, you will see the function processRuntimeDependencies. Inside this function, it uses a bunch of anonymous functions for the various stages - downloadPackages, installPackages, etc.

Suggestion 1: Convert the anonymous functions to named functions and export them. This way a standalone offlineDependenciesInstall.js could be made to call into the step after downloadPackages. I could download the files and run this script. The downside is that it would have to be kept insync with main.js.

Suggestion 2: There is already a local variable that indicates the installationStage. Any chance this could be serialized to disk and loaded when the extension starts? Then someone could do the following. Download the dependencies and drop them in a known "downloaded" location inside the extension directory. Touch the installationStage file and give it "downloadedPackages" as the contents. Start vscode. cpptools extension reads installationStage file and continues to the next stage ("installPackages" as currently implemented in main.js). The advantage of this is that no extra script needs to be kept in sync.

Lastly, all of this offline install stuff gets easier if there were a single bundle that had all the dependencies. That's a nice to have though - making the dependency install easier is much more important.

@delmyers
Copy link
Contributor

@Jevyn, Can you explain to me in a little more detail your current workflow. What are the steps you use for "downloading them, and burning them to a CD"? How do you install vscode+extensions after you download and burn? And what do you download and burn?

@Jevyn
Copy link

Jevyn commented Oct 21, 2016

@delmyers Workflow is as follows
Internet Terminal:

  1. Download vscode (Windows, Mac and x64 rpm)
  2. Go to cpptools marketplace (https://marketplace.visualstudio.com/items?itemName=ms-vscode.cpptools)
    right click, view source
    search for VSIXPackage, copy paste to new window to download
    ex: https://ms-vscode.gallerycdn.vsassets.io/extensions/ms-vscode/cpptools/0.9.2/1474583577220/Microsoft.VisualStudio.Services.VSIXPackage
    rename to cpptools.vsix
    burn these 2 files to CD

Segregated Workstation - using CD from above

  1. install vscode with rpm -i (I'm on Linux)
  2. install cpp extension from command line with: code --install-extension cpptools.vsix
  3. open .vscode/extensions/ms-vscode.cpptools-0.9.2/package.json, and print runtimeDependencies

Back on Internet Terminal:

  1. Download all the urls from .vscode/extensions/ms-vscode.cpptools-0.9.2/package.json (typing by hand...)
  2. burn these files to 2nd CD

Back on Segregated Workstation - using 2nd CD

  1. Essentially I now manually do what main.js would do
  2. unzip the dependencies (platform based) to .vscode/extensions/ms-vscode.cpptools-0.9.2
  3. Set files to executable where needed
  4. edit package.json (rewrite manifest)
  5. touch lockfile (install.lock)
  6. run code :)

I repeat the steps as necessary any time vscode or cpptools is upgraded.

@IchiNiNiIchi
Copy link

@RyanSung I come across the same situation, and I go around it by changing URL that causes failure from using https to using http.

@delmyers
Copy link
Contributor

@Jevyn, can you email me at delmyers@microsoft.com? I can try and help you come up with a solution for your particular situation.

@tofraa
Copy link

tofraa commented Nov 17, 2016

@delmyers I have used more or less the same solution described above from @Jevyn. Both of his suggestions for offline installation would simply my workflow.

@Greyze
Copy link

Greyze commented Jan 13, 2017

I am also attempting to setup cpptools in an offline environment, I still haven't succeeded following Jevyn's instructions unfortunately.
Despite having the right files in the ./bin folder the extension keeps attempting to download them anyway.

An offline bundle install would be excellent.

@sean-mcmanus
Copy link
Collaborator

The ms-vscode.cpptools-0.9.3/install.lock file should prevent any additional downloading. Does that not work? Where are you hitting a problem?

@Greyze
Copy link

Greyze commented Jan 13, 2017

I don't understand what this step is;

4. edit package.json (rewrite manifest)

To the best of my knowledge i've followed every other step correctly. I did just notice a typo in the lock file (ugh), fixing that has stopped the output message.

@sean-mcmanus
Copy link
Collaborator

sean-mcmanus commented Jan 13, 2017

Open ms-vscode.cpptools-0.9.3/package.json, then open out/src/main.js and look at the rewriteManifest method and do the steps listed there (change the activationEvents and the contributes.debuggers values, "undefined" means remove it). Changing the activationEvents makes it so the C++ extension doesn't load unless it is in a valid scenario. Also, simply copying the entire ms-vscode.cpptools.<version> contents from a machine with internet access with a matching OS to the target machine should also work.

@Greyze
Copy link

Greyze commented Jan 13, 2017

Thanks for the info. Ill spin up a Centos VM on the weekend and go ahead with the copy/paste plan.

@sean-mcmanus
Copy link
Collaborator

FYI, if the target machine is CentOS then the language service will fail due to our requirement for glibc at least 2.18. Updating the glibc on the machine should fix it though (see #19 ).

@ampatel21
Copy link

FYI, the vsix extensions can be opened as a zip file on your internet machine. I extract the extension and view the package.json file in order to determine the links to the runtime dependencies I will need. I then download all the runtime dependencies. This will reduce the need to burn files twice.

Now back on my offline network I put the runtime dependencies on my local webserver. I then proceed to install code, and the cpptools.vsix extension. Prior to opening code, I modify the package.json for the cpptools extension to point to the new locations for the runtime dependencies (ex. http://localwebserver/mono-framework.zip). I also modify out/src/Debugger/packageManager.js to require('http') instead of require('https'). Restart code, and cpp extension will download the runtime dependencies from my webserver and do all the steps in main.js for me.

@ghost
Copy link

ghost commented Feb 17, 2017

@Jevyn I recently installed VS Code on my Windows 8.1 and I can't install any extension.
I have this error:
Updating C/C++ dependencies...
Downloading package 'C/C++ language components (Windows)' Failed at stage: downloadPackages
Error: ENOENT: no such file or directory, open 'C:\Users\Me\AppData\Local\Temp;C:\mingw64\bin;\package-6184ueCuLKhZbbn2.tmp'
So I tried to install them in a offline mode, BUT CPPtools is not able to download dependencies.
Then I tried to download them as you described. Could you please explain more about these steps:
2-unzip the dependencies (platform based) to .vscode/extensions/ms-vscode.cpptools-0.9.2
where exactly? the same folder as package.json lives?
C:\Users\Me\ .vscode\extensions\ms-vscode.cpptools-0.10.1?
What exactly means these three next 3,4 and 5?
3-Set files to executable where needed
4-edit package.json (rewrite manifest)
5-touch lockfile (install.lock)

@sean-mcmanus
Copy link
Collaborator

You have the right folder. You can skip #3 on Windows (otherwise you have to run chmod 775 to set the binaries as executable). For #4, you should also make changes to the package.json (i.e. manifestObject below):
manifestObject.activationEvents = [
"onLanguage:cpp",
"onLanguage:c",
"onCommand:extension.pickNativeProcess",
"onCommand:extension.pickRemoteNativeProcess",
"onCommand:C_Cpp.ConfigurationEdit",
"onCommand:C_Cpp.ConfigurationSelect",
"onCommand:C_Cpp.SwitchHeaderSource",
"onCommand:C_Cpp.UnloadLanguageServer",
"workspaceContains:.vscode/c_cpp_properties.json",
"onDebug:cppdbg",
"onDebug:cppvsdbg"
];
manifestObject.contributes.debuggers[0].runtime = undefined;
manifestObject.contributes.debuggers[0].program = './debugAdapters/OpenDebugAD7';
manifestObject.contributes.debuggers[0].windows = { "program": "./debugAdapters/bin/OpenDebugAD7.exe" };

For #5, you need to add an install.lock file in the same root directory to prevent the download from being reattempted.

However, ideally you should get the download to work somehow or copy the installed extension files from a machine that is able to download them successfully.

@ghost
Copy link

ghost commented Feb 18, 2017

@sean-mcmanus Thanks for the response. After doing all, I have this error in the debug console:

You may only use the C/C++ Extension for Visual Studio Code with Visual Studio
Code, Visual Studio or Xamarin Studio software to help you develop and test your
applications.

@sean-mcmanus
Copy link
Collaborator

sean-mcmanus commented Feb 18, 2017

That's just the license text. It appears in the Debug Console window when you start debugging. Afterwards it should show which symbols are loaded or not. What feature is not working for you?

@ghost
Copy link

ghost commented Feb 19, 2017

When I use the launch green bottom, it gives me this error:
The system cannot find the file specified.
Actually I can't even build the code, and there is no .exe file.

@bobbrow bobbrow changed the title failure: Updating C/C++ dependencies failure: Updating C/C++ dependencies in offline mode Mar 9, 2017
@sean-mcmanus
Copy link
Collaborator

@Majid-Khosh The C++ extension doesn't provide build tools. Were you able to build using VS Code Tasks or an external command line?

@jdmairs
Copy link

jdmairs commented Apr 17, 2017

I also have the same requirement todo an offline airgapped install. Also paperwork is required so doing this in 10 piecemeal steps component by component is not ideal.

In this day and age size of the download is less of a concern if it fits on a single sided DVD.

@sean-mcmanus
Copy link
Collaborator

@jdmairs You should be able to install the extension on the machine with internet access, zip the extension folder with a tool that preserves permissions, such as tar, burn the zip to cd, and then unzip on the target machine, although your source/target machines needs to be the same OS or you'd need to change the OS-dependent files that are downloaded (we might add a command to make that easier later). The size matters because VS Code imposes a 20 MB size limit for extensions and every update downloads the entire extension again.

@jdmairs
Copy link

jdmairs commented Apr 17, 2017 via email

@jdmairs
Copy link

jdmairs commented Apr 17, 2017 via email

@sean-mcmanus
Copy link
Collaborator

@jdmairs From what I read online, it sounds like they are very similar. Both don't have glibc 2.18, so you'd have to use the workaround described at #19 for our language service process to work.

@akuz
Copy link

akuz commented May 8, 2017

In the old times, it was impossible to just install an additional component from the Internet. In the modern times, it's impossible to install an additional component without the Internet. I am faced with the same problem of working in offline environment, but unfortunately no spare time to perform the above manual manipulations by hand... Would be nice to just have a download link per platform, copy to offline environment, install.

@sean-mcmanus
Copy link
Collaborator

sean-mcmanus commented May 10, 2017

@aaronniemiec What version number and bitness of RedHat Linux? Our Linux bits are generic to all distros, but require 64-bit and glibc 2.18 or greater (RHEL 8 works, but 7 requires a workaround). The easiest way is to do a clean install of the extension and then before VS Code is reloaded and it downloads the OS-specific files, you can delete the runtimeDependencies sections that match your download machine OS and change the ones that match your target os to values that match your target OS. For example, for Windows download machine targeting Linux your runtimeDependencies would look something like: "runtimeDependencies": [ { "description": "C/C++ language components (Linux)", "url": "https://go.microsoft.com/fwlink/?linkid=847739", "platforms": [ "win32" ], "binaries": [ "./bin/Microsoft.VSCode.CPP.Extension.linux", "./bin/Microsoft.VSCode.CPP.IntelliSense.Msvc.linux" ] }, { "description": "ClangFormat (Linux)", "url": "https://go.microsoft.com/fwlink/?LinkID=826081", "platforms": [ "win32" ], "binaries": [ "./LLVM/bin/clang-format" ] }, { "description": "Mono Framework Assemblies", "url": "https://go.microsoft.com/fwlink/?LinkId=816539", "platforms": [ "win32" ] }, { "description": "Mono Runtime (Linux / x86_64)", "url": "https://go.microsoft.com/fwlink/?LinkId=816541", "platforms": [ "win32" ], "binaries": [ "./debugAdapters/mono.linux-x86_64" ] } ],
Does that work for you? I tried this out going from Windows and zipping to tar via 7zip and transferring to my 64-bit Linux machine and unzipping and it worked. I believe that is the fastest/easiest way to do an offline install. In the future, we might be able to add some "export" command that does this automatically. The other method requires you run chmod 775 on all the executables that are downloaded, then modifying the package.json as is done in the RewriteManifest function in main.js (in the out/src folder of the extension), and creating an install.lock file, which I think is more work than the 1 zip method, which creates a zip that is easily installable to multiple computers.

@pa872d
Copy link

pa872d commented May 11, 2017

So, just to be clear, you downloaded VS Code on your Windows machine with internet access; installed the C/C++ extension; then followed the steps outlined prior to re-loading VS Code. Once complete, you copied the resulting files onto your Linux machine.

Does that sound right? Did you zip the entire .vscode/extensions/cpp-tools directory or just a subdirectory when going from Windows to Linux?

Thanks for all your support...

@pa872d
Copy link

pa872d commented May 11, 2017

Also -- will this allow the "Auto Indent" functionality to be accessible in my Linux target VS Code? Still wondering if this provides a solution to that problem I had... Please advise!

@sean-mcmanus
Copy link
Collaborator

@aaronniemiec Yeah, what you say sounds correct. Let us know if you hit a problem. Only the ms-vscode.cpptools folder needs to be zipped/copied, since that is where our extension installs to, but you could zip the other extension folders too if you're using any and don't have the .vsix. What is "auto indent" functionality? If it works on Windows, it should work on Linux too.

@pa872d
Copy link

pa872d commented May 11, 2017

I'm talking about what I see here:
https://github.com/Microsoft/vscode-tips-and-tricks#code-formatting

If I insert an extra tab on a line, when I try Ctrl+K, Ctrl+F it does nothing to fix the incorrect indent. Why is that? Is it related to not having the C/C++ Extension installed correctly?

@sean-mcmanus
Copy link
Collaborator

Yes, our extension supports formatting via clang-format. Format document and format selection should work fine. Our formatOnType support is somewhat limited currently though (only formats the current line on ";" and not on "}"). Our formatOnSave may have some bugs.

@pa872d
Copy link

pa872d commented May 11, 2017

Here was the output after changing the package.JSON with the text above and reloading VS Code:

Updating C/C++ dependencies...

Downloading package 'C/C++ language components (Linux)' Failed at stage: downloadPackages
Error: getaddrinfo ENOTFOUND go.microsoft.com go.microsoft.com:443
at errnoException (dns.js:28:10)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:76:26)

Finished

Doesn't look like it worked ... Or did it?

@sean-mcmanus
Copy link
Collaborator

@aaronniemiec Yeah, it is failing to download for some reason. It works for me. It looks like your computer/network you're trying to download from has some problem. I assume using the unmodified package.json also doesn't work? What about manually downloading the links in the package.json?

@pa872d
Copy link

pa872d commented May 11, 2017

I have manually downloaded the links in the package.json but not sure which ones I want... Can you direct me?

I've downloaded the following .zip files:

mono-runtime-linux-x86.zip
mono-framework.zip
LLVM_Linux.zip
Bin_Linux.zip

Do I need any others or is that the complete list?

@sean-mcmanus
Copy link
Collaborator

The mono-runtime-Linux-x86.zip is for a 32-bit Linux machine, but our language service doesn't work on 32-bit Linux, so you'd only get the debugger stuff. We don't have a way to automatically install the .zips after downloading. It sounds like there's some issue with our package download code that is preventing a download? Have you setup your http.proxy settings correctly?

@pa872d
Copy link

pa872d commented May 11, 2017

Is this the right code to have in my package.json file?

"runtimeDependencies": [
{
"description": "C/C++ language components (Linux)",
"url": "https://go.microsoft.com/fwlink/?linkid=847739",
"platforms": [
"win32"
],
"binaries": [
"./bin/Microsoft.VSCode.CPP.Extension.linux",
"./bin/Microsoft.VSCode.CPP.IntelliSense.Msvc.linux"
]
},
{
"description": "ClangFormat (Linux)",
"url": "https://go.microsoft.com/fwlink/?LinkID=826081",
"platforms": [
"win32"
],
"binaries": [
"./LLVM/bin/clang-format"
]
},
{
"description": "Mono Framework Assemblies",
"url": "https://go.microsoft.com/fwlink/?LinkId=816539",
"platforms": [
"win32"
]
},
{
"description": "Mono Runtime (Linux / x86_64)",
"url": "https://go.microsoft.com/fwlink/?LinkId=816541",
"platforms": [
"win32"
],
"binaries": [
"./debugAdapters/mono.linux-x86_64"
]
}
],

@sean-mcmanus
Copy link
Collaborator

Yes, if you want to download/install for a 64-bit Linux target. It successfully downloads for me when I try on Windows.

@evakili
Copy link

evakili commented Jun 6, 2017

The worst thing is for every new version of vscode-cpptools, C++ dependencies should be installed again, although they were not updated at all. That's too bad for Internet-disconnected or tortoisely-connected environments.

My workaround:
I have an Internet-connected system. Whenever a new version on vscode-cpptools is arrived, I update it on this system with C++ dependencies installed. Then I copy the extension folder from this system to the Internet-disconnected target. That works fine on Windows with no need to install or update C++ dependencies. I doesn't test this on Linux or Mac.

@AfroThundr3007730
Copy link

So it tries to download the dependencies again even if the version didn't change? That's not good. Maybe they should consider adding version checking to the dependency installation so it won't try to pull down unnecessary packages.

@bobbrow
Copy link
Member

bobbrow commented Jun 6, 2017

VS Code deletes the old extension's folder, which requires us to re-download dependencies even if they haven't changed since the last version.

@sean-mcmanus
Copy link
Collaborator

And usually, the language service dependencies change with every update. The 0.11.4 update was an exception.

@Jevyn
Copy link

Jevyn commented Jun 8, 2017

I've implemented something similar to what @ampatel21 mentions above. The only difference is I wrote a little bit of python (inherit from SimpleHttpServer) to mimic the way urls like:
https://go.microsoft.com/fwlink/?LinkID=826082 are handled on the Microsoft webserver. By using the content-disposition HTTP header you can send back the correct file name.

My steps are now the following:

  1. Download new cpptools. Open as a zip file and look at package.json.
  2. Download all dependencies and RENAME them as follows: _.zip (this is how my python knows how to map the number to the name. It also helps make sure only the exact same file will get installed).
  3. Move results to segregated network onto a network share.
  4. Developers who want to upgrade, install new version of cpptools.
  5. Developers edit packageManger.js to use http instead of https, and change urls to point at localhost.
  6. Developers run my SimpleHttpServer on port 80 on their local machine from network share and restart vscode. All dependencies get installed as if connect to internet.

Writing the python is an exercise left to the reader :)

@boormat
Copy link

boormat commented Sep 2, 2017

Using URL's with redirects in the package.json make this process much more complex. e.g. https://go.microsoft.com/fwlink/?linkid=847739

For other tools, such as Jenkins, I have used a NGINX to proxy_pass from the internet URL to a local mirror with the downloaded files. This works best if there is no redirects to mess with, just real URLs pointing at a versioned file names. As a bonus you can mirror documentation links aswell with the same approach, as the only configuration is to point at the proxy.

@bobbrow bobbrow added this to the December 2017 milestone Nov 16, 2017
@bobbrow
Copy link
Member

bobbrow commented Nov 16, 2017

We have a stretch goal to enable offline installs for our next update. I'm adding it to the December 2017 milestone.

@bobbrow bobbrow added the fixed Check the Milestone for the release in which the fix is or will be available. label Nov 27, 2017
@sean-mcmanus
Copy link
Collaborator

Fixed with 0.14.4. Offline installers are at https://github.com/Microsoft/vscode-cpptools/releases .

@github-actions github-actions bot locked and limited conversation to collaborators Oct 18, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Feature Request fixed Check the Milestone for the release in which the fix is or will be available. Language Service
Projects
None yet
Development

No branches or pull requests