Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Registry installation failure with 1.6 on Windows 10 #38691

Closed
PetrKryslUCSD opened this issue Dec 4, 2020 · 23 comments
Closed

Registry installation failure with 1.6 on Windows 10 #38691

PetrKryslUCSD opened this issue Dec 4, 2020 · 23 comments
Assignees
Milestone

Comments

@PetrKryslUCSD
Copy link

PetrKryslUCSD commented Dec 4, 2020

After the .Julia folder was completely removed:

               _                                                                              
   _       _ _(_)_     |  Documentation: https://docs.julialang.org                           
  (_)     | (_) (_)    |                                                                      
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.                               
  | | | | | | |/ _` |  |                                                                      
  | | |_| | | | (_| |  |  Version 1.6.0-DEV.1647 (2020-12-03)                                 
 _/ |\__'_|_|_|\__'_|  |  Commit 49b8e61a80* (0 days old master)                              
|__/                   |                                                                      
                                                                                              
julia> using Pkg; Pkg.activate("."); Pkg.instantiate();                                       
  Activating environment at `C:\Users\PetrKrysl\Documents\work\BoneShaker.jl\Project.toml`    
  Installing known registries into `C:\Users\PetrKrysl\.julia`                                
     Cloning registry from "https://github.com/JuliaRegistries/General.git"                   
ERROR: expected package `Arpack [7d9fca2a]` to be registered                                  
Stacktrace:                                                                                   
 [1] pkgerror(msg::String)                                                                    
   @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:52                                                                                       
 [2] check_registered(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})            
   @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:1130                                                                           
 [3] instantiate(ctx::Pkg.Types.Context; manifest::Nothing, update_registry::Bool, verbose::Bool, platform::Base.BinaryPlatforms.Platform, allow_autoprecomp::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                     
   @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1312                                                                                         
 [4] instantiate                                                                              
   @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1265 [inlined]                                                                                       
 [5] #instantiate#246                                                                         
   @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1261 [inlined]                                                                                       
 [6] instantiate()                                                                            
   @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1261                                                                                         
 [7] top-level scope                                                                          
   @ REPL[1]:1                                                                                
                                                                                              
caused by: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:SSL, TLS backend doesn't support certificate locations)                   
Stacktrace:                                                                                   
  [1] pkgerror(msg::String)                                                                   
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:52                                                                                      
  [2] clone(ctx::Pkg.Types.Context, url::String, source_path::String; header::String, credentials::Nothing, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                                                                             
    @ Pkg.GitTools C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\GitTools.jl:119                                                                               
  [3] (::Pkg.Types.var"#81#84"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec})(tmp::String)                                                                                             
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:985                                                                                     
  [4] mktempdir(fn::Pkg.Types.var"#81#84"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec}, 
parent::String; prefix::String)                                                               
    @ Base.Filesystem .\file.jl:729                                                           
  [5] mktempdir                                                                               
    @ .\file.jl:727 [inlined]                                                                 
  [6] clone_or_cp_registries(ctx::Pkg.Types.Context, regs::Vector{Pkg.Types.RegistrySpec}, depot::String)                                                                                   
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:964                                                                                     
  [7] clone_or_cp_registries                                                                  
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:957 
[inlined]                                                                                     
  [8] clone_default_registries(ctx::Pkg.Types.Context; only_if_empty::Bool)                   
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:860                                                                                     
  [9] clone_default_registries                                                                
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:848 
[inlined]                                                                                     
 [10] find_registered!(ctx::Pkg.Types.Context, names::Vector{String}, uuids::Vector{Base.UUID})                                                                                             
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:1228                                                                                    
 [11] find_registered!                                                                        
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:1189 [inlined]                                                                                    
 [12] check_registered(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})           
    @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:1127                                                                          
 [13] instantiate(ctx::Pkg.Types.Context; manifest::Nothing, update_registry::Bool, verbose::Bool, platform::Base.BinaryPlatforms.Platform, allow_autoprecomp::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                    
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1306                                                                                        
 [14] instantiate                                                                             
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1265 [inlined]                                                                                      
 [15] #instantiate#246                                                                        
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1261 [inlined]                                                                                      
 [16] instantiate()                                                                           
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1261                                                                                        
 [17] top-level scope                                                                         
    @ REPL[1]:1                                                                               
                                                                                              
caused by: GitError(Code:ERROR, Class:SSL, TLS backend doesn't support certificate locations) 
Stacktrace:                                                                                   
  [1] macro expansion                                                                         
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\error.jl:110 [inlined]                                                                                 
  [2] set_ssl_cert_locations(cert_loc::String)                                                
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:997                                                                                 
  [3] initialize()                                                                            
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:990                                                                                 
  [4] ensure_initialized                                                                      
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:970 [inlined]                                                                               
  [5] LibGit2.GitConfig(level::LibGit2.Consts.GIT_CONFIG)                                     
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\config.jl:50                                                                                   
  [6] CredentialPayload                                                                       
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\types.jl:1365 [inlined]                                                                                
  [7] #CredentialPayload#24                                                                     
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\types.jl:1375 [inlined]      
  [8] CredentialPayload                                                                                           
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\types.jl:1375 [inlined]      
  [9] clone(repo_url::SubString{String}, repo_path::String; branch::String, isbare::Bool, remote_cb::Ptr{Nothing}, credentials::LibGit2.CachedCredentials, callbacks::Dict{Symbol, Tuple{Ptr{Nothing}, Any}})                       
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:562       
 [10] clone(ctx::Pkg.Types.Context, url::String, source_path::String; header::String, credentials::Nothing, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                       
    @ Pkg.GitTools C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\GitTools.jl:109     
 [11] (::Pkg.Types.var"#81#84"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec})(tmp::String)                   
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:985           
 [12] mktempdir(fn::Pkg.Types.var"#81#84"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec}, parent::String; prefix::String)                                                                                                       
    @ Base.Filesystem .\file.jl:729                                                                               
 [13] mktempdir                                                                                                   
    @ .\file.jl:727 [inlined]                                                                                     
 [14] clone_or_cp_registries(ctx::Pkg.Types.Context, regs::Vector{Pkg.Types.RegistrySpec}, depot::String)         
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:964           
 [15] clone_or_cp_registries                                                                                      
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:957 [inlined]           
 [16] clone_default_registries(ctx::Pkg.Types.Context; only_if_empty::Bool)                                       
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:860           
 [17] clone_default_registries                                                                                    
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:848 [inlined]           
 [18] find_registered!(ctx::Pkg.Types.Context, names::Vector{String}, uuids::Vector{Base.UUID})                   
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:1228          
 [19] find_registered!                                                                                            
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:1189 [inlined]          
 [20] check_registered(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})                               
    @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:1127                                                                                                                  
 [21] instantiate(ctx::Pkg.Types.Context; manifest::Nothing, update_registry::Bool, verbose::Bool, platform::Base.BinaryPlatforms.Platform, allow_autoprecomp::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                                                                                
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1306              
 [22] instantiate                                                                                                 
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1265 [inlined]            
 [23] #instantiate#246                                                                                            
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1261 [inlined]            
 [24] instantiate()                                                                                               
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1261              
 [25] top-level scope                                                                                             
    @ REPL[1]:1                                                                                                   
                                                                                                                  
julia> 

This result for

julia> versioninfo()                                                                                              
Julia Version 1.6.0-DEV.1647                                                                                      
Commit 49b8e61a80* (2020-12-03 17:00 UTC)                                                                         
Platform Info:                                                                                                    
  OS: Windows (x86_64-w64-mingw32)                                                                                
  CPU: Intel(R) Core(TM) i7-8705G CPU @ 3.10GHz                                                                   
  WORD_SIZE: 64                                                                                                   
  LIBM: libopenlibm                                                                                               
  LLVM: libLLVM-11.0.0 (ORCJIT, skylake)                                                                          
Environment:                                                                                                      
  JULIA_NUM_THREADS = 4      

Curiously, if the registry is first built with 1.5.3 and then the environment activation/instantiation is started with 1.6, everything goes through smoothly. So I guess it is just the registry, not the packages.

@PetrKryslUCSD
Copy link
Author

Actually errors still occur in 1.6. The above information about smooth sailing after initial registry update was too optimistic.

Cloning into 'FinEtoolsVibInFluids.jl'...                                                                                                                                                               
remote: Enumerating objects: 367, done.                                                                                                                                                                 
remote: Counting objects: 100% (367/367), done.                                                                                                                                                         
remote: Compressing objects: 100% (226/226), done.                                                                                                                                                      
remote: Total 367 (delta 196), reused 292 (delta 125), pack-reused 0                                                                                                                                    
Receiving objects: 100% (367/367), 9.68 MiB | 4.62 MiB/s, done.                                                                                                                                         
Resolving deltas: 100% (196/196), done.                                                                                                                                                                 
Current folder: C:\Users\PetrKrysl\Documents\work\FinEtoolsTestAll.jl\tests\FinEtoolsVibInFluids.jl                                                                                                     
  Activating environment at `C:\Users\PetrKrysl\Documents\work\FinEtoolsTestAll.jl\tests\FinEtoolsVibInFluids.jl\Project.toml`                                                                          
   Installed NIfTI ─ v0.4.1                                                                                                                                                                             
Precompiling project...                                                                                                                                                                                 
  Progress [========================================>]  18/18                                                                                                                                           
18 dependencies successfully precompiled in 17 seconds (14 already precompiled)                                                                                                                         
    Updating registry at `C:\Users\PetrKrysl\.julia\registries\General`                                                                                                                                 
ERROR: LoadError: GitError(Code:ERROR, Class:SSL, TLS backend doesn't support certificate locations)                                                                                                    
Stacktrace:                                                                                                                                                                                             
  [1] macro expansion                                                                                                                                                                                   
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\error.jl:110 [inlined]                                                                                             
  [2] set_ssl_cert_locations(cert_loc::String)                                                                                                                                                          
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:997                                                                                             
  [3] initialize()                                                                                                                                                                                      
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:990                                                                                             
  [4] ensure_initialized                                                                                                                                                                                
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:970 [inlined]                                                                                           
  [5] LibGit2.GitRepo(path::String)                                                                                                                                                                     
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\repository.jl:9                                                                                            
  [6] #ensure_clone#2                                                                                                                                                                                   
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\GitTools.jl:70 [inlined]                                                                                               
  [7] handle_repo_add!(ctx::Pkg.Types.Context, pkg::Pkg.Types.PackageSpec)                                                                                                                              
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:580                                                                                                 
  [8] up_load_versions!(ctx::Pkg.Types.Context, pkg::Pkg.Types.PackageSpec, entry::Pkg.Types.PackageEntry, level::Pkg.Types.UpgradeLevel)                                                               
    @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:1251                                                                                      
  [9] up(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, level::Pkg.Types.UpgradeLevel)                                                                                                    
    @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:1286                                                                                      
 [10] up(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; level::Pkg.Types.UpgradeLevel, mode::Pkg.Types.PackageMode, update_registry::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, 
Tuple{}, NamedTuple{(), Tuple{}}})                                                                                                                                                                      
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:254                                                                                                     
 [11] up                                                                                                                                                                                                
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:230 [inlined]                                                                                                   
 [12] up(pkgs::Vector{Pkg.Types.PackageSpec}; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                                                         
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:73                                                                                                      
 [13] up                                                                                                                                                                                                
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:72 [inlined]                                                                                                    
 [14] up(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                                                                                                                                                           
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:89                                                                                                      
 [15] up                                                                                                                                                                                                
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:81 [inlined]                                                                                                    
 [16] top-level scope                                                                                                                                                                                   
    @ ~\Documents\work\FinEtoolsTestAll.jl\testall.jl:44                                                                                                                                                
 [17] include(fname::String)                                                                                                                                                                            
    @ Base.MainInclude .\client.jl:444                                                                                                                                                                  
 [18] top-level scope                                                                                                                                                                                   
    @ REPL[1]:1                                                                                                                                                                                         
in expression starting at C:\Users\PetrKrysl\Documents\work\FinEtoolsTestAll.jl\testall.jl:38  

@fredrikekre
Copy link
Member

caused by: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:SSL, TLS backend doesn't support certificate locations)

^ seems related to your recent work @StefanKarpinski

@StefanKarpinski
Copy link
Member

StefanKarpinski commented Dec 4, 2020

@PetrKryslUCSD, can you try the following in a REPL session post what Julia outputs?

import NetworkOptions
NetworkOptions.ca_roots()
import Downloads
Downloads.get_ca_roots()
Downloads.Curl.SSL_VERSION

@PetrKryslUCSD
Copy link
Author

PetrKryslUCSD commented Dec 4, 2020

julia> import NetworkOptions                                                                                                                          
                                                                                                                                                      
julia> NetworkOptions.ca_roots()                                                                                                                      
"C:\\Users\\PetrKrysl\\Documents\\SublimeText4_portable\\Lib\\python3\\certifi\\cacert.pem"                                                           
                                                                                                                                                      
julia> import Downloads                                                                                                                               
                                                                                                                                                      
julia> Downloads.get_ca_roots()                                                                                                                       
"C:\\Users\\PetrKrysl\\Documents\\SublimeText4_portable\\Lib\\python3\\certifi\\cacert.pem"                                                           
                                                                                                                                                      
julia> Downloads.Curl.SSL_VERSION
"Schannel"                                                                                                                                            
                                                                                                                                                      

@StefanKarpinski
Copy link
Member

Do you have SSL_CERT_DIR or SSL_CERT_FILE set in your environment?

@PetrKryslUCSD
Copy link
Author

Yes.

julia> ENV["SSL_CERT_FILE"]                                                                                                                           
"C:\\Users\\PetrKrysl\\Documents\\SublimeText4_portable\\Lib\\python3\\certifi\\cacert.pem"  

@PetrKryslUCSD
Copy link
Author

When Julia 1.6 is run outside of the SublimeText editor (which has its own Python installation), everything is fine.
Well, except for some MbedTLS_jll errors.

@PetrKryslUCSD
Copy link
Author

PetrKryslUCSD commented Dec 4, 2020

Sublime Text defines the variable SSL_CERT_FILE to access package control.

@PetrKryslUCSD
Copy link
Author

PetrKryslUCSD commented Dec 4, 2020

Discussion of the Open SSL as implemented in ST:
https://discord.com/channels/280102180189634562/280157083356233728

@DilumAluthge
Copy link
Member

If this is specific to when you run Julia in Sublime Text, maybe you can do this in your Julia startup file:

delete!(ENV, "SSL_CERT_FILE")

@PetrKryslUCSD
Copy link
Author

Yes, I set "env": {"JULIA_NUM_THREADS":"4", "SSL_CERT_FILE": ""}, and that fixed the problem for me. However, there still may be an issue with Julia use of the certificate in general.

@StefanKarpinski
Copy link
Member

That explains why this is happening. We switched to using the native SSL/TLS implementation on Windows and macOS because that's the least friction approach on those platforms and most likely to be correct and secure (the OS should keep these up-to-date and there's no way to do certificate revocation lists otherwise). However, if one of those variables or JULIA_SSL_CA_ROOTS_PATH are set then we use that as the CA root certificate path, but it seems that the native TLS system on Windows at least doesn't know how to use a CA root cert file, hence the error.

@PetrKryslUCSD
Copy link
Author

PetrKryslUCSD commented Dec 4, 2020

OK. I guess that solves it then for me. But I believe other Python installations (conda?) will also try to provide that certification file. in that case you might not be seeing this error for the last time.

@StefanKarpinski
Copy link
Member

One obvious fix here is that if JULIA_SSL_CA_ROOTS_PATH is set to the empty string, that should cancel having SSL_CERT_FILE or SSL_CERT_DIR set, which would allow setting that to prevent Julia from getting this setting. However, I'm wondering what the right course of action in the current situation is:

  1. Ignore SSL_CERT_FILE if we know that we cannot change the cert store for our TLS/SSL engine and just use the system certificate store anyway.
  2. Error out like we are now but try to give a clearer error message indicating how to fix the issue.

On the one hand, the first option is more like to "just work", which is what we would usually prefer. On the other hand, what if you set SSL_CERT_FILE because you know that the system CA roots are jacked up in some way (best case they're broken, worst case, they're insecure). In that case ignoring SSL_CERT_FILE would be a potential security hazard.

Another option would be to ignore SSL_CERT_FILE and SSL_CERT_DIR in general and only pay attention to JULIA_SSL_CA_ROOTS_PATH. This would make it less likely that someone would accidentally have this set and have Julia pick it up even though their Julia cannot actually use the value. Then someone might end up having to set both SSL_CERT_FILE and JULIA_SSL_CA_ROOTS_PATH to the same value on some systems.

Complicating all of this is that these NetworkOptions variables are meant to be used by all kinds of different libraries that do TLS transport. Even if libgit2 happens to be configured to use the system TLS and therefore can't use a CA root certs file, some other consumer like mbedTLS might be perfectly able to do so and isn't able to use the system CA roots.

@StefanKarpinski
Copy link
Member

Any thoughts?

@DilumAluthge
Copy link
Member

2. Error out like we are now but try to give a clearer error message indicating how to fix the issue.

Probably this is the easiest course of action.

@DilumAluthge
Copy link
Member

Another option would be to ignore SSL_CERT_FILE and SSL_CERT_DIR in general and only pay attention to JULIA_SSL_CA_ROOTS_PATH. This would make it less likely that someone would accidentally have this set and have Julia pick it up even though their Julia cannot actually use the value. Then someone might end up having to set both SSL_CERT_FILE and JULIA_SSL_CA_ROOTS_PATH to the same value on some systems.

I also like this option.

@DilumAluthge
Copy link
Member

But probably I would lean towards the "better error message" approach.

After all, if we are using the system TLS, we should probably also use the system-wide TLS-related environment variables.

@DilumAluthge
Copy link
Member

On the other hand, what if you set SSL_CERT_FILE because you know that the system CA roots are jacked up in some way (best case they're broken, worst case, they're insecure). In that case ignoring SSL_CERT_FILE would be a potential security hazard.

This to me is the most convincing point. If we're going to rely on the system's implementation of SSL/TLS, we should assume that the user knows best when it comes to the whether the system CA roots are trustworthy.

So I would say keep the current behavior of having an error, but just have a nice message that says "I notice that you have set the SSL_CERT_FILE environment variable, which may have caused this error. Unsetting that environment variable may resolve this error"

@StefanKarpinski
Copy link
Member

Ok. I'll do that then. But I think I should also allow setting JULIA_SSL_CA_ROOTS_PATH="" to allow explicitly opting to ignore SSL_CERT_FILE or SSL_CERT_DIR would be good as well. That way the user can either choose to unset SSL_CERT_{FILE,DIR} or set JULIA_SSL_CA_ROOTS_PATH="". The error condition would be that we have a non-empty path from the environment but are using a system SSL engine that cannot handle that (Windows at least).

@DilumAluthge
Copy link
Member

But I think I should also allow setting JULIA_SSL_CA_ROOTS_PATH="" to allow explicitly opting to ignore SSL_CERT_FILE or SSL_CERT_DIR would be good as well.

I think that makes sense.

@StefanKarpinski
Copy link
Member

On master you can now set JULIA_SSL_CA_ROOTS_PATH="" in the environment to avoid this issue. I'm still working on improving the error message for the situation where this gets set but the TLS library cannot set a CA roots file or directory; functionally, however, the current state on master is probably as good as this gets.

@StefanKarpinski
Copy link
Member

The error message is now improved as well. Please give it a try if you would.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants