Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

what's SSLVibeStream.close? #77

Closed
ZHANITEST opened this issue Jun 23, 2018 · 25 comments
Closed

what's SSLVibeStream.close? #77

ZHANITEST opened this issue Jun 23, 2018 · 25 comments

Comments

@ZHANITEST
Copy link

ZHANITEST commented Jun 23, 2018

hello, @ikod 😄
when i using 'requests', i got some errors. here my code:

import std.stdio;
import requests;

void main()
{
	auto rq = Request();
	auto content = rq.get("https://google.com");
}

and got this problem:

PS C:\Users\zhanitest\Desktop\postboy> dub run
Performing "debug" build using C:\D\dmd2\windows\bin\dmd.exe for x86.
taggedalgebraic 0.10.11: target for configuration "library" is up to date.
eventcore 0.8.35: target for configuration "winapi-optlink" is up to date.
stdx-allocator 2.77.2: target for configuration "library" is up to date.
vibe-core 1.4.0: target for configuration "winapi-optlink" is up to date.
vibe-d:utils 0.8.4: target for configuration "library" is up to date.
vibe-d:data 0.8.4: target for configuration "library" is up to date.
mir-linux-kernel 1.0.0: target for configuration "library" is up to date.
vibe-d:crypto 0.8.4: target for configuration "library" is up to date.
diet-ng 1.5.0: target for configuration "library" is up to date.
vibe-d:stream 0.8.4: target for configuration "library" is up to date.
vibe-d:textfilter 0.8.4: target for configuration "library" is up to date.
vibe-d:inet 0.8.4: target for configuration "library" is up to date.
vibe-d:tls 0.8.4: target for configuration "openssl" is up to date.
vibe-d:http 0.8.4: target for configuration "library" is up to date.
vibe-d:mail 0.8.4: target for configuration "library" is up to date.
vibe-d:mongodb 0.8.4: target for configuration "library" is up to date.
vibe-d:redis 0.8.4: target for configuration "library" is up to date.
vibe-d:web 0.8.4: target for configuration "library" is up to date.
vibe-d 0.8.4: target for configuration "vibe-core" is up to date.
requests 0.8.0: target for configuration "vibed" is up to date.
postboy ~master: building configuration "application"...
Linking...
To force a rebuild of up-to-date targets, run again with --force.
Copying files for vibe-d:tls...
Running .\postboy.exe
2018-06-23T11:34:51.838:..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\ssl_adapter.d:_staticCtor36:92 warning: failed to load libssl - first access over https will fail

object.Error@(0): Access Violation
----------------
0x00420D04 in @trusted void requests.streams.SSLVibeStream.close() at C:\Users\zhanitest\Desktop\postboy\..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\streams.d(1271)
0x00415963 in void requests.http.HTTPRequest.__dtor() at C:\Users\zhanitest\Desktop\postboy\..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\http.d(381)
0x004123C4 in void requests.request.Request.__fieldDtor() at C:\Users\zhanitest\Desktop\postboy\..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\request.d(20)
0x00403518 in _Dmain at C:\Users\zhanitest\Desktop\postboy\source\app.d(7)
0x004E26BF in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll().__lambda1()
0x004E2641 in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll()
0x004E24DC in _d_run_main
0x00411F0C in main at C:\Users\zhanitest\Desktop\postboy\source\app.d(7)
0x00549C8D in mainCRTStartup
0x759D8484 in BaseThreadInitThunk
0x77312FEA in RtlValidSecurityDescriptor
0x77312FBA in RtlValidSecurityDescriptor
=== Bypassed ===
requests.streams.ConnectError@..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\streams.d(1188): With vibe.d you have to call setCaCert() before verify server certificate.
----------------
0x00420856 in void requests.streams.SSLVibeStream.connectSSL(immutable(char)[]) at C:\Users\zhanitest\Desktop\postboy\..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\streams.d(1190)
0x00420B3A in requests.streams.NetworkStream requests.streams.SSLVibeStream.connect(immutable(char)[], ushort, core.time.Duration) at C:\Users\zhanitest\Desktop\postboy\..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\streams.d(1233)
0x00417C82 in void requests.http.HTTPRequest.setupConnection() at C:\Users\zhanitest\Desktop\postboy\..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\http.d(857)
0x0040B91C in _D8requests4http11HTTPRequest__T4execVAyaa3_474554ZQtMFQrAS3std8typecons__T5TupleTQBsVQBwa3_6b655AC7EB660644949FC46DA0F67BD6A7CD at C:\Users\zhanitest\Desktop\postboy\..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\http.d(1357)
0x0040B68A in requests.http.HTTPResponse requests.http.HTTPRequest.get!().get() at C:\Users\zhanitest\Desktop\postboy\..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\http.d(1512)
0x0040B62E in requests.base.Response requests.request.Request.get!().get(immutable(char)[]) at C:\Users\zhanitest\Desktop\postboy\..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\request.d(291)
0x004034DF in _Dmain at C:\Users\zhanitest\Desktop\postboy\source\app.d(7)
0x004E26BF in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll().__lambda1()
0x004E2641 in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll()
0x004E24DC in _d_run_main
0x00411F0C in main at C:\Users\zhanitest\Desktop\postboy\source\app.d(7)
0x00549C8D in mainCRTStartup
0x759D8484 in BaseThreadInitThunk
0x77312FEA in RtlValidSecurityDescriptor
0x77312FBA in RtlValidSecurityDescriptor
object.Error@(0): Access Violation
----------------
0x00420D04 in @trusted void requests.streams.SSLVibeStream.close() at C:\Users\zhanitest\Desktop\postboy\..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\streams.d(1271)
0x00415963 in void requests.http.HTTPRequest.__dtor() at C:\Users\zhanitest\Desktop\postboy\..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\http.d(381)
0x004123C4 in void requests.request.Request.__fieldDtor() at C:\Users\zhanitest\Desktop\postboy\..\..\AppData\Local\dub\packages\requests-0.8.0\requests\source\requests\request.d(20)
0x00403518 in _Dmain at C:\Users\zhanitest\Desktop\postboy\source\app.d(7)
0x004E26BF in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll().__lambda1()
0x004E2641 in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll()
0x004E24DC in _d_run_main
0x00411F0C in main at C:\Users\zhanitest\Desktop\postboy\source\app.d(7)
0x00549C8D in mainCRTStartup
0x759D8484 in BaseThreadInitThunk
0x77312FEA in RtlValidSecurityDescriptor
0x77312FBA in RtlValidSecurityDescriptor
=== ~Bypassed ===
Program exited with code 1
PS C:\Users\zhanitest\Desktop\postboy>

first, i copied libssl.dll and libssl32.dll on C:\Windows\system32, SysWow64 and my dub project directory.
but same problem..

how can i solve it?

thanks

@ikod
Copy link
Owner

ikod commented Jun 23, 2018

hello @ZHANITEST

Can you show me your dub.json?

Thanks!

@ZHANITEST
Copy link
Author

here:

{
	"name": "postboy",
	"authors": [
		"zhanitest"
	],
	"dependencies": {
		"requests": "~>0.8.0"
	},
	"subConfigurations": {
		"requests": "vibed"
	},
	"description": "A minimal D application.",
	"copyright": "Copyright © 2018, zhanitest",
	"license": "proprietary"
}

well... i tried re-build with same dub.json and app.d on my PC when i opened this issue.
but it was working. i got a problem on Notebook.

  • Note: Windows 10 x64 / Intel ATOM Processor / got a problem
  • PC: Windows 10 x64 / AMD AM3 Processor / np, working pretty

hmm... or libssl.dll problem?

@ikod
Copy link
Owner

ikod commented Jun 23, 2018

Hello @ZHANITEST

Probably just problem with libssl location. Try to put it into the any system library path. I'm downloading win 10 image to reproduce the problem, but I rememver that I just installed openssl libraries and allowed to install them in system path.

@ikod
Copy link
Owner

ikod commented Jun 27, 2018

Hello, @ZHANITEST

Sorry for the delay.
I was able to reproduce problem using MSEdge VirtualBox Images.
What I did:

  1. created new clean windows 10
  2. installed dmd from dlang.org.
  3. created new dub.json with requests dependency and with your code in app.d
  4. dub run compiled code but failed to execute with exactly your messages.

How to solve:

  1. Install https://slproweb.com/download/Win32OpenSSL_Light-1_0_2o.exe and do not forget to enable copy in system directiories
    image
  2. use requests v0.8.1
  3. If you do not need to verify peer in https then you have to add line rq.sslSetVerifyPeer = false; othervise you will receive exception about failed verification.
  4. If you decide to use peer verification, then you have to add CA bundle to your system (see https://superuser.com/questions/442793/why-cant-curl-properly-verify-a-certificate-on-windows for possible solutions) and add line like rq.sslSetCaCert("C:\\Users\\IEUser\\Downloads\\cacert.pem"); to your code.

I'm closing this issue, but please feel free to reopen it and ask any related questions here.

@ikod ikod closed this as completed Jun 27, 2018
@ZHANITEST
Copy link
Author

ZHANITEST commented Jul 2, 2018

oh thank you for reply! Almost solve it. I will 😄

well, i got another problem about cacert.pem when i using it.

same dub.json but "requests": "~>0.8.2".

here a app.d code:

import std.stdio;
import requests;

void main()
{
	auto rq = Request();
	rq.sslSetCaCert("C:\\D\\cacert.pem");	
	//auto content = rq.get("https://google.com"); -> DONE: 200
	//auto content = rq.get("https://marumaru.in"); -> ERROR!
}

Some sites got this error when i crawling it:

C:\Users\ZHANI\Desktop\postboy> dub run --compiler=dmd
Performing "debug" build using dmd for x86.
(... BUILD MESSAGE FOR LIBRARY ...)
Copying files for vibe-d:tls...
Running .\postboy.exe
2018-07-02T22:53:05.944:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\ssl_adapter.d:_staticCtor36:95 warning: failed to load libssl - first access over https will fail

object.Exception@..\..\AppData\Roaming\dub\packages\vibe-d-0.8.4\vibe-d\tls\vibe\stream\openssl.d(381): Peeking TLS stream: error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac (336130329)
----------------
0x00476C61 in @safe int vibe.stream.openssl.OpenSSLStream.enforceSSL(int, immutable(char)[]) at C:\Users\ZHANI\Desktop\posto\..\..\AppData\Roaming\dub\packages\vibe-d-0.8.4\vibe-d\tls\vibe\stream\openssl.d(382)
0x00476888 in @safe int vibe.stream.openssl.OpenSSLStream.checkSSLRet(int, immutable(char)[]) at C:\Users\ZHANI\Desktop\posto\..\..\AppData\Roaming\dub\packages\vibe-d-0.8.4\vibe-d\tls\vibe\stream\openssl.d(339)
0x004762CB in @property @safe ulong vibe.stream.openssl.OpenSSLStream.leastSize() at C:\Users\ZHANI\Desktop\posto\..\..\AppData\Roaming\dub\packages\vibe-d-0.8.4\vibe-d\tls\vibe\stream\openssl.d(247)
0x0047627B in @property @safe bool vibe.stream.openssl.OpenSSLStream.empty() at C:\Users\ZHANI\Desktop\posto\..\..\AppData\Roaming\dub\packages\vibe-d-0.8.4\vibe-d\tls\vibe\stream\openssl.d(239)
0x004215C3 in int requests.streams.SSLVibeStream.receive(void[]) at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\streams.d(1263)
0x00418E66 in void requests.http.HTTPRequest.receiveResponse() at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d(1006)
0x0040BBB0 in _D8requests4http11HTTPRequest__T4execVAyaa3_474554ZQtMFQrAS3std8typecons__T5TupleTQBsVQBwa3_6b655AC7EB660644949FC46DA0F67BD6A7CD at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d(1397)
0x0040B6B6 in requests.http.HTTPResponse requests.http.HTTPRequest.get!().get() at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d(1512)
0x0040B65A in requests.base.Response requests.request.Request.get!().get(immutable(char)[]) at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\request.d(291)
0x00403502 in _Dmain at C:\Users\ZHANI\Desktop\postboy\source\app.d(9)
0x004E305F in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll().__lambda1()
0x004E2FE1 in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll()
0x004E2E7C in _d_run_main
0x00412AD0 in main at C:\Users\ZHANI\Desktop\postboy\source\app.d(7)
0x0054A62D in mainCRTStartup
0x73988484 in BaseThreadInitThunk
0x770F2FEA in RtlValidSecurityDescriptor
0x770F2FBA in RtlValidSecurityDescriptor
Program exited with code 1

of course, i copy the cacert.pem on C:\D\.

i downloaded the pem file in here: https://curl.haxx.se/docs/caextract.html

is it my mistake here?

regards,

@ikod
Copy link
Owner

ikod commented Jul 2, 2018

2018-07-02T22:53:05.944:....\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\ssl_adapter.d:_staticCtor36:95 warning: failed to load libssl - first access over https will fail

this means that something went wrong.

Can you,please, show me complete output for https://google.com ?

@ZHANITEST
Copy link
Author

ZHANITEST commented Jul 2, 2018

Here:

app.d

import std.stdio;
import requests;

void main()
{
	auto rq = Request();
	rq.sslSetCaCert("C:\\D\\cacert.pem");	
	auto content = rq.get("https://google.com");
	auto f = File("temp.html", "w");
	f.write(content.responseBody);
	f.close();
}

same message about fail to load libssl 🤔

temp.html

<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="ko"><head><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop="image"><title>Google</title><script nonce="PhstcLKTQDUWyLvTKTmQxA==">(function(){window.google={kEI:'jDM6W--wA4qY8QW8hKaQCw',kEXPI:'0,201837,1151910,57,1958,582,435,2,278,267,76,367,388,512,740,297,309,52,64,204,133,2339594,266,161,32,329294,1294,12383,2349,2506,32692,2074,13173,867,316,1264,7,7344,3232,5281,621,1333,7094,2192,365,528,24,664,2102,113,2201,3191,1067,44,243,1088,128,132,3741,1366,444,131,1119,2,578,728,310,2121,1105,257,1343,369,1376,505,277,453,377,1240,479,609,685,8,275,1295,222,551,94,1138,1020,279,2,2537,288,758,878,392,133,22,42,96,42,424,2,154,272,233,441,296,904,193,3,540,395,5,312,79,29,272,614,283,509,1587,296,447,897,679,189,11,43,25,10,52,110,334,123,7,326,24,767,122,105,386,8,806,278,7,28,463,620,29,673,32,3,186,4,4,4,4,101,384,458,62,57,512,807,2,133,74,127,331,140,194,2327641,3685997,2554,8797786,4,1572,549,332,445,1,2,1,1,77,1,1,900,207,1,1,1,1,1,371,9,304,1,8,1,2,1,1,1523,8,7,62,1,37,28,1,1,1,163,46,5,100,4,6,1',authuser:0,kscs:'c9c918f0_jDM6W--wA4qY8QW8hKaQCw',kGL:'KR'};google.kHL='ko';})();google.time=function(){return(new Date).getTime()};(function(){google.lc=[];google.li=0;google.getEI=function(a){for(var b;a&&(!a.getAttribute||!(b=a.getAttribute("eid")));)a=a.parentNode;return b||google.kEI};google.getLEI=function(a){for(var b=null;a&&(!a.getAttribute||!(b=a.getAttribute("leid")));)a=a.parentNode;return b};google.https=function(){return"https:"==window.location.protocol};google.ml=function(){return null};google.wl=function(a,b){try{google.ml(Error(a),!1,b)}catch(d){}};google.log=function(a,b,d,c,g){if(a=google.logUrl(a,b,d,c,g)){b=new Image;var e=google.lc,f=google.li;e[f]=b;b.onerror=b.onload=b.onabort=function(){delete e[f]};google.vel&&google.vel.lu&&google.vel.lu(a);b.src=a;google.li=f+1}};google.logUrl=function(a,b,d,c,g){var e="",f=google.ls||"";d||-1!=b.search("&ei=")||(e="&ei="+google.getEI(c),-1==b.search("&lei=")&&(c=google.getLEI(c))&&(e+="&lei="+c));c="";!d&&google.cshid&&-1==b.search("&cshid=")&&"slh"!=a&&(c="&cshid="+google.cshid);a=d||"/"+(g||"gen_204")+"?atyp=i&ct="+a+"&cad="+b+e+f+"&zx="+google.time()+c;/^http:/i.test(a)&&google.https()&&(google.ml(Error("a"),!1,{src:a,glmm:1}),a="");return a};}).call(this);(function(){google.y={};google.x=function(a,b){if(a)var c=a.id;else{do c=Math.random();while(google.y[c])}google.y[c]=[a,b];return!1};google.lm=[];google.plm=function(a){google.lm.push.apply(google.lm,a)};google.lq=[];google.load=function(a,b,c){google.lq.push([[a],b,c])};google.loadAll=function(a,b){google.lq.push([a,b])};}).call(this);google.f={};</script><script nonce="PhstcLKTQDUWyLvTKTmQxA==">var a=window.location,b=a.href.indexOf("#");if(0<=b){var c=a.href.substring(b+1);/(^|&)q=/.test(c)&&-1==c.indexOf("#")&&a.replace("/search?"+c.replace(/(^|&)fp=[^&]*/g,"")+"&cad=h")};</script><style>#gbar,#guser{font-size:13px;padding-top:1px !important;}#gbar{height:22px}#guser{padding-bottom:7px !important;text-align:right}.gbh,.gbd{border-top:1px solid #c9d7f1;font-size:1px}.gbh{height:0;position:absolute;top:24px;width:100%}@media all{.gb1{height:22px;margin-right:.5em;vertical-align:top}#gbar{float:left}}a.gb1,a.gb4{text-decoration:underline !important}a.gb1,a.gb4{color:#00c !important}.gbi .gb4{color:#dd8e27 !important}.gbf .gb4{color:#900 !important}
</style><style>body,td,a,p,.h{font-family:&#44404;&#47548;,&#46027;&#50880;,arial,sans-serif}.ko{font-size:9pt}body{margin:0;overflow-y:scroll}#gog{padding:3px 8px 0}td{line-height:.8em}.gac_m td{line-height:17px}form{margin-bottom:20px}.h{color:#36c}.q{color:#00c}.ts td{padding:0}.ts{border-collapse:collapse}em{font-weight:bold;font-style:normal}.lst{height:25px;width:496px}.gsfi,.lst{font:18px arial,sans-serif}.gsfs{font:17px arial,sans-serif}.ds{display:inline-box;display:inline-block;margin:3px 0 4px;margin-left:4px}input{font-family:inherit}a.gb1,a.gb2,a.gb3,a.gb4{color:#11c !important}body{background:#fff;color:black}a{color:#11c;text-decoration:none}a:hover,a:active{text-decoration:underline}.fl a{color:#36c}a:visited{color:#551a8b}a.gb1,a.gb4{text-decoration:underline}a.gb3:hover{text-decoration:none}#ghead a.gb2:hover{color:#fff !important}.sblc{padding-top:5px}.sblc a{display:block;margin:2px 0;margin-left:13px;font-size:11px}.lsbb{background:#eee;border:solid 1px;border-color:#ccc #999 #999 #ccc;height:30px}.lsbb{display:block}.ftl,#fll a{display:inline-block;margin:0 12px}.lsb{background:url(/images/nav_logo229.png) 0 -261px repeat-x;border:none;color:#000;cursor:pointer;height:30px;margin:0;outline:0;font:15px arial,sans-serif;vertical-align:top}.lsb:active{background:#ccc}.lst:focus{outline:none}.tiah{width:458px}</style><script nonce="PhstcLKTQDUWyLvTKTmQxA=="></script><link href="/images/branding/product/ico/googleg_lodp.ico" rel="shortcut icon"></head><body bgcolor="#fff"><script nonce="PhstcLKTQDUWyLvTKTmQxA==">(function(){var src='/images/nav_logo229.png';var iesg=false;document.body.onload = function(){window.n && window.n();if (document.images){new Image().src=src;}
if (!iesg){document.f&&document.f.q.focus();document.gbqf&&document.gbqf.q.focus();}
}
})();</script><div id="mngb"> <div id=gbar><nobr><b class=gb1>&#44160;&#49353;</b> <a class=gb1 href="https://www.google.co.kr/imghp?hl=ko&tab=wi">&#51060;&#48120;&#51648;</a> <a class=gb1 href="https://maps.google.co.kr/maps?hl=ko&tab=wl">&#51648;&#46020;</a> <a class=gb1 href="https://play.google.com/?hl=ko&tab=w8">Play</a> <a class=gb1 href="https://www.youtube.com/?gl=KR&tab=w1">YouTube</a> <a class=gb1 href="https://news.google.co.kr/nwshp?hl=ko&tab=wn">&#45684;&#49828;</a> <a class=gb1 href="https://mail.google.com/mail/?tab=wm">Gmail</a> <a class=gb1 href="https://drive.google.com/?tab=wo">&#46300;&#46972;&#51060;&#48652;</a> <a class=gb1 style="text-decoration:none" href="https://www.google.co.kr/intl/ko/options/"><u>&#45908;&#48372;&#44592;</u> &raquo;</a></nobr></div><div id=guser width=100%><nobr><span id=gbn class=gbi></span><span id=gbf class=gbf></span><span id=gbe></span><a href="http://www.google.co.kr/history/optout?hl=ko" class=gb4>&#50937; &#44592;&#47197;</a> | <a  href="/preferences?hl=ko" class=gb4>&#49444;&#51221;</a> | <a target=_top id=gb_70 href="https://accounts.google.com/ServiceLogin?hl=ko&passive=true&continue=https://www.google.com/" class=gb4>&#47196;&#44536;&#51064;</a></nobr></div><div class=gbh style=left:0></div><div class=gbh style=right:0></div> </div><center><br clear="all" id="lgpd"><div id="lga"><img alt="Google" height="92" src="/images/branding/googlelogo/1x/googlelogo_white_background_color_272x92dp.png" style="padding:28px 0 14px" width="272" id="hplogo" onload="window.lol&&lol()"><br><br></div><form action="/search" name="f"><table cellpadding="0" cellspacing="0"><tr valign="top"><td width="25%">&nbsp;</td><td align="center" nowrap=""><input name="ie" value="ISO-8859-1" type="hidden"><input value="ko" name="hl" type="hidden"><input name="source" type="hidden" value="hp"><input name="biw" type="hidden"><input name="bih" type="hidden"><div class="ds" style="height:32px;margin:4px 0"><div style="position:relative;zoom:1"><input style="color:#000;margin:0;padding:5px 8px 0 6px;vertical-align:top;padding-right:38px" autocomplete="off" class="lst tiah" value="" title="Google &#44160;&#49353;" maxlength="2048" name="q" size="57"><img src="/textinputassistant/tia.png" style="position:absolute;cursor:pointer;right:5px;top:4px;z-index:300" onclick="(function(){var src='/textinputassistant/11/ko_tia.js';var s=document.createElement('script');s.src=src;(document.getElementById('xjsc')||document.body).appendChild(s);})();" alt="" height="23" width="27"></div></div><br style="line-height:0"><span class="ds"><span class="lsbb"><input class="lsb" value="Google &#44160;&#49353;" name="btnG" type="submit"></span></span><span class="ds"><span class="lsbb"><input class="lsb" value="I&#8217;m Feeling Lucky" name="btnI" onclick="if(this.form.q.value)this.checked=1; else top.location='/doodles/'" type="submit"></span></span></td><td class="fl sblc" align="left" nowrap="" width="25%"><a href="/advanced_search?hl=ko&amp;authuser=0">&#44256;&#44553;&#44160;&#49353;</a><a href="/language_tools?hl=ko&amp;authuser=0">&#50616;&#50612;&#46020;&#44396;</a></td></tr></table><input id="gbv" name="gbv" type="hidden" value="1"></form><div id="gac_scont"></div><div style="font-size:83%;min-height:3.5em"><br></div><span id="footer"><div style="font-size:10pt"><div style="margin:19px auto;text-align:center" id="fll"><a href="/intl/ko/ads/">&#44305;&#44256; &#54532;&#47196;&#44536;&#47016;</a><a href="http://www.google.co.kr/intl/ko/services/">&#48708;&#51592;&#45768;&#49828; &#49556;&#47336;&#49496;</a><a href="https://plus.google.com/102197601262446632410" rel="publisher">+Google</a><a href="/intl/ko/about.html">Google &#51221;&#48372;</a><a href="https://www.google.com/setprefdomain?prefdom=KR&amp;prev=https://www.google.co.kr/&amp;sig=__HvGkW_sT-DSK348qBhUB9Zo159g%3D">Google.co.kr</a></div></div><p style="color:#767676;font-size:8pt">&copy; 2018 - <a href="/intl/ko/policies/privacy/">&#44060;&#51064;&#51221;&#48372;&#52376;&#47532;&#48169;&#52840;</a> - <a href="/intl/ko/policies/terms/">&#50557;&#44288;</a></p></span></center><script nonce="PhstcLKTQDUWyLvTKTmQxA==">(function(){window.google.cdo={height:0,width:0};(function(){var a=window.innerWidth,b=window.innerHeight;if(!a||!b){var c=window.document,d="CSS1Compat"==c.compatMode?c.documentElement:c.body;a=d.clientWidth;b=d.clientHeight}a&&b&&(a!=google.cdo.width||b!=google.cdo.height)&&google.log("","","/client_204?&atyp=i&biw="+a+"&bih="+b+"&ei="+google.kEI);}).call(this);})();</script><div id="xjsd"></div><div id="xjsi"><script nonce="PhstcLKTQDUWyLvTKTmQxA==">(function(){function c(b){window.setTimeout(function(){var a=document.createElement("script");a.src=b;google.timers&&google.timers.load.t&&google.tick&&google.tick("load",{gen204:"xjsls",clearcut:31});document.getElementById("xjsd").appendChild(a)},0)}google.dljp=function(b,a){google.xjsu=b;c(a)};google.dlj=c;}).call(this);if(!google.xjs){window._=window._||{};window._DumpException=window._._DumpException=function(e){throw e};window._F_installCss=window._._F_installCss=function(c){};google.dljp('/xjs/_/js/k\x3dxjs.hp.en.1PpFRaDyuas.O/m\x3dsb_he,d/am\x3dlMLY/rt\x3dj/d\x3d1/rs\x3dACT90oHnRSd4DdAfoAxO9BbcpBldNRVOuA','/xjs/_/js/k\x3dxjs.hp.en.1PpFRaDyuas.O/m\x3dsb_he,d/am\x3dlMLY/rt\x3dj/d\x3d1/rs\x3dACT90oHnRSd4DdAfoAxO9BbcpBldNRVOuA');google.xjs=1;}google.pmc={"sb_he":{"agen":true,"cgen":true,"client":"heirloom-hp","dh":true,"dhqt":true,"ds":"","ffql":"ko","fl":true,"host":"google.com","isbh":28,"jsonp":true,"msgs":{"cibl":"&#44160;&#49353;&#50612; &#51648;&#50864;&#44592;","dym":"&#51060;&#44163;&#51012; &#52286;&#51004;&#49512;&#45208;&#50836;?","lcky":"I&#8217;m Feeling Lucky","lml":"&#51088;&#49464;&#55176; &#50508;&#50500;&#48372;&#44592;","oskt":"&#51077;&#47141; &#46020;&#44396;","psrc":"&#44160;&#49353;&#50612;&#44032; \u003Ca href=\"/history\"\u003E&#50937; &#44592;&#47197;\u003C/a\u003E&#50640;&#49436; &#49325;&#51228;&#46104;&#50632;&#49845;&#45768;&#45796;.","psrl":"&#49325;&#51228;","sbit":"&#51060;&#48120;&#51648;&#47196; &#44160;&#49353;","srch":"Google &#44160;&#49353;"},"nds":true,"ovr":{},"pq":"","refpd":true,"refspre":true,"rfs":[],"sbpl":24,"sbpr":24,"scd":10,"sce":5,"stok":"jF4FqOEh3YAf_OT3RIhNvoH-2xc"},"d":{},"ZI/YVQ":{},"Qnk92g":{},"U5B21g":{},"DPBNMg":{},"YFCs/g":{}};google.x(null,function(){});(function(){var r=[];google.plm(r);})();(function(){var m=[]
;google.jsc && google.jsc.m(m);})();</script></div></body></html>

@ikod
Copy link
Owner

ikod commented Jul 2, 2018

ok, let's add some debug. Use this code

`
import std.stdio;
import requests;

void main()
{
auto rq = Request();
rq.verbosity=3;
rq.sslSetCaCert("C:\D\cacert.pem");
auto content = rq.get("https://google.com");
auto f = File("temp.html", "w");
f.write(content.responseBody);
f.close();
}`

and compile with -debug=requests

Edit: failed to format reply. Just add rq.verbosity=3; and compile with -debug=requests. And please run for both sites. I'll try to check from my side.

@ZHANITEST
Copy link
Author

cmd:

dub run --compiler=dmd --debug=requests

code:

import std.stdio;
import requests;

void main()
{
	auto rq = Request();
	rq.verbosity=3;
	rq.sslSetCaCert("C:\\D\\cacert.pem");
	auto content = rq.get("https://google.com");
	auto f = File("temp.html", "w");
	f.write(content.responseBody);
	f.close();
}

and result:

dub run --compiler=dmd --debug=requests
Performing "debug" build using dmd for x86.
taggedalgebraic 0.10.11: target for configuration "library" is up to date.
eventcore 0.8.35: target for configuration "winapi-optlink" is up to date.
stdx-allocator 2.77.2: target for configuration "library" is up to date.
vibe-core 1.4.0: target for configuration "winapi-optlink" is up to date.
vibe-d:utils 0.8.4: target for configuration "library" is up to date.
vibe-d:data 0.8.4: target for configuration "library" is up to date.
mir-linux-kernel 1.0.0: target for configuration "library" is up to date.
vibe-d:crypto 0.8.4: target for configuration "library" is up to date.
diet-ng 1.5.0: target for configuration "library" is up to date.
vibe-d:stream 0.8.4: target for configuration "library" is up to date.
vibe-d:textfilter 0.8.4: target for configuration "library" is up to date.
vibe-d:inet 0.8.4: target for configuration "library" is up to date.
vibe-d:tls 0.8.4: target for configuration "openssl" is up to date.
vibe-d:http 0.8.4: target for configuration "library" is up to date.
vibe-d:mail 0.8.4: target for configuration "library" is up to date.
vibe-d:mongodb 0.8.4: target for configuration "library" is up to date.
vibe-d:redis 0.8.4: target for configuration "library" is up to date.
vibe-d:web 0.8.4: target for configuration "library" is up to date.
vibe-d 0.8.4: target for configuration "vibe-core" is up to date.
requests 0.8.2: target for configuration "vibed" is up to date.
postboy ~master: building configuration "application"...
Linking...
To force a rebuild of up-to-date targets, run again with --force.
Copying files for vibe-d:tls...
Running .\postboy.exe
2018-07-03T00:09:50.173:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\ssl_adapter.d:_staticCtor36:95 warning: failed to load libssl - first access over https will fail
2018-07-03T00:09:50.173:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:setupConnection:809 Set up new connection
2018-07-03T00:09:50.173:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:select_proxy:446 proxy=null
2018-07-03T00:09:50.813:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:setupConnection:858 ssl connection to origin server ready
2018-07-03T00:09:50.813:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:select_proxy:446 proxy=null
2018-07-03T00:09:50.813:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:exec:1387 GET / HTTP/1.1
User-Agent: dlang-requests
Accept-Encoding: gzip,deflate
Connection: Keep-Alive
Host: google.com


> GET / HTTP/1.1
> User-Agent: dlang-requests
> Accept-Encoding: gzip,deflate
> Connection: Keep-Alive
> Host: google.com
>
2018-07-03T00:09:50.907:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:receiveResponse:894 read: 596
00000  48 54 54 50 2F 31 2E 31  20 33 30 31 20 4D 6F 76  |HTTP/1.1 301 Mov|
00010  65 64 20 50 65 72 6D 61  6E 65 6E 74 6C 79 0D 0A  |ed Permanently..|
00020  4C 6F 63 61 74 69 6F 6E  3A 20 68 74 74 70 73 3A  |Location: https:|
00030  2F 2F 77 77 77 2E 67 6F  6F 67 6C 65 2E 63 6F 6D  |//www.google.com|
00040  2F 0D 0A 43 6F 6E 74 65  6E 74 2D 54 79 70 65 3A  |/..Content-Type:|
00050  20 74 65 78 74 2F 68 74  6D 6C 3B 20 63 68 61 72  | text/html; char|
00060  73 65 74 3D 55 54 46 2D  38 0D 0A 44 61 74 65 3A  |set=UTF-8..Date:|
00070  20 4D 6F 6E 2C 20 30 32  20 4A 75 6C 20 32 30 31  | Mon, 02 Jul 201|
00080  38 20 31 35 3A 30 39 3A  34 39 20 47 4D 54 0D 0A  |8 15:09:49 GMT..|
00090  45 78 70 69 72 65 73 3A  20 57 65 64 2C 20 30 31  |Expires: Wed, 01|
000A0  20 41 75 67 20 32 30 31  38 20 31 35 3A 30 39 3A  | Aug 2018 15:09:|
000B0  34 39 20 47 4D 54 0D 0A  43 61 63 68 65 2D 43 6F  |49 GMT..Cache-Co|
000C0  6E 74 72 6F 6C 3A 20 70  75 62 6C 69 63 2C 20 6D  |ntrol: public, m|
000D0  61 78 2D 61 67 65 3D 32  35 39 32 30 30 30 0D 0A  |ax-age=2592000..|
000E0  53 65 72 76 65 72 3A 20  67 77 73 0D 0A 43 6F 6E  |Server: gws..Con|
000F0  74 65 6E 74 2D 4C 65 6E  67 74 68 3A 20 32 32 30  |tent-Length: 220|
00100  0D 0A 58 2D 58 53 53 2D  50 72 6F 74 65 63 74 69  |..X-XSS-Protecti|
00110  6F 6E 3A 20 31 3B 20 6D  6F 64 65 3D 62 6C 6F 63  |on: 1; mode=bloc|
00120  6B 0D 0A 58 2D 46 72 61  6D 65 2D 4F 70 74 69 6F  |k..X-Frame-Optio|
00130  6E 73 3A 20 53 41 4D 45  4F 52 49 47 49 4E 0D 0A  |ns: SAMEORIGIN..|
00140  41 6C 74 2D 53 76 63 3A  20 71 75 69 63 3D 22 3A  |Alt-Svc: quic=":|
00150  34 34 33 22 3B 20 6D 61  3D 32 35 39 32 30 30 30  |443"; ma=2592000|
00160  3B 20 76 3D 22 34 33 2C  34 32 2C 34 31 2C 33 39  |; v="43,42,41,39|
00170  2C 33 35 22 0D 0A 0D 0A  3C 48 54 4D 4C 3E 3C 48  |,35"....<HTML><H|
00180  45 41 44 3E 3C 6D 65 74  61 20 68 74 74 70 2D 65  |EAD><meta http-e|
00190  71 75 69 76 3D 22 63 6F  6E 74 65 6E 74 2D 74 79  |quiv="content-ty|
001A0  70 65 22 20 63 6F 6E 74  65 6E 74 3D 22 74 65 78  |pe" content="tex|
001B0  74 2F 68 74 6D 6C 3B 63  68 61 72 73 65 74 3D 75  |t/html;charset=u|
001C0  74 66 2D 38 22 3E 0A 3C  54 49 54 4C 45 3E 33 30  |tf-8">.<TITLE>30|
001D0  31 20 4D 6F 76 65 64 3C  2F 54 49 54 4C 45 3E 3C  |1 Moved</TITLE><|
001E0  2F 48 45 41 44 3E 3C 42  4F 44 59 3E 0A 3C 48 31  |/HEAD><BODY>.<H1|
001F0  3E 33 30 31 20 4D 6F 76  65 64 3C 2F 48 31 3E 0A  |>301 Moved</H1>.|
00200  54 68 65 20 64 6F 63 75  6D 65 6E 74 20 68 61 73  |The document has|
00210  20 6D 6F 76 65 64 0A 3C  41 20 48 52 45 46 3D 22  | moved.<A HREF="|
00220  68 74 74 70 73 3A 2F 2F  77 77 77 2E 67 6F 6F 67  |https://www.goog|
00230  6C 65 2E 63 6F 6D 2F 22  3E 68 65 72 65 3C 2F 41  |le.com/">here</A|
00240  3E 2E 0D 0A 3C 2F 42 4F  44 59 3E 3C 2F 48 54 4D  |>...</BODY></HTM|
00250  4C 3E 0D 0A                                       |L>..            |
2018-07-03T00:09:50.907:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:623 statusLine: HTTP/1.1 301 Moved Permanently
< HTTP/1.1 301 Moved Permanently
< location: https://www.google.com/
2018-07-03T00:09:50.907:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header location = https://www.google.com/
< content-type: text/html; charset=UTF-8
2018-07-03T00:09:50.907:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header content-type = text/html; charset=UTF-8
< date: Mon, 02 Jul 2018 15:09:49 GMT
2018-07-03T00:09:50.923:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header date = Mon, 02 Jul 2018 15:09:49 GMT
< expires: Wed, 01 Aug 2018 15:09:49 GMT
2018-07-03T00:09:50.923:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header expires = Wed, 01 Aug 2018 15:09:49 GMT
< cache-control: public, max-age=2592000
2018-07-03T00:09:50.923:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header cache-control = public, max-age=2592000
< server: gws
2018-07-03T00:09:50.923:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header server = gws
< content-length: 220
2018-07-03T00:09:50.923:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header content-length = 220
< x-xss-protection: 1; mode=block
2018-07-03T00:09:50.923:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header x-xss-protection = 1; mode=block
< x-frame-options: SAMEORIGIN
2018-07-03T00:09:50.923:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header x-frame-options = SAMEORIGIN
< alt-svc: quic=":443"; ma=2592000; v="43,42,41,39,35"
2018-07-03T00:09:50.923:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header alt-svc = quic=":443"; ma=2592000; v="43,42,41,39,35"
< 220 bytes of body received
2018-07-03T00:09:50.938:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:receiveResponse:933 Body received.
2018-07-03T00:09:50.938:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:exec:1439 Closing connection because of 'Connection: close' or no 'Connection' header
>> Connect time: 640 ms, 672 關s, and 8 hnsecs
>> Request send time: 15 ms, 626 關s, and 1 hnsec
>> Response recv time: 109 ms, 383 關s, and 1 hnsec
2018-07-03T00:09:50.938:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:followRedirectResponse:739 Closing connection because of 'Connection: close' or no 'Connection' header
2018-07-03T00:09:50.938:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:setupConnection:809 Set up new connection
2018-07-03T00:09:50.938:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:select_proxy:446 proxy=null
2018-07-03T00:09:51.188:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:setupConnection:858 ssl connection to origin server ready
2018-07-03T00:09:51.188:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:select_proxy:446 proxy=null
2018-07-03T00:09:51.188:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:exec:1387 GET / HTTP/1.1
User-Agent: dlang-requests
Accept-Encoding: gzip,deflate
Connection: Keep-Alive
Host: www.google.com


> GET / HTTP/1.1
> User-Agent: dlang-requests
> Accept-Encoding: gzip,deflate
> Connection: Keep-Alive
> Host: www.google.com
>
2018-07-03T00:09:51.298:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:receiveResponse:894 read: 1401
00000  48 54 54 50 2F 31 2E 31  20 32 30 30 20 4F 4B 0D  |HTTP/1.1 200 OK.|
00010  0A 44 61 74 65 3A 20 4D  6F 6E 2C 20 30 32 20 4A  |.Date: Mon, 02 J|
00020  75 6C 20 32 30 31 38 20  31 35 3A 30 39 3A 34 39  |ul 2018 15:09:49|
00030  20 47 4D 54 0D 0A 45 78  70 69 72 65 73 3A 20 2D  | GMT..Expires: -|
00040  31 0D 0A 43 61 63 68 65  2D 43 6F 6E 74 72 6F 6C  |1..Cache-Control|
00050  3A 20 70 72 69 76 61 74  65 2C 20 6D 61 78 2D 61  |: private, max-a|
00060  67 65 3D 30 0D 0A 43 6F  6E 74 65 6E 74 2D 54 79  |ge=0..Content-Ty|
00070  70 65 3A 20 74 65 78 74  2F 68 74 6D 6C 3B 20 63  |pe: text/html; c|
00080  68 61 72 73 65 74 3D 49  53 4F 2D 38 38 35 39 2D  |harset=ISO-8859-|
00090  31 0D 0A 50 33 50 3A 20  43 50 3D 22 54 68 69 73  |1..P3P: CP="This|
000A0  20 69 73 20 6E 6F 74 20  61 20 50 33 50 20 70 6F  | is not a P3P po|
000B0  6C 69 63 79 21 20 53 65  65 20 67 2E 63 6F 2F 70  |licy! See g.co/p|
000C0  33 70 68 65 6C 70 20 66  6F 72 20 6D 6F 72 65 20  |3phelp for more |
000D0  69 6E 66 6F 2E 22 0D 0A  43 6F 6E 74 65 6E 74 2D  |info."..Content-|
000E0  45 6E 63 6F 64 69 6E 67  3A 20 67 7A 69 70 0D 0A  |Encoding: gzip..|
000F0  53 65 72 76 65 72 3A 20  67 77 73 0D 0A 58 2D 58  |Server: gws..X-X|
00100  53 53 2D 50 72 6F 74 65  63 74 69 6F 6E 3A 20 31  |SS-Protection: 1|
00110  3B 20 6D 6F 64 65 3D 62  6C 6F 63 6B 0D 0A 58 2D  |; mode=block..X-|
00120  46 72 61 6D 65 2D 4F 70  74 69 6F 6E 73 3A 20 53  |Frame-Options: S|
00130  41 4D 45 4F 52 49 47 49  4E 0D 0A 53 65 74 2D 43  |AMEORIGIN..Set-C|
00140  6F 6F 6B 69 65 3A 20 31  50 5F 4A 41 52 3D 32 30  |ookie: 1P_JAR=20|
00150  31 38 2D 30 37 2D 30 32  2D 31 35 3B 20 65 78 70  |18-07-02-15; exp|
00160  69 72 65 73 3D 57 65 64  2C 20 30 31 2D 41 75 67  |ires=Wed, 01-Aug|
00170  2D 32 30 31 38 20 31 35  3A 30 39 3A 34 39 20 47  |-2018 15:09:49 G|
00180  4D 54 3B 20 70 61 74 68  3D 2F 3B 20 64 6F 6D 61  |MT; path=/; doma|
00190  69 6E 3D 2E 67 6F 6F 67  6C 65 2E 63 6F 6D 0D 0A  |in=.google.com..|
001A0  53 65 74 2D 43 6F 6F 6B  69 65 3A 20 4E 49 44 3D  |Set-Cookie: NID=|
001B0  31 33 33 3D 4D 2D 30 48  41 4F 4B 63 55 6A 63 67  |133=M-0HAOKcUjcg|
001C0  62 37 68 6D 6B 34 41 41  49 35 50 71 4B 78 4E 33  |b7hmk4AAI5PqKxN3|
001D0  67 6A 64 6B 76 39 4D 66  71 6F 7A 74 6F 5A 73 4E  |gjdkv9MfqoztoZsN|
001E0  49 42 78 5F 71 43 76 46  5A 53 49 48 72 57 6F 6C  |IBx_qCvFZSIHrWol|
001F0  4B 62 66 65 34 38 56 70  64 37 50 4F 76 69 76 55  |Kbfe48Vpd7POvivU|
00200  78 33 4F 4A 56 58 42 62  68 73 75 4B 32 34 79 73  |x3OJVXBbhsuK24ys|
00210  39 66 50 4C 52 68 64 4E  33 63 32 58 31 75 71 6B  |9fPLRhdN3c2X1uqk|
00220  34 72 53 62 62 6C 44 42  35 66 5F 38 39 68 39 79  |4rSbblDB5f_89h9y|
00230  6F 39 35 7A 3B 20 65 78  70 69 72 65 73 3D 54 75  |o95z; expires=Tu|
00240  65 2C 20 30 31 2D 4A 61  6E 2D 32 30 31 39 20 31  |e, 01-Jan-2019 1|
00250  35 3A 30 39 3A 34 39 20  47 4D 54 3B 20 70 61 74  |5:09:49 GMT; pat|
00260  68 3D 2F 3B 20 64 6F 6D  61 69 6E 3D 2E 67 6F 6F  |h=/; domain=.goo|
00270  67 6C 65 2E 63 6F 6D 3B  20 48 74 74 70 4F 6E 6C  |gle.com; HttpOnl|
00280  79 0D 0A 41 6C 74 2D 53  76 63 3A 20 71 75 69 63  |y..Alt-Svc: quic|
00290  3D 22 3A 34 34 33 22 3B  20 6D 61 3D 32 35 39 32  |=":443"; ma=2592|
002A0  30 30 30 3B 20 76 3D 22  34 33 2C 34 32 2C 34 31  |000; v="43,42,41|
002B0  2C 33 39 2C 33 35 22 0D  0A 54 72 61 6E 73 66 65  |,39,35"..Transfe|
002C0  72 2D 45 6E 63 6F 64 69  6E 67 3A 20 63 68 75 6E  |r-Encoding: chun|
002D0  6B 65 64 0D 0A 0D 0A 30  30 30 30 30 30 30 31 0D  |ked....00000001.|
002E0  0A 1F 0D 0A 30 30 30 30  30 30 31 0D 0A 8B 0D 0A  |....0000001.....|
002F0  31 33 37 32 0D 0A 08 00  00 00 00 00 02 FF C5 3A  |1372...........:|
00300  E9 77 DB B6 93 DF FB 57  D0 F0 AB 2C AD 69 89 F7  |.w.....W...,.i..|
00310  21 86 CE E6 70 53 BF A6  CD D9 5F 37 4D 53 3F 90  |!...pS...._7MS?.|
00320  84 24 46 BC 4C 52 3E 2A  EB 7F DF 19 80 A4 28 D9  |.$F.LR>*......(.|
00330  6E FA BA 1F F6 39 91 70  0C E6 C6 CC 00 D0 93 83  |n....9.p........|
00340  28 0F EB DB 82 49 8B 3A  4D 4E 9F E0 A7 14 D7 2C  |(....I.:MN.....,|
00350  AD C2 BC 60 3E 21 BC 83  00 3E 59 D4 75 31 9D 4C  |...`>!...>Y.u1.L|
00360  AA 70 C1 52 3A CE CB F9  E4 37 16 BC A5 73 46 A4  |.p.R:....7...sF.|
00370  84 66 73 9F 2C 73 02 18  18 8D 4E 9F A4 AC A6 52  |.fs.,s....N....R|
00380  98 67 35 CB 6A 9F D4 EC  A6 9E 20 6A 4F 0A 17 B4  |.g5.j..... jO...|
00390  AC 58 ED FF FA F1 87 13  87 48 88 F3 84 5D AE E2  |.X.......H...]..|
003A0  2B 9F BC 10 E0 27 1F 81  1A D9 47 31 89 53 A0 54  |+....'....G1.S.T|
003B0  4D 82 92 66 51 9C CD 27  F3 3C 9F 27 6C 3E 51 6F  |M..fQ..'.<.'l>Qo|
003C0  DA E6 45 55 C3 1C 2D A3  8B 30 4F F2 F2 42 D5 9C  |..EU..-..0O..B..|
003D0  A8 18 17 D9 5C 08 51 94  79 E1 13 8E 06 B0 D7 71  |....\.Q.y......q|
003E0  9D B0 D3 57 7C E5 93 89  E8 3D A9 C2 32 2E 6A 29  |...W|....=..2.j)|
003F0  CB B3 10 E4 9D 7D AA 5E  CE 6E AF 3E 95 89 F3 F3  |.....}.^.n.>....|
00400  47 A5 4A 7F 4C AB 9C 5D  FB 3E 39 1D CE 56 59 58  |G.J.L..].>9..VYX|
00410  C7 79 36 1C AD AF E3 2C  CA AF C7 82 07 7F BD 3C  |.y6....,.......<|
00420  3B 9F 1E BD FD F5 99 F5  9B F1 F5 DD 79 FE 66 7E  |;...........y.f~|
00430  F2 8E 1A D7 F4 C5 A7 97  EF 8E E4 E5 D9 FF BC 85  |................|
00440  69 45 56 75 53 73 2D 45  B6 1D 5B 36 6D 59 D5 14  |iEVuSs-E..[6mY..|
00450  68 43 C3 74 74 D9 D0 4D  59 73 14 D9 70 2D 59 53  |hC.tt..MYs..p-YS|
00460  0D 59 77 1C D9 54 35 D9  D6 0D D9 92 2D C5 96 75  |.Yw..T5.....-..u|
00470  D5 95 35 5D 77 6D CD 96  35 03 96 03 B4 AE C1 3F  |..5]wm..5......?|
00480  57 73 0D 40 C6 3F 74 40  A5 E9 06 40 9A 8A 05 73  |Ws.@.?t@...@...s|
00490  96 AB C9 AA 89 E0 8E 05  4B 4C CB 91 81 9E AA C2  |........KL......|
004A0  87 61 AB B2 6A E8 9A 01  F4 5C 40 6F 59 B2 69 AA  |.a..j....\@oY.i.|
004B0  B2 65 19 B8 4E 56 6D 1B  3E 54 40 A7 29 2A 12 57  |.e..NVm.>T@.)*.W|
004C0  65 5B 33 65 5D 46 EC 20  09 7E 00 80 AE 00 3A E0  |e[3e]F. .~....:.|
004D0  CE 30 80 BC 0E 18 55 64  53 D6 14 00 B5 0D 19 99  |.0....UdS.......|
004E0  D5 55 05 48 68 C0 88 AA  00 31 10 58 73 00 58 01  |.U.Hh....1.Xs.X.|
004F0  14 BA 85 68 80 90 09 F0  36 E0 D2 6D D4 8B 01 7A  |...h....6..m...z|
00500  B0 5D 90 DA 91 2D C7 92  1D 64 1C B0 6A A0 26 57  |.]...-...d..j.&W|
00510  91 2D 55 76 4D 50 18 30  04 32 BA 40 18 34 A1 D9  |.-UvMP.0.2.@.4..|
00520  26 92 75 34 9C 71 80 49  55 76 50 B5 D0 07 CA 86  |&.u4.q.IUvP.....|
00530  26 83 5E 0D 6C 00 07 80  0E 3E 6D 0D D5 09 8C 83  |&.^.l....>m.....|
00540  CC A8 30 47 95 35 64 07  F4 A8 CB 36 8C 82 72 55  |..0G.5d....6..rU|
00550  54 86 AC 83 EC 9A CB 57  58 60 1A 0D 74 6C 2A 2E  |T......WX`..tl*.|
00560  32 05 DA 31 B8 B9 40 20  C3 30 65 07 14 69 01 E7  |2..1..@ .0e..i..|
00570  2A 4A E0 02 E3 06 40 A0  CA                       |*J....@..       |
2018-07-03T00:09:51.298:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:623 statusLine: HTTP/1.1 200 OK
< HTTP/1.1 200 OK
< date: Mon, 02 Jul 2018 15:09:49 GMT
2018-07-03T00:09:51.298:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header date = Mon, 02 Jul 2018 15:09:49 GMT
< expires: -1
2018-07-03T00:09:51.298:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header expires = -1
< cache-control: private, max-age=0
2018-07-03T00:09:51.298:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header cache-control = private, max-age=0
< content-type: text/html; charset=ISO-8859-1
2018-07-03T00:09:51.298:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header content-type = text/html; charset=ISO-8859-1
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
2018-07-03T00:09:51.298:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header p3p = CP="This is not a P3P policy! See g.co/p3phelp for more info."
< content-encoding: gzip
2018-07-03T00:09:51.298:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header content-encoding = gzip
< server: gws
2018-07-03T00:09:51.313:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header server = gws
< x-xss-protection: 1; mode=block
2018-07-03T00:09:51.313:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header x-xss-protection = 1; mode=block
< x-frame-options: SAMEORIGIN
2018-07-03T00:09:51.313:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header x-frame-options = SAMEORIGIN
< set-cookie: 1P_JAR=2018-07-02-15; expires=Wed, 01-Aug-2018 15:09:49 GMT; path=/; domain=.google.com
2018-07-03T00:09:51.313:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header set-cookie = 1P_JAR=2018-07-02-15; expires=Wed, 01-Aug-2018 15:09:49 GMT; path=/; domain=.google.com
< set-cookie: NID=133=M-0HAOKcUjcgb7hmk4AAI5PqKxN3gjdkv9MfqoztoZsNIBx_qCvFZSIHrWolKbfe48Vpd7POvivUx3OJVXBbhsuK24ys9fPLRhdN3c2X1uqk4rSbblDB5f_89h9yo95z; expires=Tue, 01-Jan-2019 15:09:49 GMT; path=/; domain=.google.com; HttpOnly
2018-07-03T00:09:51.329:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header set-cookie = NID=133=M-0HAOKcUjcgb7hmk4AAI5PqKxN3gjdkv9MfqoztoZsNIBx_qCvFZSIHrWolKbfe48Vpd7POvivUx3OJVXBbhsuK24ys9fPLRhdN3c2X1uqk4rSbblDB5f_89h9yo95z; expires=Tue, 01-Jan-2019 15:09:49 GMT; path=/; domain=.google.com; HttpOnly
< alt-svc: quic=":443"; ma=2592000; v="43,42,41,39,35"
2018-07-03T00:09:51.329:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header alt-svc = quic=":443"; ma=2592000; v="43,42,41,39,35"
< transfer-encoding: chunked
2018-07-03T00:09:51.329:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:parseResponseHeaders:650 Header transfer-encoding = chunked
2018-07-03T00:09:51.329:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d:analyzeHeaders:593 transferEncoding: chunked
2018-07-03T00:09:51.329:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\streams.d:empty:308 empty=0
2018-07-03T00:09:51.329:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\streams.d:empty:308 empty=0
2018-07-03T00:09:51.329:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\streams.d:empty:308 empty=0
2018-07-03T00:09:51.329:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\streams.d:empty:308 empty=1
2018-07-03T00:09:51.345:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\streams.d:empty:175 empty=0
2018-07-03T00:09:51.345:..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\streams.d:empty:175 empty=1
< 674 bytes of body received

object.Exception@..\..\AppData\Roaming\dub\packages\vibe-d-0.8.4\vibe-d\tls\vibe\stream\openssl.d(381): Peeking TLS stream: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number (336130315)
----------------
0x00480A91 in @safe int vibe.stream.openssl.OpenSSLStream.enforceSSL(int, immutable(char)[]) at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\vibe-d-0.8.4\vibe-d\tls\vibe\stream\openssl.d(382)
0x004806BC in @safe int vibe.stream.openssl.OpenSSLStream.checkSSLRet(int, immutable(char)[]) at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\vibe-d-0.8.4\vibe-d\tls\vibe\stream\openssl.d(339)
0x004800FF in @property @safe ulong vibe.stream.openssl.OpenSSLStream.leastSize() at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\vibe-d-0.8.4\vibe-d\tls\vibe\stream\openssl.d(247)
0x004800AF in @property @safe bool vibe.stream.openssl.OpenSSLStream.empty() at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\vibe-d-0.8.4\vibe-d\tls\vibe\stream\openssl.d(239)
0x00425F4B in int requests.streams.SSLVibeStream.receive(void[]) at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\streams.d(1263)
0x0041C86C in void requests.http.HTTPRequest.receiveResponse() at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d(1006)
0x0040BD5B in _D8requests4http11HTTPRequest__T4execVAyaa3_474554ZQtMFQrAS3std8typecons__T5TupleTQBsVQBwa3_6b655AC7EB660644949FC46DA0F67BD6A7CD at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d(1397)
0x0040B852 in requests.http.HTTPResponse requests.http.HTTPRequest.get!().get() at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\http.d(1512)
0x0040B7F6 in requests.base.Response requests.request.Request.get!().get(immutable(char)[]) at C:\Users\ZHANI\Desktop\postboy\..\..\AppData\Roaming\dub\packages\requests-0.8.2\requests\source\requests\request.d(291)
0x0040351F in _Dmain at C:\Users\ZHANI\Desktop\postboy\source\app.d(9)
0x004ECAA3 in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll().__lambda1()
0x004ECA25 in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll()
0x004EC8C0 in _d_run_main
0x00415B64 in main at C:\Users\ZHANI\Desktop\postboy\source\app.d(7)
0x0055407D in mainCRTStartup
0x73988484 in BaseThreadInitThunk
0x770F2FEA in RtlValidSecurityDescriptor
0x770F2FBA in RtlValidSecurityDescriptor
Program exited with code 1

and coudn't output a temp.html file. 🤔

@ikod
Copy link
Owner

ikod commented Jul 2, 2018

Thanks! Very strange. I have to investigate how vibe can receive control (see last stack trace)

@ikod ikod reopened this Jul 2, 2018
@ikod
Copy link
Owner

ikod commented Jul 2, 2018

Wait, do you still have this in dub,json?

"subConfigurations": {
"requests": "vibed"
}

it must be removed if you run requests not in vibe app.

@ZHANITEST
Copy link
Author

ZHANITEST commented Jul 2, 2018

Omg, really?
I will re-try again with remove a line that you said... when i off office.

Thanks 😊

@ikod
Copy link
Owner

ikod commented Jul 3, 2018

Hello, @ZHANITEST
ping?

@ZHANITEST
Copy link
Author

ZHANITEST commented Jul 3, 2018

Hi @ikod! Sorry for delay about it.

Sadly, I’m hard to test on my work-day.
Well... then please, just close a issue.
If i got a problem, i’ll new open it. (not re-open. wut ever..)

Thank you very much for help me! 👍

@ikod
Copy link
Owner

ikod commented Jul 4, 2018

ok, thanks!

@ikod ikod closed this as completed Jul 4, 2018
@ZHANITEST
Copy link
Author

ZHANITEST commented Jul 13, 2018

oookey doookey. soooo... i solved a problem finally. (🐸 yay!)
i used a 'dlang-requests' 2 years or more. but i didn't catch it about fixed or changed points like a 'subConfig'.

i wrote here that how can i solve it. just for the people who got a same problem like me.
first, i was clean my environment for build.

1. remove another compilers about dlang

  • DMC: a digital masrs compiler came with 'like.exe'. sometimes, that's make a noise when i linking with dmd and dub. or, crash with VisualStudio>build-tools.

  • LDC2: a llvm d compiler came with 'dub'. when this older version, make some problem.

2. re-install dmd and dub

>dmd --version && dub --version
DMD32 D Compiler v2.081.1

Copyright (C) 1999-2018 by The D Language Foundation, All Rights Reserved written by Walter Bright
DUB version 1.10.0, built on Jul 10 2018

3. install a libssl, copy carcert.pem

4. check a 'dub.json' or 'dub.sdl' file

  • check a 'requests' version
  • check a 'subConfigure blaha blaha : ["vibed"]' option. this wasn't need it now!! 😠

5. clean and build it!

  • a test code here:
import std.stdio;
import requests;

void main()
{
	auto rq = Request();
	//rq.verbosity=3; // for debugging
	rq.sslSetCaCert("C:\\D\\cacert.pem"); //
	auto content = rq.get("https://google.com");
	auto f = File("temp.html", "w");
	f.write(content.responseBody);
	f.close();
}
  • ... and dub.json here:
{
	"name": "postboy",
	"authors": [
		"zhanitest"
	],
	"dependencies": {
		"requests": "~>0.8.2"
	},
	"description": "A minimal D application.",
	"copyright": "Copyright © 2018, zhanitest",
	"license": "proprietary"
}
  • just clean and re-build it! 😎
  • if, still installed the ldc2, add --compiler=dmd when building with dub.
dub clean
dub run
// ... or `dub run --compiler=dmd`

that's it! i wrote faster ASAP. so this tips may any misspelling. sorry. 😅
Aaand, thank you so much, @ikod !👍

@ikod
Copy link
Owner

ikod commented Jul 16, 2018

hello, @ZHANITEST

Thanks for detailed description of solution. I'll rearrange it a bit and post link from main page to this step-by-step solution.

@ikod
Copy link
Owner

ikod commented Jul 18, 2018

On windows:

  1. Check that you have fresh version of dlang-requests (at least 0.8.1)
  2. Install precompiled openssl DLL's. Use this link: https://slproweb.com/download/Win32OpenSSL_Light-1_0_2o.exe, or any other place you trust. Do not forget to allow installer to place openssl DLL's to system directory.
  3. Create local copy of certificate authority. Use this link https://curl.haxx.se/ca/cacert.pem, see details here. Download file and place it in any convenient location, you will need this file on next step and later. Let's for example we use C:\cacert.pem
  4. Use this code as test that everyting ins ok:
import std.stdio;
import requests;

void main()
{
	auto rq = Request();
	//rq.verbosity=3; // uncomment for debugging
	rq.sslSetCaCert("C:\\cacert.pem"); //
	auto content = rq.get("https://google.com");
        writeln(content.code);
}

If everything is ok you should see code 200. Otherwise you will get some kind of exception;

@sugoidogo
Copy link

sugoidogo commented Aug 11, 2019

None of the suggestions here are working for me.

/+ dub.sdl:
dependency "requests" version="~>1.0.8"
targetPath "bin/$PLATFORM/$ARCH"
subConfiguration "requests" "vibed"
+/
...
int installTools(){

        version(Windows){auto url="https://dl.google.com/android/repository/platform-tools-latest-windows.zip";}
        version(OSX){auto url="https://dl.google.com/android/repository/platform-tools-latest-darwin.zip";}
        version(linux){auto url="https://dl.google.com/android/repository/platform-tools-latest-linux.zip";}

        import std.zip,requests;
        auto rq=Request();
        rq.sslSetVerifyPeer(false);
        auto buffer=rq.get(url);
        auto zip=new ZipArchive(buffer.responseBody.data);
        foreach (x;zip.directory()){x.name[15 .. $].writeln;}                                                                                                                                                    
        return 0;
}

I always get this error:

object.Exception@C:\Users\Josef\AppData\Local\dub\packages\vibe-d-0.8.5\vibe-d\tls\vibe\stream\openssl.d(566): Peeking T
LS stream: error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac (336130329)

A couple times it did work, but I hadn't changed anything between runs.
I have installed every recent package from slproweb to no avail.

@ikod
Copy link
Owner

ikod commented Aug 11, 2019

hello @ZHANITEST

With vibed requests do not touch or link openssl libs at all. I'm afraid you have to enable vibe-d debugging messages to track source of the problem.

Did you try to run this short installTools routine in std config, without vibe-d? I have no windows right now, but I can test it for you tomorrow.

@sugoidogo
Copy link

Without vibed, I get can't complete call to TLS_method
With vibed and without sslSetVerifyPeer(false) or sslSetCaCert("cacert.pem") i get the same.
With vibed and an ssl option, I get one of:
Peeking TLS stream: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (336130315)
Peeking TLS stream: error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac (336130329).
How would I enable the debug messages?

@sugoidogo
Copy link

I should mention that this is all under windows, with Linux I can use the std config, vibed with an ssl option also works.

@sugoidogo
Copy link

sugoidogo commented Aug 11, 2019

With the following code, I can eventually get a success with vibed, after a great many error messages.

Response r;
while(!r){
try{
        auto rq=Request();
        rq.sslSetCaCert("cacert.pem");
        r=rq.get(url);
} catch (Exception e){
        e.msg.writeln;
}
}
"success".writeln;

@ikod
Copy link
Owner

ikod commented Aug 12, 2019

hello, @josephsmendoza
I'm glad that it is fixed. Sorry I can't be very helpful with windows.

@sugoidogo
Copy link

The issue is definitely not fixed. I have to call the same 3 lines of code over and over until they succeed. It can take from 5 seconds to 5 minutes, and I've had many runs that I aborted after 10 minutes of failing. I ended up using vibed directly, which works every time cross-platform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants