From 6d8778d8e7af50f0a7055d239df796e34f188239 Mon Sep 17 00:00:00 2001 From: Nick Terrell Date: Mon, 9 Oct 2017 16:44:31 -0700 Subject: [PATCH 001/145] [doc] Add images for release notes --- doc/images/linux-4.7-12-compress.png | Bin 0 -> 93913 bytes doc/images/linux-4.7-12-decompress.png | Bin 0 -> 56215 bytes doc/images/linux-git-compress.png | Bin 0 -> 105620 bytes doc/images/linux-git-decompress.png | Bin 0 -> 54333 bytes 4 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 doc/images/linux-4.7-12-compress.png create mode 100644 doc/images/linux-4.7-12-decompress.png create mode 100644 doc/images/linux-git-compress.png create mode 100644 doc/images/linux-git-decompress.png diff --git a/doc/images/linux-4.7-12-compress.png b/doc/images/linux-4.7-12-compress.png new file mode 100644 index 0000000000000000000000000000000000000000..71f3381bdef8fb1e0d811513c588b03dfb03d0a2 GIT binary patch literal 93913 zcmeFZ^+S}~6F)8p0v0JCNP~a~h!WCRbc%EeQZ6CV%~~`FA|S1Fcej*uFVYH<3#@dr zpRTB`+&>;VjwNQ>RW{kPyHB@YE?x)l;W1 zu4117e^Z~(+YA1KX7y0)?y1c7E3@E#aGr^)S)DpXLV)@Q%_ycDydrW+;{KgS_Gt4X z=OQVTTyFo&8z>j2ruiNiDnd=~eHMFD1ivnIlb8ta&bf~T>a3SEmK3OzAO)%q(snb9 zA<(l-o*BuTb&{B-cUh=Ey~U!wLl8|vL-Ql2X@7&~@#5&na79VkK-q_q4X0hx#o-UP zZcRJ8S&?kmbwthai9g!5QvSI)dY`Gv!8&S78IF26m*mnusU@y~%oV*TAQAQV^r{Zw8~qGY!WrEkejgdJ#i>?}|%8Ae-llsX%SRC`}y>0T%ri(VgfLaeqL z30ajTc?pj%mclcwhboI1I?ry~NijNp85iW57?{P$mRsAhLw%y4@3#NVPX3W|CF0E~&HmnPiUpHi5<1 zR=Zs;UsP0-*?Zl+ez(2;)fS!o-0_u3HI+ zGe+C%IW9nSBC?hTKcg?~@*9cOHk*rKF325PxNkm~uec9C6$&cmiJ(gpjY&zud$Y5P zWe8c1E6=~kn$))6Hu3%G_hW9RD^5{GrT=aCvu{wWM6g^%i>t>Xvs%scYJ|T?)CF?`EW$!!UI;(TI7WAzCL&JUinIU8t-7#$U2c3<015TUj^Wo z?`e#Snl-1)w_2+GXFdK}b*C)`av8L&yB?_ZXs?9#KN9_Uu`q+(zUJJ+-Ch>U=$PFf zj3z}Rwo+IW*76mpG6US2E`AO5Ougc}0$@%9tb1AZ^(@^6p;|EcGX1&NgtE73Dt0H? z#vVJ47|Uub>746 z!!P&)&Y?GPNNdy+jZIh%HuY__O}|!X2)j;n^hWu#fkfln%LgxhLa`{nP`VfFES4qN zi+8p0j77F3TwXD6SQrg7PUD*U84k&mSfWmj^XI!JILR!RB4k=v?zlE*YHj3e?z_c9 zhfjhVxm|m_W2E4s09rkv8z$j*3 z-QR@j>HGbdmtT1Z+LxXcZ)($X|jOmcIn zEN}b!%eLH>4y`>htE-Jhkmg|Kfti&#v%VG%0>Q-Fp1kR4!ghYzxP6VO1;aCzwm0>9cN@33kVUf*#cs*)P#^Wvu%=TkTcL`2fC#j^_~>DOMOTy!DJPXIAIQjwwr z*F|lVLlJJsoqu+x~Xt`VkuHReK8-NdhJEEHz|qs{7M! z`$o3Oq!MH}pTrB$=#(Md|vL*jKG4 zuqUYZJJ+qx=XH!u`Mli>`t~W>gl)S+@rs!vTw`p|2xC)s4z?3TxIpE9`@;H$(xNh= z03Q*?O-B&F#GrsAs6yyr*b7_|Vz;c=ZFJie+vj%j8;f#qm{W`yWULvyUbgGpm$k&} zwerBrb`^=ZmmUC_VA0u-Ae6%a;cN_r@>2+}u;z z8|zgifm!yyynBF2>UTh3dFfL!?t#ZLMOV?Zd=Rqgsi4Xc#34GV<+isrTU51Y79#I0 zjIA%*PQkj=nIay!KQgwb6@sxdx=|TA`tuFXql-@8Uy~R%^@%fGSYO{KeSAqGxmac| zzehIhO5Wvjfmi$$d=>PbjEjc`&cprJ7woEbv}`@?e=7M3ndR4E*RZ>7wWzHX@6(!4 z;tYxpH;T6pnU}GW-yy}d^vShf8ejF2bEcJYu4ig3cGX)o_09EHh~jw29z@Qu{WEKS zE-Tk%8)>yQ8EO=F>hSdv-vWVX07Ht)! zWet)$tyL6g?B&r`pA*2pSUgW!wZE2?*P%v#MSp)Nz-@P8FCw@ipiZkbkl$r{)imC8 zmD&GY=*$kv2#majNGUeDLn}FV&Wq{sY!k&9)=r(%*}Q8IZ))n(p+SFEe}DL=AO2Vs zau1OgIKAck(bB)Bw{3{C6aK6&WuPkS%(6fJCO$IEinn}x(1%@xHibf>zZh1v5`}m| z7c^m~8ayqG+#0YUqEprX@jQ4lilkpTSY4vc^q!yFo7&9nY>tP#Q!hq}E%n#yRWb_e zJZE>Mf@Y+Gzn4tORQyMB#VisMgU+53S+lfJL z#muiylbVpnO+Otm7nJwVzpv|C=kt8ri_y@lcwXk+zL&|xPpuzrFG%+Xs5pV! z4?3zQg0LJpdf9Q8d2%`}EXnMK_5HY-iNP2?o2RhMZ0~rf3^;RPy){KwmU_{)*x1A^ zJTotad_NZeX2%SER#$8*-1ON|u*G9K5_)kGd8i}nzGm*C?4@j=goI?%|(uidK!3F1`|IyOEghmO4a+R~})_@tjN4MwFIr-`&j6OW`R; zGkbG8X$4=9OB)|=o!hu$6JZU9;QhLSE`p}yU2fz%S-zB@`PPkqx=t$O5|vR?lPXD^ z8XYt7WU!*_>^sdi<@H^{xHI}cOoBH*$0~c_d-XN-B@@P#&`Dfo5M^_nxkr^sucEKz zEK93E#xTW4nOtWy&%Qs-zU|!b$=sK+?Oh`W!`%#m1%KN~iSRi-OF`w03P+9W9!k-x zehTFdO-Y!Z4wev?!nAka!Cp{R!Qjv8QFb-w&`2T&#ww5Dpye98G+Bd1)vAI!1b;TL zW+6{M8CS=&upf7AV#BKS!4-z827QO(OplM0M%gh5A6-RlpH%JoBy8^WA+5+`w&wVG zK0l&4h%yZ>z13xf{K$_$xyW!@Jh$wrqIBgL>L4ivz5BAh?&O&SxI3O0YQf@(F3ay? zTFpoN@$Hrxr*$O-wuD3(MNv`DdS^<_yztl3t(m@JuG>>O6w+Q;33b>-`@%Vufj7Hd z%;IHE_jz?ib97&qtYgp{HEw#Bw&b2d`naSbD29g`8-JI_FTf(v|8E6{b#?IbEDa9YaZ7c%gsoM(;$O>&va{7uIgw?A=>tSEtPh z?@)_~n9AZ<@H)ntY9h1@yaz5QRsfoK?g!oe`29cuvUUQxPvap2#WI|7ly$r5f!O5I zUK;iKm0yZ8c&SWohwWi0@(@^Ud1l({KFknahEpK45RNL`0Y*r^KvjSG{`F?@bsHzM zlp;a%+JF+_xK}-aVw7rG+uTFT7;q7Z`hET%UQG*aOW};vX@q`z0ZMQeB);(;{E<#x z0i|nil`C??L6~kyJFT~@{dt;raJ1|8Y=Lc-ue=|Ii!Yh)M14jJhc_E#HBrAceGe@f z1zq(7nqHs54V$zLc-8Ry?F;>gG{@1dq%a?zL^My2tM!_#OOuMl4aVOL-y050HuaG9 zQMfp^XpBDm&`dWf=EcssOmA9Fl?i`b>`!RcER_LlU6}6P2s`ph<0=)$XRSk(%eYQm zGuYhK$8(SddyA~G*4Em*u*X)QSB z%wqR^Lfw@R1-*O=*P%6Cf5H_=!@ZF0(6CxdCxiH%(u(!-`GQl8zDWaOURx9$wGevzby7X*5xMjTbWkYB@@WGiZAN9YZjC|$#uV6lA<2-jJKTh%2jl(= zE+IQhiaPN{;Q03q9t!!1z`ipgFZ;u3E^=u$HVrAm^Ei7uzW6Vy?A1$+3S}K)TpysJ zG*~}v-$TVKbZl=KZ@jsw$4=(dA-M4bcf@N_T)vK}H_K%KGHZccw_u@5*3+mUkPGg3 z_i*Xesv|=TE@#jO4u%?CseE=faT(uAKZgfc8sjt%?y=Gh^nWbmFi#N#X+z!*Y5I4_ zWp?Cqajzr+G;Nm4T4`nudjdF0X|m}pghgAdE2sO&XV)Bri+IqBl*^V~#vOeM|=JxArxH<(S)Bp3H4zb~UO;d?3vrt2#aYU3Y60ud6Xk zj1Nh06|wTHg1d)JC~fJ3r$R1=j0nA|MdjAieCF%TZd%G+<+S;kwYOAOKua3a*3`)6 zjGiR#Od`!Zk%mX;--kJz)-{I@6FyLpQW2q2R;tM!@8L(+66>TshNm-Un@`VGF9)uQa*cF8w$^=rOkxTrs-Qa*m^9&g>n zgB-cPsoy7C>$U zVU~fXqK(rId*e)>c&2FN=#szPszl~`mIh)^r2lS(OSoC6aNX1IpI;9kCY>GK!;BoA z=r}wa1y>OkagCS1Si~ONkKCI^628-W84x&!mHBB z>Hb}xOCA_Z+7d5sx0N=~)sQJZxPSKsK@-OdF3F5pv@=iVbhCcD#W$rolxdsiM?eZV zHP(r?&jXaI}s@*0W; zdk&vPMIoBReycXw)86#M!n?J?Pvkz10mi8Zr1w#RHUdG|dv0gQ@CT5QfpyVf!wQonaq zJaooMItH&uS1jndr1PDnNp?404S$7#4u^KD+{qdc&b$r)X{u?ceYUEzL|0Y1MiCqN zWlCIw=9a3xp(+E}J+bb3A~c2ZbHScQ!64YTCH)oyw!l5nC~czUB@>5|$eMEpKA?@J z-P(gp)LxSze^gEDDf(V)%Yur!sqhT3iEd8onARb@HY4k~o^{^AGV6&rjOS}yL*8s( zzC7o;gsHi|0|%tqf>lP{BX+8tuZ`WylPyJ-1uqWNJy=24`Z9S&+aYwSQ;SwSc$ls1 zB|Oz3jNDa^cw;t#BG;~WU5X=~z@1FXEWH-N&t(qxzdt)Px?X5X=DyG9>!8eI`t_}! zwKHHr)CR5;ji5rsj%Xn&oHl)b-m4VPztFs2XjA_p_N`$d^07Lcdm$~Wl8@hIIFY~8 z`BmG~v1gP;+B$TzEAO+M6P}kkn-8fj0Ujj>ptj!5R)?F=B;E43^0K|VYH%^}C~|+p zsHI@v%j4xcjkm9Cz^SY<$+svgT8G2=j ze`~zw$yWaJvuV6#d50LWD8Whxa<_v#dZI{~>P@Nn;&918^1`O(8!g|;wwQ;MbS2Xr zF*+QS^SrXWlfkCe=2s9_F*};JaK~YD3OUaJ2wx(#Cp+%=I#S>yn5IdC?wqnMneN9KOS0I3fw|2nBCGTU5-b#>o2QV>q10^Qwlg&E^D#R^ zR!@bB#hkpOsPS}y-V=T2knwQ;R~mE)1GXIAT$R3G#2!_B!(>l(khypeIGvx&;|%T) zuLJNNWjkmxfXDz$Ep3E~Kq9$k*SEmP0Iarp#K&`N#j%0eX$M{q<18z%xsSDRmIpQ` zN4;O{TPINwBrL%@cmE-a0Yny z?B-Awbu|>0R{#!FqjU|Xk!NER?J&jN=)9Ju6Gh8G#2`AcVC=Qhuod^iW!9}#;5!jr zajMvd$u-(pBDxE4oZ0?j<5j?61p$4PJo#CIE6hXcbZKrt>ClTnpr;2X7<+BET{7!6z z1u}N3tSy7PuEWswh#0ycyRMDR-TR-}gPJXU?0gOVQt)X-*_{SjZbmrAkaAEqEa_&8 z2Q?;-iDx!2wUNHQ(o|4>KGA$$F>9pFSJ8F!*#dIs8bYd~-x&!lpV+wP48^OO7(eGJ zo+$c>GqVUV7$eLCu?lAfkr@^)5Mo3!p^m7)=C_y3q7urXj%Hu+%jT@k=U-7Yy8lT~ zQnQiWvCHSFGR6Fw{0g-oi^s1N-4@rTinL}sFu+-D^R51ygnDMXbVZ(ei0L zB>KHx!jyDqcujB0@8_}+;P>q`%w&m+oDY8G8E9)cWIF$jk$idL!X(!qUSlgER)If0 zPL}iiDUqQG-0zN!$pvHLS>*ne{?-xXygHgmfxABUEqw20*syL0%SKmwZ*M79?DRK{ zSs<+N`k!gO!$(31k}WUP7tXQsjBf&J%qg$1(VCPo%KD*g^9;z~HN1BEJBEm|h9D|7 zBXg7DqGISgLB5s|cDv^vjY~`i?s^F|U@tgYQSqqT-B_S(zx{xUe9MLFZN2Uwh1ab3 z+jSnF;s!*aBhKiImG3Gpd>3l&5OJpGP0iQI#<5X|^ksY(;~yIqas9RH?E?p_XK&So zms`G%5d6V=xsUp`p=WR|y#sxtWUp-rNw-T14xc^*XUeA#}i|9Xa(%mT_3 zp^d-Zgnq~SLZy4Fb2y{c@?s-(=S6fOsRcX(7kX53l#3JgvcMF-NBd|T5gg2M=$$3J z7<+V=3M{Jk*^qE)-TRo%A)>)UO@}0|7S#D7tjh*6AbwbC0MpYY#X!;E<6<--x{Vml zA-1g@qX%<~y^NlSZTu!xpWrVU@Tfb9LHW%L-!KX_lLNW`_z`%MAiYiAM*3e1FnAYG ziL9T~-~DR=IPZanms9;E{{E=Dhz5#*eh_98^w)r3F$}>&?t6V4|Ejzjp97WVLhL-* ze@b#5JTy$^yLwUz)D+BUKzc>|7$)&ANy1SN&nq?la|u{MJ~Bkx%J)x6Zh?mpALkA$ znI}Zee+>%!8n4tC|0&5E@bK0nig*87f8g6>@(zF~VFjRL9j}Zhl^RWxAtd z?*H=HVDySwHmQN#?~qkgwe$grRg9_{KfywoQfSvuhtptI<75EG;dWEr7Kqfj?ucJ1=hPD3Z#WQ>)d4l7dyPy1JW zL})O5br*12KuqrD`;>D>neVq}&^o}o;kMcmpM0fp1#nHB4Urfgr&q^2HH zO#hvy2qEU6D+a_D!;klo@a}h9BiuByFODrCtjtp;3p%2HgW+-QI^-Gs$lLcE5acNz zlLtKhHooegIQe_nFdz2H+w}sephRk+US?~y-qY9j9x#^(ecoo!ClCV5&?xeAZml9? zh*6L2avZwy?_$R}FT@CdzV__-7xsK&0Ont)uIIVjC)s`c`tie|Lye|s{J$EiaLWwS+l zL5aRh=I1y!l0&~C{^do!doU>q zSGs5l{d*}-LA%f9ySyk+{xJ?!+PasvyudNI7Du&b%!k}v=^?4wtnVEeiAt1>3YQh$ zTSYZ@O!hqllJl$n)3o!^51O+4VNSFz7Uw6>RF-ZzS}*Kcwbqj17J1`mtVzbL%~8a` z3L#k)+!`+PIgPMEQ;L5fs0^`cxxJBl7 zUK2VD7*0sV6i}VK%&8>xFGQ!7zYhG5Uj04Df7SBv6HI{s@4a3mr2H3F4v~vU3OGyE zrB1ScDiJ-1BtGeyT>FX6@(Jth-Zf|sfJ7b!pGl#5Xk|a%jXhq z#y`RCJ^(w0vmuH9a|8pqRBC-GToBxY~ zK!ErE6CUVV=Y+>`VGK&^wloAQJ2NC;vB}IXjhQd())~cfF0?VX**i(bH56XL8l zYi0n^V5Yr^84HeSF?+&Tc_L$uqtuvohFaLe-So|(;EVgNk`sn&Ba45Bz9Nq?r19m$ znN{dmfCmtLC=jojwaU$H0j?pg6{;+xmW`j*i-SgR+hMqPR6n#lgY zekpW01T-ARgO+P1R2OJUS(#I{Zhhgt@M)#?fXi8Dp+e{!(@*0-A)Q<>(edvvxR$na z-6~sjd1Yv9-?nrr9MT=m1-aQfVmrhd1xoa@pdWSar88*qG?Mr##NssS4ESF;tLgTP zbXU-O)Qw4^`PRH*4VM47_6?`&?&iEI;I|FQxFluE(EHlYZJo{_;~(GfvPJc6MDqKrqB#VEK#n7aEU8 zu=>x6(aXWjJES9yizU;ElP#f1;{PN8vU&^tna`DA4hq$jb_ngngQj(MKMH-5@?OmF~t7TqQy8(kx zXQkRKklTqL(B1&_HV zae;X3x?E<%8)vP3>WuVUw?XCDGJWY%i#e6<^!FB2H=4*$eN6w-c|Bkq>vaaGhFsP? za&A*`RA+bT)>K4~kh7c8J?`5VWj^>hlSi$w}$5-zq^X3mQ$qy$l@hgDucR5+#(+Kr)Fe?cjmdm z)J%qbi;_-@nnjTpXp9^1a% z67BP|kJ)C`y{^NRYM@m1dEceKs2HYlkm~y3QGiZu6GX*^EU0Uzm7 z#bX;dPXS`DL&YGAMLGS8HfYD_5Q10Gyv6a&lq|QqFeXr8{da~P&eFM#11nCba`XBz zgR(*fwH=EzS1LLs(6T(11PC(~Wh&dRj|Yg*iiW$b;C4!CpwHL)$6O?@RVqxK{%5?1 zkrWEvVe={IN+OZ4Mx)saKEIpt*oxINea;OTr{aT+& z5~_3CND!_zV78Adw5h9Xfjaf@F{ych$$Z3;C|%Q_ zetXV$R8VK`gHIneO#8glNd$!IodY1W=Fs7!UAnhEq$p6GT2{vIqddHhEG(*EMbUrn zXX-W6zzGdGMSArW0y-7_N?4NMg2`4Y(Mmq(XP|2H-vEVI4t~Qch}9lBxkd^*&4g%7 zbQg?mX?$G_2o?F?%pNB3jB38ILWq*-24i<){wYxMHum;-v;dCVoPKjX@uL0iQllch zOrsw;k8Z_RyhdLS4Ve!&6U-}W%*)pYA$eNDR&Jei(*O(Y@8}M5|VVZ zdqbX*{KsBV%Pta7Jzno~%cGF0CS2#jL}P#)AZWjgUEJg|>5eB7_6xpkxXT2((sjMv z7YI`Rf=s4QB16JIQa$fZufa@i7CBi<}@UcCRCaXCWZP606R)BxX2Xbp~Ubg#x3CmL#ilPZZx z42Yd)&p1J8Uz)Jf5UpS9gpq?@P|znuv&Nax8&*j zL4o}4cR84%noO!5J`3~_Qa zZ7js*o!r#V6Wa@xvKEwV_^x>S-_gB|k(z{k0}KAllkd!l`H1{JTXx~>BSzISWv=3P zPyhZHzoN(s8za#F34MyDi17%b`qwT8qiS75kGsDV;vH$-gVw*wS4QvZ{)3 zo>X7zY!|Q}3*;$xbe~{|`bnk4O*2YvGz1r>kCo2roKO`&1%5__a@?F`un_b_9OIU3 zIOmC_lGjGZ)J~UcXF(Fc8h$dSPRO!ynspV&12q>wu3m z%d&@?bz$-`ZWLxiln}WLe^2LEWLb#b2MBWVn!&<;s6Xo9+@Enu0zObmnPkp7H3Frc@X=RBwWc>VCVb6`hc8#(W*znp#4Q8P!QLI**Z{v|#Qs9jz}Yh%L*zuIYp z0Z&bgYJmRj$=-+ob+f-h@w)ma_Ueb1efKioVD%jy&iPaPHgJ;HEINaZkhEN6jR;MP zApIp){z;P)(R~7-P7F3QfZa<13vE>aU~bc^Qb80kM*g-0F!=(3obGO0ycyD{?qn~yb!#ihi2lNRS!;oRP9L>6 z^6EgVy5GpwYADBcFe&4fJ{`CXCq1EhodhVUz zZ=hC3r+nV0zSK*8!XRQ3z!qxomGh}c0{98OX-X#ZA3Fe@{z*VArv|MopX&fL*50Pk zib)Rrn&|N#BwqAA5Q#NLL5BMM{J$XMMlA4D?ws6sUnRg#MkId9N}U)6j}(M%@NaGN zcVbIWD`TDUbg|@0)X7~H(e(g!XXn|`juB3M2x5fp6n_=-Z#aW}tu>kz$9N+K)UtMB z=CF_TXwb*6cD9c10yrpF;Df?81=^D&>KjbZ_7TTaJJS(^mGI!wqVrGf-v4byqO!n> zexyfviRhzK2vW6NZzAg}Qykn?)!#{Za8fYp8z|RAC%gCxgADJ6jL2#6)W1yJwh{>6 z9sniA`>GtfeMdzrt!Tvj!U<&s5ey9w{v7@pKk807po$3ZOr?sJza2QnSyZN;{LJ_~ zlo8Z7ZJ7TWPZGE=b?+M{Plj*+q?CfV)r`cA;-tYzq7M6q%bOT$fRmkr%HLsXC$*;> zNH*yg6;hOoayh)E+b2)(=f4Hoa}HBcMP0la8%BIKG4teHj+@Vmmd}Dtl=7UO=O+ir z074I48|Kp>1O2s9gx4;dgqT4MfD089SsM>7-N1q<(8V2{di||DfB+y!ce!0oBaa82 zV@5AkCW%)Fzw_5lb?WF^BQK2-Qp=x0{ft{Go0yt5`~Zc`XtPrwQl;B&i*mwcvmaM? zcKQt}q6J;CAYb0oVpxcQ2^gvk21uc9*>$X@&C8Ug7Y6!7(@;tS*`9$Zua zUB6^+zxHPJyIY0eNXhP!5VqZQjfnASF&?3?{rwEA1VP3Cc;3h_V}-60|Gebv)To2- zxz@>8^ArQ3P#b?BWG5dw$Mdy&$mKG#ptL?g-#Q4rZd~SX$4^c2R=16zoPa4vWI9tY zV|IA(6RThmqpjVg6q~`rpQ+1ICt?;u>S4@ib2e;2=?W^p>vC^_>zN|Z#vziaW`ccs zaQ~nUEcP{~UVEQti|!n^qx9M_0$?`; z?}N!~#$NkHas?}cQSv|b{0t#Umd&>P@UmcXBID)3^c69+_jYGZ+FQMb%xYT&8mO@P8NpuRo#n!u26$7<$Kgt_Q~biCgtZqq~->P^J0I$Xa7*-MPGtQY%&Tm zpFYX=K`E>HBi)XsiS8{Us-XSxS9?7@>myu#RQ%U=c%Q-QIL%v=dq-BPJX9>Cb#`h5 zS+B)v{E7nZ;gbpmN6p`Jug1m(B7?c$pRzk_40!v{_le;5nGXYU1&tl9Yr1?JxeNoT zX_-2K+SW6av)@i%RZ+>2KDLFe zZ3jz|2(PpynCvyQz2Erl$_G`z61wPTO+O&H@j#@DF5U$-y-q`Pgpi4B`0dJD@HKBS zSdv@C;3G6MTbvoTBYcZZ1a=}r%z{HCQhhJ;EFOl;VK7#K&LIZZDc^T*_71!D4aTpc zGOF<_Of!}P4tCwQOgr>WtfTiSFi#r2!Gn34QUA;}+K9QPE8(d3G6j8*dbRZ6@4;k9 zE{RY=TfcXUr zcQb1f@Bzq}+)em+Ae}b4lQdbxsKed9M`%35G3IKesTFvuA;-`YOA9pdW41`FF1m|o zcKBF^Zdcb86u=o){{ge@&H@pi-Ejtg@ zLH?fZ(~d`I2tG?$gO&hu=NqiMBhAfD=TOodAl(A0BaPA$){=WR%%dPZu*dvhzJyTy zm;xh5g$_AwA-nqS)vu*8TN;PwW>C6gMh)INa|c#Wa1NF1Ifpu^I&oY-M7we9a(Lb8 z?%t7xCl}O93>Ep1Lw1*vEY*L(RHrrA%tS(dG$;k$%&iCRNrVXZc&!7!E7F5;RfhE4 z{_a08y1v)M0^3-w5Y9rHn)kkIY z1Eqth{$vH_v2%gKK)~s8vk-K6N#F{u$~*oD6nkiAXE)+GZ8K4zZIZ8Dq5l;&1+3F- z<($X{YA&(T=m-Hp6*KcBt&hNqmyY_y!0RlB^n8+o$%jCq|O zTuIRb`sjzPM{N2N{K$*ibNY-cnxU+C$&80_pdj;MSPDuF?aqC!ChMy>YECeV;UFQG zf$@9?sXgZ8vpx0~emOVNe32nWRsGf3uQ~cV)sui@$(2Hgzr;L89PjV%e{i`;3OUX| z7y&gw8t5qNpQawgBNYeH_Ca*$yI;7(G`E$8Pb4<-j=fj=O4QXHa zWrkllpoKBCvBG1gaRuI1?u;N3K_?jth2zY}_id;{k%NFwd}!0n#eKj<=Q*yy7|TJ^ z-t%u1s>zg(%T9abK%khmT1flwttUIM%u;sNEHWjnZ-x-E!^Q2S`#DE4QAwCj^N^gq!O#?L_k%ib$o#x zkkO)H%Qjo5bICGzp3~SS-aKcz__d&WS79VTDTMV19_mT#XY*_&idcuj7R%ify)UD) zZD(=KTF6EQHTVzkT?NV`)_|2wjIgNWb(y*qBY^;n zV{TwNT8TkRh^aPgKRFKY$_6xUX8GF4iSv^c7V&RBDd)Pt(q&90Vcc@0Wz0@^8cmL| zB?n*?n}I2`o&@h15_XbOFM4d{_-t*;x(&hCQzj>rogrg=2<*TEJw-PY#)4pQn(qmJ ze?{2MoD><<&O05s3dMD%MqN~)W&i#4dwWFTkh#f7sm=S1t-GMT&S3dR6RI07w|So? zTM#&}m&r1u5CJz_Ix(^jW* z#G-P3HV96F^#n0UUDKe0lJ%TQ*Qu(^Wg}JweRmaOpocTBUleZee(#>H69`m?2y+h; zA0q8<1|6gP6jn2o?`@+|s5=wx_otg^X8lD>)6X3e(<^2k6t=3|52{i3gk&{qldx%q z(UzdBecmN*ZTm_cIr?cFYjcXrQD)pPY^WLFhcNpcvmJQpLH+Kp=}~Dga}>7a|1^V% z{MyNe16rRYtNi8&nC~p8W5}XTnk~Hrp+xC5@|5l6(5muJ_M6{5jI0V8^3gyY$K(VP zecZ1nA_!Fwlp~|$`e;P(EiC#IOG2BV9PMJHss-**W(m?{>F^s@D-gXUvwJ#QX0X!N zMZ)2gcD3HCgK>sL*#+Uh((fCjP=5dyYXUu-1q-!Sv?Y#R4@eT~eMpiP7kz1EqWHL? zth*&kya^ev86-nnwO z?kDV`0tR69Y)e{ZLXHaki zcd+=?N*a3wmDzf}Q!dN{ne7no_V$6*v$Fl%HIkOea;aIBWUpF{D(iN^6;}B3y-o05 zeLx^_VHu$F7F;oBgi$b^UtB$;!^6K&w!x;x-jDK0Hch}MS#{K^c#$+9XAsAfu(BUfRKTbio!LQyP$6jV@_qfv&S_g=1lOt!aOK7hFf9d{pr^ zHi25|^YwU)F86XKTERopkIc&H-(O+qELrpVtd>;PT05v+)30%0Au7<*hO*Lb>w^RD z7eQTpayGGk-%SLw4*f`LVyS^Fm^iNt!Kt6l={qV@k24?!hox#`8fZV2;4Q%o6UA$S z+p7jXJ#z&S*AT~k9NfF+47NRr>NG20^r4{t*epbML@12xKpO$Q-R{r>HQCT=d_sYW z*|+B>rPPzG9FJpmV>IwGG4?06e5gnae!BSu)0dub+JjjoV@nU1nL3W8Sg{~Tb{D&> z`_@?gGp;6TTsUGLgN$(I7FHU5y5Gkm+Du*S8?RswM)~)!5AQF!Ogh;mm8l45|6yOc z)1a~VpcvL!e8f!q)utTUGMd5h6 z*PI?aLU#Y^w1CKHZTp{;?=ME7y!_9(3xA@gDBvb+o>WBnRd<4F1Rp1l(_vmkd}$NT zLm9R{=wb>hYotU-A~*dQOj<$~i5D`=k8i~EMx}xCo5mE-4D_gXP7HuchCDvJwkMy8GXrWIoDwCic?x6g=SgS{3+ut;%61*n( z^D`+UDbfe#f5~E||V8=c-whE|cQ4^7BgnQR)c|!Lwg_?36GO zooAQ8u2a0+P86s1tBVOb1z0ogv9Fi0=Zi%IXp5~LA}oFN77C6_22aq9D)!wuAstHt z%cU10dW1D|naoCqWqz^du}xt9GG}b<(QB{jQojNNWq( zqjSFFab5sYTCb6FrDoYS{=&Rfs zTJ?+BquJY*MOJ^m`RJjE*oRfiNurBKP5i++-+t&DCDcxt@97nm8NG40 zV%Lggs_=*IH#rc`u>5t z&Q!hoTW$%I+%IfY(QDZ{OPmlifDPP)mpP!Riv1zPGauNFHPstMlt-!><3P93JKx|d@R}dd!jooDTyF3zTR%*31F$)ncPIBWy9K_uym9+&0G8?R)^OE44<+d-X z#DZU7ck^J!<@KO@M$*z8L0s%(`-n-9@7r^_!G9fS(!~Poqa9x7B8I3msveN?hjHV5 z&8%Kxj|`07SY>|i7_&7Ml>c-g@lT@X9^i&4N}~jEeA7`7o+i3J+JUYZG)navaY<<`Jy&qma+!=3opzP=zj5^ESxJ9NE1XXh*7VXOO!(@Ks-Ki9>2 z>@s!!^t`&pV1w^j<&i)GNkAdD?@Bry)hVOHqmPp$x5rb;lHO@^=5+^Vxgr*=H@q8J z{%}|4wZSs|G+YOKGDhmO?n0362=AR|hYKz53I$5-YSNzWr6$I0W0NGOm3!N4I6q;R zqUZEcd{r$AOyi*Rq}zEw#=W%ZbhSMq6mLQ~;=~egD;>1Scnsk&=qT*pp4)Jf`=y+^ zo~XIM$|Hy3lgvcEeb@-YQjA|*V>TPOJudU?DC3Rg!dQ>urm2|N++&FQvsCW;mtR#JSdN(!|_g=t5D3w6Nx;$e(ys36#M_u$)fb6gr0wJS(ehdcOaG`wLe6 zR)Qr14r+2$biX`^ek(m|454&#ub6#;g{&n5fF= zKVd{Ly{Z-_Ml-!G$TU7B9QLckGGNFG*(&q>aF4g>yZ?+pxO(iKNiVAa`6eS}B+lpprdV~%J0ff4f+~v@A$6B=anY+3*r%k>i zl$C!gMm{Pz;uq5yYhms*3U^nUoPT+6K{ME=1VZ~+e5WM+iz9K}iyhjPoEhpvE3dPD z1=m(!bndEC)>W@iMVKCp;2ZAYhb5Y3mS^h{R^62E92b*knFBTVk}OzREsXLbkAj~Y z(m*w8F=lU|T!JGdEVGZ!R-N&g{qE-@5v*7Uz7v9|a;YRfbI2nt9QEh!og+et@ z4ip1YqMx#MXgjaaJ6z|2S4-m(Rn+$rkmW3b+0R{mE2Kynh{_8!>eh;OQ8DIDr@_VF zZHCq2IJ*N7!z48)Gnb!JsDQ_)vV;?yT@L7-J=ze^t-%DwzWe%~J~k({ydcUmTyRf0 zT8{HY)tfm{V7uC-;`jmbVq(5%nPxY z9}E)O^8*$^tcV$WBlu`Huz$DU$%3*FT=ucu!$Sqs)qr+I#dYtN{{Zospvmj~DY}1hZN+@$&-v7gq zVx|(J;_{N5Bc~ujO*nIUumxAvS45gJW+aLIri~|vNl8~rCwdg-ih3iX!hJhz?3e63lx_XY?F5Gh4U*y?VI^M0=anr!un`4*B z>F%65{7K8Xe4h)ULs1Vg%eOF7(675GUOt76P3;c;)ypK}ULZ<&k@nkbbhH{;XrQo7 z`u+G*G`Fit?mBK%qymReD%nKb`~DAK?;Vfz`^ArQ z8&6e$_&k;^0! zE?~{B#vyvSIv96vb#Az~$N6B!i8@4wyLM1XZAJ3Ni9e4_xUZRFvv_@HCz31pXG0Z_ z##ZLEuvm#7?tix-%ig^98flwcj8+XY6L^=rj%jnaUCU>81Eb1nS*Qb{%Ui5aV zh>Q#sAL9`S4Ze(dX+|UxI1cSmlT=ECAv47pgYHY_g__!h&#@OUWJ-CD@NtrKB_-uL zDtoA6m0!C#{h(msN+NDW2|F~9c#BoX9mNnzmn2ROWU9`_|5;E*)l+^a9lrB8sj)Q1 z9qn^SoxIrUWG!t6>q~_5c;=0N2jX;>oL~-UEWiM=Qy!nn^E-*9{zxp@z*yJ@(pPiQsXklg?Kw zNVsR>4Hfd<*JNg>V3@(~Z`eucuw)8jH-(peAv*8}OdJb$4}Rg}{dsAQt6ix3WK~wC zdNLu&yY>yH({COAd%5E=%RPEyHwbw=)IVwkAA3`6ZQ}v6#0(|HY}(sTY>Px~$coke z>|5BpsN)pIWebUEj!xWL zL5DlaiNYF_b-f-@M*WGZy1G`weITj0c-H=8VYq*PS}ETP>`r~dvf9dCEw64`ST#}f z`i{H%&bu!)f?9X52cJ;%s_J0@?J0iq&*psFJJ2S!30fjD4-E~OlXQ1?^Ek~Y(%yU= z+Zqa;8f>Yqy9_WmIRXl%Ty&Sxo;h>1#Js&&Ynt8MlO~t$oyI9WX^N z(k^4y%q;<3kU!5{aYhcQzrVl1Q&i3w z8m*+q4V9IZ@}==l@(pS<5sypABErMO*q5CZeq#5v2(oG*WE6VP;2DTxk!Jtu|)oAMatd7{Z3BD5yC-V>%D@LS7cx( zFEDpO7q#j{2^&HRK^Cw>QQK>4Yo^e4vnSs`WPY$9x%HlRsZoeG^hQq&dN{Axe=kis zP9OE|-Mh?uVpaBITAN7k@%i2?gW)Lx#OB#+Yh7xyG!-dT$wN9aF?9t`}4Ry0Pf0q+6TU&s$=`1`piZlcP0MzgYHzR zR8}IsLXU+$R-J0+_g=g6S(Mb&GupNC1O@RQ%N&a;%zO*HYQ>Uwg4DhHyyo66ElCB( zgk*VprQ5{w`M%qDSJSHc$BwKj;3wK|ehf1xzP$5{_Q-Sb_%uV>o~$hIpl{3`(@B46 z?Ab0jTe#7n?tC*_EWaRRBPMDn`ogWoweK$(;-6=mcf4h-^Vk$d@h5)B!B-$IiqJc;V-Q?G)8u<4@4ixI~z zx=CF0-FI03T7TeXFs_xS%Vbx#8LjWLGws;S6jSN#fzr%NFD&f3XGD-qx6OI&n*ER7 zH&^qAin#W6kP>x7PxUn4>MA135dNIF4^=Fo7(`L!ZAH^aakwd!lb*zpoICNg z`LCy%kitBb?Pzs>n;JczWvIVxb2w9K$sLQf*h^Z5?iq2-)lZgm8-G?mo^wCiZWK5D z_fvr5Oe`ap64f3kL^dgK{3vuy&ec|Eh7oh1~i1`I&By zy~)i-j~=Do9)$F@JQSKeeo<^Qa&xHOyXFLgSgo>vYiH6ejM)na3W~UR%cDtP+-p)$ z{Mt3y7v$&7#h~o#D7vjpv1;NenW9lJgE&{b8Y2sf4AemiHww`a1_uO$#Ta@tr&GQP z385jMutH--OPD98@th}l8-!E{M7B9S3UE4&`^p8BU6J}uU~!HP2HVTkjW3TcHHF$$ zWVEjObXp*H{cd$A|UQBh`36QJI(-g4FHpJ$}OHQ7}v{9knj9k*8> z@J^}|WUAcSjBK5cx6`mlfAQi)ygn+~_pr(`rQgtbW&vtU_1DU*k_IsCic-l;ZtYA( zFbm{np01pZ)?0}5K>Lu7ozB)o3!!!I1R|{x1^+-cJVf-Fx$bvWinP&;3WFE;4}5n& z9`J0}o};cUo*GT>{bSX)%aKfoT&P(MYig2(hUE2e^wq8_hP2{7wL=3V_jPsI_I|Cz zgk6yt*s*;>%PYO1q1PRU@;z8IViFP2b98c&^3SAm%gcd2SJKp6`j%^ObO;i=vwfg1C3OeQGvBL}Yg}+U5f)ElZy!!sUYLUwmnK*$IQfiU{B8JVxm3Y_ zz-SQ+-DQbzrP3n5-*AxdnzqXv3eZevC!kQO$rruMY>iI zb;4z#ax~=*`g8A3Z7Dd&y`SVgkNquD;{I{B(aTyNU!ID&B>AH0%bWA<9GdL~)gGHx zkA{jfbUw+`6fg7zeA-gl$@o5Oc$wnNqs3Q;Ec%c2<~cjJgs)J3ZBr1I5%>%7I*;#r z$c(@+F9!skeOcSUJ~^@t&4@yU^p*LxZrEjO9B!xo$Gox6P1|ANoU$0 ziLVgD#b3~oD3V~j`0Df(#fh?ulQU-sy(Ae0Td0k0X^2IKbLb)Y2Lxb76YiFN@(9iZ zfe2bEk(Lq$hU+qh2R}j;WdkF!We?H%0f8a7b!bA|hGtjpss;0TWOtvxcp?95dDQG% zrGq1gQUYf5Mr&LPpqZ~Sl&dq>xUNJak$3?iAu$ui1+Qs5_oFWg+M`Q{G8AY+DwO=G zsRY+@InnVtWAo*%Uk`=5TTBR2YH&-6+(Xs!4PAhFh)*}KE-bU#80x?+Ohhef8w#~yEZ>sQ$a}V zVc?;oGA2HJCq*BiiWe1j3D6;gmKX7t7GwS#4jkvo-_ReURrE>@)#goMO@$8CNAGHsuD3t8D$a`g7l zrnwoaJEx)CI_|EFOpteAzQAO!0AYs?<5wZ{?)@C3lWin1?sMX=P*<)8=2M4}}mdEk`{!Q0cgdx3gduwaNS$0iX zE2|uxXVzf6dy35z+@appZ1T&SlPE1CZw(*l1T5R&b8unrs|1>xP~W_e(s(n-vx5<5 z9Z1@0OC!9Bem=vUfffG7slYIs?!V}i&H!yqkV`~F`%)4Xxju!vuP$_6 zi)?$#yH=p_p2u3v!k1QT@-7yBai3ltzAA-rCgnCGEw5>L)8TCM*iT5r-SRAd(EAp( zHIcI{2tD80XVYX_ovynFaE?l(d?#x9FueLjA0M}8H5so8s@m4&Mx^zX?%Ih8_V$Hx&fp z!TABPo(tDR@3SdHGYHpEdgAt;AMqKup0%7@0H5_ho7o(lZ2!Nap8XjJ28vYyu&cWx z^cJ<)W3DPE;VTe(D_LfS49FHbQQqDi0PZtJC7{}W5S9Gn7MV}3bq}I-u-Jrrg4#jGvgcs_1oG8aw_i}6U z1;@w#A;S0~k%Jw^?d|PRR4gqFD-N!!m}m^TWmM`VdXQ|tKz5kia;o*>_^fmJQh5qH z(ceqn_e)@Lj+|Z$R=DTz{)h0}E2xUh)~3U}19krg*pDIc5VK~R(DFOs@kn`|GV%Q^ zV_(&D$0y*+E37hCpVr|1TjjDeL1M&za40b@YAww1HCdBIS}NA|@p8zw(L ze3!=o3%KP&dSaIEmeb5@!ehr;LoZ3D1JbsAlj-Xl{=68}Wr!esjX0Y8Fgje#?($fCM znAqq{*BSgRwL8du&rr#;I9c42!Gb2aei6GII1(4iwLX&J%usRTX#>iPI z9+Bl6ikH_-QWg}5axi7=FHhh-z~Mtt=7};^#9Nu{D(#mLO&GGWvaZ31foABuVs@~* zNy)Spa3gxcDvsi|*OV5II51t2)zZx+&j5F8%Dg}TzjRBvZ6=DnkEgLZH_O%u&| z6)?-oc`K^3K@JG)*V@iQY5{++knjBC>ph>t_2gFYl4j6hqFhw|_8RouXfkjeWjcZe zB{{;+y-Bvq0qT119r6fcZCQ4Ogqvlk{zHYc#@q!3f*cubCS^{mIQIA=WuZQ@U(7bK zpYHVOOYI2)0yTj_LDQcCsJ1@VUYQWbct{{u#eXG+;SjqXLTlK(Ncsg)T=h$psaD^c z$Z8k%M72e0g|uJA#T7207xXG(yD>*dT4GWbV;kTQSG~fZ%#F=| zgScCIG88hr_qMniJ2yYptkknx59Ef)FnDk1CxpXa$7umV?X1fCVwrMjZ&7aWMLFi| z!tbxbfS62L-xw}~DQL*S`$40ENl7`SN+Ed^sQ~e=kXKlF&aMWuj$z1_E=KEaop{_>}=5+gmH17$uZ%Hb!!+IjM zNGx@c@lR1(%2u^BQU%;}EThy>Le&2pmhkOPYbvl z#rS8#yWhXf0AOq{tFRmA`uzE`xk0+fO6`5jkaII%mmdP7&Ji3s#h*4RTowm@Tl>NX zA1|~g2{}o_!{3Uixyj3WU$@MPNq9)D6)<2MhoOvF7P-yd8-cM<_>>Bkmf7<`10jKd zjye6?37F5B(1+40_(JnNkVIRF5xte{kise8@F*#7mZ|)x^(<*%>F~8VZrpkAN?{?}E#v>AIFR$~>r1fue7Wcd6){$Fdi`62r!9|4llh6X zUA|`l3D5$;o+9%8Lw&kf@{=c{yVGQXh-u2w)6+4O(!->Q@?Q#0Rl+KQedT~*&DV;9 z&)vAgRXulEr_Q5{H08zf=W$yY{aDQ>N9W~9`Zk4;zC$lyXnCv$ zI09b3j)cOSIOAO6=Obm-(7TImZ)+(6LkM9^Z50f*HU&eJX+ti31mjV{(dbOib_a$&;_=$H*Qs*M`1WMPhLVcv z;jn!LgYd``pz_Q)=yLme3XH_p4M$HIyj|EDv5SHJ<#B9ZeEt=|^#X{Oo`Bf&2Cox; zQ;r#gt~^hEXv-YAXfDINm!sH}ybj=`ELF?Fntm}Ns3Rag5n5R(h>4!A3JDFhb>uD7 ze)&4+{BHEiIr>2fv=N~!)dVIdj02jth%{|Qzi-94ykC5hrv5vI$<~lyc)-oU5m{VZ zEZpGyBE3dxabT9tBY3MIa|pF1HaEFUXPlQm_qtv3<;ggSY_2uK``t3%mp*9ZW!*L; ze*)Q>)m&$Sttard5M&!%_kn3s=IyBv?2Vif8M9t|No);7gYDBQBk#>9xziJZRq^CY z1-B}5G$<)46_a^+crIAXDH*l)m8i6XY08} zo(XSuI!+9H>6_6d;fs_AwIN2;ohj*xN_^^yg$fdAF8hf&6_vfOu6?V9G1dHhA1`5k zR5 zApL1Zjf5{2nMqO@Ly5?Ja(wJBFZ<`!_9dZzmsO}O%w+E;*5R+raVfRE9}TX`knfrs z)DrdAoUgWwnsM&m&*+bW0oq>)Ir$k-TuB+Xm1a{4Y7V+e5!Ho*z;`MdYtxMM7|+Gt zc$ib;oVVw)m6s{A{j1@KQ<&NS?+0{hk67p-u&mc2rp2K?mUu+Kv+Mk$GhK~A>_ z>Fxm1%n{ON#T_&}1Q)`9@^R49YMlkO!FJ~NzY$Is2_in(33zhz(>Z{PwhC5D?4MWeZk*rz>E_v8n_Z2J?l{a9W$!nXp?TJGA#YIM>CHFC zR?cGIBOl&`exF*g14{@#bmruhHcubyQYYT?oIvAO?XR^?!`SAUT1Xa}|IC3tf(66O z{L^>5ylQnwP{Z;{ZV=Nez3R+3y&8)_GSYIA*k_+eK!87N4we+MC0Abjll9GIO)0tW zQpDlyuGjrG4Mx|;$K~nh>vN3z9@WjKb`1aMdw;)dv|k-BM0D+yKFkYdX2=oSN={1R z68L5>88thXwhfc|-ZpD_%xA;|&balz&8wd1e@w!b9TKUt+b0kVbi{CW0~ zavNS=<;|kQ-Du4`-MAqXbRWwmbz6L_?fuB7AEu7ob^d_LX@Qx(k9y8nRwI@|GUWuH z{D^)0weUl_UbgL)wT6wm2LryQN>Ih=ZVf+GtlOZX3v%uQYG$$My~2E`5n%QLQa`;X*+c3{hb&W z`1*oSPxp)UFM(&-uQD-7QFCZd%nj!1l$egyy06V7*u_iamEXTF{$uG88v%K-*eaU9 zJZ4M9KVgfggzZNnF>4aDxUH`dZriMd&V>1mK>+`~r<&OC}qghf;6$53a78VI{C}D-Zs?nB%y!{~`dV>^s zf=pn4E2BFK4ZvY4boBFE55#)YwE<75@+MeEM@LOn)tI`%aXI9`@iiY95VZo99oP{$koBnJH&R@Eh)+&8Y+=DBIo;{gsQ8=O&Ypk+m<=i#%xp&bqA}5OhKAK(9K!ZjQA= zh?I;>LX54ay@BAf)`JV%5ugVP0 zGw%j#;kx_dD--uI*_`(y&BnjKHt-e>MES);`b+o&DuWC$dfbawPd_l+L8(ptW6%HF zlQ7eC>1RQNY`x~=R!k7@y9vYU{hO&!R4n|Peq1U2P~OaDG3K#0Ox_>Obl6|$z4xS> zaWP#kBFFh0@V8g5UTuPz7o>%nte=#VY3!9xtG=CI|7eE98B`<-QFwMC#XmvH3rj58P%&I@~)G#e@v0OGTa~a?VG-k z!@qZ3UJivI?Z^bPNFZpp8@HCmAjIE{-DUgd_a!!g9lqJbZ+QlL7QF7GDe(s&$tWu! zffR-Mc*m)x(=j4pF)=LAz%mTdjPnC|39UE87$*b}T)f%_>oj|B@Cl@_Bq(9WZR0y? zZ=XGnJn~2nGO`(MYd2O+)t+Sl=>q@BAY6=#lz4bt@qE$*Nn!q2JQrK30_DZO-&BR6@;l2uZ;X~I>+8fxA#!Wv($38{>CLb`F#bwh$hC)vY|BCw5DGTo+T|jQ)2vJ7d zjfXGVl5Xkk{#se#d5uO(%zvvYnd8ucGyyxsw_=kyrgV~Qv@wper^rq~Iw3bjMotWX zad)vs4&&Ut7Df+{nUN7kn*&HTZ)=7S;`!URZ4_x{MF)!_ZC)qDfzt6e2 zN3r?{6D@Bnnb~9S-&Ki-c&V{(TeM#@$;H}2SHYrBjy-*~2AI?L>C@eSSFb{|l6`a` z?A2Fv8CbyRiXcPMQ{!qE^UzBIT|gN7VL_`R;R6oOe?0#B$GC3%%VTjt7v#7@l{{rX z^cDUKLVV0JBVCAEVj4Hr4h8JSzD*f=uQjtwL-Gv{+BsG(uIQch*@SPoZ~#;12|C1& zyto-iM- z`*^#T2fL@XgCtC9Wao#9s(#2fhw(C*74z&o)924gY&emOi-3>`i0D97D^(XTotPx} zP^ClK!&gA0I?kN%ReFLf{GN=stsIhjmdbdulN&z`r?OEY6YMlOa6WKQ5m52OJs|-i znzS$z!f22(Ouc(~F?=@3Q~B`lSA)oQh0O>v44X`lqy4Q(frBgRU-k?S_1_1V{K|E6 zfZZ6+?pe@~b+=F)JD zW}pJhU@9_?=hox-@rH)hrC$A9APlGzr$OmpNAwiJwTfR!ez-h<|8aeu^fzhV?H_g3 zbZ-oObU@;CQ>%IyRY-47Puq!pK>V7XX-}U%O;1Tl8J@f#fu1FTDd+*n}p#h?6WKjRJ`)TvAz66PZ0#BUTSY^>)6^)Kr^2c z^JgEnf9SG?&5Ld%Y6@W zDZWCRv67-;xlf3`3 zNrD1?0sibggkB|^kyF%P4`Y@7KrlAHo*$Fp2j4oc+6@_{Pp#ezOZ8gsu^G*3EDg?T z#ZaY?`6`f!zvZ$J2_lZhD>mJ-lceD6hJnV6@N@x>=QwEK+6*-PHDIlHe1Tb6IXbk{ zDKdF(xr+pbvf<}cB_(3)n#4s-yIS@QQk=h+NU@U)s7`pAnS2c1Sk7q><4+a&H@XzJ*Q-f-^RUwQha)&~?w3WkP;-qrsI zDS@mB4>fFww+X_to43`KNK>YvsZDrjC}YLAPfZ&zn?ELxA$&MNWPQWye?-c!)F@sRn&JispuDH{H;#(F6fsGIzXXiO#-uFxLwZERR(*QFepLCN=Ip-&FRby-W_ z11XQ6Uv$8!WX{funE2G}9Xqv=C5g!}E<<~C_ME_V&7?ichREoE$D_%dEG5T*nC$bt zeNG5&2s*KNr~c7Ymya(-r))Q+^9Hk27`kboYgP2|LgC&84oyUE$K}UYjHxhS?aoQf zdYt4030TcU_gNFstyHtgd=5xD>S%fUs^!!7jzz?zV1#Pu0-`m|eS_omIwpEGRS(wn zt(6DvH!!G+FQyoio_7ekt5}H|UKihAl@tQOnc`c)doY!jwj`iJ_k`G+%}e zujW&y#r*rtu9%|?I)Ksh?M*n${GRa3$=AI2nstXUgxU!UH|-4wDOT_KJuS22bF<%k zJOB8D(UOG{zNSI86>w?AX06mz6MbdR@7%d_RZvhHlkBkQh@3?#$;pKQI@~HFPPS%^lgS{XOEor;>&reLk z;KD7BzE7Rs=4GaD?&y~wnUadQOAg$(A-TmLqz5?=6_bY#Gk{y7q@iH}K9XmT!ShE^ z$ss~@y-O$%c?#6w{m1OsBys$`9p$KUf{|t%Z5kah0kv9UIjq20y|UKI9m9G*Ggbog_P`{f&R3LYxa)ul z5{j`7V5TDbjT<+Ly>`~vRJvK}Ejp7ptG5Rl4mY`n%B*8@^{doutLK@W488a6O@IFS zh_H5FPtPZYFSVe1$LC-82-kLVcFiN`I4qEPE=r^dh>^cigAL! z9+#Y%g7zYZq(@^@#fWm8wb#x8>CO<-bOWEjSE)xB*Re|F_bR~YuF)o|Fp3Y%(l=p# zlQp~1nmtD z99F!U%l6#d#3WVj(kdpm_-ovk`C@_N}!-aNz84O!aZsK|v>;M@cYmfXek2hT+ zF_5#4Z6Ka=wtmxepJLS-e-@7&g2i1FdIB2|qD3xO(DFkODaF7 zV=Yq}JYX}Idb=|Rk?djMt$qZ{bE8CupF%g8^6Xi&)?+(_=hAKMfjGcijDag zN^A7Ius=4W61#FeSLHP(_+g836gApPpV@BvdX5Bai?N0ua>BgFU78^}sazi4%yXJap>!Z)I^Px_#%7>H-oF#7rw*22e2GnVRvzMdN#c5pB+62A5o zlNyg#XS|;C=I+5DDwP%Qsqh9BDX9SW=zzC};v%0}H_ zg!yY&WLc?Ah(l7>JktttW)B$wZ$*F{RgUQiS?LPL9fM?9r7F(aVEIzIDzkW21!fy}iBDFcLL8r#cc8`_Mp&g zRDrv$Z|Z+(y3GZdD4K^^r!g_e;y~U^Q2MQbIZq8#`lBgBqmF67c|1ml_R;-rfXF-raQL| zfqsJw#VN>n`R*uueyZ>q`mDmza21HvRl>-onjcAOZI6^Jy2UF|mJx|!LGW~u7-pTQ`H zxQoHGykA;9<$uu`Fxq7`?HW$UkXIZCc)wfX{VNcWNV!dTxUtKn6Q(Z~wE4whX?pg4 zHD8m5`Zl8cGqP zFfz#1^DXa8C&`cDvQ|D*(tAq#B9D5mcD{bANwI2%iJI0%%^u@0wizO=GsJ~ZyJ0L` z8QI^ud}TSa&MO-M*VnUGDIqrHa8Fc`kcP&G!?k+SVh7zJjLSxcw zu+a@BEN?UJXYGPaVEI1ju(}kxQqM z#g|7=U0eE3m(HGBP(i%LsS_zI`TRmBBxna%r0cg2)2lMMp7iG#n5G#dll|U8pcjI9 zs2N5LKK0_NK_OjW99_cL6Ekw;e3#5X!OmQ?0tv>c7WM)@UY*5=tp2f!Dm@g5<( zW;A&yaL?@Hcds0(N>XtqlEh6d1%d}2(D6+NwMUH2PY`}bTI{3k%e6vd4}p(aS%RX$35dIBm|w~wjwUUc@DzZ zTWc#}E)UtJs+Ib^t2b|I^6>Dq-O&Qa>Z)C!QzFg7%WL+v@IkQmV~lHs3bcO}8_bL+ zusan_W8K)@PwfYTfr#iT`3rGoC> zCsn0b^~6^r+&<94!^x*C_5EHI8ds%wmOlQfbJz$FL6Au+`0+?(;)E6Uc3q~J@K`rN z`R-l95BG%t4`NE$vnTcN29X z*XPj8cpLYk+K8g=caDaQGsV12GV=1RQ#DJpJhl`;t2d*?2mgp3Qdlu{Sstqsw?U1i zGS#EN9R*xwY}|WHr3r^&t+uf9oUx&Yzw?Vlj8lNB3+9wzNY7!Sf6LL76vjAy>*%VG zFh}LDN{iA1~06_#)tHCtldr5Bpl0F)% zt(trh{a@8!gxapl-&ipsYs>^49>*!!`&yl&4ZaN+MUmrryMO`+hb#3u9d6vDSS`gQ zr#R(t+ybUD_2XVybc~pg>4HzimHQoMV&qFE0$E1CSv`}-E{(Aee#4Q`-o%7)%tX%E zhsw$|8TY^-_BG}_Oz9T_ag~*|d72q~iz*)zl3240?v-&GC|o(%ZfBK$fcCmnykH2T!%Fu%#trY=iVPpFHjhlqE z%4i<^0SGJg#(a+s=>%vTX>L&6qU3D%9wtFy!_15Xe#^ea)!dsyA2BB{<`?DT4> z#3PB%;{Ot~wlu^f)e|paZwok1owKXr{wuE{{_zX@-&Mn!Ugazn;Sxn&G8nrM8F_|E z-v%;}`Y;wLbp>+sB8yz5R{aaP-Y%o>s-o;aS7k)hEMUp83OP@`9n ze~^X0%hvdZ0LPc1NYU?}|G46GrWBTg07>isfPF!Wmr z8N(HIuIl)`NT6J}&68N=0SU*}~c^5IXHpfUvS zUoAC|GzfU|pHIO?S{i&jYTpu=^LX6yq*~hbqX6Z*@>2Uxi^}RX<96yPl&A-M$l1_4Ru;Si(sY^yRf<8}!H z_+{s9Q;I=%QEdPH4shT!6+N_h8fRS>HC(uWD1U_kWRzf$K1E9Z#TUXPmKQZWkuJB~ zwHFy>H2yf6Tow#!fxxpfo@qF;4~nBw3x)%0Hm$5*PLlNLV5AAG*pTq|E`Gz-g?i}p z==5ohto6WY49R#To9}FMekj{}!UX4D1TmXVGxHiS;<>pT0l~r4?VTVqgmN3^V~Z+# z+EY1{CmC>`%I^gY$%zx8=}DYspy-}4S#&j}G`4Yr9(7JmX>9Q!pa+;Tq~ z3ITfRJYBmtpNJZadH*udDKUQ|x;vLP`)vcbm%B#8TBt}Ef7c8=$24#R74k8+##CDG zvhdHo=aSgO6lr7h8Yk5av~$4j%n%1(5K`l_U;||S81aFV{Tw{@e|i;=@TE%hxSz!z zm@=SsJovix#?6Hb)5l;tS;EjTX|9?-E-qiiF*7SGsD2BA?Vh{RckjBLVF5Q?x)LA_ zI#(%-GZckFg@5OL1~Pnwi*B&7wG?H0D$QEGRrdeqP@jV;%nPbGrr&2Jg+nwbS72CY z3UCOM=vg0pg;DFlT*iMpe}xrYnP7wZe&mJR{F`1l9)*xj&bW{hZN-f_fZyXN_hkM^ zTtqok)zs9?pxBLv&aEFWcVKgX$9>H-PRPkTf>_IicohQpM~J$yg(;hZ;!h_u-#l%A zAUHKh8`A@r`Kj$oj%(Mhf9Qu9l5K4n^#l|I9pws_=rd+S^b{S+J)WC zXnBOc4Za3c-$GM^2s~7j?L}|MoGOr7{%8W;@jMTD7g*Ck!-r6TnyO+jJ2lk={GZ(# znoGM7{i*Wpp0Ld@Uw2=OuGsF}6{<O%ggCU!h`F z314L)VFixFSJ zq(y?(xvA@&WColDF#RUfRt&d1N?~&0m$wPw;U)v;9hG%byhXgW3)5_Dtbq@laX2`A zMY;*2?*|oSbc~VDkfLzk78{KLMSb}qhi<9t#`{$vrQ9qm2u2kPdj?V|s zJQ-6lO5qt1U}3KWnw^l?IVjxx13oyUOCyQ{RB_KK2YLF~ccob{-`XwE=^M?V&peOY@)py=5cXoCP2#W9cGFn>7kWJ^;U2P#pX=vffGxe1Qa+b z;+0evap;vRo}?GP3VKLtZ=RY=N;|x7 zOSIW^k(l!As`|PH=$mkw9@RjD3AdYexP5+-xo>pKOjk4<skU!H_o=eeOu)c&~X_)15loaAIexuia(g~+O!A4c@I zp^=6&~RQ1Q))hxtyT>Z)Wbdsfd>_Rt>Ri)Myz0 z$k0H4?lI<-E}rS4jr07q%pWo{*q2L8f|3{4=@m*6+k(leL0glcdO^ANR#jCoXIYz? znwHu0InyUrxc zyhK8bj1xcZ6&=I(r6K8Q?AmB}!xI+XTP~pYbQN#Kr|q9{JK9N14+hM0j-Q3R`kX^i}j&krePUhv@qFiaZfP5V&mk;lQ=|wa}Ci_Mg=bo-rcl866C&j{-6T%gq zA!U&_9ONCIwl!J>rtag?8G?@4o#FBf$jofV+dYM%&3+Qg>-^czIVqqa4JDRm`;&iOt%dRC>#N5^1TjPjz+$-h0WKB z5dv3dZ-&IG;m8jf2I6zjl%3ZblQV}}_pxuK@MH;c&-+JjD-o?WT?uJ|jvjVjQpi*@ zva(vq1-_|Hvz8gi^x9Y*_LEXE%L0Ois;TxiMND&&H1XXAgZSh!NWP|$InMvH2A(vk zV$1_nHaaHMgFUn#?P`C%81eg;O=R(kNZn0zwe6^I_Yt1Q<^mIxQJTlQ+RQ-lMyioK z0JE=zd?{9l-0~%pk)v!1<7w-^3JShBWF~3hF>Qh6j&OFFGV9`c`v>stpq7LWnZtW) zspQuAhq)d~vuGROAjL2eK&fH=?Y)?W{m=Cy92YV|7Omee=qJt#LuZ)o#%+GLoMbyl zLeL61rl*n5z8TI><8|k6+68rVZ0|-q-Q%*X@xJFX1=R~|?y&Wy0J}tzao2VcYGXJh za6*w#Y8s1Y0}&x6l}~7ZY86=g#%(lqvQX(pxsn}KchqoghNZ+NC{n2E?x~3WeD-*_ zRPJzpR(x={4ss_=8}q%^c@~?;g+~p7lt(fdRrAohhnL+)T28UpP(Kn&8P2(YBRB{t zEVcbUHm*+(h}-bd1e_oXccnQQl_JrT$e$~mhhbpfTuBM}vB;s~{n}6g#BcLb(@sVn z^Gv)E>f`(x62Po~`utg=2B~R~4+Yyc=J^YzpL3GV3~s;8B<-5)z*kT7M(y#3pr{Y@ zds7Wv<|FzStaOPu4TbzQsAdhF+WC8OG}wFP5>vfeIV)n9;(grihgN3#IG*s6K27pO zcSyye@IO6<2;&t~Bopid-kc1J(2>HB#-1l{ z7Rw}pS%lZ1ROckE3l^5(`Mo$(2^3SVtk?stpLLiMG4P@<6H9y2si4a#gI$7@Wt5eb zqhY%SFz*AIg^U1(rcAJR9xnlltzbQsE?v*I_)H}|zVdzCLdoIO?}k3c{P^ubyRAlt zJyxSzHHRKcCp2e8-U%^7=JFbpmhrMj&lf?T5cj!vFBLnE9lVW>cw6W<$%2avW$q>85R4v+Kl^1x@2oK{nwurjQk(2bf19{ z(@+DnNzG8da^Y7I#BSjcAB~XojPTSKFrv@;Q@Evv0-_Uhnnk+n(RD+H-7Lbm^FPas z=`!+P-7h|U`qVU5=P4Djzfd0C)1x*7EEK4D*S_%yp2s($w3j=L{y}0?UG^GB`{%M> zXDl)yUOaWII3qp1?WX%xbTvWhEcV8XKt1A`Szsg@aAb7mtadl%gGVaN+v%Fy+9F}n z${$@BStW;~)Je^TBQZSqh`%pjc&Uq-S)Ee%d3^nuGec{I7iU;S3o36o|Di+toG1zb z=+hfP=1_Ed!e8U03xxc|mfe@T)e#W2Z_A;Dq$F zu|%rVMa@HsM_h!Fk-joWHc?64wH%=c=SfViivJ?Nt*v;$86xMfZ;%* zq|o@<PE3n|e@|fGuQ1_7ifPe)A1# z{eM7;nwXq!YSp0FG+=6W7E@1zrTYO!EDClIBYe0gD2KVH`>&7JHO_i!vHXBg zyKcI^>(n3Mxp3wGy@$>Aw||=J9W(f^7A}oQCyG@Ae1o~1ZS|e)K$}O1__D3hA|4(r zgq1+wK^#}m^ zy0!B~QsZ&Q&2vBbr=;xz*!d_pSv-Z&T_4>7+Z-MTq`SYXM+jMKMm4vIZ4MCV_-slE zdGGwop8^Eqq6FWsEo6J1v)}#>sew2%bA-P3TrV9+RpQ@B4;%XK7m;RuU_ik4N8E8C0H=ic z08*KDMMr0h!5#RV$3;c2f)@uX7)tCTh{63P)478F8NR!P(G@BQ77b%vd^|OOPAn+; zS_H$5WO#_O4gZT zdwolgCbt;0hz^pHf)>_tw)X>sNkvhKsUMw*ylh`ZP!Y5|0q(o<&@+%7MC?ngB9Y^S3 zH~nvdD2em`C5T$1qL#Tw9|_JyES1Rc`wP3OT)yagd!l%@`-XnQ%vWJQF`)RR?)os0 zuy>qBd6gGC7+ShnXCFEo`5$LZJQ9TQP`<|uWN64OCk?>dAW(R5?$FG_V#MU-O+>wF z5Mlzki^n5{!zd}I1-4;!!}KIq$4UAn2U()Ar0D=mCNANAzW}(vD{XO={>q8`=Q^0g%|s9Ze9?Em6;F z)gLrmcW{*GpHw}}Xi8e@(W#0@fXaXXfSPfWs~g===mj?dqG+9^e82v1uHrpM;CnlQ z3PZ)wG5?(ZNV?x$ayIH*LCeuLhk@)V7A!y4d7N}{&vx$&7KF(l;RyoG$sYluj=?Q8 zU4l9)nl2$_B)#$!^tg=#Par-Dh$L3rZs5mp*;mX92tPtwi{KC)oeJbzQzA-jXKH+i zE>MqDFU5uKByBh|G}2fC`Tm5xhifR6>zACI)WKeu=@cPQ$s)CvW4gMZUy06E;#Ap7+@@LA}2=PdHx0!PHJsOHV4&syn3#5ZTsj=#hUw zbjF`c>?Hz=fx7g$MIXp|X_-cE@I88PYgYh;#&PKOc+<_2LG#6%A9K&hL0h!5a3B0s zX#D!LMO!5Hb0BhKK7RZ-TzacBA7S(;kP8pk$tt$%ncJ8d0v$D42k@$R>~t*~3DNFhHD9=yCn; zHgVrHF*fKFd|s7)5^!BXRw4r4gJc>n_@*elPg?1%T2Fhw`JvcZZbz?13p$HW=I5!f zE-}G-UkaM(r^l}idb7XD6KLM5SmtOf?AP5O^~!{9Dg2ol@o} z4(@>dQ=Zxu#ny4XusHf47qFkS!fFQ+U#)xlR*^9C;>C-Irntbw=MI1}NGL;85x_`c z{0~Z&`$ZOM2TaochW168b{3HFitsN{sG*ZoEFV?PDSa+%{F<2qxtO(s_LQYjNnrC? z-O(qpWnKb@cVjplC$J65{nQ-SV|e1EKzIoD5?|D{Iq^AIFQ_4oZCpmUb!mZ$i>1VJ z88=hj+@y&oc9Jzj47uPfBk84zrsi$Cn?^}MDxL*ipd(@c_@&N2u+1Y2q}4hW_0|r( z#z5QV6s-L7be(g~_?YG@>%!f(V%z>JTx!wZiDX}rf*f3-`6nhydeMG?oh3vjaP2zq1p zGYFyeHsHRa0le8lw*=JaHMJ$ZRthglCj1f#5JU5=a$okeBefCgzJwm2@7Om9L9&sR zb0c1_Lv54I&eGv|zwY=*y0MNH`s1YoQ2!nJR}coa?xGy;(iN5x^)*HW|)A<2pILE*r;}}6OXfwdolu_C>)9}u(_g$HKtVd!N=b6tf zQ9XQQ$$6$2YjbgeQPSx7Q6h`^W0`u2gV!!lYr2WF?WH%-3L`QGIG6MD2b@M({T-td z_8Zc``^+s>u=ET)Gi58H7k^lCF`EDE^0{Rk$7;a4&-iIDB_yXZ1i>j0b|(bJ zLj8B3-if}S?VR-z&g3Jxd3z1|gaVzJ^WO|^ekwe5^V8DADu8LgsbhQ>Sj-O^9bVor zzr;(8sjs_J#?#;heM~qNj6rBu!i$dgEgHw3)*p|0JolUq- zGBoTY>kW?{FG~Red`o$JJL7-C4hL8+J2kktCS6^7HM3*AsW5|LQs`YvCl9$%lpZ(^ z^^47Be5JH+SjI`(>P=_e1>@dJ9v8dxE}RA6?bsP}QxO6pQ|epm6@DCn%p0THJ(ry| z8N;i|9=1IsMZJA^_ihzVBth{E2bl^3`_YhndKcI|=PoT6Yp)ND>&#{uRO+~tXUtYE zNP7o>IcHDh_#F+>rfv;cP-2*Z8>{(GCtxa=mwfu9uLo@t#*(g;jJZ4G(#hG`>c-y7 zW+EakpjLO4Umu%9yc3w_y3loy^nOz1s(=^r9s2p=HhFf?Kk)`>zXn4MtIeGk&A~Ws zR3@TK_sb}6{)FgrXJ48Bvb<;F6SA=XGGVU%zUL2cCTOO&zgY?JgIZ*{QYvW06Od5G zlaK9I=o_?{F3izd1z*L-Oqu$DMQpmp6lp_rCh#`h7ca1(kojd|^SI0A4cH!Bxtrmk?n5QN z+@$dVFg$FEE=b)DN1LE0&(-%aQxtP?`njm^n}q=rNK5$px!vucK!*Ck^^JL$y~bLlbc!wRubdapv%l#<+TY!d zy-c}c;}u{jDmBk<%sFD^`=H~F=UVIFo0)Gx6g<=+)bz2tQQ6n<;OS;`g%T^?P=II& zQT3!~D-dSs`?;7p^p-!WuMovZ3rlO;6zbP%&t&8%FXm;}I?bYGsIjsz_P3QySS|8M z#`^l4PvqjFtK>{!cfCi96N$S{O;l~V80K})-C!DmQ^J_nrj7nc?ub-fzI+E*%xs;_ zigbsQC^F1ycbtyx=zc0b+SjVyE4)u)$UQMCtsLBcKK}uq*)1u@{AhElQ;az4z|l7M zB5=kls2+?WM9Xk#&NX6V9UtMIk%1oJ$SO6qbmz^7SfG>z9BiG>6WhdP$#rj)wm;L6 zEsER@0}gfMDS5LMDNjOg0_ygP(qa`~Qgs|!(=Kse83;5E+`E*KEQ%3W?=)rJCC}@u1E$+#Y9{%psgC>g-kfT7-+faA zuW_@2c`c3CjjF|djf?Zs!Zd*s?{=VtS=yh9xT5lMeHfU_vNUmu1D(&_K@}ZImdnI! z#A@tjXzu0`L6SghWA6HASFMG1WKS?dJte?DeBph^h(f8Be`g=C8^M$o$M9LxAO1Kt zq8Bch*M*(oA~orZnRRvErq|VXcQei&9qerDTx6#EPU81#7ersiZ~Jr*ru5!{2v>kr;ms-Z`bD*?OC>Jcu;;EY02$49=S**>SA1ilLz$2z)w z3d(8NzxSol=2x-(Yu}Z04cwGls#h&1_Ax^cUDLtZrEx92=(dQQP6TgN}3MJ?N7 z|MmPgWg2v%&n&*-2X^U{DeSOhHcn0jYd@6bpJIi0pqEP5dX*dXTKcvP%j_Q#lmXE6Hp=$g3bPyc{B z8`oOL%cQ4r#Wj*CY(6OIDiQPU3N4`n(u-=7$}|X)$`su}MjUdGv%;&B84?}Mw56U} zQUT=1Tr*+lvVW1)Nf!*E61h%BXsz|dpFukSCP}Hz2nnZOul47}KNn==Ji<(-yyAAZ zSP3eQhZ@ZKPlAVL5#kYH&WeX=>$g=wfd;Ea@<|T`^KWA#3Q2ueXaFP|s zb=ueW3GE7o8!!2Cj`@F&YMmJyD47jtlJAzPR2G<2sV7(RH24xJ<}+81Y1&uo-z>v*0xgS_dV zqy2S)aV6c)ON-Ahatj<0IJdpQ*zJjsa1R8X%UmOmXbAM}7sBG#k`T(;n368&qrX0% zN-EOJvh6!1?Lb37NqU+vQ!R8bs`Z{)w(##)5<4fKeS`+y;QjCYx|H~9gS4T`Pg-wXj^mB1BnM?e~W^komBMR_-CiaQbh$7 zTsRGZ__6(a58g}^(Ux<87~`gFx^Q4YfbCS>&X?cZy6dKM>k!MVC}$TaN5rkzq}WHa zeLa+MMUvmh@RQ#Q`@f&Gs-@n0SVv^Jr>X4*Ta|RhiFZDw6qbVkKRIa)1J0cc-_>hU zHC8E{vyAiV9&0?ZdOFMRnOe3~_Xp( z;VPwkq4cy9flV9Y;^iOuuRGk>-Zb?PjfgXXz1-a~ao#IdNEgZ!0LrncyLVAh(r(!m z%*oqk+nNsv{yqVzdqjJdp1TU&4Dhx(1!QybMB`u<|kAKA|o zsw#A5=6N{oK!or)HD)M%f0uRqyONONnTb$IrH=3|dmN!+1yQ+UKA!88^K04FjT_k# zE$U7uDZ|1^+d{%3U+~txeU&HQsE4LFQzJs%1hd;Uz30Q?8-X(ZJvA}-sn1%2&$AoB zl1ft2(xbr03c-mu zfH~nCTt&BBOB)}yUt;6xkgbmz@s0Jdb59mtRK5197|1RgV_MGHT*)a+p?cDAoF=NY zLe<11KC1CF{O^m{f<7dZT|?eNw`|@HMOCI<4CjoAD>%YA?DWk>!!)|M51OLZhkcm4c^X5>69^!V0bi3C6yp|77Kl#ldE4bKcTpbxtU2p zCxxjyR>a8kjLcQ9HCJDNtMrBEY}tJ1lRNM3{03w+5=>J2XEDac0_5c6=4|W4Zt4P3T>g-qqPZ%GYNWzBcVDlx6jjjE0|n_iogJ(lnzdq1~Xw#kn-!%WBVAa@I!$eL`nv?M{F$y(y?D-xPAqE6d|P8x~~{Kfqb; z-J1F8=EmG+uv^)SxFty75;=dlm}iwp>5!sP2r%0Zym*Yg69vIA$>~dIQ@Bk?a+RP+ zieWp*wcB8*qJM5;!gc)oqw4Y^S8^3))5awy^Scns6jD(7PGW-*7l)m5Zl|u-AH1tR z8jTn)fu!hC)$iE3MyuDc;i{0(=iyv={t&6!!=}5!PFnW(xovJdJ zSYK>osYRqcs8?)!T;?)ILSA0yy)IQeu{@%OBF9bH z{fgD7P*-0&D96QA&=?w-y=|UNPjZN^@oCxU{D+uFQXZcl3ls1@abD|hTM)q_zAy?N zs3*278n>@E&m9wIQJW*wnobte%kdt0al%VVlj>P-vmE<4t5e$YGLl&V^3rN=EG_6> zOGN+C1`v5JmmXXZgA}g)8_{kF}TSa@#iFCyJZ@o%c z<+i%m-oWN_=c)|CB{Wy-bAx3N?mV3}Wh0&$!NS1gI4;O28(Nz#uc*KI1+hpEQk9>b znGtKZxon)(sdBES9d+LKdfEwlq1UJRZdpB^zT{JWBF6AC2fNp4j~!VN?05RpGRYKE z@0cULdlv?6UH;KEVV{pg{~DR>QHof5M(pGALZ`bX$sA+Nl`1D%rzLVBD`(uM8pz1+ zGw*wVQE!wdX>U)@LvMps$UntIM2nu-eb4Ue3{ty^n16zm*yhWRVw#z!B;Xw^81`iuaO)c@jHlO*Jc>3?iMuo3WKN~` z&8;oPS6A(o3|y!$UN^|%ZFN}_hx#73LIJ_VzX`GECS7CiGi=sEf_zZ<>1gZB&Z%%- z-4D<<`=NBB{$5St!5;syO9pcUzoYfJu&$$T|76c))~YlaMDPM>TQXzBSZi;G6yxsbyA*(5b*6zqE3A3>|@1-}beN0y?fp*xM@Un<8Y(aZv zsdCQCao|=%)H)Ce)xH@p{f11kHMKMXFWdXE4-}p^i1pe}eMy4OJ|Q9{ zHsz9^Vp7)^q|=Hm`UK$Lcfe5cjnzLFa0vp4?}xy#Zz^yux5!f``OZsJ>YK1rX`YDrt0`fuiM<)Ptr&SLpi^_lp)z2td?$8h!EW-ETHFSl>5X^8yII5XT7ZJuZS zSdW~^#&!;T{ijk#eqs7L_7xCYWVr%~JVwk)Y{Vw~O72keLP3U2u6016(q@A-Q*GZ! zI^Ludksw>HVAfCj#Vd27`mT3vPKPxLYZw?R)hJA0Dg9Be z{;jCYX*|HrIoVg0-zTm0Bz=NjhEBJ=t<6Q%%9Mi2mC1}SJIB#!`OZ8pJ}~8a+%+_0 zXa3rH`EID1L`fIjqa?fRR02IORZ-pXBIhc7=>^1QG3vr{L`lVYCT8YXqb{jpRECR7 zcl84+a_n98Y<}v5&^3qCe2`QuqU$s&Twvlz6QJ+*DSS37LgWPUq*oa?Pw`MZYsA_r zcCZ7tCQ zqQSilOQPPR1DQr)q++*z`tCmx0tUNn z0~b^XGwBT-d8K?Ec05I=$xVsv{kL0Nrk@bO-`tthif0V*?V%PZFEwr=ExEz zb$=~X9Cq&MTd6A0m=_RnONuclDk{2o`*xWn4la{*#eJaehRafLulW(DNSI2brx|FT(n;Nrw%4e{=dl zr7Utgv$jjpiGu59@~yAU(K95599lKz61wTEcfPk zi+u?fUGahhD9Q_%$->u$orik#51%`C?wf1xh1JFe_gJyh0|Nsx6G}$kyyAY71aWsz z8Dpz^5B1+<({R_coIZ=ueELIJhKFV@K3Cd)FM%T|)litq)qngroA1C5V%C_2v$4sA zK4DZoaGeXH1T8#{Zyj)m(Z-e;jGcEPdW`Q@h zUABg+94zcYz6ySq;dW6Qv?9hV5Jtb3>Us7z)jW^x_2y^pDer$7I>~o+>?3Nd*R0sL z_FGEbkF(X~hrFaujH~-gZ)fo#%mjVJm?FL!ZC#_|@~i#rh;`p9={PIi6DLlIoQ>5> zDG%(28uOG{EnC{_y=CDHpej zZp69_M+y4|1F3@ z3C^;&ikSm1IRCFH6$pmyrMbz`<=;@%#9X> zZE&&iDsGd8`Fhl}PKwDo^rFU(vrxpmdHr~RAUiqvytR!@GVpH9_(ebK7+8$|F(bF@ z&p$qTq@q#K_bcXO+|QK0zDv2yp|RH0VID#y^6dE6i?bf*IrsywNW-+U?PUN=XaVJ9 zjmpnBZXNtPXUaI7lPS#Vp9A*Bw&-$((i~VfhDV-Us#N&l~$&8OhNcPx0+_ zi}yL+(2_V=UovY$d1p){vK>g-E$DA^{KT(&jz;lIxAZGrKU^PA7&+F|6179un>}vfL9eyGR6i+q$2A6`2jhCIi>zDh?!!j-(d$HHfui%1D zxG(!%VsVLSwE4kIAIlD-Rbi2cB=^YOGV^p~1T5GYnVKNDfzTy1V%-cM1q>Nl@@xcQ zJ~4jVx8^F*LDGZCRUCQ_87E@-bfAm)p>=!@9PJ~ydOskkSU4}Qw?@-&u{ep#t<*e$ z487fBl%Y-_&dBRP4q|q;fM=6|fx##=0YsY#SisiG{31^I$ZbU_DXH&S_Ffs55!PB- zVOt>2g<~9SYa{z-+OsivXRWQRkupbEUIKgokx{>jgxQE7=62g`3jxZX2YP9XZq?U} zlhZs)_%frCQmi{Jzv95(dgwO(Z5LUF+MEVs`|N<_43nsRwU^107JHuNUd_^-*3fN1 z=;{5XE(L#XGkDXBlzznL7G4(V*n|8qEj3lhCTjx*z|fHw>Z^(9<^c96IhmT6srAZ5 zzZ%S91MD6yK$4;Pz~BFTGYM$Vk2~V^Ec(|^D`2I-^NkUI&4n0~7~6dco&E1Z4TAW@aFn(t7>0d+z}B= z{VP|5$HtkMm|ASCS#<^nR7CWxNLubeDH`Xc2M49_G6(hwhtwX0*_g@!inALgOhtby zg-+}Yy22^LJ}@tJ2oDc`^~71<$|~)L--8;}1ETlnK1SLFOy@soki~{&+_G}tOMHJm z#hK^AE5KOoyw?zE0`amBv9fMTBOydgK}?xCHo?0rU zAvy8EmTPR5Y$0(QseLIm?%Sd5=VOpN4f7qRZBq7~|C)RC5Vvr;|8>Hy&OYH&=Q={Y zSVhELMXb@?d%F%@!A(VWC;;kLrKo+E?pkcWvV2%YvDL8wYG13vmL$?2(9RlHxDwde zX-L>Ad<>7_WwUO4x&CiQ^wVO5CG@zk`^`}K``dT$eukq8I2SYt%uS0{ILD*Mm2l!| zy^`(;7YwMKjP^`zN!T4Uu@`TY-ORfo&urAsuyeLT%wW;tcFX2Ffm;RbD!2q6(6(M?MV^X zAL!AcyJU90gT;AHL6xGA(%l9XM7JID`2O=pE4uN_X!dd_>F=+P!`$&y$T*7m79 zWks2w~I$A*5vY;>8chLZOK%hq?Q;DNIfm{I?n+JOXp&`&tyPOw+X z@%#q4BfoKK+4ZJhb~;ax0-pDM@5Bmdh}jkzf^;iGynNp+UCj3}nT0E+n@32xLP6hS zF zzb$;G{$IxqzS!d?GJIbXcHt|HusJ-Gl5a#CLD^!kuIcJfkICuE6?4TuzkU>`&Bs1r z<9E?M-tFFZuKFr8>90a9Qrps$4TeT?RyH=wC7+S<2thc&k{~LE&3febO?%H6LtvcH zQ1~~jzgVb8NR3E!32^#F4UM;X6%_*&_5N~yj)LFfsW)0Skqknn_Z8yUl8DV7&9Z7+xOltJ&`EyrLYI0k1REu;>jr9 zCzo0}M}*Rk!vX_qL=_FO$i!}#k9Ix(B4G{_u5#3=*(j_j-M7X5&Nt?H1ToTqJgDhe z!}KaZXKgZ@QSEn>&b|zh`Sp%=9=vx{gczP7IQI?jdZ{w7N-sl#E?jx@LrYb59|+Yb`bqsRW>ebFx)knQMDXsZD!x>I4j^=Kp3g$RYc%3>QK4 zoTt><=(1ntr=`l*V*0NQe@(BIufHVwK^y<~-c{}6)ny(nry3fO92zi__@DXvDOf_@ z%$8pYKK+$MhO>jp5sd+c6>Tv9D?p&>2$0x2w3H-2lysS}y(uo31Gyg9P6iGXpp7`Zp<0R6 z@WjDf>j&eICA&Oo_;#YC8Ys!rIUj7W^HAg_S3p5xu9QlAJO^pDAQE3}lJ`%<`~~S9 zIergS{s7cLJL9Rc@i>tJr6N9@d6@T~L^idu28`f~T5T^A5nVA(7m)MZa4BhDACssK z#qgO&&^Qq5W>I*+(1oYWwg%&|?JH%V&1Ga_a*S&Wya7?-dQ?=>Fwxs&u`5h`ksFpd zWDHNTH^kHUV#O890oi%Ib96PS%;i}HF|?4|k*;-VP9uHCnSw)8ipeWQKf zY^Wt)+3hY>c1u~jo7b<26Gl=DJRy|X(cF>op3j$WlgmI~bi!5hGGP>gR5E53+Btxn z@q&*}b=1XlvCCq0S`rCT&>S(tVijO>aYkE7f&HwNt*R;B4hpWS&GotF%gU_Bp)H+f zJ6NubaM45*Y>XsgEh^{YWIauRct`8nH)Z3{SH5{Kw&C_~gp@gc`@@OXLEUxLuMgy| zxZRA>i#uQ$g_HPP^1J{qmHBNyCd}k8m38h|zs7-WsuQGR@(1~D**YgPZc#dFGEkIq zMQ-(e8UEIZwI-~u4`X|$DuFmDE||fXfRN=_Tt8lOT}(GGwvO%npBOk{I{5aO1KB!f zQg=FQ5scD@1~ez5ZL&kZe!Y&M87=#MAe3&Gi&M)QJ}Y5l6ciuc02+R4b!l-in%Saq zAGiQW$-N(;a57~C40GVpy60@RL1 zv>6KGE!;(3tAw@hc5X3tdo&$0W2TO$*SofU{Ozq$S&*b+si*7}<10Hmsj9&GwBZ$h z`L#1ccYdpNN~QWL)VxpvH*!9s@~cMx&b|$9-`#Bry$RDWt)pcmeKZdl`TX3fGHzP0 z0r4RF08FzP+#Du8uc~W~EX5j&5it?t)Mbk{%ZM?1Y|F%eCo?ABXM@uza1;BDznv8t z|LdELFlN0pz2$@;(G~IX=u`@u5Hftl8o{-$o!B;F;44|6k_70<%KmR%{j}kLsH7dB zUv;fgB?IS1UWfO$0~Oh{tSA}vlI6WEmG#-y<>!oKZi->Dm}kLC6515QQT>@9+RRgr zX^#hT?v~TWfX7spBO+->2B`q4AlR}DmltjBXWday{weR`0*00QbYFk}iQ(0urKI9w z9XTk#)jPCW)bdWAJc$gDa+Pn*gt3g()>U}yl(O7B^{VW+FB~?t>_2OW)dv3YVD2@) z7Y=U^v}ISxaryQ3H8+2sznCzT*Qxzmkuj5?)#3EtH{qpujAvyw12o3F+licZXCG{m z{^--rI2jQ1h$4pZ9NTi6lVh(>1rL)gLqLjM`BxuubGUeyPddO&+4$vgQ_#jQmcl0h zzq|eiixx1_(&98(>5BXgQ&hyOujeouufvof*4}ohm$hm)Wo5N^S6{`iLi5(fyxDt{ z*CbSv$JEGUz+zDGQh~Y4xvXhAVN4FOh7>o1W(&-3K>0??3m>=#Xby zv&~4y&LD~|gt-Gy=q;EQ$+*-7YDY1N@-EzX1pg!1t1uz$W^|Et5rix&OG}{~tJv;w zU_iNa(GZrW^}--Do|&1M`Jx;zwaKZe*hq$@zgw{3sYv30BCe^cF^4f(85uE?$7Pl! zRnYE^{S2b0L};Kk?XxkM^M3R`k%ot5eDmuT*TPDtMJjV*oz(4*k>sukE8H-=fDf-n zewJvSTCLv<{OOv=HC5zR!BKL@Zn<#(AxAO$StOwg;?~sFzP_1`x|>5$PL2lRjnGNf0UcGiiX{ABo3xq$fmkmLf&RR5Wn~jfw-3x89NsNDDww|vxA8Jmxb(2 z*UmMhE-_?S#bO2@P604W)#@QTm08e|voZC;7^LaHId6oEb}U=oMh}i$^2_j9`^J_G z49hw3awhH-VLCA=7i(Hk?|o^~PS}JjR&fC~l$>l4-$KDHQKiD@2{|6npg#WG^BV)x zC1NVBgW3As9qbv4UUR2o@-ge~2z_#mJ%Qlr`jS!cMlAg!MMb|%#GfU`uDpq>hdjNI z4~M_pTk{QAd^T0DV@#}a7oZC+YT{pat?b9Ez?PaZP>FGK$2vc?#4X$<6$pV_`u22Q zLkoGnIoKK_gg8d$-p2*?Hc}Kfkr7JE(`){C?_Sm3U!V5N{Uj*3=L^5THBXdT#{jV> zbo_9_d=N8LpueaP6CvG^a-+*hKWqV5xIQ1o49)k)ZzhhwqcGzo1q_N?#pxN^iQv2o>VY}H2_#Nwv!NZ(%j%E-a0am-lIM>UD-m*Ar5{{K`C z0|E&yIz@*Rq-Ij@ALY!L`s-ZYwI9jjvBc=|^757z?N(cUfg_dFFNsBpRB;IYYfYRM zSAvRO(I0kOjPm^wEmjFNcx#sk^irAJ&SgiHmPBS-MSA?~N?_!L1cR@GD$agCCWhG0CD`8pMuN0*)uZyF=FMt$0$KhrmK99Xy=dI{5|yE zvFB}wECzpQB~X5!RE3E6 zg)?O)dY#@2w(eK4XU;{*&JUt7H%WKUwdWUMKZv9?7g0C*Dp*~6;xR9jkBGJJR|{!bXa8A zXdj(dTKcjhiiOvW8%KXoL0vJ>vkjJJVvZoZxgHloNVv@cudzY2$;H=$=zT@eFU(NU z<;rNxH$M(fJ8H%w!bj@g|H<3<^G?$1$rdGVYsZXO5m?YoWt>dU%{-ltK0ojvFcd zuSYQ)Lq!2_L;sP@GI%=giCN8Q?=NhM#p~5d*j|zpItKfRL<_I42z3sjA?$3wxWfq) z85WVj+w(Kruj2M)DANaD;}n7~_GllkpLT(kiBvmDiXFpd;>ysa@xYKKsVeHZy?-x2 z|9o!XM|u_j6uzFR?RQIdfRElSbGbgy0T z2%aPE5KySBpLpuh9!J@d?XJ{OWTnu}X4H+z3OC;yp5VK%HvA?r=R9$u(%4&H&n~tr zaA>Pf?vZW@fODUlP}Ne^Y-gSOiEd{b32DI_wor+f;s`nCtMD?78tjw(tnDqTgd3Brdxb^O2 zc!m0?v4g`(xZVQNrkbq+RtDl|FKzQXcIrmMgnZ|SXXxfu%0*?*jm1OM|o}YP1ClIvx!{F2Ak!=}?Ire?e+OMeHPsyp0dv znEmd)sI<$FHi!Lk)&<{NDs0K5G~U)@gzi7#qr?3gTVPwbqB65F zhlsa;Qtu?7bs44{U>2Q-b?CudG=gEu-n@)(#oqgSXe3 zQTLJRIWd6+D?X)hwukp=ZS-UhDV-teSK%LjFgFk%nP-d&r#bq_AD)X&199>HANED3|xVt!tYcX z0_`Tx^G#T_>;L%;=WxOV8yIEt?M1SKMVd8cQ}VAlN-BC<1uc4BDT2hoG`C)ga@p8$ z_f~}4(g@WiOu!FpxxTMLOk3Y}7&(cpSA?n2^>onkQ9ekD8jFeb;umI^(Qq{l%Sa;n zmjzG~O{NFT=;d{WtJUrxliep_kLyd8F`#B?Ne&Sw6<^Pm%^hnrUd4R;pG^64z8(!q zy8N+zaxac*ZLb>p62uUvYT}@MzKryiB&`K=p;!FHj{6~ueVTu;ng$vWL~|ANUsLTU z;;k#`Gh9jKH_tOrqD0t3$~(goDQo}-Hpj(W+Uo^mQ#(YEu>19jHNREZiXnH9vt&W~ z!I_HTQi7aWV#Z*TfN3rl|LMIQbO%A1{R2Nd+xmG!ZuoyLm)_#2o_izx<`g?1pOE1v z`o{Jo+Bi=nE1IqiskPQ}*uL_Bwg~Buj=I1DoHWtox!*IsMT#uPQLuNUr{#8T4df^P zoebY%tn7BVb0l#FhM0!|vw*V-6nXh;sH_+!WCjiE=3n(yF+^ zAa#1N^g`z6h2LA`+MTO93J`o6Y^!R}(Dsowh4&>UBIv<6qDNqXD|#eM5ND*kH$;WR z^km4>+D{jWZBo4d?A%4dunEHs{E|b-QCpI5gZ$_TGj`@rlkN7RZG8;xxF)keK1Fho zP2<00Ekzj)`k&K|htYFpqk;~4+@QDj8|t?DEf8RSUJ-WQ4lqbmo_H(72ib5<)ca4SH=V<2 zfZkW5TDqJ6QYo-Uq6%1I42#?-#~gVEc6|J{CPR+q3!i|)3OBMr=02N*ll@U-`2U_# z4fNa5GY6mx*8k^EgQ)}0Y*^GO^6keNKrT(zsTv-t#sHj1jg%{~jj~RCt=iTr+4>`& z8(d)6#e(_p;`x{Y|DB%I{8>=48S{yRDY73O0ok3hE4RATCGYDalhZ^<~W#4GP z^LQt7S8hFuqDA(m0tMtJV^r}5oFO;2F0hE!(8@>O*dY}=w3CWK+>Ad6O(p3@?83xJ z5q{)BeGQRZC|qXgJo|o>%S)xz>@3;UV{$CCx6E^uny^K_r?;Xl@&}YrGd$ECM;PZj zTb}PcN*O&gAy&n(XcAMkgkzYK+Q~&@>)?`&cfu;-AdlBGAhX+x#io}NxUQNGXZg_( z6cg)V^*dV6VaRytL-}n3QNfj7#u15-266B8i zxs+VEmUiLH#aDGd6E0N9_eXJVFC4NqCL~w2IBMvp8jo$Q)W8W?%h;*6sZ`HT`S$Fa z8hYo&Lo0M+$Z&W39hHM0$e)V*)v^b2xwaWxHS!Q~*9#(A(7uJpfuRG&B0nb99q*$A zLbXT#5jIi!KHFylBdUgB1k9Z;(g~@fUhYINmmlH7Q&6BNR zbu^LqI4@3ArfF_Uz^^^B$j>T;gj@yK&Q2V4O14FT!URJmSIxIVEs!B0P0>3@d{<#{ ztGeM-V%4$EHK&Z1-b*VaX;NVqXW{#A+zEWNo0WBQAU76683}5LCY4m|6QNxvfV@nZ z;w1l8SA!`1Z16r_7mQc=2i2{?n(Xw*UG=g}mbx)dp^~%a)4&Z|=Wx)(iD8^7WCo7x z6rNPiseRJ}K;B&GzW+GN{N3}wt>E?jH7xU*p__T^IPB-!#R^WQrpQhdr}+1oPtfTv zj-%idbbi=QM0^b)e`;Mg=->ltUQLrzuN3#ehvgoGQg&xkG-e}ZlrW|suzhfAPu$^$ zJ%K>CR15|Q#n*!mF>H3@hd%RlzIDjI0n)OIWWifMhy1Zg5>h)EUu4R)74K(h08h9s z>*48XW@%~puD>Vpm%CwfOw9RsZtID~m>*0P{xzy3Xu|hbjCAacJuqLp=v>w^*ny89 zB{Uwrn*}Ol6TJ|_yzxjZJ3_IveJ#{TpqfsOS4DO8aCHq23xX|LHDWB*Ci7brVKFKj zfh=Cb4km#i?%QN!WEX%XfWTgkU=3^?d^c&=3*h3lYr+cQcGZ_9M$jm|L#Yyzxy4Ij zQEs+JSe_D34FQm}OQ%DQ%7Lp=Hb5S%(Y0wd-TKCkyzpjz`oK`~?~9-9BNf=$>+0&p zZhX)GvCM2&dS($E!HKWZ3mWqwbL|J*hP}qd1whgBn{#53`8JT0A%2x>CfZsV+Yc3+ zCCK0QL}gwi{r5824kSGz`Ek^*SNT%&HvHqr7oz&6<(Gov?Nt*%-nRhJU-irlto6u@ zGVs(7f8sTL!x1JaLi>*NJ6P&I3saZ680Rs0f{}$qR6fS8`j-7WW)jnj3WUIUn3&3^ zIR6T0KL1Sul@szgqD;C$+v!6W5w&%^z8k_Z%|jzopA4Nl&x24)boQ>y?-mho_hRSO z%~qXodV7HTVFbFPCcW5sHQDeUJ;`^4gzW3tUVURj;;WcOtlEDw)eE5S{{$cKL?J0eT}Z^2ffDPt%EaQO-^I@7u=VJZ zfm72qZj{AMA@^qb?$-rBC0Nw^ci*5Z1T-&4ukm6vs;RA#ccKMk&LOWNxUR_@iNx@U zr@LLd_LY(c?^M37WMp}e`|snPLmhuF1Ss?dOHEAMgkIxxD&jq@JV}8!G4gm$eTUy5 zx+{`?_ZWa3BYkR`p^yE*#%IW8pf+M)Dn#ko`8Vv)94JQMBjU|6G!Ue3_^l-MukpbsPQyvSBSBbuQRhKGVqD8aLqq4d0mY?wLqO zYGd{gPB<9CennzBPg#f<|JR|o8bRM*FfsY~KRIa+(bMXx7b{{in@up}`GT&lqgFf~ zuXFp?d;0TJ=|_OGwY5{X)#znF&u-}hFYO4#U0-wrn)8v-g5fPI9lXbS994cGH~9M(vF$8b3nH9#?jSTBz~<4FS<^3NrK4sN zx|v*u4<81_6W{RKn8YZ!9)+@P{PY)=k>LXX8>GNxifvL;e5(Z$iv{vE1SrWvTOUeJ zi*5a)zo)=fU}r|9Z7H?Moo2UdG9S2~>Y02%L#A$%%nU@TDt&)#&=myX^YqKt>sR>3 zsTxyoB7Q|K4vl}``!ZBxgp?G>@wNhlOM4|rJ{OhiGbm%kY^Dbg=OGv*i|WC~zC}Uz zKbYCj#aLj_yRF?JNI{%PCGcbtWTF*jJrA>8>nZUh35p8hk-naLm!TdlR7Btsf&W;x zrC-LiLvFTm$2aN-DhzV}mZCR>_06;mfM?SFJ1X)AK&EY)yJBS)wdG=W6 zM^K0`BjRK2KAX%sBF-Bm|2BU&zh>0d&G`CrLXMhxVX2%)!qal90yusTn}wYrfAHBMq9h(otoltJDL%q;r{vYcB)4Brh&C*i-sqS) zu(Pq#WRORc+b1*Z$-ISRQ77~=E1{40Oma(cUSvGq>m4v{2-I||%zUv?`&?Q3=%wr{-*q0jlJu*oAfqPJ;_J99i z(cJ93ePy!4Q||u(=|;(co`@$`dLNsxMs1IwgXhTcl|tB{OkDSqGB z6Wsq}?=8ckTHm)}7zZRoKtTi~Bn1T&K?DWHpi8A2krD(Xl`fm^j*UvUbYl_H$RLf1 zv?8tKdo708zkeLh=l3}tKX|aW%$l|Cb>CN>*Lj{qQaN8`7EdM;*tgq;4<;O;2wrv1 z_Q!aUiPiR<^+FpXhQaigw`V4F)XQGnWU1};{V=lg0X(m3RLnk06!?F4#L-w`Uj*QN zWd}AE3V-s(%!3J$(1lp&FhXm1w9k>xkN%>O7=F38<%#~tc^*>p$mV>rUE^w~!kk^X z25A9wN|D*`vJLCupsM&>IY4EJ`U&0Xo*#?NZ76STYvfeij8KW}?w#GQ_ArlFLP&gC zP@}3U%5OW|tEGlLTW&~u9F6LwX`lJrzqRaezEosrQ1EvKz6QT${n2mR6?7!?9TD}q zugTw*5{s9)ax92)J9{mKCPn6S#3THZK@46sRhtJ{OUv(c#v*gf|EhhipQPYcBGh1o zxII4&S0Z3eMPkvWwie2U_}Ybv&5-4m^OLd55|jJ1mO)3oZfMq4TfY)AXR^bUCh$&HIRW#z~DdBx&R+6X7?b9XI;v0VT&ZfnZ3+|W7^ zgt=|#(^Y4tvv_5(P-iixjg8T_NYu$<)J{{<|eBWjGBY1Oj5-05pViJO4biM(}n} zQO7CPqNibVM^Tzr89?eCv7hM;VfZ9`DG4q6LIxdyK!ldo9_dT`xsFPA;SZO}s+|B0Q zIc>%8se8V5;%J>SP&05mj*tCwqrz8ePZb81CWU(>tB!7xvsOe^N(|Rk^@QTck8$|7 zdtS+y{72r;8H~SwtYIu)jhR{5Px~o;2ja1#){#thJEYpHB;uS@{JtUvQS_|s2V(oYK&lntaCa)I-o}#qSC;A`WlzLKm)=+{@y+Mr|Aj@&<%k1gt zfxc=Sa0n}Qp=`dbX=lUfK=rQWvB=8x+7BqfA9gbx4u|HRTGIb()um7pbyvB#WY-NDOKv~hGB}Yle{df^~h-bb`WW@UCtiQ?IdEe7u0pBRLl2P8T!8n|Ssd`q+ zz{;4fmmr3!wV!(f6|k7=tl;Y4bbDaKaa-$Or<*2? zm#e7$OGl64e{K;d17W4DKZ_r|gG;}O_TQHtMH&}z@4K0a*}VU~?In-AC~ip)^!ycOjXOqp%l+qnLI+^? z-(eg6_C|(XDGFlWPpQ=`ew*GGuhYQ|>f)6ZT;>u^%=lFOCQu6Af$={9p)>N)1Rkbar$EqdNQgxL3bayH#Xq70EVz`lLJr z6Ffw(I3d)grp=oCr(?T>h37>H4(&MpTNwX1i$;Y@l3}mJCTHe;*nJ&d_lSXBMx#!3 zd;>xloG;|3f%v;tQhlF1>ku5-_2Yo_F!S<8rE_x^v^_Xtup6E>Bml_ycyyTvcCcvM z6Jy5Ui-6_;OlZgm?Y1d^U5hN-j2*(ulSry^jI+kPSGfJxLNCAQ_URQOF>C3z?+1ZH zBY@V*%F512+1TW}>5TsQaJ#(c6%*<&a}oe1p`8D}Tr!MGUFL3G&>}gF%o)b;1TWt7 zQ|XI;vmn!fm+cvg&LYe1q*VqWZzWWqzB}@qt&aZ(zo(Ir?pe|5ME+8FUJ#$Vzwamd zgkwaJoHYmyO_y9FPJG=KnnZ$6{|wOKtCONFUKQNOtloJpZvSAp zJVW*FI`cHD&ODv_p>hnH(NOs(qPz=gJguu|OHXbaq)iePRaF_FZqdSn%(cL%@yMlT z8D1*W=Pq76)nO)(J?+;cBh=CP)TBwV=L=bb z0&0e-s?pMG?IUS~TJs>74d+?_JDZbYDs{4=@ z*?aT6Zf8d;lCjIYJUrBH6k(#DsqMDL8x0(`=+5EX#0ql2w*SkP^?n{h= z-B}-&q$?dD;nWCnarN7T$7{`t>f?g#57X)-Py1p?O10Y5M%47R>#++Bc27fOGV%3z zY=UH>4>%NgXVWD)GEPs=&)c3i%N}=N&7U+kFS6{v&|7Agsu3H1w~?sD9BZ&DpSxn; z@{Y78-Tt0!cekC{Yk|tM`a9E8P;Z53npidGIRvVHhp>71jp3kC+@E!AcWyYwlkj(| z*xEgPbicr|PUsxG;!aj3TT>0n9C;3=+P&=%5m0#2(%b6M|JeHFaEv6j^z*)vr zdMd!8GMrD#S$eqik7f&#MGs}{Yd|H?h%@)T_>=DdaA*^I!s7($(QU-SN^ zWo1#h!K}9(FI{{5sXLmv+3c)^frLX{WPJfb9Id{OnnFF;{T8Lz8JQy9zMcFe`4QK~ z@N%V)-s}-zGWm3Z62t7~6-=9T0GjDcfjj6F$k(~+GXh8fiYKotoL9;| zq;@(eqb*q42*AE{?=7vYE-)P(yff;el^BjUEgha-1=#)FNeji6U?p5tWIwKckCG&` zxO-dv1slh;BSCdvJaPPg7v{zv?uzivD`ILgtlc&|vKlS$)O@m6C#I(Og>pJg-OR{( z(*-ORGSgin+Fp^UsG*ImExzff&j#7lCa&Ci{16x0*S-h0%DMbi7d@|D+W=D^^tweK zj>bycFO|r-Jrln_K1)t;9#^vm@Ee_Hw{zxdl#sPvlZ3#6YR&xc&=4>10|`{8Sh*^R z`lkpQ3*Gy|4D{`{)YVNtlkv!ZSGK2>OKQ4#B&n;xU^fTDn825wN9g_jvcf!&IVn91 zCBsT$#lR35uQW>v-7-3u{x924!j0ZtOzXDCq3-$0B9k$BnQIafo7h8*BPDC z8-Soy$P*;HWEzE&E@?fqYFzG9T(M8^zfIHPRPsGimRD)N~K;V~~P|!JjZ7 zl0UCYN^+!P&?4Fkw2EQ_Lqd)SBcy*Y>>C*q^Ipi-#JtkI%u4lCS$a5j{|ok?gB^xS zXkjp6ni1-WntRk_U3_`=lef-GXRsgOFxN6S_&fijz+=n3_l;>_SzSc82Y525h8v2R zw8XGGE07Hk`mA@?^V9u}(?TMqV6jRZXb*PVX4tuCidoj6HWW-AW&`mk@5^0_2>vsQ z3qQNN97Y*5D;%>mBOS8VV4y74tuPB5??sq@f&mT*WFn#$)V*c%in6m6kV)}3YSTan za1!V%Lksw)8UyXS6>_p9pYIPgcG5_1Mha}SwL^F@FHsB1s>gm6nf5C@$LsgrO^Kpt z@yX)gYP|(JhRoj4F-E8}OSR>Cx1S2xdo6`e=+8Ypv4Nm6|k zm%_WD+fzf6oB3SmvuNXlVO|SYS@D=N_^YLs6argDB<769K?r`eC~PwtD;LW zjIwj>W`16@R6vm)EX8dBc+I8Bt~Y> z!POZP-hD7TWNDIX&V_GDXVxu}4YS5)K}SQRHkvwCN`6cw^J-(wF-HJTl>9KQ-rmk# z%t7mXlL9(^au9E{p_>n>Ns*~RZr6)C&ww)@>wHO?*;<*ZOfU-^3Zur2S-lzOnds<3 zKub0D2a%%4BwMtoOYzo~k5RW((;CF&$kbwYUqt7zQ}Pv@3}6qp5WjHaYm(?}116a; z#%=dEAgsQILbRCPw2H_L%>_J=Ha31OIuBhEr}epdAdB@$u+#<#S8|>2cbu zzPABq)S$HUw+-EeGeEVRsK=5`DORr>fP^6S$APT7OEB}^n?--&-dT$xt?`Xbou8+U zws!#Qrn|oN*mmXMY!%pm)Sl4HlOudy$#iyN>BcCLbwMeUUCgc%&kZ#-j?{R^n7^mE zxbk@}SP$I;9=nkk@@$T{=21)2mzk~_wqx7b%vpcrVw9nBCr2)M)z%%tdIOj2-~qIT zTS=?@PI!)PKBLA%pX8vGjU5;`6DYQtR#;S|nd-WxbCkPy9IHr}X}Mc+ngma-AoW~~ zXfzWprUq~#NK0Ke?ivi?eC;1S%o|PhY%O+RkgGJ)y=56ww+afonjjhfWBPBZQJ&-Y zu%v#qo(>1@QNP38nZ`xYGmqT%@)6WfmP6Y+KfHPfQTg01c|>Vo4rJD!u=C3TIXT~HRO@jVsC-*r}s?Dzbm-#0fzAKAv^7M^&GMdxf^g2kgj|RF&&UI&gcjQB^O4i; z(M$`t(v1NJBz3K?czYykr%?n~f-xF^T4;dMPi1wMAJfj`cGAH642H2sWr3I7=CKzn z5ltu!KDRj??dWC$_Sz|5c4J&AYrn$5N#Mtie-fFqnS?nT+*^Nb^SXP42#vDs zyPNtNty#pFNvO5a4)D$+r_P*BJpw$eBR2-fy9^KGzS0O#6&7kyWyC1PV`up_PLn+G zBR&)xI>9g#^5?9_$nx(OJD4Fb{K*)YZbQaw`5*k!StHTnHUJ^uWs6)!Xdy_;GN=`3 zRF#MMZ6k#MHUlZNXDbio?=Vj&+@CrgVSBzI<2Ls8B*V<|XR708b{?7etS&01(HXkX za@Vf;xHTeln27gSD^e#1a&T>t-ifBn>4wtnHbDWO!HmH+S1<e5a4MpYy1`exPEVHs{^!?k&gXNPsj7w}GQ|PEqqr*_pNy z$HQOtL2`Rx5ye?qw3!@n9^Jpf2IgS`P`hrCSIgFiSXN6)>M3a4s2S#ed8n&nQ=P*- zXUWbWf1Vwslh1Kz8@^Xd4=Z1lujqHgYk-(alN}Y}P|9+Ep^TpV0nWiAwE?y~n8c=K zRJ=zh(vc_7Ftyy02irR0Fuw+;meLMaNi72JNk7+^zgc&8^bXJ`I5^0#9f?jOojp0? zj$d)THb@kkP{;eP@)dI;f#a`%=;g)jWFnXmAF}`Ik359J*znKGUFFP zz*eaIkphY%#X;AXiC7AVPCcVnqoTRz)%QbHzk9h?%jt4rw|ilBZZ@_rqev?&m%2zx ztFKQ@g5ZN#`ZJbf&oc#&8Bksr&qr)Q4nylol|rFLAr`h``4{y-KrT7zNOQU##_*fvQ-s+d%w_H-kMk3OvR znQnF;o!L+Rvs7k$jVEy(cWdu6pzuxMjQu_r6@Inf8G(1OadJu_lt&%wF($2XTq^^1 zfrwBvR5-BMe&Hvc3urOYyp#`uij+6>T3FOur%##h0@OMvCa~GDVPRot2lJNc(NURa zk-v09U%kreLu-XefYuRt^Hn+hStqxXLkb)g?cM}FIW zZbKDM%B++2_uSm7h(kX9{T18cc{fqa&9$VN?F+kxM_KEqPj6N<=|#e8|GD-wEAqAz z@2OKna>7*J7;Tiz5VvqwWSz%h?45A-w{7~`<|lR=My)JKz9XQ(*YNx(XWzX7MWGa@ z_)S{wmadCDpJV(h*k315W?4F(IY@awtv2Z7wG&$BGah5^LF7hOpucqBCK!>+5am^u zJ)3H-uovq__yPFMKc8CYiz(C-HxN`R&#kPi{A&dygAehobqYSqCBOP0q)2(cAR^JY z{=o_AodTcNVajKLfsu#{DYhEyGYtsbZgUFiiGco;=9s_UQZ^P5OQFTWYbW zr!LaAE%r1 zfU;HE(lYz6*vMkr?{|TWVI^X{Cac+6o$n`?ehP>i2vBX?-45jLW*Fm-VSF-U7M;DJrIJnR$QYgtZ?$a=j$L#gknDwQ9B74kDr`^|&& ztHY~gUhnjWnNcVahcBQ|-}0{JAwv{0O8rKz$D*OFjeM<^bZY^od94CJiYLvA4=fiR zOm*QM!YPttvWOl*@R^l{?hOD=iP$)hLKbL?j*cEHD=jURAO?PevkS$=7i$lr2bc|b z?5TS$a;i&Elj@fJGo7}}`gUF20HobR;s)~WhaE2e0%^Vd>Cdgbb}koSC?#CLwe9XW z&k3A$P3nS@j;KEq{sw6whlSvc_)tM1A$3sw88ixb4S16VP?0hQNzPQ83qvn3R0P|{ zZjC_0ja5Tt=P*DvYDxO=GUIu3kTmQfx+uIf6Tv2v@Fe%SF?G=`n`CTb_v@YF)y1pe zy(y5>wmD-e9tz~gF~Z?Uxe90zULdtae%n#$j=vFEuk4DB9Xr+(Cv10pAsf(d?hCQQ z(lC~dm>{(a36%FEJ*f@+Ep#R_DqrAbR)_@m?S$8uLbN?qX1G1tB8lrm*7pu~YS zvw&Y|cR3Zm0;aIy(E_fQjPVrK@R?xS$h6bain$C1#>Cs%&39iGHlhWTIFY!?4Gp!F=!ZyQ0Tm@?e@76P2e3cs7t97*-8?AP%r{wB|}pWg>@^gFwdS4^5hq zd8VR1w$Nc;Mp0)99YDEveLyV}VLx)AxHB|In?8ujcJJ)!Bp-s+<)0QFPusnYD<*JY zr1G@xwwbbOX7}te-(isP7aoN#kVCW(%m{1H)07z=#*RtvWb5F}wP>VeWkFUl>qNPg zkFs*j%tyTGUq0cA*&Iu_8>~GCN0On7S_O@NkW>R7`>=)*5xpEa{L*}9$BkU~I^TNn zT@DuP^o(VyF^%wLi3aJ#w=+DV*=kdYMIa!W%R@8jCKA3P~X_OVtWEcQ86Iicx!WG zxj6KKxvMhj^XG{|JiR&Rs`*bfR^gg8dZ!nL`}v&xb{(JsWB?bIMyR3rNUHL8j8I!t z&!}hX*`Kkmvd>mzBTJt!XYvhkho9!7=(x&6Wy7S@9TwJ$xegN7-%_i)W7_DRVsD!ceH=Y9fCvVeq{)*36M=eA`srkArG|y#q1z)!LIN38gjfObcxl4xVb87|;3DS5NS= z#@P39EQP0of`XWV60vwe_%7A-+aCvwq;9U^>qAvq@5s*D3P6tegA<54QKTp0XzqK2 zQfBFaNmoAYM1OdGU!TUa1_OS3CQ^|kk5|vFAxGVX{-06H+%o~&c7%Dv5jMQl?3JlN z%quka1@swVxFr#|rT0%RXOEI;;9c5940$Q?jsWaD=fKZEy`pm7uu81UL!1$2R)aPC zXx`Szi)LX9+U#Xf*K&QVNQv2lwNVhsFaux|gm7D26ca9UNd+>JO>P+N2FRw!0F0W) z4SIAOYtV(u2D+Uaa59zAg&{%)_%nMwIgv7qj!xX4Kts@^wtdv#DVBbvhc&G73XQBJ zZX4)u!w!GVAXP;9tQ50MhOhKh{9vZ$sH|Ple_n_pXkb7``!=XwEISGa@iKLtp6GfS zA~bQe5!c!Dl!NCfFZ`Z%0YypE@V8jHZ{k?$-E-cm3ub$(53>&Ks7-wAyQk!Gs;a8( zv;Ha{E%+u_eeAmWc=Z1Di~o8}@7KoE8S;}Pf{Geh)H}0J2LAh%qj{3-@|I_MyGB=> ztDus&nKVqF?)N<5q0?{iCq=YI-6I)e9hJKq-i%_TghiYf^lZy9wh0YU)GUABNR%Hs zGWWth-29S<)79cTzu8J+TWRR@Rf<7d_|R2S^+rwV=T+<@j_ezlD+$X9CdclXF?jI(x9y_-LR`}}C ziN2yt``Mwv%5>1K0a4;>r;~ha_4zYya~}{L>@0eEX(B6*5EsO+z)5jX(!Z&UPCh+C znum4%ZZ!Gaz1nox-!M@$L1$G|v;3!OdUoz}tBs47b}iS)Rj@^{5wype!lW2 zHhmEJs%=bX@TLmaNe@*U9@?*wcZLib3k!Pi5lYul=I% zJs+4{5I^5kKHT(ucHm^a;@RXnhANp|d9GS66toH2RAXNF`o7L;c2Yk3nvJu;$y9Kl z<<*kTJaOXR&t5pg=!pz4ZTCO5sM|Zz+p^ZPXL3K83ASx^GR!w06gOnE$h>+2S4BbG z6v?ZUlowlu>QFPVh;GDp;fU}5l=Ff9^kHdTw|8MTmlZGIR&xT$hcDTTciE%8ss9+% z(-+j-Q>y3{Y{+o>SAdA#899PHaQ*)&O9lHWG!KSyWtz=5b4B;EGT3Y z555Ju7C8u-H~P5fmEkbXk#&7&;?$NiZmW|`Rq=PmU6cX=f8P8tKDv8_ZExk-0D^Q9 z<9E8D*~$(@D*F?1xy?q=6S3U2vL2c`W|s3fEi6pxG3MI7^oqoftQ2vWM8C~COb@|$s30ZCG(vvEJjD4dIWb0 z>qoMZ>zOf@XRj?D-*!vP2%qLhcc!kZXhm&7OV7t2; zo5D{s^h|v{aNcO3U0ylYIk$mnsBgMd#L?*S)TbA(@3q#*{h$w&)%}`UEn}|~_0(!w zOf%9=oC-mR zT)e`38>q9yJ6sxDzSE)e5Mm=DHY=@I#tldDH;nY{W9N^~6G$ed9VJD1C7Ycq9c zPf1hu(8rX(+itB#m0Xh-=rLrwjvJqb2-2JAv$ws|)7`P=JdhU21I4wW zTwXsE_Fc7_GJDq}!7t?zYm}TQR!<_-EV1mz>J>jbQPU{&qDV?IgllbLj_&V9Yz3<= zBMGi7TssMNCw}PKXL5W_Vm5(+j1*PbKnOcElR@sa__jY{|`_B#~4o@CRdj%IV!L;hL+wjJ?Vq$v^SCxOOdkY8$7P z)|B`@e!V8^TuJoh`KhFSs{ifk^mQ@E)Q#0klQWu!O!jC^F-dmBt}CDZhC~mYW%7{G zEV{hflO2y@b{bd;X*ZRwuDeW8YzSK%i2iGrxcbBjpL`mwGUKD3v1hD2dPTsFV=ZE~cQ7v)_wpRSw_rMd9Q`qy!G zVbOB?8)V6LbPVx*#&2t)rR>dXHMe(tGEXfL)@%iDTndWHo2(ks^x`w;4`#-S1efoa z4^MjZkt0WpI$z0zL`V0UJBTs{zW?wcx*qBUKY+R`2>7aA5bC}RJ|a3d9~%2qMy=;b z|JP64=0B>(FUWpq>@!n&YKOa2`)$O5`o$bQNpS#0T&h_3>xlY+&QWG6R%Wl>R zkqA*L#@swZKAK@e^Z!Lf_;rS+2^V4~mEkF)1!$1TmqMN-Qr&LG z;IPt;T_qp-xc^~>G(yo^iX)YD;U8F%aC#Ih0tas6! zIAKy$KI`w^%UQ%WdNRub&8a`aorN{KN+(oj1R^4K4l^FAm8P9C8)3ydM1P@~u_eP- z4`~z>7}TMlajx`K=v#rEOJh&!Vo}=~c$N7-!zB`ytEdW=op^u(A6z#j=Cw51P^;FO z5vvlYNZTI79mP?byaP6ab)LA7&oJo>bY@M!t7)7QQ&d%4pY!TcR@9l(;mj zxX|vQ#%)f5?U(`clmt9*BPdOAjVF3kL)Eu`96ORhs5$I1E6T;gQ?w%G{QZ*t@l(pJ zvzCct;VUmjdkHFLdUl_N6vc3fR_V;M4nnj@1?H-dHgmVN7~iq-0?jXbfm||o(!w`T zm6n6Z+6bhb3mm4^I=Bu9{2Z)M2mfYV>xs{A_?`$@TLaB>i+G1OZdJ!+WrRNHiHNeR z`qL_wo{=8epGsDkD)7vV{nUx=h}esW^sKkw+ghqXOf&Hxe*=ZN!51d%x)U#Ub8txP zPk6n0#fE5q%g2j!qqVbRoT~H?FjM8`>h%2Ar%bX=PDZoWJ0EBNTH?9yQ1|&RRz3v;#2dwhKCg#AI&opvNX#JLkgRTVmpy~g}b z^G1!Wt;NG8{${#>I?<$SXKs${s5#;Msq5ac9~q9#|t&ui9v>qE6djMdIl9Xbr~YuM*-&ZXHlK0Z*&!nwNOnHOGxfsrD0=-^+0+qr z%bsVHSr-WCcY{~Y)95VBG@|BLR_2edltwz11j2Yj;lc?uIm!dI|5{qH1T2xQNOtb{j5R3`S@=idpK6YFXb?u!A>rp zxP0=#==nkMJ6(v?$X z-QBC9(BiIphL0~25e!@ZyMW+eZNFWA=$(WF`QoKEX%Af&-k5WXX&FlvSe-V)S5@Dn zxOo@6Oj0O`z~;x9S9k)@CcR39xYuu%>mzcWg4d-Aa0L)}xWm*)GKq+B^IP@_AR?Y@ zTwIMgcO4wIqewlsB~`>ko)C5ZEasR&A1b*LmgsI5WiMm?$JsfnQu5d8u#)l5Smh#X51YCUbpCTuM67?+N7_!6LoCbEL7O@k)mYbO~v3!R50j zn0JW>U)q?oLdowJ)o=1VY~&Y9gBN)J4lXzd?B_C})EOfM)XMijp%SVbcLH^~owNV} zVLWEWaQR{0AqJ7c#yv%qp2^CNG(Z-tW_cO;MSd}#hHi`6KB_OiS@H6 z&&|W1-zWZMk(93^dY^|eB(JbZf1bX4p$q+mYX65v>dxD3W`f?tgJ6( zDpo+M#FvII;xQ>r6UdghvEHQLJUh&$>`61|)`ec5zcenrHA6Mbw0ywmcYl$a$K!#O z8IJVyC9(%QWkd=BB+JE&hY^+P)piY9U42n66tnb3MN`>oHU`X@VC0k4jGQq{c^ zI1~gk3#z$}j#7<8g83r>^c0+r{QRh<##B<$ySLvdCR|Y>N|`XfeN#=%*jbqX?cT0E$|FL1d4AJf5z33K75)=0LB9^sjH(xd{;)y8f zLOyN-1?U2c9^M7St`Qd|*nFg|uFFu6%G4s&BJiP=9WrQQ-I;u>L)wpOFI-`xLsqg}+Towk5K& z8`b3t{}J}%d0jx;4|PK;wrQf{r9|rCFVKWU(1i~s-n4d=-|gZMelv~-Wj=rJ^~1V3 zq=vyeJKNiHxTWgm#Y?GB(Ro0^X41c;82QAh&|XSHHN7fvZ{eW~)pt}{K|$~e_HI{7 z|5S0UJXHX#m%ZVY;PSes(fB1liouWxm+nvkp9`Ihp`s_|5DD8- zK7BZ^|CR3Vs`WD)`=Jft75?6%8O=77oU)gWRU9m zG4cGm@_jW~|3gjOk^fXRDKIa?gPUaFlYMqH>U><0q8QI@fz7&W+Xtx0XJYPKYs1@! z0Y1LmMjKp@;e7yD@-LHeTHcs?Vm3iSjq_70Ypwmz=yabqtMB$YzhBsWUx;yZES|XY z_3h(ZAi!>F8W}64?LSmx5lKtjua&DHFi{lsZ&-%??@($AplO*=jSe^-% z?b4#9(LmnqYxGjFj8KW|dN99DDD$rhLn_IuL&X*m;a?td3Ey(69ctT;;qg$Zr_UPH zdM5c2{i}%Q<%p(x$@0@^^ekc{uh-JmO;Jy37PX406l~41EOjqi?37QppXMTi*ZR6^ zV{`Mp5~7xuPM7$}n8YqmbUohhDU1+-H4c~{)PJh>7Y*K0{ft*s|XUNc&-HnGdDd^dp+wo$Z0%vc=qcPRXh%6|fdN78SxmI?Q&CdhsLT3|vhHvBYN_zQ3cbFLHq;&&x!hGvuDERJFq$D=i`2yH0)*Bd_UbAI*~*-Ml6&s+esTBn{_B^@Cum1bTeRPGfc{&7XU zTuE{(X<@(Mynm+dheDfCXz7@PL@hB&c=(tMJ+y3uwKL=!J z%GNj4(Q(WaiER9aUh*>?y0%A~`m%3Ym0aPsX&3xC#(Dr}oA>VwazsfaJDCxj^1~Do zge<&2@A!B?#I}5tqq|rumN!AA%;DRCZ_kzP36dY&f9$=VgphuZ-R#W==wz-D`oPZ-x!A+Y8$FZ-kQi_w_=15-B{d zC>rRDM2>n~;T8U_6CD2X`cy#U@TaGZ2URy*EdxV)#NAr`464{Xw3@ZsF}u5MZ{IC=lAWC7a!%*ZKHebF~$8d%xvx-$%0lI-I1wfzi@t$^ag7Ad8u#8;rV(fY5_gCXcS8GIU zc~0nb*9BSc9Ooagp+!}NHS?ved5a{$8MQBU`d!59_ao}P=MR91YB|T0FaV0_7$FsO(B!i??HG*F=8{ zl_1=dHyYryEv>d_&&CoIyfXBc?&$hIezX_e*-H6biMXo!;P~TB#-V@tmEEthPeQV( z_hTwVP3S?m6!iGak6SqzmQ~} zLfu(&=baGoQ{W}}A5Wi`8Og(Br+E$06;Q1h1>xY#6PF*h3!DQKlNBh1HMX?{c5`&` zqWh?$E>6&C=CASe>X*N4sVpNl=Fk5$^yKp0cm2(uuKc&njA8CW`5BPekt5_=2pSA9 zo?%Q|f9h?A-Bv*1CHN_$v-DvG5vs8_H&38`QT~3<37&u{Xy7xJrlNCXDw{U*!_jd| zGmT}kUw4@W|NB-}1iY$X^I$cU?iw2#gZr&(be|k%;70mzFfE{rwE1EBgmj_z)M5Mw z#M}K6&Kl;f^PgVnzda`tqP5Web22@4wAG49*2Goy+RIK!-%;0oU9wljyBm4;RWP?t zqN7-t{d9A1C`X8(%q6q6Bp^jncV3?15-E4+E4+)u%@biLMx_`X+pohvLY9;zxYf`X z+DE3I5wOh3cm=-MD>m(0zxyzPi9PFh9THP69*Dps(tCS*Zzr-CG={MAw7vt78Vdzq zplm=TNAG;`t43bspCq)*2aOA;!aek+BCVHxb{*Wsk?iboW-2e4P>#nT@s%Lxg+K?D zj<#?8gdv?VV_Df3TUIXmkWBh+q!wJ`3qz|bNd&B1lu6RNT%cqU^)z%!0teOEJukRo z-3{3CPm;S8^>RyY-%eOvT|FlJ{rh(brmwQ=g4{YGVY~AV|FLeh>7>W% zu-#Y#q}UDA2w7ti9|S|6rI25cInAtMr}gZL6W2wb>edU^Z+Xt8ijq zMIoat$Uw#66Bn65WYPLHu>SFAeO&s7K&JVp+3dIhFagYifLV@21v~)P%8-k-@5api zKu>qB7;z6=cVdI}1xMB40sAfm{@ZUKagcS4#BQEIXxK6`GG56=^4flVOw+_uGQcu7 zx2{`79PvpHK@D%NkBfhLLPjrt1G+eht(Rsm{{>t>mu`h)k@mZy+LUKyWL3v17dz#U zs^TX=$SbTB&LR&OyEVQ^Hl{U_OR3$D=$?pgOez&X;$Lz5&PTssYhW|eA-xyeS~T}r zS*a05Ef8Qk!r8{*Kd#mO$cKjLI&l13Tz7h>{s)v(ji9o}@4o)R)2^H^+3HCqXe>7Pv+<8^*?*)p{ZnIfNfN!t*RhJf8vkG@7*w)i)ClN; z8;`KnK? zK9SRl90 zLGlPOx7Da+Tb^g>R@NO+>UvHRfOH)N0j+A{w5hiD2=BvCrPG*8@)DsKjmDAujVurQ zt`T#<0KZO2NdcY5V*+1C2moJ(lsdlsjx(t9CK%(pwfc{ixU@x#uw4qwTZxW&i&*v* z{*HTg>jYmCt172L%;+EITXkJD8fCVxXP^PZ==%GLCITaX9s@q;uEI|u}6BIVvP(2_D=;~QmAYIO0Xl>23QQqWonBbBUOr+PZq!4$yD zFN)+!kA0y!SN4@jr-&}COX|=+j>*=}ojVIwf zAeu;f{K1zd)~RRlKq%o&-s(Sw=X>;QZ3`~ZUEz7hUAuHLD=vpueq86Xg#Y65%_?L0 z&;Z-@k3!WaH}xxZ2c@~K(e^*gHguDGbrbAbg^OS4k!!7GB}HdR*vts!eKp5tC{uxf z3UwVDiBQM;JZ5hx?f9VTV3KmMquz^H8Y*sX}c_`W|n!yB9q%*O7GLkznOxt&&YMOtw=e zwU;J!X_@RywL1(%wFpqsX>`}9Ob^6bS1m$4oIEd=qKQK!@4d`VnWq8$qDw{MM8jjXl{uZiBsTq-y4 zH|QLvWR;A9*CzL8b!qxQ)nhkas)tM6>MfDnRhx+pSWN}3B;|VaYE}~KpnzcFyEl!o zPG9Np8G;u->2G$r9&QizZ8_TGJ;sv1q$R#J`K?(0Z<9JA02bjgTcs{AvI(k{{bWD1 z43yu`e0=}>XiFGWkCxPHR>S|g{RUM@3HwKH!b^{Vaa_s5cZk=oLTaj3YE>^)$BlYC zGhmc`OpC(BP02S7&>U)}D}m{{CMGFPyu7?tH?^w-rL8I19Q5#YN3Ke}2@y%f>v`JR zJ6l(*l}jFDuR_|mfdWyzV3;($DYo^u?O-MQjRR>SWS+|DQn(3<#MEdBch7;JRR6cS%=5k49n$ zxOV^$S<+{fLy+BBI_5d_y3lT1QF++fZCaaW3V?ie-n_1G9Rl)86wlJ|ST$Ii=o8*L z92O-@&Ou`2ey<`UxbyKrfMs2MsCb<#=V*JdtjJa;SW+GJ2izwZ@pH((+W2+fTKp8j zkwf`w_O>puMyhMOwwSgykA~En2`c>Z_Gk~Se8b0O;r1{tvsuyW80|^e>QEjCVQCY& z`|ks5j=v3J>WY|;vyYE=blhzT-PD-~J4t6$bfgC*$8sU@z{Uv;U&iV2x5Y<#yo>Dz zY~*Kem2W{4f2(0^>-dFxQQ*p|GjqzD1fE!UU0My(`WOIDqT3k>uGX3$>bz-ye8S~k zzZ=n)9_0KzD+rE1ve4T+A)xl{W73w(a&Q|LYA!ome>23LQR3w~DW@+IpX`)-jX=QD z3exm&Z^6lf-(lVW(#QN7c^kL;W(S|7Yt^lvm@p%Z_fev$>gxX>&;ZLX;u0Xa(XL>d z>p3+47Xp!J&lp~_H$XL9agLanc$rD7-j9}5MEvCtcUH+06H`-PLKDG}j0O8?u@D!6 zML0^FIxatp3S3sK7V@-SdR%Y(^y&NppB+!0is7uL`k|ic9*e5O)$Nlki*EqCmMR}@ zC=`WQP&;~O`ad2UJs!~n+4jNLbVnV#QWWOhWXpx_9 zxoYQ7s|0^R=aFJ7on?`e^Xic&Y?xQ$edG`?|eKYqoCj~9NGGrIi3!)RZDkjvT~qJbS3U@ zNOt|wgGap2(jJse=<+#Vn)sZq{IlbXSCc{FB{V`&0e|pMuRZ2-1>xYvB4%c#=9$kU z8SckE6If+c^et}vS?bZZJ|?_qeX0lL;s2#?vwwJ#0~(YUhKY-eS6vJSN^QpM{t=`; z)Z_hSYQaG3W#WOQ2Q|D_Q;DTG7h{B8Tq)o{Wd#FzDSh>;*0^#9^Xc>j;t5Lwk%O`B zNmx7*VtfzS^_z6@*4k3V6K$RW2JOLqei}2b;s?}x9j?dnTD=xO_^Oq&wurfSO=rtv zT7^=?utjU8py-$#kD0>9!8O6!9PtCIQmSMbQ-g=RU{_lBCn>`aJTj}Opm>7xr$s08zh@8J6(g59Yjwc4i7J?{FwWI z$`HuvPSKPq%}zhB(cup~@0Vd4PKr!#8I!Q_GM6;E6DS5imZ99YIX9G1jhlH-jIR%2 zHXENeH($D>9afj%UZXHNI;skxs5}`@oJRRl!lHo*;t1K!;y{i+oPkq*i4K6x&x14F zcjnZo+tJ2QC|seSdM5rgoI$3@!ZP6zk90gn!oQT!y;y_9yQ)L}4CVP+?t)&Y{UA_% z98R_SjCVC0q*5dGV%Cd@H*E=;JE^vR0`N>XdTz+jcz0GpXPSi*BLI-4@=qZ&LVmNg zO=gc_^i(aqw!a5Pl5w?n#5o-wh|S~I72|KeGv7ig=*0`%fw=H-QdTvUXhBk?&GnYo z$U!Q<^iDKRWnnBm@?P9xwC%uk>ir{#T6qh5GEi{BZt&-Qv_wm&ZDO_NbifE?y=Sd|Jv9^8w53{%VA?3`dDtd@?+mQh)i< zwrikK213PGmEOdAC=DZ|(!sVo>nh(OXS7+QrG6c#n400#Ju;q^qN(2s)zjP0>ZdY& z>60|n6Uw--SFO)gR-LRwOs#}+YXHp8v2A%iiZq@vkuM@gWshq z3%Dhqh_k@2C**OVfN&f-H-0FP_X1dCN!Fe$&$n%fLch&(jU1Pi7# z3Pm5}m#>BY7%U{IF%4+J8{C*PZjCkZLYJjq6>bB4GayLpK24-U<@U7O%k{yZ-O z1_CfSA`t^5bXG(l1)W@Khbg3=#!2taA!E4@3$32DB^@&!OibK)1A)TTL_Dft4^K-s zYb4nbG6UrT3qVmU9p;O@yGwH6L3<`kN9R1D2h_k+8_;AtMBzM z>3wCStyo5)TIh0VI2EM>2OgwwCa%ag5if(AL7$*M*cd1y9Mqr>eB}aC7d@WKWwx{3 zyakCuo)k`T^TBPcp5*lsO^{^R2bAjog0SG%LqW8hC+%F$cWZKLqPggRY=<YD1?Xq@t zI4X>-s1(J>w8zh)oW$CcHl5iUG2}&ju@XlU0AXp zEKf?ZatHRdI^SYWT3UV86Y&pSbt4XuI`W9s!#x>2+sw0Zdrsc|vS_Gl@ai$@l^wsU zc3aPCjNf!5XP|{xH2`ZjLpgFp7Ym+fM2@Sk;`q6NYl^hDI&0A63;fEp_b*_UWtIru zEt?u*w5iZlv@7UR<~w|+-`VMhv9T}N5B#JLjG+!Y#QM0J;i1rhvyzLbS!kp7`e2Q% zJ$kh@b~HJPI%v}ouL1*jYUTx~dsX0QV1Vfm@_g+$wa|!l(hi$UkJMGvPRVUcA&U5F zY`M6A*U2{CUwSBeW7KMvLslL_>SqQ=x|Uh|fJV^zdxjGZ2757EkCgWxmVT349?Lj^ z33(*stl`_o*8GU}`>n{dJxd?QwVo4*CN_HeuiKQtYTvQrK*9Uwwl+;Z%Auh7=BcOK zOvBq$ay_Fq?>hdn>Txx7aa8;H#B43MT1qF6&oOE1iEdyKwtQ>1=)L*!0CrO3@!-g6 zUg|&PXzD)O_umVbF5~GTl42e6=cZ=@lFq1gBxY7>^$5!Z*jpWgEn5vk2e1?@BP5FZ ztFK?H6H7BzO@4b9LGjpGn=gPZ%1+S8KpBXYsz_HOH$ zq!~2sdZTfs6W5@*r54poVxu0EL6PFuR>LiN1rY|lon&&`EdAEZd|!$iZ`BETDPb>2 zn3z>jUedzV;rX@q(~Xb3akPl)vu< zzH@E3F2t$L%B>s9ff$629D;r0u7bE%MnT)}xp!|=1M2AXvA|!$POC+6=!?k`di{z* z9FjXUU@r6@WyM)6$s~pq`J{7lm<6yC1Z|g=&eg{l`x7`IeHf`^%sf?Tkw*4x=*Y&meUX&mSr=<#ux7_qtIHx1&ejCw_4SsN+NG3E$eoF40;Kec z525;f1$A>L>%w#Y6nqW*Z@O$zc$aTo6&H5&P@S##(E7^Q=!pK77cU-x{MQehf+ndf z76Qk|&7f&YZj6fL{bthjmz<&rI*ecV3ysB6=ok z7PvDvmU#!ZUjH~|z75?w&aa*m7S9qk_71YKNwQ?0&Dw4zqEw0xJUm(Q;`b%j-sRlm zdSLOiSWA*oxjW~V6Z)x01fR9nVBicY6@o<y&*w z$PZrX=A^~F4hb~EAHLS!TGDDa+z;oFn2ZJQjQpT)BGFh3k=``)E(e4qE_Tc*-?eO= zZ2qr3zRHzcX_ZIQSl+-x+F>1RUIa=^u_fxdA~X6NVp1L=7#X1 z&5w_+oQ;n1fkkTQ-^mUaV8bZgx4PJ_9u+?Qbpp5xs6CRe{z+?C%G{lSxKeJlusQF> zmf}p&G&r4!lA9t{LZQ#X&kK+$UR%*>?d`*6CW>TAH=OeM`{kvE;ac5R@TSD_yeU^E z*h*Ws5EEmA!>f$<&pyDjN#`Ao*7jZ20CK#$Z$4bJx$+uDMx_u2!fNY^gT_jmy#DP* z=(A$?CO&I(q)`AJXQB+ zqhb%v1OFfLe@?3Z@sQU!j2=30pit9q#UbJ!cevq%>>y+MpvWE>%YU6V$V6rf1{6_d axiUXUIpA*IvP@bBFRY=7LH-H*n|}ey=Oq{b literal 0 HcmV?d00001 diff --git a/doc/images/linux-4.7-12-decompress.png b/doc/images/linux-4.7-12-decompress.png new file mode 100644 index 0000000000000000000000000000000000000000..c99ff9809d540440e09e1a88a9bbb75bfb52af7e GIT binary patch literal 56215 zcmeFaXF$_i6F#Vj4MjzobgXoc-irkgQL1zZ7^L^!K`aOfRi#K5L`0g@&_N|Zx)6Gm z4njbBhdn>MdarNz@3;N1yM9n3A;~#2bLPxEGxH=sNkNvFh??lYfdj;Hw{EH&I6ycH z{;(c90)F!2;8hav&p`(j+3N?gnrNrMe~#PU(sDR(fbuN#=b%YKD|m_20lAyk?ztYE z=|BEbPx{q?Z$!t>JYbR{Jf3@tEa1dJlH>JP@3%82CHe;+7aC#Y{Kg}aEORjE5rXl{ zmowz&hOhTY9px@MofsgJ@#xaI3m2}PPh|I#Vx-y?b%^QYDlc0t5$#9KYD_o|sILxe zR95OY)h4{xNwzM<6*H({$Nt)02<> ze17j0zMQ_?V|f9jOc9FWXYt11V{=F)~vFot9pY*$MHYq%;XKJp{&!i3?%rk zKF4|RY3a~&7rp;3>z}vqI>y|$>^pLCpQk=wPP7%&UC5IEpACUmCY}cGC}Hb)6aOiL zB0{oRKluTteHOWNKGC-GXmH3rQaTy=@Ke8SZ7Js;vim6;3Gj{^Rr4hKJT-JysfdQt zt@HB#O8pGu*Jlqu=`jdNT)= zZqAlsvwgOw$#xm_eYTgeyz%paOEV^tqE5BOh2~gd?dZ*JO<@;tP0dN8cZax6Klzd4&6qP zvYp;zeKTxxN5*)^a489x{DQVj(;$ zOYe5cYf4`Kw&eDdVm)fZ%M&ado1vA5c;!Z8`QciRQKVjLk=+QQn-A6RtTVr(QLzvX z?-=ntO5ZiN?#TExOGNug4c@NJXQlat?SgTW;PNer~u5uLSOIMXx>Kz{}IZ(@g}jw zhNZ7JPLs>)oEDw=ur$WyQ=DQ~b9NM3#T0nru5uY$FPG1Hy<>-#2{~7?>#B>-stZd5 zxdq`7okVM&X5zwx#XSe?D{`z$t?-l<#YLdxV}Y)|nzubW^;}ZuoV(6%Zx8NnhXx56 zr-t)l-zvn3u&-*;G-!@^KLn8fE#UwApIE z3zz*F82D!}bnJ7$zb_F9)>uoTS+N?`u4BX8qaNxz0EOq)TIB3l*6%H+2vo^wZ!_{PiA@LP4lIyjK zhznCg&lq6r%E8>*t3JCf^Q%&8H91_KLnl0R2l|Dwxg^adg5FSO2u-QqhjZZ>K*pIH zMi}aI%x=33xJ{(Qdmp{f@j^i)t!A*7`2d=qriQLFfcQR~a$>b?LcG7?gtF--@Tf66 zYc-N9Lb1%w^(NMaPWwa!OKeO!j)uv;J$!=V%?Zw; zL}t%Oy5Uu>iae%xS2$`;$&;QY{iZLAvqAHtNs$|@_6HFd?5Gsh~$nTN-Gz)1nKMCuEA(pf8|Aa)DC@b zfCFQ3ke%c6JYQPD&^cQlYYC=3cZnln?kiw>E4g>*&-oTROi=882N2L>(unx^{jMF)Io9%WByEBp@1QssU>Q_-NFvh_2GhXRzVNgy*ERWbQp^_ zm!Y#(S506;yz>W+=5o`Pq@|V+?|f!;^i0{~QzGxy;)3BFDUmSk2E^tUSqYVyy1t^c zFS}dKyKf*5k2V{2kszv4whX-B*dT`NcOEIp8IsR0Qix?T4#i~Tti=UU`Eir-6B%<` za1l`gm#E(=I=J3695pTp7t3*L03IX;z8K7GEo*nh*uV=oHbriEc7z!?!uRJ(De~?m zf2HKzN=l^6JK(XIb@O1Z*y{Obv?{S97#bV)ieBX}2oLIb_wQ+qH&|h^E#k8tYfGt; zwrRBq~Vi>`w6$*HJMMz4eGbyDI|JaSR-#?t~DmWq5mpV1gD$L>XGCnG?}X&@(Z8q!vVlkb|W=Zwf^mP>iMg|X6B z^2SS51r{RmoP(bY$Yw8mh{_>7j9}PZt3lsM)I57mIs{yQ6X@q!6ZQQj9x@hh+ z+(T%IwLteNB|2gleP)}a-YXebw%C-0N(sH;IF)p+0BGzMt@2vHrifcq~ZDYR7w8D}>_t2~3GGBZM6XagCBI#~<6a57v2rqy7IY&JZi zNlVEQ5r%O&Pt@1~H?tx$HKcNmO}1>2L@05;SO)&nhkG-B3kaN>jmP*A`U;)$mj5G1 z2jr)o(b^Z-TBd|Pr*h2*nx_>}+tg?rvujhVa2~S+W}ZZ&EZES{7-sXdbgj${W9d0q z*ab(zb;A;vZ>}tf(k7r)*8S7-ZD>81Fv^xEvzen!OV%T}j?2h)} z-pra8$$3y#j6P@hRyUv7d$kR|0yAvk+^7!hWWFkcq(e&n@ICEFjvFptvz9r?sohtZ zi=r5uMpZ0l_E&Va2yWG_!Od(HIWTl1-_uMx)9i}H1ZJJOu!BrNPy3xWR`_-n%0Jd& zmA2`+&0=xmh1gi)(ISP)&qlLid%1(}M+%VbSSnBK-SP!j&g>owyzJh{L3FGIzH+lc zi8_NqI+*#J-5HINRKd;@O zf*dOrFNhi<#|?PQ;Wmocd(fCO+!r${)xp)11zf43RrzwqbdQ>#0DLkHeX=drzhlj< zI6zh`V#I2zY|9#Rge7N6UXi1sQ#?)-5jr+g$y~SEHcbDhyuUO@b>y64zEOQdilWa3 z)@zohPhl_~qhBGY=|=s&@}g)EZ?FXuDsDUjJ91=!j?-Zx?B`)Fe3+8^nB%)Al($`- zAsDlKeeM?7u_Jltz}H<KviEtl*{3XB zOupqFphUkGte6i%yG8FZn|Mf{=Q9?Qrmr7yXH(}-NM?6kb{VY5b?wr;WGSJcnC%fg z)zOtb?{T(XvDeBdHpOXi-fP=BQXNNKlRFXhbgj|GJ8z91?RCIV%eLWpP>>|Gj;I$+ zh=sRAn;Z4yVAx}F6T?37F<=@_?z-z9?X@mN+!W1wabT4JxO|A;zIA{C&?mmsi7+AB`7|1d9q(7GS0e=%%R;na&3lQEJYNQ*+D`3xXVtBE<8(&MOpVxJ@-- zWjjMmz&Lza(1BBe>%_e=u=dcg9W@3CUWTx0tgbi^a*+Jw#-?E7?yjw=oT{op>b9;ex|O70i26nM}1e@z)#n z@N;I9)F+HEdFz7pS6#E%eJO=rqU{rLIG!zhCgfSwl=1*7K%g8J~tRM4loU55$W8Vp*56P)set78A zt%S8|CIppGPOoKpUN=0zjGccPr=4?h0VKh!ejgeIdf!AnC677i*y{b`F$3A*G$H2? z&Rtd}v=+P_&KR|opvwUan_V?3ec3jjo*bVlpYyWmt&gvF2uDW%Cpf2}MaV1}QZqn= zG4h=OS=prakzQ9_o922C40@rNS!_j6^-ojbA zSFuf9BK+1BGM3j#^Ja!isp_i?sWuSgY>2rjsSN>oou^{_!6>yC&JMq&Aai!$SI zWR_O8Y)O)~2;`r&xz}W?GxE8zd>*VfyC_?aeaiha&47~Sg%2R^e%m#+$fm!3JEuA~ z+$b*xIU_5v5Ejp=Ttk`G$%(*0}O|}CMnCr2& zVCzsACYoUGc{;Ke)EM8ld77c##6Tl(6WRT;at+PZdhIA(8}0-1%f?r{sl6U!NX=kc z4tomG9|8ds9=x=d#sX1Tek3&M-V1dWy-qM6!)z=ROt3EE{EB zOC7~(CIYVQrngiO!FbnRj(pkwY-@fP$;D~BU7g!B zV4SlB;`3>3Zvomo+>sNn!!>)$)ycFacIHDUMW0}TGnnEWRdVud$U38al5E&Gh?2*$ zQuQ}lBTM>AuST3qu&-lD>nq4z_vG%i0U!a}Krk^+(z(9A2rm72xbcLI_hL*|ND`wC zj2BUpusFhknJSy5J&su)>XAZ`G)ws&Umf&Zo;wRM*6#!RN@IChNLEfSzIIn>Iu~g4>V6Nl^&?wWQBsj7dufcA+kz6Y(U<04 zjf};KX2sokhTRr^zl8zcd5rKxES8^!+uj4f5h*bZjSGt<7pEDIe5rjlUO?$_oWtD!v`Lr)xc)+bgSUoG`? zRuo1}QHa7Z*5x@T1tJ)O>BURF*s+raUBI=-GQ>IeJ9TC`RL?XsJ4)Z@%A(7gP6{RC z#lal{in0wyfIotwH8JJDU}9m;vLBz|0>)*WF71KCnt=snOb8Bt<_r z74OeW&u$)$T3BQjd0A?M*=$MjKg&A}r-FIHnzoQWp*4Z;x~k4c#A8gxTp=tbHO9Im z&2_G5Bv@x8mdm5N9zh@CCM{-p1i^BDv?)I6a$8L9aF%g{@^ICX&H>Shx zb@F`djd{<+9GUy9MDNEG8+h1sMob#>#^G`fX1XhLxE#B{@@1C6kZv1ZlO}cn(tyR9 zlTBR>jNXIrLob+TWE4j7#xFP7`D8Lzq32ZlklaLOo%%jr+?$*c*_f!)5^LYc&=VMn zdIH0hZ%}fu)r8qzhWqKWKMSIAW7zre40}p+=Bv}Kt)^{g}%g!vu8ETyE1ySny2dnV!Ea)$&?b7bFDrtHNMC2ep#hm zc;{rbka4@8o#iCS;lZesbJC^E!!>sry;izY#k|Z@ZD`v%i=Oc)jyZiUSfBfnWw7De zY8^Y4CQ@s~B#fMP+EbKKEd8%gC2*s+HL>09ll?2SL405FUPVAEl|b3iA1<%BN;^HB zx8QE{`U;U#dBG&9bokm)d$Ox_6w--N#qF+^>L37qK=c-8;wLQFrdfQMce|p ztA!1n6>T^W)MF=e<2R}&=1(tHG__#uEH=wO`a4;L6o*fPD~!x_Si&c$QeIg%uuq_( z6T6;tfKoc0IiuRXVQnfK?Q$`TO1eIZzZ1Kfx=zbI@&g3992+$=g%#R9gXL=j)vJu& z6Ygvc=s6?L4$XHV7CZHoTk_m1nh5VUWm)pi{+dZX?Ds8a2Zfq;FzsEac{foEz#7Ql z7co)jp3ZG{ke@ac9pzQL|Fv!~EJ$5k+=jo&ceHT51UH%i0)k+gNtdGejd)V1kmKuo zxK52JqOOoFbq`d?AopcfUs89RoX)ei-6oe#(qCe+fy3x2L&V9CxzotI={7JTv{W9q zyHnp-EZ$`8ksjQUu#p(FEvFCEd}cA@r6=fJ3BO|EFrB&9+sB14-6n614hfjU@FuJ` zjH5O^>GEdkJo1;j=WQ5PR{N3H?R*BqoxQn|C%bP?+vt|ZCDV{j;35YoI3h3xxjfYaU9Cftng?B+sEM_$z-5 zR!*18F7uBRp#b8a?bd-xmKvwgkvn_W;vXwN3RW)iKr8U~E`L7=Ck5Nptvea@?;HK| zj)|wh%ALOCp2OcRD-+mm@k^?!zd!uf11UPN^6#NW-1z0RdkSE?|6f!TwMrqVq~zO= z)%m%&dsBPO7Ei=4eYqI2)Hh@^m!)063hDt~Xv6Hdp2^NmVNV%6^@Ju=MQvR-mrNpO zQ-1?c$+vEc<0)yXsc)gkx8}uO)$F-EryQs;-%i)a=G>U6+})XaLaH$NeHSX6EjCL^ z_B&$lP5~5B9;yPVzP%$?L+7HaZ4#II{Y~b|v`7xqZ;k(cP;mC0*U;>EjEzE`NwY$- zYzQa9&gfd!I_et)*@*z)vvsA{CShQ`$%P{1wAc?RL_#h)CpE#_UAx55O!-|esODjv zI`tws%a>Y*w?IvSxumcsTP$1lmr4GziUYnI$J;7w1x=b_9l!Y=eQSn&n=*w>UEQFu zokvrq+KpgRRx5Y6(d;FYMavyJF4L8^L!TU>Y8a`a7&oqTt^#dBdgab7`3QD74Sg>s z5fCe3-KR4nwQ-%eQP>}6XU~q?ZhuQFh$$f2OfxWPj1CVLTlfMMEhUzXv3XE&OF9P! z3Pw2A9)2elSW|+eB7c1=6_N5QgdwlG#@R-49-5=|=&uggkFSpv&NK*6MKW+WZKz6NCr` z^IhhZNs9+|{5|IXr!fH>JW?iwj2lt#j8{^-YU$CwOJuX4CaPbyJ-53vhqlfI@H3r} zVFauX#Tn5fgZ!lH(84*uHR!Q6^lZeaejCxWgWqQGE(JJ_6J|bJ^EF7A$6UFK$jpb& ztui!LFX`PUUptP!GI@*SEMZ2gbb&0CJrkI%`1;X1%uQ~=8MF;y+b#>BtZ-Ke|#VT`?$xdCn81rEteJw%PiY;me9D4ud4 zYyQuAQYx}I-wz;s)s|_1T+eir_FZ-cX2mrSayd`Ax^&2ores;?gTXaKQa^wv`qV3} zndb2uKoZpn3My;*L0J{C*&ePH62zP&MY)Rr}`Y4Ic7aj?#tiv(G)swz^NmT(Iz{(v&)dVVj+fcz(1y2hbcI; z?-6Qke$p#V+nLti)wYx(k=L)MaukodIO{+eqw;dKW;#u=tkb-5Ab3^c_W}R)Kng}+ zG?cX>Z`6C&qM?e-WN6vRvf&FdVMjZCdqXtjXZricRkLvRgRnGgos9U`Tl{`j>I%4k zUR_c-hc6#DaMg{4E4JXD{0A2jZ6oC_JiUhRWkC;%-ro=UD>}%WLjM{AbU^+Z!(U_g z?Op#G!(U_gYYe|7@YfiAi{h_m_-za~BqBj6L_JHl`-j4uq6X++RnOFJw|KTTQ?WTa zou^3KXI=^88a;#`TeBa=JfKpGxh73wJ=<4o20FFOzPF`Kt&N1tEuNf+bAgqB!U1gf6;I-5Of&LK8%>s4knayxnF5Hn{B*|Ry}0~#?|y~Z!}yW26`{F) zQR(`440fhF4?EeCjQ!r8j!dY?^{MATkx7?kg0`ffZa2HFsnmq{2u?k1vhU6SW^e?x zgA{qL@_eD)Kq_8@Dykf487bSBv?ps2<~=laceiZo&EX0}(| z>3GmI>ewPpnR4iq5F_rx=Tl?7dLbt$Iq#qby*;8Oyxeahpw{kA>mfs&|i59O@!e=_-%n631z5?1jaRfndc41=d~=C;-s zu~54(y~j)oZV^B(;^@{c<}N73LcMUsPY0CPe0#XU>3HArKLU-9SND3LWa*sl^#CBn zW9zb3p`{kJptWv~!DHeQFA2~HIqVdB5o9rlrbBCnpt5;LsOR+`i^hTFHx?o*b;-s~36F-tP6zBI>LQVI;-Joi&H=d0@^uOPO#KTiv0%!$n|F!M zpg|Yu(dGq8!R!LBCNo#xtEH>-9nCHdFRhIT0Xdw6Vrlla&5cs{1|GU76a0|Mg=2IW zlm->-5=wBWK?(7NFBj3x08t48kb3rrTYiw>fDng%g=Y{i24v*n@|HR{6egf-A6d~H zoyVB`@PeNK=Ho_=TDr}oJk2}t_^QJCSN}*IScG;HI>Guj2pc@ zdF5{OOE3fy?hG2YvzqIfa{8Qrt>1-ujMvqp#oX+B0M>wgN(lp6ZO@ z8lc|`hp_aU(I=n(avZ;3oM;V!%k@RB`+&<}3`HY=kLnWMD!?cL0GA`|=Jx@YuLE%T ztQpTOd^wZ=te~A^WEg)4<=qp22wZM`5Q@KYCkPhT!JzS{Ym|d)P!%BT%Hw_N7cpSv{XCp7{O$g~Sy6#$ zI#qw@sY>ZM?bfWtv+r$9ItP8{5`DjZ7qnq!6PH!M1seO68bR^IbQI}v}oYGA}FU8OKE9Dn5kVC4mJ1ycCieSZmT zH^q{Q4S(fUVCCj7%zg2S^92kopeeSsuh;<#rmb2*#}; zmn^ESppYKfp*4{vVg?eXxV{tn-LRm3?gECoQoHj^Qd^Q`Q=^nAB^z35^L3GX)12DoSoI8+9BG zV0sIU5Khw6aBAgV((_twpMu&O!7woYZHyfU>EbMtdxm!)D=tw8%v$Mr#-u4a}qjfB&RCES9C6hNy zHlWOG_m(#C`DIc6=8J7Ai7ULbNz9#cr9dEv&!7pk8ya%P1kpr7h5CieVY{uhdGyHR z(-N9upr@f%PQ*0_Olh48wm%=l3Y;Qq;iQHJ@1^b z4b8z7_B;3TnYF&jQoVUUtJsv1A!0B8mmUOV>$ombe9`wDcNFN)aK6)jDA=PHJ(~-) z?4sh}hO6C0t9lM>E-yG3ba0)X@y`02gMQBFrJsH1kSo2aw4)Kez*0MHb^B5N65q?xrrCvU;(|lLU0{xb52K)%K zfDBa8FXEsv%T_Rw77d$6XuP>`O87z0#pHhy{}x={UOB#eEJB9@029;BXSx2B_AmLH z1IB_*c!?oX7i7agv_fp4r`SY($M&cuUx?YI?yB@8b^ zQ&ynyCH_}U9&}(Te}P7D=3$ON4xQ5b+vK~aCn&Rm-%Y!d0@%ysEgDIg9k#~Yr3VHE z%svwf-pRQs5D(^#J_kUPr)FS=mrJ4oOnGBLLr}~tK<)DXQ;JXdd`dKlEQCz@Qul=(^W1Gj%`X&Tw52cV45QXx!3 zLi)z9yy^Gp3=IIm{FMN*BYzwLNWGx!Pvqh@rpou9Mf{`oS0FbqdVcHvFDv-hE=GYp zME?DtzoLW8>8~;TJx9M?#9w3hYYcyl;kN|-8pCf<{Phffjo}Yt=nQQhtB*_#XVVZU z%Nxh4rK_g^9w;ufp}y6J&1EN!1boJ|QKqX`DnolhOwkB(v;Qf@>yKOtk=QU5+1*+M zbTR!eK{Yvt&v~YcODBdCz4ICZ$a-x2QG?l|W!_-`otYf0wkK>@PAQE)I8O!dV< z`G$HnOmBMG{A6(Bm>9oW zh6J|@(h-nAOjR8^eFvH$E-4>eg9FOCDbS&>h)q>g+?=cQiH4H%VgHM*Fb`aKR>j8m zCD2?z*?+WDf8^Xh)5`yi&)-E#;ZC^Z88_aB#vQd?`kg!)xBgp3&upG8or{N?S(42p z5j$5}Q!eh`8?DbOYJ6vxus(uAegqRF4^c&-^n>R|YACZIilSk=51US%qL77CZ(}Gz9`|PprAh9i zc^0JrDwn+^8PI^df(C9=?VWq2DfM1vm3)7H!7S&&vfiA#2E-2LcPT(Q3OV+HO+Dil z7}vm+V8mDZRK*7On%O_jb~>HkzqSoNKsnK)$sr!AmXDhCN>=64&%I4xU@)q)=Q5{`l(i+<3~FCAvv`bN&I?VYmO z!f34-0H?SRkLN|4U&KlJ{F6?%FlF&~ZEI6iO8nRGezrWg5#39&ML~_}V7{aDiuT0hhz@W}$Sh^w@M-RzwbAov6S_Joc;HyuT#rGJjPESz} z;E&6F(N8@4wDj|7$+9lm3lXW-1?}okhYCwXx{Nz~r5B9W^pmYdBJZ;Lm7vGTf@tn7 zIhO;S_d1pBTagEA1+%wu0NS#g$Hw|;wr*6cmS=ZT**0#M<4=b%DjoJEDM+6IgG2EO z66>RUVB!WYXPN#GP>b|EQ@~#>V<@u9KiZCRH4CYC-*C!e>rT|w^EfbG#4>Bw*(*L~ zFZVWW;{0Aui#j2zdBM6RGZuV$nI3{;V*LGg1Dgm=JP$GmT5MoVkHWsm6>;-)wtrmc- z&k>A7rvf-LlldDOLnb)r>7WfU`~g3wMm8V6**Edlse?*6-K+l0PcD`}0g7{>RsQUK z;UoYaf*L;Nc>zBL)apdeJD&w;`G&b8niXF0q-ZLX0;n$hnHzq+ol(inw*_LFPz+!O zb|IMbH~_Gf*l`j$9i?+r+H2zg5O0k4JW~J7oX_s<$%-8bsFB16o`-Qs`P<@d^Fw z+t-P7?c4R%k1*wWA0BX>T-jb}@s$!KsE_1Qgs?uSb!2ZsxaqaUk{-aMH+pfci15Ii zXuCJ4*R*p6C8xqipn0I!lIPQR1i_S%W7qv>DO!2PDT>|;FQ&+|U8UUt@mA~QNZscO zV)NB}9xF4i?H%x%1*GZ;a-u)9Cyc+4n6D%d-UX(>rt%JJ1dzbm2UM%J{aEkuwdcwC}<#EpdV}#Q2xLjP6Xk(=Wn}&5CrN7 z0PnXDRbn_}T94~9qJrm7OWz69Y|kbQmx0eo0l$*@4pw8hWs2`A6pv?shBdREd^5#z z?oSD7R%JKP!bvAwDgpZ#lAu31L?Rlt_KET>L>~Zgc*#Mu1FeQ3;vGgjVED!n;$H;d zo}-Z7$nul_T_9jB?2EwxazX=$S?ykiP++5Iee@Ry#rFmt##wX>g6YAiwRRb04Y-4O z>D)%c%mBuGcdU3g!mx6^(WVt*UDK`y9FQ}PV7`jd4(_a=rqcDd%{KrYQzX3k2^iMOSThA+^%QiwbZ7P&Rxf4Rc(HM4 zeqcmzkHMc#0OGCqm2Q&+>+OyeB#F)xLca(MdcCV3NtN6-b01^Ve>EJ*)8SXcr9Hz+ z6Qf!^Q@VEexDuWoLz5vfh4^e4mi+j10y(VJCR10q*d&??&0}#P;hJ}zu^VfeQw#WlC|X zUVq1auaW_crESp23dg&aK~0R1*$l6k?2%jQx%7vFj>R7Sig?T>V3B$4?*-Z@(XN$? zadU3~m2av`*$5rlPT6r2aMZyvz64(A2hDHs6+lgIAz}k&O^P`8wUJndCKl_D6evUF z^+B+yk)`ft@4`vpSad0gRuNJ^KHCdtVV0g*cpC&)T<^;uZih%cXRU)2YW-2ZlpJTt z+sD(hE3}8Q&deYE)7vnH=1KL5up?B*?&SId3buJtjw{0Apgw~R&PUUDZIQ^UBxfz; zr;2Azl7~?0d)k%kY)oUJKob)NSWPv!>$wp3A-)mB1$f{{d$zYz3o6czT|56>6bYrg ztD&(jS?7jD%iIly4W%A%4I{IZ-yx%w$rzHqx&jdT4YOvv?Wg0?ca+ZrQn_=V;KB^3 z0){w<1CAnHxL$%}6Jml}!rAzEXqxnAYKh=j9u;gsQ;!;bm7D^x{wxVOV|Mq%3^ztf zrn7oMq;VfTQ%YI(?XYyaE5~W<7~XYYq4y26c}}b6ThL08EyE9w0DrI5XP+^j7ay+( z{A#)>Wr8f-6>WB2Tqd|$?a;DN%n232a_%{SXlt&cJlyV1tWZvrxC@w_E!6Go2De5D z!rAU4&|HO(Yy(B$3KYqPb%m-=T?n!QJUQr>Uoor0ZucJ3%(-=L6?guxgYnfR+$gE>Vb=+wt_6Ls zB1A4A_U8y@J8@q;?m=IX9Sf1E zD4EFdo;GSazqKG>b!<5?J|u=B4;nsT@nlR`a-t9NvE|(xJ-SJoBo-Brherq+>iZ68 zvnvL{HOUG9v4rFseZX&rK3SRTBRb;mF)mGVHKMc$wCH-x7&5TUzSf9O@2@$hNVYgE zU}+NPq;5A@zE_8FYEv$cH-B+gfJI^xF!{y7Z-j)k$vha!B190g-Y#?H>Y%N;R}Ycv z8buCszrAXXC&Ch<>a#6SU8jp&VC9f$fpi#$@mutRG*i2fBGa7jIVg%4L(2~zkD!AK zQsbPpbNB?UZ78I(6)+^ED0ph`yY*^7k#X6$k2UiJ#p-+HqhSz}?L;v7pr3b{=9JEY zL}Z}W=-EsZB&Qdos+=C?c899R)#6NGC5@e0&@Lksd0$;YQkJiXIjzmQyMg3;1WDlLwu^Y*hWuZRp{A? zPJCwWIHH#{tV=6ht^?30G1thBbr8pQHqqH4!_M)OW5soV2@R>IVHjYZ4`h$9Z`>Zv z)AXk19fmvW%`nq&mt6~J?o%>lw$0klGv9d2Qx zTs&gJY)1_7lKU|iQ520;zH-iIOR{egtHsSwjhvCruA0OmR&-;xw{lxYxkwZ^a8)hK zAZ=BBdCPk4lVc=P3#V~W=->~xMnUuOq2$!4D0bH_65Y=@so&Awzhe!%d1=xRrf=RP;IW7JK>OiYLj6HdxwC zk*5Ea6QOjmYB$$~>6ZR3zqxUe#6NuYOtiyC) zPYAsuJX2g0eBxczv7|>AK3`yzZxecGnUwhCjMh`a>s{yBYXxoA}$`^Qhl$U$y%bj z0&>O|I@ZPL2Uq3+2@UBHKMBT%2MCV+&tHUv5A~1CI+l!sd4o|fV(>CWKC%u3YomY$ z>LrM(q7DpzCio~e4>^O=7t3W293(s=_2*x<(a<}PX{B14Cn`v$rKROscJs*Hx)n-H zBXA1SFJ(n_Jf#A!QK4W^yM;sr7+57KJv zy^4}-J&<;%I7~FzgR9V^7>WJkJ*vV&pC5YbjN^a!q|)4O}^9WCW)uzgA~z zZ#iT1U^~&C*G;;on1qG9*^~9k-3tJ{BxWW6kT<&~oYfj#wro1Tk7yXbDH5Q!(zu_R z$!lsxO~PE&uXGzJ0&l%`HX~rhNtj0hE-dNZ&T|ILYu*!#n;{UZr zj~)?WO}<~)cde@g-ZVwnZYPV*tZ;z$l^S=dWooN!tl-#iy{Je4t8I#N65sq&bw4$k z_>RDvsz}s(t0?ds_jMi~o;NoG&bHIHuKo#bdyB5Lg^;puCq5$Ex=J zQl<_n;>Qo$?h>c;byqC$sZ$a3;_j}c4Q~5N%JTBJNMBL5+MC+i(G3ZbgC<`&^-7|s z+$9F3r1yQn*1amOLoQC!9pQitqZW{3cp0BKe*C=YL>>h>$^Y2oo+|X9-$$tD8Q=9m zC42p0G{4S)u7EE*5?3pBZaVneZC%za-FWa074&P; z6pXeS4@9i@75TS?<7bLWJqyLC_WNMp3n$JLvGLFe?yG6rb;9eMgUQI1QP5EH1rSD@ z@N&q3quI7lEm$K6~ZH`ntO3Z{A!=(=K@bl2dOZDIVRZp-?qP;KI$#y@0CR zZoKggOnfW=`;Bo35nJ$o;^$YA)f^+J=)Jq`;x_b@@>oyty)|DvdG0XUx;m;R#E{H? zIucSc(VP%@Q8xHX?F%OHCdx5`)!F_l&aIg`g2m7`0xTSxEhZxHrYr*L4+AI5U8x~X zRKxK&4!E#OJUp!7JmU|wUjPgxUg`qr9HGHrCWb@LxcC#ZMLa$fI0E%BzS?DpY_6v&Cl#x(;spW zxNOJ*SaL8*@(K9rigv>g(SFT&Bv(Z!k5S#3{6&!lUHp?(dSWu(<>)Ee(9@DzUkRJ} zj+un_6j;=QexHmRTKT47u@YWAP8qA_*OlUy-k{w(MCZybMi(Wy%njIYt=X+F)N_r2 zE~)3BDNhaDKai2VA!q0}O)7;~@1CaBV~uc*aRCt|SzG`&S@c_;Yn z6oV-w3?WGY9iV|vP1A039`!t3<8y5ERNThvz^1pzkdR34?G3q<+u@fuIPR*bsNAW_ z0<{+7_a?oCR@)!O_>SYB%A$M%F!NU``?YC%in>iNk1W}Kfo}PQU z(L__+xM;Qyz3aDw4(?#UG1qO;I4{2t_-J~y6TPQGVuWpvcfm1nh3cs~-^r9#%@VU< zKMvu4*s(JQ@3u{4PUfmLlG*|z@JN3?x1MWMA3a@S*`brB^!kzGY34f zAYv?j53>}t7r!c{2)#eQKneTGGY0RgG@Ql{9%!cCg(GHWEIYX+|2H52cbODv7q|s7 zbc$z=eFB9>AZzFh;rc&YQmaXPA&2u-HgpQ5_oUU--&)wR}$#6eh4Gu7LU9lDP zSh36#@BnbRab4<;ucmaWF+d*CBBM0>?1=d#br`W+H=aEfxL*D(* zeo4of%gFCu=e8TZIn|QpZT1FH@nSBK z(h1onk`>UoRRq3(Bdhel>6g3B?Y?A68dWcRnD@#R%p0MOv}llwJkZ?drt3XH#r7MFa(fgEd9H@;+Q}WA61DCv@N{yf)v+M{>eUuv#%Z-^Y<;`y8#~ z1kvwQBVYUZ`9;r;iaF0EXI*a#K@HW)V*7G2$L zn1eNvP0P=(oFwj1<}826@mv*jxqf+b4p!;y#pydb`rwtQOIF)^HMT+i-eT)fLVW)X z?mcgI10b+g*z5NF8pcRYM z9LJaC0pC7?XwM|QQs-hWTTC{gB}tJgH}L>CBaNO*LTqeo%_o`MgC61Cp1?vM_8+z+ zEmAd&GeQ~GoXWdXD2=3mj@yXOY@aY`7nYEGAKigVF_zIFrm7#zKpTNtsr}F=fK9!6 z#Uk#p!dD5}AlX`4T6kQ47%&AM_rkvjzy_o&!_=I?m(}q2?0Dwg9|jKE@R4J8cX!qh zw%Y}mJISg;7IJIfmsa@^6h_%aMny4)v#LD@4ckc0XL=~>xT8*Y?gWB6M7tG>z59tM z1WEhfsO$IM4Q?FNX(;MN*(6kBFwu#6FidpObtGp(%D%t-jqo4q8UyrVuyU8#KC?&e z+9CUkk`xlWMccbDPmpkc`+q}sytcNM$7NPGO)Wiwj0+~#>7=ml#~8>S)S0pCEOoKz zF0xC$6U|pUbKhlQRIc{RmrziEzin>*ep03T+T$bLEeQe__X)ZfLsK7e826bOST{s* z+kW2kfV=I)HQxOoTV)J&xTsvaAY*q?-DHx@@{5GiH6#PF+2M zjLjHXlq?Xr?{1}D6IxQOm)H(ne=Qlb zCvDmK`k-a^GBAO>7;&8ReqmPCUYsH7GW!x>xylL&5**-?d=9QSd0X2La(CuIz*7qV z4K6o;!KVNh3KOl{CrASCJhK-BOeFYlLIL9W8-D&*l5GcjtwDOY0CJ>SDA35R+$o8h zF=N9&t3cYY!#&Jl8=S)TlZ;hEKOZ}c8czKVBpDYx-o)%U+jGL zgY-9r$Qr+=Pu-k|iHZ3;9o6X%nH#6YFM;sKF6bJV(YP!k z(&N-{dVhsMf{HsbQ)n}2t;+-kyHn=?8Zd1KTdknshCithKDCcuB|P&12?d1E*{@|H zdR%8&`pHz8a-R)W4MGP58<@MlOCgY}qVN^9Ig|?=k+AbjR9lvw4j^S1$)ujZzrB3Z zFI=_gU{4qnHK;leQj)3=_uM4jO=e?tjte9l$eMyr^bd@T!?%)Bazw$UvDuq;& zsnb>9FW_J)QLLm7ESOgK0+V8V*(B%8K)DXgb?7h@sB&#+Q7%3N9LHaveAB%ec&W?$ z{Az9Im5@MEkcZz`o|eyn`|xR8tahK>n9hy_K-4`}^$cbP@3s4h!RNPS@Q=g-`B&Wo zBFS^0zrG1IFjlb!4sYz4y36|{QpXq>RRHB8h&^i5Qol4cQEV%Dk^;l!yl~B^Pc*td7b007LxMhYlDW?Pm&e7H)*YaA$?y81oB5R{O^z$YS? zGr&px!d1C-K}3#~msb@8W=w$IG0~L$N_G+(lgw{H2H8Je2DOC$ZV3!Sw>E1h*v05R&! z(o3?)ef;=wiVn&;%T8WSE-W#T5ejanY4-_K4>BE}gxP>x=4$U#y&~I>ghLrSX2S~o zl1;4CLJouS3UmFHK9!RdcG?*x<)D;z1rPuXltXuj@#!*9*Qh1z&fuueF!}CsZEhVf zg**U-Hhb_rX>ye=##TP*9h&;%Sqv~uiLdeJ9Qm`(%uh(YF#AAcW;9X z%f27>)q{`ASD~oYsCYv!Re`RP*YRvb(>v_9`yl1kn?PCjoAV5uq7_mCGfU16u*aSt zRpWnAq)UG2$1`vMM5I*&l^R}AXw}G@^h_<)0Wjj|w;wI|x5zQRgT&UwM{#|d+1JX! z!IXm#J&+yEMlNjiXX%M*d{tnZY}Fy|!}S&OHdn9v2cEoNxDa2@@Z0gDtY_>C%)h zk+%l(!*3))2GSv7W#}_HDRGZUgN$R#*fdIg$lesHdb3;{ypTTq2(Kn65$%k+ECJ!}m<3mtjv#NMK7Fzf`Fwh|FA=YQQHxNME zM55=6E`f424v$J&r)l;j@#T4ycp!{GSa*bn&jwth7crFfle1-(rl@U~i4Pb_kND8n zsR_^Q`n=fJ^UaUXV$ZR7NRQZ5jgDH(-sjgZF6~W=F)y<9H>_yN+3d`|6#BpP>3_M0 zL8^@5@|@A&f!PXgNCA=J8zbSmgYSE~JB4t8wa>>8OXgr$>>6vryi2{6I}d8^hsaoY zh3Q3@ta~oN2WO%c;04JB3yRXZp+D>L(?>R)d9tv#@=C8pj-E)^br;yJXl5_PI2i$B zbYKd|0STAMw*`#;&X4dYu6v^1q)Ed^x_>H~7F}CCv&Q%_pDq)#dqF`D@tyS}yGPJh$UxVFZQsf@2{N0JIUgde08j1&$^3Mgmw zOiMukl!UfMm~yishjX147#kbAlzfi9#Rki!?>sj3JYXaD(W6J3CQBe|PNqNd%>xoX zDZ{9T53h73d3Pk+Hrl^Qs%h9Fu5jVPiOOdUcguljkGo?7sA;@7nwOwXy>X*bUzIn? z_iY)k10`v+&Wn+0Oww!|c;S4g+LQ0{6ZL5MA${#@!np1Z{T8m&k_BBQs~pc+>jKT1 z+P-4C?lIwd=pc+9wf6~l+$td=uKago`2CadL`vj$-`o~A)gZ{8wC#Wu|&Rk_9`K#hmM-r0|sAT6R2z5}rV#iA%dk7+yWU#9;@Wx_> zAbqCMWHFl9{+)}LTn2yY0)9tpr(UtmkvLJ?N8sTVp-**Rn82x#!=tQC5c>uty%q0^ z7%zZaq!tD(bH~KQ497rN7C90IwaE0>`y$pIZ=sxmG@(5{^!|O(u?QzD`$ADd%}X(I zWf0v?|0Li(Os(zV2u+k=05UH;c<2yYXBEUquBIeI|C2k^ufV}P;WVr~)}9rGfHVRI zrChMd?eritEk1;NKmpS41{UGcQ-C^Hwr55-Pj$7oL9^a%B%pdgAi0&PEmKhX^R=w; zK42Qm~ztNQt`5y>unZ;(Y|Oj%d*83v9T^v@Cv%Qc+PcJp`#K zD1=bSu|_TIu7E`@Y95&>0x07Sv4BwR>)Wzm(b5OYJ}c-C>-m?%Izxram%}%+pLkVt zDACMmOMIzk)3 zsa(B}&cy-4a3PdK%0O14QXPf^J=J-ZtUI3NFJUwh6F9S#Ox7-gXMpS=%E?*yds>=u z%n30;ROCQ{l;=Bu{!D@2n}L?wS}dyE1Y8D>ts?~FdWqK7)`k+1<;Kc=P_ab@Y}~pd zGhumEZq-&L>*)Auci`W)Eywbuk5M7GgOkV`zhQlEwu52YdAuU!%EiZL0_z>tyN@>-f|7<{_`hB)m4%}6C;p$9kRYKx^e;< zA-9CZzwqdTbQ5Kp->BQyrONj*J9jSIbCFCs$wa4&6t~W|5eT{AL@OkUF_X{s?c2xW zlttW|;o1DUrY5A(rhJcF*hP!a^tOR!rK_%Iw(3@SmAUtUNW{!A2D4hj^D5R~)V#&} zYj)*Vubik^b~&-(^^fYTBE-KOKtD6xWHohD|F+`o{nY^hzFgO_bPD2lbIpba z@4IFzYAnn~u^TCUGU_gca-Oz(Bwc7tP`ze) zsY&V(PIw-UOESAnhDsa{un|$fS<1mJm>MFgB?Faar;dC>qlqC8x{CleR@;ryEgslrk9G$9`C~dZ*dvSI|&|S@W8i$}U&X(Cli5S`m#OVV~ z76tD()-{q(yU{BnEp5o^r9ah13Yw1%ye$CH8e;!G$&<0| zb1+$qN#)$*+P%&@3?ML%zSrr|Pz|s~XdHkFj5@ZprLsao7KuRg5jHPAZbI8v0DA2~ z%|4qsVO?249NP1@1Ni<_=7le_<)Y;#%Z?$|J0k0IOO93k=F87yqOJt_bc@Hjl+i@0 zShEAl~=cL>+i zGb9ijo(#4rU73F77DH5Fg4)&;W-@Na@7fRO&K7eeqt0Y5EF!{`(sh?Y3b}y8)qjpc z|6s;j`dp=J^j$-jKmATc-kFxY@^cozU*Q9MjE~rW>H@Ykl?5_9;pN(?4+Zk~_ew-9qMn`Yxki28=ZHNYO&&CC)iCegR#>NY$C?Btham34{G|?$0wX+kS_^2kn7J@} zaz?Lo^a0p4Vs2oG$B&OQe>AKgSUKS^Vjbwbm`(K7E_bnoN%41cOD=9DjU1&XDCuRZ zst<{NT);`UNjeSRH8D3=a&vR@FM?pf3yK6Y8WaDBrw*@?x1wKU6>-$CqVt?LS}{8k z0SSN`Z?Ipb@>&|p7=jA^WX(zdY5qcu!$XTa&#nhR^lWuz_&C2FdN?}qq!E=ft`D=l zPzka#Sqa8ADMoUz@-15SD=I04 z-MK?I_OT>a*rHj+(0ypTlK>x|5?~{C<2A2eD?-gP2xB=;Tzj+mLxp?6&6_tBz(aEj z3%AWF`+N`mK@0|#*!M}Nr2m5#96{j<7p1R%ruLUO@SIf1VeCrL(`vNvv>=#v6i`0f zz3*VG#zBn?)d-M`DeV%qISeZ6hoH4VA^8=^A$UeJ=IpB$&74BtXctzj%+$@mN;Sn? z!%F4-42)ZVWQw+488AW;%_2zCrPsb+`%q*Zh2S$c{wnYVYQw`>=@}RvF>L28ZTw1x z^vWynqa}VzF`eov;*rh3N*4y(ai<@D1X)(}v^aWPCrY{doO#prVadQx(TpW>^=24nA>~xrX$_`AyNl3{WH~bwn7xZ)q1( z&d-1;)&YQYl}8Mq&d#={m(qger4|AZNcn67o6J{V1W2EKQwne$f?CH|Y6E~|-+qcr zvDkS@CaM?vEXIl{d5n;#N)nLG^dFddS;b1ua0BVWIMg({{Z0!TNfTJKy*d$gGE z9F5KK0XxW1^WDB)Wj|>b57nJ~_H_`eCx6f^9Odjm_2z!67Vl7X8EP~-G}okhSY#(U=sAQ%6@88o*=A> z6muHBgSATio`@VUs02A44k4VIu1l<<+)4Ge)9)aeha^ru)u7CAEkikkG~@9~u( z`EB=cC4SbR|pu~Ry~h2rd|lL&T~@vtxS^qP*#R2Mg-JCXhU#$&)-Y&y=({rcQZ z^r|Qsh@Fm=;@-o-C^;?jF87cq9VIJ%z8fx)H`l#B?sss7wLZ}etlGJTw(0(eX!{a0 znWRKxF#KQ@j}S%MP{G<+B_+~F3z6k84tX&+-Oqj$RP~31QhUZ7!G+<_U zTCDXjhzq`yG%k6t}XBvt}X5A>tGJTDRM=s%wMb$5)Xpz z-4l^5uXvENI_120l+3$wEhGkeMG7c5KMgjt8XnAj?XB{*gnf}JyBFckH8X9si&IP#J z`)h^dbykvyV^XDZLXu1vJF9&m^5~IRsrfITUJjQ-QGNo-=`xtVJUhvCh!pn&bID6V z$hsrhw9_C%GL!a*kBH!Y9sFTG@ks-D8dCi*h5orJL$9xL>>%Tjuc7O5cX0S0*zwEp z>OY2*bo0a>KEM{xJEeNaqcG)&xgxqd@Op2FJ>``PH~GlE{!guHd!}F{i)xQ_*kQH0y!c zME3hS<$wb!H>~uieS9`>Mx>qHm3nH1`}2Ns$?`oUErYth?4ANSi4{B^HbLW%K~}MH zci^XM-@SY19(#?%VIy|C=KwXZ(c(-0!5MfPgWOcpd<0EnOcBKgaJTmj$n z3s0#45+FI2i#nzd{eNRB(Xr(6VG3#+{x6Ai(VVUa03Z{Vm4nnw>V`|#AlSK5qO`Yi z#|I!PPZMcW&i=*P{GJozU4Osa7Z48f@@x{2A8CtGSydY)R-V;jlpI2b&0NaKJEmX3fQumHML83b)EW^oG$Rm9$G**e^!8aDtTI|OAVdZ4y4kux6Uo-wMYcT| zW&=GwSCMk0-Z-KqImCHC0TJ3!S;7*NAZacW(T4 zsi`lnQwt<5J^QCAaox_ql5;e)?b!hC{W_+#zdczl1kvUJo|0acukT@nvd389_;Q9<#@bYy7@7VM3o-H4{Qlub)Y!jvqdT`-Z+(L1#xz_Nm{ zk!}5P#OH)`mEG=8{?~JOQqgX9XrG-wyPUCrr1s9f;#s3>uDe!Roj?+iuj;wl;r(6s zcwi%ITWIxmh)OGr_vU{IRX@X5l)em=>rzWTC}_hbKOwTpO0B4H+Kn6I(z2Fv=yzu$ z0$!mMRr5mhbZ?J1KM=GcN*959JzUbaCjWUeOQn|HRmqvmd^>FG^xzQ2c{fZ8Q)n-%7+CH zZ@xR#Q?4zXpaGd@L7`^A-Po1=Qc^t`>7$OiJ%{WeaHLmNnS)lB=VOjQ#+(+&1uV?= zS>f}c#{uDn++pek0_}fwsne9OhlmzyIS4(|<+VGKck_Ah!NdrE`8T@ub7U(tIb$F? zAAyDox>~B`!*+{U9R!_jUTv4SP3JM8I^~JWPcOYrNm%wb(`~;xH8Jcnb!mMK(d0tO zD>8)zIJ#y)#3G|$LU<=a@Wocn3g5{H;&-0!gBmLh@-dEuLCNOS9l%N+7g`c7&lZSl zFMUWmIKu~cC`SR4$jfZA;&xr%B|BeU@GTATVj2Kcnx_Yj3oD?OFRzM_l8% z;5R5$hj|+(H$n@N-7~{36euKuWERQF}4O+sx1((0+ z7t~r^+KM_eVgDMWH5gRJQ*S5it@)G6&cDyQ za+am@J-bURy{37(5qvac0)--hnE`-TmjIYk+5#tt3={I!gt(Bg~rX!B!i zkXo#W7L_aDQ6jP;sPp>J7s3u=sF=s1kmiS71c<3PSxF1+TNbY$R#vSYm51B-T{!v) zXXOas++oOSOCrkL)Vke^hA4N)M8xv_D+D4Q6{dSqFqA#Wtfx)ou|8+CK7@XQv5QFO zTzLOf;}DQP-yeED8!uC%4!LvZ0d(TTRav1>;sz<_83)W7WSo89w6E>`Hhm|xc0MUa zVWnuatrliCw3_T<7q(DO@cb}lJlH+jY_xo~{O@`d|A~^mw;)=qV#HaE=4$034XU?` zz}&5xn+45!Ds6RQ04iEwuIm&)iG|u}qd~{9FqVcq^W_UXu=}mM+Z0Q>F zEb_)x5C}(0a1|$WKmYjfxy`KPXoDh z{ct`~{={l(o!qqt$|-A%%R|rld*tg8nWSjOuEV(NNoHqi%X~g;vCSa=i>yfDBGDk6 zZ0WHTbT8r3j^ud74V1+!d9N?&swhFLoEg&B@=9Mz7jusQ>C8vQ4UFViIvFlOBrksx z$|(qSRQjq(M1%ECv7dC}uUFkKjQk=ZtAqgjj*yU@zc}Nt?e`@AN8*n+L(nY|``&J| zY~bEVW7r1&5D9EF+GWaX{>7;q0HxJNLVZiakeWe588HhbE zfPRr1bQMppovS7Xnoz1EPYi=P={Tq$RELFixfjof->(?<#=DFag)^2x|A;TWM6t^)JeRh#S7%)Wig8m^NQM%z$>0Rr?^c zF+r*Pe2?}i@;ciDJ- z=%)7UR;j_FuwVOi`Y+ILUR{~zo8#>rh@wnc$=4KSKt3P;6)>0bg~>!nbCFzB^CfZ?%;azbxhaI5xXrlRV-9*? zCXn{aK6?^MSB7^(<-~?v`+=nq-he_uImB_K^Smmb655ru${5*AdmYD<0_h9<_2G>oe+%ll&;$Wymee8m;@MU^`qG zWQyhE?#quC1amB zP!NS@mLb;)LY{9tlTs-zv()mu9gl*AQ&d;(jvgSrHgyYsf}y91Fc%vqr_rdVQ*WL% zK|r^{?HPz?G@;r$fPmR*(u?PMJtp{(j;O`KiW+fXziz?+C|MYIw+|Ss_a*j5EPkyq zo`}G0c4KVCz4pQr=FTPZjz0TYv;clq@IOiIRy*6ELdkPT7sy ze;pbYuud?r?Eyg1iXcxLO>h#A%Yj~pnRk4-(N4H?=;sQ`ev*J_NRilQiT59$E zFN19&bnb^f)dL7!w!PR>8ReRjFb(nMPBudx-WR!`|9L4ZkKZqt>c zat4^Mc*_vv9FL&e^&QO4F`f7OSYRHy0@D3tp*Cm5Oxo4LHGnN_h|eA={>oC;cbMp@ zX`kFtoSK>%t$+3E)rwx6_p*!A6wFups+W!j6kbE|lY z>O$#6Z)G`Juh_1K*P{>bPc;N`qLU*cZ!@yk(VP-SA&p(Feu-GW3YZL6(VJdpTy#!O zu58@Gz#RNPc7U1tLUcSW(UV}Qe#x9+{GAd;<%tlbh~+KMy#%T~D=&AHFOG3da6T+y z#fzgf{-eo8mHCwWOsaC6s0vKd5I}?{Xcd-I7RD1WKY*#BZCpuFRaLaTTj=>Xb0_Fm zzdM#^-6z^+2z^2}L3jmNy1%x9bi06Rt^hi23*KE~(J5uaM5hVSh;Xz4|JKsq3wsw2 zR7FpQ2sUOM*R{^7)EXqyx?wEj4+u<5?S&r&aUB%ICdG`) za|9#P&b%upU<5!|#q*R)_o-A?p1$r95~wjv+yY;sw21WJBR&bVA0wWjMZOq;Pem61 zk2D#4`*64wD8z}vnv%xm)|eZyeKO`7??WrXWX(d0Jg0k`(~>{Llcm7#@GWtNZ5kt| z2yjSg-j}v$lO&K1r9TD8$+hC>$k>u({X3Q&TKA$xnYHf4fSC08DGN<~5aoY>ngSo&wdWw&zwrP=1>rsPTgO;hKUNNqnQA#$sXG!&Gp$A2`5RdL>-V zN2Kr5$vG>Qr1n`pA`-Gar?Y$?MjWG1zk-qZxAYE@Y zY;ee0o&jO;o6B`E&SwHbPp^ClC`-Sd1hn}1EqlceB7N|4oqpdDla)WS9xZ$z*r(7! z_|!;J>?U$jlT$J>RGL3D&n>$fHtc3$xpu=s{3Q{Rr(qm%(@E%M(XKdG@W$#dRr5(d z+9s=x>>~%x-#tdp&!ZY|kzX{P`UG^vb&$eT^3Z~^A(!vv1Plvu4V|~}N{6bA$Ny%{ zt##ididP8V48G5*Sk`IlQaku%;k*p`6WPhKl}N2RKPram&6+E;?>H)(g>9`40Bv++ z4yH}{;6I?HzY-U8jL>tuI55e3Ot>-&6{lJ*=#Os-N)dRj;dT>6BVcA0fm&8B&#m$^ zi;#kX^yaNwiqO8|C~{ar;$ya6+XMjp-%7CsUBbpYcI~oioXD<(WDU@tJsWpfNIDMQ ziP>v0Rpu(zdu_y=hnqWQczN}1Ih66W2%ZZpakgp4^LEo`;;#z;tK+bIFeS-VxR)e= zG9AwL6VPbi8B~(2WR0#auLv$m?ZCNk-*8zHVB9}UFw2tiTKA;MJbfVBsM@gCcio%i zR=dkMXUZ0=YE)Y&(bW~;$mq5jP}8n;Z|}POlhFOO@|Af^y&~$6(JCo|c}6rPq{we= zxwlh<{T4Q^L)SG{5ot9>;L(<%_7wThdEAzA_5GL0czwvww^9c$%I==AxnXvy76@X5 z0YC;7;2u7XxO>LD2LhoIG8~``Hq{LFEq{yo;OZ`YC+y#W?#H!ts^jgcT#>eUfQ!TS zpE^xI=w@_FEcdPdC*hpwCWvK>@`G#zPV{3}FMmu(>P>H+EG zT_b1bLbdy`rlS1(4H>wo z*N`JJ`UEj^d+nlD(av7}glcqzHrBAs129qU5fBZr+P4A&>t>ry3yN5^IaGQy#EKvS zM(A@5+vVj2qcDeHGMiI-3ujfwY-tb~8Uuuu$Tefqs-4F!9nmk&Mvu;ULV{_Si^|*t z{u7&RSg8k-PgP+pO^`+2F)=Y+VK?&#NHS`Hr$(aNiRa>&hZ%~ZP}?F(EoOFh%giOc zCJzC!{JZ}ln08VQ!V4j0dU|^4&^zeRrjxA6pFuPIgAS1d`U)AjLcoYc%?Rk^$s{Ky zo4kFnkC}O9qb;ONn>nQofV@wGAqvBwhAlIEMOGr<$AakXK*nqWs=ix#YkE~)i*=%I zyJXs8d;wP~Ah?biq5*i10LHHLhKpVqWRghs(W2|Jt8}&~fxvd1Tm|#vtvIl;vL1vn zA3j1Gpn>C9j5Rt@g0b;OA-C@~1ae*5qSTCxxM}FAkF{U$EhK^qYB&Pd+Go_(2O@I0 z%82E^tFQk3)Z>P)D!YS`Lu713v39bveQ!Aqy;zOccBOV(9%4f5X=rrxwd5?2Gs&{$ zFx|C&Zl}uai_oNm^yq+)_&iX%R@ky>cXQ1SC#~DMMiaUVGxx|GOQvW_YNFT91(+3Z z0Vq+)weJI0<-VYs%0auTG~5t(2oZe2D1+J5?Cfju;Gt5Sn*{X`c@TXzu`;91ZLSrK%SO5L z=P$rH1!U%Vk&Tuhq_1y*rB4MJ)2&Q%-dS>h7YON@N@uT(N}$J0%sMJzWt|ykrpDU=P9iEICB5fM`lBc)H-I1odh>y~GU-37U^5%Ix|#k``fpGI;t2@h<}R z*>~nWmW0V$$SGzQb4)G!QR7&(9WH3X%-sQXQ83YbpO^0-A2&=c_>t6%c^yBFJ! z5gL7LniXQ*%BXLryH*;64ws2CeHXhP&ZwuzA{d2a90G(CcL7Q|>S!5gN*y~cL}WPr z1!R{jJ#~#ymmZ&?G0F_eo%8b889@$e*mD5S;d-C_g7{C9`qgFk!$MzI)mg(&hrV+Y5{?zsTec~Q11P>bNI}E=j`airC{O>+9-Sx}%SQ&GK zAp#BjBM^Yq0*@mAT9i4r3FyHv0kt2DWUy+K2c{XMw!9$wM?D;O?|wxAZ}{iu{QQ>; zp6=KH_Q6>x?V^UVkg{T@-0$2fhN7Xxs!8PP+$azcj$^%px(GkN-g=^q_gPr|BfQ z`!r_AtHFd0ti5J4QUIO6H|RyY_8|-H3Egyyy08yqB76d&XR)4MIU~5m{V*FuhT-?m zfw95;z5yt9b9;Ma&`60=yga^`68fO7j6ye-Akr&Ry)q=oykke?$X*z&Dl+o&z_op_ePtm?xh9CeTTc1L0@g;&fGD~)aec*H$XeAL0bbs9 zN5sU8Euk_mA8~5L4X^1)Aq2kDJAwysVfH_oZlkYj?`Ia-4J!BOtYMgIdCq*m z8yI$EIHr>nKR>^AB{Jt2AOf-J|5G0{45>9Porab;1SFH^XKbv5#bWP4e~Pqv!k9Dz z7p^7<$7%cToo_J5!i0a_RpD%m5er;(>k;50K~3$JbSq^ zo|ng?zjYEk+b$mm5}g?>@6H z?vj$Qfx?xjQr50}lxe88{dcrv`3~H`7FTd7EEH$m(b8qIO;%y0wq;00In&r)Iji2R zV0KYgyf^Q()w-EoUs-&B)9?-RHZi9v*93|fr~7@bjSJHDyZajJVdIocy5A>(0_~@y zTO9NRj7zQE{RI+92ETJ@;2okqWL^IEVg(9Hezs%XMHMnpWY$9+F)po})a3p52>1>D zk(+iB^$e5hep$EV_t#gxWj)qyI?ucPw*mdgfOnZXn$52Kg3U1ZS?2HG{rL|QhP&o( z9tFb(DDizs^tILR+Vo`{{u$+a6vp$QfBE|T)D;5)$>;Zx-uZj#&90XB-jObqX4+?5 z8uH{K-7lZ=*I)Y!Q@lj!rOY4~_xs~IN4Jx>--?xV86B4jJj9MClB5rD8|CftJG*X? zd=!%c%dze>5#Ft&&*LV%#6P$4???In*;@%X-SB9`hTS_A&&gi?%@+Q=p+Dz_7CEOr#*fhEcY*0&BWDg2X4H4*=g?Io`iz(F??F{)z>6TSC}kH{61lOR?9p{?^GQ= z&EnZr(ygwA`gjKdam_%A^iH3ECoj}_MjDj5_#qs-`eGAE?v(2?-hoXoTZiFL>YcpgMBXrQ*2Kv>n}*#V7B>d4k$ZV6(A+91iIpAG(p z{l7jD4q+Vz<>?&E9`Aq!HwFa-1tOxNS|vR$`clBU7eT=(mg@mhZx0YpjU#h%C97Zz zjR&X*CSW{zY0|)O{NNUnGlS3H_T_##MW)Zdy_4edpt8(@QoWUZN*Y*959r(+j+X-N z6-GiD*g|V_g4^dH4_JI?4Jxh#ubEnL=sB70lntH0^wX00Xh^vj)fW$hN?ro_V-84h z2(W(R66`tBQBhWIgGQNCBa_JFB~hDB^#;5vTtWw^aWBLC1T3)J9k8#%vu2Bj!8A3Q z4a|`kzfX34@8TX3yK{$8&M*!3DGXE0S#X$@n1T+}ur8RTI)ZZPsqQSa)PUY{Y^>`C z5u*SOf>fU?ska$A((l@hl9_Ih7)4~$jiIkx)J}sobz~S#KoPv3EO>)>xso$t@3+$4 zTDI%gtgOcOH@fitcF#x8;j5qTNJfu^LOX3;n(w711>+!^iUA@egB!H?N+cfjg!*x= zQ5rpdd1M~+1s%dLXSU}xpl#b%EURdxu%5)%UJJgv1{p!$kqWydW`0MLhsOjR3@DjvE z`!+MV)}k((A=tGNDpyS6hiLE3w%JJ2%}Ikm&Kwg8a{QW&oL1~H` zxGy0-ve~1%g6#{*=lS9DnGw5N*d$n?4xQgvn5SJ(_5E{57YMAU)n>=?dvb?g^1{~a zEDz{V9EXRH1u}MzwZ)MM&}iqt^;UZ7F#fmMZOz!3Gy}>fm-`;eQzavNy%dikQnyjP#C>*H9QsBop+ZA{FpX*l?kvzt&hS1-E^5pQDAifJ zA)p#_U_d0S(eX@Z-`jWZ^ao`-Uf=MIZy)llBB5A{Qe$ZEw<*hs_eNuVCk>WHzTM?#{%#$D zZBJu{Y!Uw!fWzAcE%44(zt(Wh8Wt^E=J!m`ZYlNKxYBoflZ+F?zmss|bJQ#M%lEN8 z17txQwxCpK+xk?aW<@|dKRr#ZB3yyg{4BA8eK?jg5Vr43#%oKvOzMoRL1XQ(B#{jC zFas17W7)!R>$Nz}2-t}cKl=`-{A{zra(KZb|M4eZ;2C!>LkG_|AR$!hMXD%GOXSm(P{%y}2{I{|0T%HXRG>@B@k)?fUJpz+pieQ40ICT6sG&XjC zPGKC1wj57lnYkO=rwCH3K7Yv=R$n=xZkKasXDmv&HUwT0p2m~2grsewTU>oEyU?$D z>cIFM5nnG-17jLx`|^o(bw805m}S~Y0rpQ94%5m>vO2nN@G6(&x=A5mu49x?X>B|F zoSXEI;LDD6&xMqdqM0(r)bQdCsNYZ`sVV@}h5B-W?K&xq>4&ESa_)Q*brdN!Ad_SS z5Ut>6>3^FP0NC4@j&GM~5#Ps(T80x7;tch?VqCF%>XEQ9pfFTt_=uNUV3l_;XD zqJp+31LVK~WZskw(>r^rUlIC;$b2i)Df#}U24dejP4vJGuZV-^?`?qp6y;RUVb2)f G_`d)y8@(I= literal 0 HcmV?d00001 diff --git a/doc/images/linux-git-compress.png b/doc/images/linux-git-compress.png new file mode 100644 index 0000000000000000000000000000000000000000..3fa0e88f4f169b1f23c43c189097af7be1d08729 GIT binary patch literal 105620 zcmeEvcRbba`@c?v782P+C}d{OGK(l>&&bFg*&L*@NwQ}lduNZMh=gPBRrWqc92}hA zeX958Tljo`|NnmPKi<7PoY%Qu_kG>hcwW!zx^MpX6lKqyrZ|m-g>~+>oRl&a7LGa= z7B(*4DexP76^JbO-wAtV*;`n7aH@IC?`-5W?6I&eUBLWz!X&8+d_@B5w$#ms&L@^e z@#A%jTsT)vie6#Msb9c%!+V8g6(MaNc(pHJ_eseXF{>us!^LMMR~P#Ga^I;GUMEdE zZA$XWEO1CQTfLC|VZ$>Osf&#Gm+5Z3VPwu~Mk|8I z>RHitlqZ6GItE#rz8I-xwPU5G*1z_akWRu2>jVTF@2?*uh9RS)qvOk(l~12I4Vz@h z9rFzMRt=>4YwqX3OHpyU_pAnN^)D;=E1f)FB?6LLx=a+wvrHLn3P+Iq2KF!6x zqOLVBAoKTEe$CaJ6mlWwx(yZmu?t{5f#0Vp6hd^|@{(V8VVj?!^8I_wzo*x6N8(vY zM2O)biTwK1^fKMYAxP8Z!}9w1>4yui^exjS$&N@9^Xb3KJOTOtW;frL2+jSi20ahH z)uL7c+dWq73tQN$8s^iJXu8vt@=au=?*3G67IwOinrkO6&|tKuM@_#sOJ!{%2`#cU zXp-c-H%SYt51I1B9(NqJwC%dh{F$qXs>bVs6+L z%PajLRBYF@uus2gIZMsjTQw=r6=^ZmS>`5QKv_}9^N%k&iPNf2ebpf8m+c<%1eT|x zKMpayG5cxf`Z+jx6*IBrsFkWoe$Ng5$ybu+=bb9h2MBbC9(uR`^T~p(5&QP?#k{Hs zS@GTbBd&|NT-IaN>K{z+i%efr^3Ti5gU(LF`D@pL%h3m@$?eI|(31orHdT`ZqCf7r z?=R|c&_LJSU`V$x^E$EhFAQtF`B2_ci$UdvV*QWLmfGcvckQQV1k>iOXYFc>J2!gPr6O5&pXyi{DSmkC)H1#yrs8cpqmK zK+a>_P@J^C1m&8p5R_kUq_hr-U{xdSAF+n*(~C1S(YRUI!t1u|#SL3*Th}yUP>-o- zS}@JeU1TR0%e0&d+CSHo${5DKw1=ImKF~)kyNYjr^I5B>Jt!z|-{6Uvj5XYuVivDh zT^Zi8LHHIHEmA(HNPwlUj}4jU*GveuyD08#Ip(8LFf?UF!^cS)WH}V6QEF>WGrcMbD=8xYpkmBzn~Xk z2j*d%t>`%#wvVysM?lLC5Yi7|8^SQ+82vag57#j4wgzP;N%8HE1Y1n(#Rg1n?c(UY zfod1~>h*~HB}$o=Wd$4eIZ0ZU*ROL~j6aoIKXc%WTg-=v9W0L@AqE*6uh#sGPY5>(`Mb+%$!8U0p!AfLav2+R?9g?k>ph_WEt$Wi)^$) zC&QZZtJl9y@T^n9)+nXdH{|tpm0_jD9KJNJrh70n5*@(BKHS>Rou}~0*zTIRp;7a5 zi}6%MyXV2i_MEIZMa(N2_45MXyaX0=ic3-P9*bJz0rDI*H|+3zYZ(n)d$V;(b}Cp~ zY!=RMRa_lz?(ZZ^|2}5zkuTokC_J_^_(hy-idOJcWVN2&k9U`(#Yk$|>T75lzvcy{ zBh1H`U251|^v>_?Msdi5` zI5=29!SHu7Dh%VVhbol`G`- zGLPXst*^O^?F3r%Z^uv6dC>|=u6{bh!8UdmMyx1sd27Hh^h<+1{9u1x$y2kHuGROI zkw&S7Mm4eqas7gx;29oN5|5RAqz!zbHO4h&e1@`(V}fWagb0_G{Q z5!23Q08dq*l`1jZB%rdy=54I*I&i!k;JWrBh0 zPM%1oz;=D6iDD(eaDGd%z3&7__?tGv9hYXQ?-={IRQuq)g8#5JtR`AeCod_kpGRjXgyb>RoCsg6uKoC~8{XViX0 zAO5NB_3iIJzD_BPlSl1nd=)^!zfmiWt`AApgft2SZ|?VjJxnpVP=$DX{>&xzr5ZHP zPQvXE>l?fjNXaLH6plt(JFy;LP}e-2reIMuOGX|9mxgzsk$Zd7CJcx7Ej|jVcwhy$ z8+ha^EmodWZEb(;rNXCKTKtQ@UHB4?$duWiTbGxOoZ*L6@K?m1o3*G@cy8RBzk=M4 zHC>xg+!L67^B^EUd{!B2hXfJ-$Z=;n>eBTY*WBgS8`n9Kc+`rg*m@OcwywvF#Pgb~ zhFP?vHabTrz$5iGJe%cpyW*DE@{|1EK2Q=ZkyD&;6O7{xH_$j(B|sPG*_AjKuQ8Z( z&a`&3#jMigOa(R6HuvU)o97t6=9Y(RyQ7B3$KtfxkaqDr_hT9R7pSwQOr|#TJw0IU za(!)sDMsr8QBq_%k4MPWVp8X|m`(0(ExU~yRwSqzb@7|5MW$(!e*5lAOY@_6kpHH5 ze5Db}rzWKDx46+ZUHlcATRbMFCfXWy=O#PZQ;hgI^Ax_CG;0i>K$nTK1 zFL&;W1bs;OJ|@plgYl5@32}^`p69BqHxd!3HlKIzXn*_1Xpqi@;AhMH#9(y;cvGIM zQF`78w-*XA()ApcvWIxL%@=H*@m6-OK5aJ2880WcIn;AMZd(giKP5=TTUBwGQ@iF}1qGv%{v(+UOWR9-%C%HUhy@1#gCtR2^MDO=EE`c^!3GEAl_C~~x}+|JmQ z+(}TYBqveu^ouHS9x*WQx) zkGrV5aKg}f>GUtm*^0Bv#K^P57t3h!W?F4^!@SgJow47fO=`mCPE1{EYDXyaM+#J_ z{9p`Ot9C`58$WI_I7IClagHP-32x(kCMH~X9kqJCg2dL{8|QO)1`g`TO;0ZV;H6#q zE_-s3V1xq|qsKv;b2d_eWidiE=BnuOhwiUssG7*)^!{8ewJV*)HG2zo@OLe^ucz5Q zE4(*?2*EzZ6A&(Hy?juHa5-7lA!QdgI5n3aD;SY4=~B##a(8WfMF~WqzmTt- z{)L!0Z9(!*Y=ZduOB4#&UsFGu@+A`3e#JJA+#}d=$Lq0?&(nx49C263Oq+I0 z7BOtJ3nL<9{4UXYdhnR1F^}7D5mP>&*Y!d?^QMRRGx_2xC3QSSnx3$&lr5?4PDy^} zZjNOPubjWp=l`MbLyHlF`(rFp2YTL)3HknRoNE;B)hME?Le zPg0Rz;EDZ60uCeh-&~0I8p$htYCUT;hM&P=lKmc&mkjqLmrq;CgN?V+8r&bll|0Y@ zu#zKO`>R|{l1Xr#RvBRINw*^u6pa;POxrv)>;`?byEoAX$f0LdS_@rl{9>+S;T6(UkwT>UrM5zR&I zDVsu(jyw7#)y#I07VN~Y@cZK15{p+hkhjFPEenc@lYEQXI?b4eDgW8$b=}t2uyB&! zUe*A|J?gQ+zD~lg09hd+C|nTF$h-P%y1x>)HVNgzMy$W1DHbC(-Y)e-aPbCm_}chx zUs-_E9jMM81MQAn^=7Vhu=q~6__aiJ&yP5lE)oPU2rm{U@Zt77>5cMsu!kvVPWO-5 zGmVDasgNDIVzI-1X&Pfu{?)P!q_ z?o5RTcH0el*+tIOIbw9a!htNkcJ2HKd@#G>qC`mdqAs#>tbM-DRd|SYp}1qAy0Da$ z*1%zZv;PuI?d$Z3#Pv7nOE!aHVNjY=TV(+<8Hl^{FaQstl}?M5}8=d-f} zE>^qE!5EiSaD%V^V%uw;m7>S%<{7(SqFQWvwbdE@NN>^4l#{CrO>D&9PRg^dy=v}c z6AZU5XEdQTZm!ky*zEnh1>d_$dm*xgB~dj zm|LHBT!D`e434P}N8tA<;2F~kniLq5Tex^l)X|B*M7fZq`cLFim+*}IG!(gZ)+QFj zTZAVXHSEBH#`~d!{-HEngE;qFYkuHdQI{Qm82HmSBqvG0>ApVyqL2RCpaUb8aY|NwzJdjS?kPfk`PH?^%a+DI@Xii4XRQxs@%BN zW)0=h1$0LvfaojRb+}^pL^KXSDiDIqFu7G~{cH!Mm%+7*F?oRJI9Nd(T-C}-qa&shmM=LpYfUF@5MWPCQ)m^Wb4aA z>LBgqDi);u`Y;kO0T03yPnvX1UpN5}D3;qwg-M}keC20+$-RtvfpbKzYg-2>gkJTG z@(GvmNXDiOLt4~8a`T5GIA${%)rXhM#-X2i*EIpu)mUo^-aAX>&~txB6~?WZ2GEgrWoZT&kC^?C6N7uXweCA%O03ayS6e>uME!IFgq&}MfW9NX|^BT@>~Nx)7Aim zr?yP#{Jc&xx)LKwiNVB6U)`XN(T%12U*c7vGN|6 zB=Ixt`LFFLeaoe#{a2PDB|30}HgTCZJXxNoGEcXT_NhO!ktbqZ8@dPUQI}w4G%izN z20BaHR&=n-ubOwX?HD}?be?ufyWqe0#dVQMp5Re$B2??fm{7+cWQTS(#@S5eEAxGm z1ow^ykslNBMiE#(AZ@S%)SLBnTm}5`j#nIdHy_P%q0D2L z4aFaQx6ciex&uVdrr0dCE{Yr>7G%vyc&*po7V{z0ZDcawU{)>4eoGBb+De`m^G#^# zm85IjS90f=S8WXr&=|sKfNRAGE1v_u-A^RmXn)3JHI!1^WZz3xV5$WBxNABX{mS*hSbb87$C1w-DeM- z?jnydZGW9KX(BR7KK&M&U%xN%L%5;zIiLK}CEEg?Z zqG0<|als(F`cQG{>6j@TpG;5p; z(>z?#wVz<;CA_lD6V%a}eLQ6cEYPAnkFXlwW00D4Zm{aQG$ zk=seVjmjPU6nBq}xN$1L#x2#cNZ;C6DsE>t?lXQ3cqexw>jcN$qzJI@JoY@DYGQR+ zWnp}L>q0SmEiu`GXB-L&$p#c+I@)%HHRT;;_rGw-bt{OSi;!qbeL}Ms7vCPe{`L{M z8~g0)>!|8fJRW2*6kb5B4zm__yr?!_=~2c*o`651z&+Dn-LeTq^<695xIG)Q*DN;U z^TsadCChRjwc<#Mpdz`GFmm|bKBeMFOf+1a?Cgye+cvjim1kwW`<(G)FAe!0CIN6E zFY}moXlK83W%kiG-cu`O!@w)bpUfAGcS}W7FbHyphr6`X+SHxcheplXG4G%%5eQ^$ zf6*^G9xq5cc~3Qw5h7zXr|>f@ab`hcu9CC7?O#wQml8&=#XEe(&3ZFCy{O+@_SWE9 zd&}TG!N9_V*XT|?rqoka(9$rE)>JJfIb7YUgaZ>wL7yAw;15#{q-^S(Io-bYZPc3# zCNbxhP<8sdVhyI>3cXP=rcctBuI9ZR`tp1|((!R+&k=yI=%jsBmBxQGGplCoN(L^Kn9v2d@{av+w%=a zsOf6tv_#)-ObHdM6q$xc>z+DVfv;^ z5d-1|ri3ZMeA@Yo3Uf%M81YYxb{c=GKHl+%MEdC$T08gF-^wnFY-g-f6VZ9&liG0H zUBxOF`MH>kw_d*5y*3@s6^^M@O7Ma@W;WOPcZX^azW~llkP7iq-MIYoh5wPvuOIH6 z1Ay^AYx{Epz<0qrvCYrn-u?1(oI~$@C<&@P6q9tPe~RJP7XMr!W)QQhpxRV_lbHL^ z>wZ4g3$`S;QNZunq1XMC2L<3i`~F?#|3kZZp6o+g9Dh@mO18oBP|3-V8%||bqZG(7 zH|r_$>@KSsRm)@R?-C9;?^XCetOYNX4qFAbaKJ<&uWONlu7S^=lMa5QH<^RV;+O;m zsg%#X3!K0dfPyLc8!<&fOmU0{F2L|!;mJ(aaBlx_!EfaMw+fqG1I2c-QjpDwj%_3& z8!%aJC~nCfu*x&(v;#PSDcS`-U-h28|j;>5$sb!h_3dmMuT$^iaV)wg%J3GuU-1Z$xtF(U>H<7>fXApEZQ~ zX~)e%yXmjxGqL6q(;S{IOGPd0omMr_;Njow&VP&x^FwkZqzzHh&%KvGu-gwia0t8T z0k===j?<>j=>=&Tdu3(i1R&^_ownyvLPNAK&=F^A4CVIc6A2$@%RS1;lO_cqzj#Mh zUQZw{f`OUk8*pJmkje$(oFQ!h8NVy;CW!m2#ue~b-fU#eNVK>ov=H4PNkEdn+{em- zXb9D2RXsdap$GbF6Q2_XZEKkB-SiGoDj4%b?|&|E2YFARO13IVb8m@xf8&q&?m!0> zOnyMx#N#c*SrLo-71@`g!IR@EJxz2_4hn&RJjiiRG*Rid8^^gD8Oh{H;+!}>OBSl6 z-JbMD`y|xJfCqAP5-^h?U>zLnH9xw(Fk)e9tOJz}<`#oRM#3A{9=NJ%T-6)SEz^-? zaIV+OmhqbTP2c{dC@EJv*31iLd@_q3KjRJ0*06ix^|K`DvN2P>Tsju-Nua&c~(SAFB=eOYRUhuRLkDq+LE zv{2{0B@^*L*8FPj5b=X;%zkqfG*Q_rOj-2HvsYXJDi(muuUe_3|6cjH`SnCi&VW^o zCR7vqH^TXA?GobNd5sEfEAHvxw9!B6P7*XUiER~@@65!ViI3qiY2-;<3&ODA@3d2R z3BWFi+G5C<#Ue*1rzTJXrsi<|cqYKtkc8O^1&pG0_N)nNAv>QvE6I7nzh3aJ&aql? zTOaJRwETq}4+#t*De{G-|3RAuz^mH6ytGy-dE=GEn=getGmwnxzOUEBj+(Le8qrna z9BP=9gx(6MN?l(c54i)kY4a2YP;vOaaLBO{^{#h_=NA=TU+IU(n%YeeHqWPAkvVS0 zEWjgaHa{Y|cyuZ!AP$(YS%1|(?6|Ob2ZT*kJ9TX7zclyP1SF^dI#}eqWAGb@{iROH zuK_xUpd!6_Tqn~#T!0RQf?kvT)A8`R6yDVJ z-6hIq582){q3;_VA^vO?8W@cZ@Y~lV<}!-%4`_8bny0W}n8=SeiMp`yEvi{3m^nQ~lv1LRpAXYo=c*v}NKGzph}xnlmmKbKcAWJRtBGRuddN0K_g@Lb3# zq2jw4S}Cz3k?lx910EcD;CuAJQp|(YLf*$^qnw@KLt;@#p58g{QJQ`zRek0#G~4WV zq34gxt>+$SH5^WM0Qq*Er)2PKU#`~A=4*FL6~G&1ooi~mt$)Be1naO_C>_s={o|Xz zWyX6kTD3k}R&=TuE|e`7ecoalG-mos2a;~q?tftjZIHgCbCG}a=>vT+GNZj842g@@4l0u zo83%g&0lE=`Bq@H?xY*i-f^*FHjxrEfjWiV(=$ zpG67<JN^F*0$gV zGi2j{&1Cahpo^aA6`gc~s%y-D85fp!0*QLH^BSD*yE`b6+y<=3i5R_QH0Y5eA3xYC z!<t~^I2+!sq56ZI=D}>>iK8q zRk8rx97L^VDO6hl@~G-WG~Lo?-v_oA=SN~flH00- z4CQj_oF{VI>r*UQczjg2yQkHCO1K;aeRzpeD;mw%PulFxr#GFzr6L?hA0XXi`J&Ea z+A{YcuG+(`6euM4iaV99Q|Xm)j4(D6>c6m9I5Yx`DAVhXS}OTtOo& z38G9ZUys3Yfsu0~*-!e?4MSarM-2{oL!?R)tQ48 zytM2Q<>tj`PqPLOJ&Qe5S>kq7E%!uul9a-DcH%w}?@){E*BMGMaa=eV&VAE$ zJ}pXyNwwuQkD8L;^qZeYS!z|PjT7AI`Yg*vF;$1K7QE_jI2?YnV#o@HtbRo5qa|(7 zza+g(tR!MfW|Aqi0Et5cZjU#~S{yhib%zQvP823lZ?>5JI==L$?kBrKfXT2vTiC`n z7ra3rzGsu8RhB0g3yqhc=?Km#Bqev5@7c|EI=y zWup3=a6=YOOUH(=C|>r$Wk=2ZBqAq0k@rzO0!Mb?KUK?&s=7p_P=VFEZMr$m_6-cI z$7&2yniuwBDwS$NETQ)cMBu=tt!UI(CHq8VH$!EaqTE&XJBQsi9j6$Gev^|+ED3a+ zMlyz(UM&oj?~y;YzMC_w>=1ac zo9Kb#GQN?H;VE zkkI|x-ursAh5f~4FG@=aAJ#n~(&|&;es~Pc8>9eC{IM0BzQE8sFVBxbjD*XfJ+YEf zRVv4MBc%O-&k-$4=EEQkau)ACW&K8PqQHZ<O zsYxYY=u(rh&>bBqJ4#pI(*g?^I-gKy&RDSh$!{LD^}Kq1#cw)zQ{#8j=O4QaIvhc@ z7>ehRObY3qUm_1=A1QMF0-2=upsXyLv)TrvjK8%1AobXNT?hLbK}&x^(ahTi&26w{z4f%v z?E$c|2Mncry2tDm=8STmK{U4D6P(KTe37TOpHAA^7rR!LuQU;r!Q8qW)tZj1BK$7U z&Z6LqgB~UQ{?A0d&;oK~f+z7>2l~cw`u`APzD1G@1tAv(&`Jzj?tP`~)=$_-Y&07p z9jK*~iw`?4Zy`*>fo)HN$PRX8C4ETw6@vvDQIBC8^M-&j#aigX5i#k%#+Y!?iQWW? zGp!<54LWh!&gWEe8f^B*9T(_Wn1Rk!Tqe3Fd!|*K*Isw zo$##p%er@g^zRYfl7;hmep-}^#mD&Eh>M^kG*y6?m4f5wqR=p*bOV27$D*Mo$$Dk@ z=dtxEACr=Q?gID=mt+Ki;*HVETXfjFc&%=a8uoZ%t=Zgk8s6(2>sB7#0%E;TPlpHb z!J~pg8-j2|acOGA4d$~?WlmlHYh;YG^Y#Z5OOW&EXX?14KNRoBz+u@lw~$$~`_)ec zJ+N^0=r&_FYJh!?|ELpESAe}G!ZqPl9GjjaAUqKk(dMCbc+*Zmh*{`93ccN9S4=+V zEh$b|xOiWbq$~$)bitK(ZswHCSx55*PK+Niyq@C$5oYByG6s68#IG6C*WD#K?!P@f zje;|>#ieF~LPP3t}jluNGwWBLZ0Xr_F`FiLwn4eBCBP66+2vdCn9E5|W+t-aim9SV35t zmkS%rlM{uCiJZFBb>)bMs=EjFP_o%M2`~nvp7aJo)roTa2lNwt#S4TbN2`?KGC)pU zXVXHSrq;1N0d|asF zG{z{DO!gzlb>fXPdI=|_1kZ^&~@ISq~{wMJa#qv~4%L#lH+7c)ggJy$7-oIF2t zU;GhuIA(ELftO6mEZ@QgbU^5Mh`?NKTj1!{EYV^V|8ed(C>DU1jQeJO5-+gN3o;v& zF>hgiaPVljp3WP2dO-Ot){7%{8chXW5pGDe`Aovo`&!kF$kXM(Al71y+8l^r+xqRL zNypd_6WBPjHx%^uxV%RxdDQKh%r#b=of?K{quh?UIYb5&ryVX5^4?4?f>e+$#*Z)H z=eybe3S3rM7Uhs%Cu;t9PY;MfB~}bfx{!LJxmXI@1|tk=6Eq)HqPOzwx6!+o4{g&%isaF|@kpiLyEtM1w z5*dHY=4UeDZH6iN+iOcn>p%TvpYALM3yGr3R_a8B6W3GR&gCA}KtS@8oW#|M5`Op- zxUri$LVKl&vh&S;$BVxeuoP@wL36xBawB8|eH9bBgbRK|5XI4&9{YGC=JE1}Oq}@G zqE#k<QpHf8r7ARK!>(_A)m%%XQA# zYecwcBc6s~is41;C&zBsLtqB?NUbl-Nq}G#G_XIH7~*7YiUCW&7G#A9Z$%w(8OxYm zuufz0ZobZG%*fh7^QcM95!+b;^{z!lh)_aZ>V<)2TcR`Q%E2_@ zG0MW3m&aNH6f9z2V2O~95I2#UYxc=79fdshh@DwOU$FiWc!+%pL|kg@&sx|*n#tK~ z=HI`x(7#_At1mb(Az~dL>%1bQ^oLA;>L3}ofXCq~If8iZvA2e=4Aqv?U2nYuj|QLq zV-e*=FdZ>GY`nAV1!!|%mn)@)S`3kiG=)aX-yCY>505@~4=CPwYUfd=j>tNqq@d_h zHKpEjtPEC`Iqg}0vH-un_ycIsmhu2d428YPQL=2irW?z89`q9j`FzLtQz30&zxCo+ zl}h@)5LEnywdnA;-pCpjgN=n_L@9v_Hp_AK+oF(nDGsR)8JEX+;zFFNG@E$w@&lD4 z%|HsqVCKytmAf+fAF$uI2MbKl#^sK>NjHcFdmhPmlIuabBfNJq8HGJW8jafX^Uxd2 zNi0_9=QBPERK3rWf+p;5ANw63^`rtz2nJ)SF8iUBUYLV?qMg0Lt zx3Mgh%ah3ud5I6yJb4XNlG0AT98D%Zq6eGc4Y|l||HQiOGfO)`ekaEUs#gMv8E7<) z3zlgZS2p8}z5R#ts)T8O*=-Tggr7tA2Up1r9*Za$Ds5!zCiH-?v(@ zk*vbnAB3*jlY0uQ*w^yjlEVmvWB;NGdDnxd2{| z#>k%Z<|A z6ZDp*4UDd}R#N|Ely;X-AGyP3aS1JP|qz0V(W(eSPpyT~>9Dsh#@GN(TF7>3XLHa(F0 z?NiG9cn?+ehn6r&-Shrl;)#WPnVelIqk59RZ=}MIT>@b8^_|L5Sl(gyzpYXP+4tY9 zG9wdMz+GI$0ciGNqi7cMQm#2eB)k9Im}Zp~-jR*%z#!`UmzEIhMn(Ovt&Pej zlB_<6=gv|PcrZ^y&4!L{emHVmf(+BsXe;4}55X?QZxv%V_&6L##Y#E62pu~HR%gV3 zIklZqD91MADELNAR7E@79lUZ5@Gcncu5-vq9`}%|@P>ojce&NJ-yA~B+|jE}PZxC> z*e3Zaqh$hymF>mj5;|UD;9Lg1@T2Nd2?v%7iO!&|s34s68e(PMsT^*p#_IumU5W8Y zCT&Q#%s(c%^1qp+L@+>F<~P)DF>+>NS5m;BA?0iaUsps&HPOX>?sE}GAb3U?Sak3y z`3!8Pgu0v;M7|0AOMU;{AOk>{QFZrvF00>lW(mFESCdqh8wX+}yZ=CuW0#W_?jaQ6 z-3+K=A;HG`9C$ri`fK7c3d2?D1eB^WBd(4$Ce5x%Zs7ucex&Y9c7QCr`gie#d z0TGy0;OZ^ix*1wM-bLaj0@>M6N)Ete(G zTWzVqY+kBm{pF0HL~=OJZ|()c3J)2uaXTh$PliCcSqLT5ZvDJe33H9p5;z5HJLWRG z8MqAY^70R4ue@qiIsbm3+}5x)UXTaVJT3qZ*&2L(W1LrssZ*9odev<~e4%uJGoz=e zU^Hln_0x)!+G{MMB-Scxvlu-%i#R-c>_!zJ9Rg$<|G4%QH6*wJ2Xk9h5(5#a&DerN z(2by6sdMGwC3bBx{t+8VZBX;&a#-k5FM53YKJ1^DeuF$P{>eKwghH!q_d5N24}>T3 zHW<#kcYoVFisi^myk|bN*w)(gvX>dPM3(>PlAk1Rcb1{D`*L~sMcLE&F7;Y>mw4#z z{2I7~OB8d-nR92L?nNb+uMZ|-&GY>4#_^HNe$cY6I^*nv<^2(7MnrJp8_a^HYT1(L zcz+Wu0lbstHs($^!_9@>wW6pRi;nN*;6Mh&_??viP#P>9UjxVGgzRp6DdZSGDbt0? z0I1xdssB-!!BPBux+3x=aD(Z*?<4q)KOCV9!J)DODG5U?8rQkE+u-IOLJpT^d4o{V zEwZO_D^-YE%#BdsdcExNwVjRGOQ=DaNK75lX}t;DvgufalTMrwTF1^aPK=xHVXNTZ zxK3^ENr)V^am-2FPv4yPlKazMI0GQyXgkg4z8lhBb#Odz8(s$r94OGbqNF4=pL+TF zuah1TE#Rcb6A`cU`$L%0BoI?64&Pn9i^z7Xyc!{=cFBD5#l0 z+ISz1uZ1{WB*>;X?E{*RKo6!)}*iOHlFQ^Kn$$8 zm>pGXx#Sd+0;{19*HrVgH9Bf{d-KOa1jgOKZCL6e+bsW7aSgyVcQp=bxtb~#?Q#71 z&X#4m)+hP!2+0!bawHkfqpJL-^Opq?QM3Wks84>PN@TjiW=H5bnVZQe#5rI+@}MFBT0B{7hz~ zge-_U{#o38>frGCcXgNbLLTOP)mY2d<-SGBfm2`zVu6WpB5`N;h7o z^==fE#viF6k-fznjls2V!BveQ$YlwgwdjZKX2Sx`u$R+`t%rq5IXO8kfkL_M(3AAf zx0w?jss@5ihD3d!dtvyk1L6~_o_^)iB%-tVl>!E12-&6mk+70kwnJgbs}ApX5qZ{y z-s1rWtxW{_{as2CbUdgK)#uvD{65@v)^Q1QQe{kRt$`R zp}5^Y38mnw*{m7VkZ?VmO$PnC-qx5hbV~Kq>8YjOx39A`_-XA*Ql=Jc7S2`IdO)yW zUIukCnOhFfcKo;%E0Mj0&3XY1l_QgdgQCSsd{Qw!VYI3D{07B?Rq@WCi80TRa^%Kw zaght47-~uFyTo*yijWN`hrt1PTj2HoS|MlzvCAPypI6?iPTR#SUmn52l&v=V_*Q85qQJ>Q@vE6R=`%UI^w4x`vbdh`a1}wB^p&#%mP=V z4%<6ZHh|`jsWtZLJ2D{;Or)uH=4BA+TX-Vd%Ybp|%B0vme6lI15PXB|qa z-uU^Ti(I~g6mL`hxLQ!k6@Yv0yo$Xnk!%1tGe%N49PavTXjkqZI<%1Ph2$YVp+^Tq zy%mP5(4Y99TXZ7hC*eY+x3TZ1=dQn6Id%ke1YtV4p4%ugNF<+vREdgByh7eMS(8#& zv(AYEjsw#wS*5PMaAzIn&xRDUeN7=nVa z8{~(V;?(eBO8uod*&58x4;#w=B*GvQ_~xf1qcB>WzJHFDUbWg`DKE|FkU1nXU?8PI zw8LvG?_0l_P=o(o!>h;G0#oz?xHwl^|A1Dy73E{hj49%!5FE$Syu`rUvur9rC9XgM z*#n|$m6S{X6I!@;E(f*R-~GsOH=kN<)DX4k ziryB^I$T<-QwL@i5|QHw)S(tsDd<6!f&reFsE$A@H{`QtLr3Tj6Gtozpkx{ImJkoj zpT3|UA+vf~4^0wc5#&*r1!CAxBy?VRZdoLl+NyWtDL1M&<28 zHn0>VkxQHdn$P@c4_19`hb_x^?JmE*EWB~khTS`aG`iBfAuP#?P+|Fz^J*NXa*=ZT z>0xfvR7qdct(U2^4_vV|QG@WX-h(6dJhxV`YqJ(cP}`u!hG&SkA=2)r5DI-r0dEH zUzCvX(#$6UY{IK2y*2k!XTsB#2~Jtn&Lfq77%BG@w)Sz8T_}^nI)P0n@y9>j9hX+6 zZ`gr5oZ00|)be!)-`UT!-ZwMLk_o0r%g(PIGAhE{qKgut#G~`7yM5+ErBW7D^_yYL zwXUH~-Au2b@Y=Jqe-8L_eD45;Z-H3mw{AT%>(5)(-Rf;5bQShk#Y5h6= z44o=R&0^!bpLb@WuR-wMR7*IM~y0bpG8!Z?^eDPi8dXpPJUCu`jCE*WC0RX_*?*y#Tl>l3q3OIN*iYy^E>O)@y&KM~ zqzbO}dJ`QzM_NfCX!m&Q2I+Op)YQ}~n~Ss8l_e#?wkWF6v3+7C;cPmSA%m0m^h=9= ziGnJ{kr~syQlewy^C&!%YR+RGc##bwm5%NQNa02_5jQ(h)gB9E5sEwz-cS)XPP>|Mx)ZuY8WJxCAHYWPZ8WY+mkI{D+q8 zC2n0Uy3~|d{+_B?P7xPHvFZL+evAwxF6;R?pbz$o=&~cu7PP9GU)L&q`mqlnygD3R z)*+)&52kWT^X%f1UKJ4OcL|BNKPJ8^^S*NZMgDR3l%Af4Dge^Y#@mOSDRo>K z(fI!TvYIZRz+N|$xz2Uyxi-cY26&WqzYoQ?J9;qRW2c+N(>f~Uo7nEu`fdBv)Pi&$sSILNbghE3o#2k8ge;W;9$RDX=B`MI% zQXUWGdDMDjWE%>|M=bL%*XEY9BFH<6ihbasCfcaM*jKOaS)I&bHka}Q z9_*&~mXiGKq}#F+)pjRz2fa37Zz^kbp7DQEJ~Hk%<`d(m)ImCv1~x9kb#o!_z${m@ z<~K9P zBPkF0G~jql@t<3GZFVb(JACYYRR1#56Mb+;L7@jY^(=7NH^OYVG#3Ox@vCfXtxrm= zMm0ml56W{aBtSrsl#lKN1MFP%%V2kiy`D?X#b5pt^J<*a z&o4ROz&>v^R;>Xp&*;mzpVE`1!n(OM_+YXz*q&h#{0RX^o8I$3<~r+_5kn=c*fMZJ zLW$&0cs;ceW>6>iQ~tY=%KTGH#Ye}U)}?!VGAFXo3|LeyxGKS_f1zN;DuPADF6~54 z4i;ty1yUfnD^c%4dOhJHS8{<*av{&X@weT8r!oj3mnrqzzI?ei{WW649iLRLgp}>v ztJy2UK&>P+@Se-wNtks-KN8v+&)!FemAY+e9QEk;E@R&+J(WF=7$}squGv)Gx)hd} zxKbL$rt#Wjja8UTBx*F~9C-CN1^gZXEwdfjCoLc9sLPy(9ei)|RwiW<+0Y)D*i{Da zSLX^ex6D9bpJ|J8qG|&y$>aRJ>z<9~2OCQaK)`8C9TNn~pHN-iXjjnM7Zt?kS zcD$7wDmH1q{kEy0A;Y*WmU)w02qOtb7D7@c2X|>r*T_riqa10CcjD6JmJ``Jhi$ly zNEYuLMzUhy{!C2}dv9OU0c_qbxOfq&ob;9ltn}UlL`BQ>L@(@4AwMoVHS&WR%Qv%Y zLeYbo6HZglj(D4UG8hdKb^VdSqLP&g0#VjYpHrVF=ErK>3U@p4F_Lxgmf^qcHY<;h za4Y+|{w&upO)NY0PERDegPYagxB~zx@{7O^EV|4lq=Ua^VG1rdDXN=ijM9FWDC)){ z5?@qkvWoQ4y6jaqkN0#t?n7$!n?yS5F(%7ZQrN zDXZtLB(B?;=E%gvJaCQ3!epp;ANX4sI;7?pr26;^p~GU|-AAPv3OHt@rlou~-z1Q5 z09x^Ey?GhZX!XvVEwYo$%vaN^w5uVcy-|NO{1M}W9(Q=c`N0b(DsZh!)KLIE=?<4! zJ9+oz*Y4f{WY2wY^$G9);p?lzqTHXi0a-=XM8cpI1St^^rEN;-PNhq_b5RUXX%J9K zRyqWhPL+_ZrAz5Xr6hmz;Q5}Tp7Xx`;l*`{yH9*(=AL`*nQ>awMdl2qDEak1Djev( zNq{+W)Oht-NUp2^85hj}NBSu)d*LwrNycmX$N%-3txSo^*o=FD79Oyv=Wv0|CVYks z$8TJ}KDpeb)Aexwu_eGEF_A@nn`*Gz-ha&Xdnt8WAD{^yWjzA@->C%`_9 zNxX^<9wzTBFvFl8mFLu#<$>!jPEkyj|8mS7_(Gg^Ll*2OP?(uBs}GH9fC^gYibwL*?ar51veDdMy~9cW^hxN@~YCcqaF} zZhN7(vv|Km%D3!#O6S|rg(`~QuZi@Gu5|)TU%JYsoL0*-V_Ej&=86kUN?(TlktCD$ zx6Vy5^RSIzBQzkuswmt#CZC&Ar%aQ_sL2k?Ej1dCbqZ{wwv;FeW5g8OfoB|YT@WsX z9+vw706f~@ge-WcDm#WEm|#skb~z@&eJ4b|r2MTRZBXZnz@`<-DfD*>HFF(Lngg`| zJSjrrO)lMt)DMpjcR|mY%A`K~qA8&&Ixp-dvsoO!$kStRsuci7x*yfVnf$OQIr~-V zm0}k-w_Yyy>(?K;tj?!HryUy~pVH2HxJvONak^AEr|D4H)_xZ&QsjIQViv_A#iE|- z^N{b9GuctmzBw}@|9!}(V)!`GZ%%($l#mT=6+dihb(7LEyX@H6I|pboaku_Z%Al8*c^Tcc4tGUXl(OG88H3n9-S`l?vyH2+Av3F`J_7Ekb6U z>|0POXos|8=1$+$uZM*ez1COeX0o09PTra~13ca};0k+sBxPhA4Fqfa=`v69nUY`9 zj@o$6KbQoaFVLLy@#ExK(H~FOm%j%+dx3j%%inac#5u#k0ub7CXF8i7O8pd{X-NGS z;ZDBhD6I{|ML-azIB{aC`ra0V&CtZ5h{63m1gTr`02)RTgic;Gcaq=wX-mms^{yrI zgmmKj!ki+~9+%1!mIt_aoc7tbEyvpa&f4}jcRPFlE&W;m&p~eBzY%VjLq>S_?%hN? zG#iK2>VV@6%sjP?i^=ziF%MjU2ErnOL=@wkErOH$a2i&vy(@El%W!eU#2<#M(!$K< zonC60L1id|a|T7F84JU$WMGbRC^}X-T^+B*wbnCS9C|#Q6@ojxkJIUGcCfk zhtEPN#(gEC`UOoYWY{w>8O*Ly6-~--()O{4y6d7vZ^90&mymaOh&0zr^0>0JeIclQInl7RkS=-F;Z&)pVzZB_58o78ToRD_AyC)S!vZ zR@cZi4%GPQQQ<{+e<<4qrc4hN$EF*UP z&h~qYI5NKo)iPd{Zg}B}*i;JREQZ$;f_su$>^e}e64x+fkHgWe@yq~4Up~}YB0;nLrU<;_0w1vsBjUWntrwtH{Hdv-(e}%M`6If*54!si@5>@vVY*`%ITlKK zNviG=JvJlsB4ihR!g9KfhyK8DqZ9>OD}Bqry_i+`QaH z%$|Le4l|7Y8Y}6cx7NycHbi?g)t%8^x1)vBd_wnS$;VUD=dt$(p1wutyu~wsf#r|t zuQnLqXrX{fVZt?1$Ub*%W9!YyijAiJ1C-UC)%9=GpiDr?qRkwd)a3BgOzq z>o;+_iNf)oDUBCQ^UZtWVQic{>}`6c?L0=2@1z9C2VI&Fs+m+;M}~Ek5Q7xMpBSJG(mEQe*#byRTe6s?8o@2#+TgJ z063UC2whb1Qa4+T>|Iqn?Ymol|PA zb&!6I@jv&x`_m0lRRSQ+!W^FC`}e#fFr4>E`L?icgI36?g=1E~CkokDx0nS)${1~B z{vMQj=LM~rXD3d(@(y|D=6Q}h=T8A5p{J|1^efOK#v3bhkO2R)x2d5H_ta z>cve@tB;mMhhE5{pMm=5%%Pq&_JR|d=h=3gzAx;{)QiZg{6-vxF}tFTZaZ}Rd<9;4 zpx7}JSpqZ({6rfh4fB$8qvc^7O9nk9fY08FY%R6Vezk+%@$R0A%C8|yt}nyy&0-}a zXaww7rp#`4?;^w(s(45T7%qbpQ>phiq-We8MSqFMHdx{(y zBJuZyd-6PYJi-f2)#3{?qdZG~tUg$Mhe~1BHNB$jXQuWjokKcIaK;W!%lu1}ZL!^y z&DEOGQGF~HTUeED0?gnr2>h0Ld zgBW>2*FlnJ><+Iw9C3=5nLBRQIsKLrU`V-iuYAyTczbtrNc2SPG51cQr^=6WlMK69 zSSFp$VerGHv(|2|W0pT^qC7>rMUT7XZ2`b6{-7+g?2YEM>ZA0+6=ioeOQKZSX4M@s z9OioG4f^6*u3o5E&-C1?kBepxpr~Q0aLG1ZxA&P#Z@-0AB*VQ^bO$A_gcr-qVb#V$hLM0#NK(YoEW?CV|>NjSx!{>J!j zXeJM)$ZafTe6@`GlBe9=MY^hPS6sp`h5aWxWJx62zZ0T-xIRL2qAOu9|Mq-ueJ$-EVlt9X{>Ge1B8(tO$+!_)M%F^`% z?0R_Rldj)Boi#{(|JoRj>yObpg|#Uc-+sf8Ccm`@3d==4{@}V$i#@4b|eLcv8%}sJMVh>G7963 z0!5s??%LE-`B{##*Q9%Xk7vM&36ZWbR+#$SPgkq6x;yui2$8v)co`{PtSQjPKkBf~ z<6qI+DZH~O?kgJBI^BGN+w?c2h+w2pFD|0AxxRcARn?b<4KL#$9E|@Ul9!j~HEN<} zVq!|R=*x3%diMZ66u5c;D#=FXF=>MlE&1uU|CM{jHV~f+?RhQpr5XVvuS1kdT?h0U5dQk90SH-yv0-glmuVBRY)uWJOK5no>yrI_`a-%;i) zCu9DT?(^D@-1;6GNIUCgw_rd(uKkR%MR(Q}U@#CY*(AE-rsejvDn%WaVvj`v)R+u? z|Hgtw;^nq)AiQkDj&fmXJOd+#w6@3Vp8e}^26|Z%r5W~OrKncF1#(?O7%Ry;v4)n( zW2J4K8$@0Gvld;Ynkj?0(fP{IKmWJba00R0vbhP=gOGH?2Tf%P^E-NuHP+S1TQQ`~ zJ}^hX*qB`Gm}2tHn2Lt%LsDk9T$%O|p$WrkrY@ZQ^A%mys^BN=gjZx%7s6Tu+c=kD z$;WhrQS_%gm?VL_U-rXrZm?8v<2X`%6DuSq`A#d8B~N&_MlzmSlhfk+bk4u5Pb^lN z*zKsUu+8X`Xs0jxna-XyYl*#51bg!I)lw@vA@^4Cw+FhX@>&TJ5XV$vU5t_&_gwzo zH*K+?6O_{o)vaH$G$dxe;wID~R^daE|=FE?L%Zu z1jja9O!cM8Io&XFIiB;G9k{l)q9K`=FP!@GW8CPG5+ejsyS2r;YRO*S>Njsba+vLo zgGm*Xc9D9>6Z=7))GKMa5&o*MN|=?2qoKzL4c7fDKYmuTBLSBeHIF(TP9V=`}oIxjeXS!HDg<3&ibtL7F3nQT)CU&EU%Z;4|XMnT% ze3e6|OlX6xp{}m2_06rE)gI)zpGy$iFv6bGzL<)8Q=G6ovgbdFycqdoi?{dAAnEE- zyNnqCzC&*$L(d_hqz@J&;b2ntQDImbY1*B2H~FshU3dZX38-^jp#n~0gvy7}2W*?8 z+Wa%64&Fa7b(iPTpYLarec10n=jmH7CZC>-mbJ0TMpz_-4GP0SA4jo+lD>WdG9X;& zqaZwcS#H!l&hV2%VLYqLEqr>Vl#m2Yka645xEMp&T zg}&3n*`2|cn)kivxjduLHU}GnItLtqX<}-DlY;Q)Ar+woA0x;?AAke(LdMFpF9jPy zCzNh}s|S9k3+6U0LnTc;#_ikFG&eDSTH&SynpK3ty{bW9lG0)Fg1=htFxwf=DTCZ? zrrer_$=_ZLM&U?S1AdR~*>iebMOQ^FCJt?u57n-R>Gzd;mXnc@Nsb0leqSG<+er!u z2_Y+_rMk?6u&O70P_lGxi&vM6BsJ~6p`_Mz%`xeDb_e|4NG7HHbS zgvx$8S^x^`s|yQ!1=h`TX8z zo@!?C^9Pu@4+T|3io;nn$*7beeX;XtIgC94mrIc}&q35ZPG~ann$X+guq5~C1a)^d z_Wj3x#dw$&GI5Pfm*jmaGOpFyi(S>u@dT+HiWH|#rT*mY(m52dk`ziQiV}PF9%yNO zgE^T66^^bh9nW-KR_Rkwc$-Aq+}b6G<$>W6!4!P3)h)n;Ye};!Yu_jt)8A# zXMyqFcV^G&M7623o#F%`lVXee0Sak~b#*_9eHYJ@k>tPN#>}~Hm4`tt4 z9&(xCP@_ynU6sN11P`oybd;2ojHAmhO78@)BMh|k%lGV4%qnb8tw)PAeFm3qc;USh zVN~1oD~%pkX1VrEO<^^N3u9ev55OZ@d@VMjkx-;UeaayW@LHM*cSXJNE+YTuQbZ_7 zQWe$sZVB7Jr%y$xr28*()L*n6m&PY6+=A1*#d`PyWqP)H3arWCS~|HQunBxIaM)^fN1Ah-#nh~%FF}-0L7XTI6CpCPoFxsVbQa-SCIFG@e->k zmk7N_)b}{w#ZSli&)U70KkB_1pKN;Hc%3A{b>5kAVX1|5{?GplC}EA1Ry)Ap+0CTq z*U@fq-4O%?t-;II-9?1W^HPQ^)xTcy;7U)fNpNbb3y_1RAZ36GVGgWGIwCYj86?_c zt?&8r`3SL_+yi{JLi&M<)BZwVE+~Noz7`Gygf;_9idxl-ocU0tD_g5WMlmxjaSvu< z+`SOB0CM@EH{~BsVCn9Y%r3a7`4+0D@DnloauT8XmroS-+SE^$!iXV-NB9$He)3<) zLeaKgpCQ)+YyqaFkOgm1|vuWS4z&UbqB#IB;@C0`!q1wJzAF&`CU>~-`@Yv#PG$j zUzDbI9FD=-k}^g{Mqd5efZ6@{G&O~NhiMxIjdpK(n4DN$COp1qugFVhadp9)b#+2M z=&whAyJjT&dE~`qn&mLnb^{z9f=4N6XwpG0Rc9*Oorh~^khTE%)YgV;kIJri38O$o z;=d$`Y6vY8?n0;0hOT&WSgb;vdg(UZH5&YnoBjDyh~EEOYTA_z^>u{vfhMD#n(o~c z_|gqbq%bzD-A1Hj%)c!Te|f|F_dtTY?L0Bib(n|EAwtuQ*rAI+AN6~yC`-mN<^ugu zWO9i5QWm3h_+F{% zQj?jTuurlDARqo4x$gcH2e{^B8g{?&l@`z#XdrN3L%e?80B!i3R#=@%bJ)Og%oG$b z2%^lj8IzoOrAyGQWcfXBvdx+4am}}XP|iTg$NLSX5_`)fTHzOI+X*iZwveiz%FvxqfzIKA{ z&pXDl2LwB))$x-jAQH4!Br>?xARnc^jdT~|5~*afwE)C@~^cd0r~z1*V|#-P3- z+_Se<#1XHgG6n9x6-uDVkQ5iNQs(Ok4I!=W z->a)&u#9N9_f&%zji| z2?+EGuX_U31?m_r1VGY4EgBJ^at7n?M@B@b>B{m83CS8TtZDaqoP8zEE%U7_<=)j+ zP)ACNS8Sl2OUorhH77Du1ea;>_jx)69hCNK#nk`#?I8UC@_}R0q!JHMbOzBqxMpe= zezWd{0+sN*s`Vi#>rO%&TGKp&5QNu;AGnN#s_;60(|0TZksc^a8|IJoXRW7pBGUgs zzrJSsaU^aG^JCV#BoE${5Zx}XsO=hH#tZ0xR@jo~Zpk0!OA_(~LGl2XZGyMvEIm^x zIZCROB@68r2DOkR4vs+1f)4R5GJox$c+GqO&mIte7NRzSWc+~#IU z7u!&_>;3x$pDkp-QCT;VaLMtQeW+@Syu)gGMF^bGA2X% zi@MY8odp#l{YaFGQDY<)-aIQ?yVFM;~TIO;wB+R^E zKMK>~6W7kZ%K`GQOK@lH&KEK}gkZm;Lq8IbyJXI4acx-L{t?K~*cZ6&Bzjt8{Fb*M zFXIVXRj4%*UjLB$>T7fcrGbFkniZ(UvXkzkqVux}@}7ip$Roa#XoL8$x{gMF-34`- z4&T0o>s>69)P$YhikColWxyD)NoP_;u+Vt|)0Yo4`Nh|=%-24@{qv-~lECPC;i;I2 z)bq@ffuy9GvK3G($g@%c&9Yy~WBaVKLaL&3v)8KTRJtd2t1k~r#OA(Z$IFBBM9Gg& zza5fl)pia|T44~i`vW+_y+{C4TcMtK=k*Bf2d<}|Y=7c>vLnXJ2 ztNY$X>o_-yUk|CMDkT2Xv&V59q3!*^=43ZDH6l+`Hn8{8N^K&r2=U#np$j;3U95#9cJcs1KX&~QTXO)1C=ieMF?)My)&Z`&Uo ztm!Od31kb~zVTK%A|7az%q2_m_l}D<_NM;miI;~o4F^5>G;{rh?Q@Tv(aGIorsRnA{)H75nmB)Y7>>FbUrIT`kb$B_=89Yd#PO zjW0ruIcJ1j5w#D8F)al}=)~sK{OnNqkk*b}re2IEDDrS_KWq@y1Cwx5@ChB@YC$nF zmVQ@riLsh8F{i@n#{IIV;!QE4I?ckXuMlqS&T9b#;s6DmI*ay%0MBdmB3$^!?lrIo zVxTKZeF8Nf^RsCuQ{QX#kIR!(tH@SBlV*&UrK7adWlRjR$IH5s?;k#OQChG18EgA+ zib2Z@I?+Lti;O)j#hh(ftuBsbt;G4ezMtv<=y*B3X*c5$VU*P}6tNwysG!S!mT2n4 zDoisRUU6^;3d;H|x12Mnr^8;Xp{zLffWcbo*B6&`oQRUVA*N8hr2*3AIk568iS?VR z;hb9fnX<+!yqD9R5VQ+3#cxDJL==sMX?!DAvG7FzObv+8;~|M2G{~p4oL<@`GFYrJ zNh{i7%STNfv>V!Lwp1xVo7Vgks8lnEo3=)S7KP@)slu*&K4;^oMoeHZM z5n(jaHG%9)><|Amn8)YmTht6X>49a~$2ih?X}Z5Kr(S()_`%^^r+4`okGf_}zCtHU zyV?Lqsph-euu?hoj=gRkU(pMnZbSkBaBv&-emb491F#aL4X3MWYN&pec?gJ% z-&~oLTXQ+Kawd->1@xtouVIOOirkw=kkrLP^}z%*S^}W^E?xz;dBEjbp%i6zH$b*P z;fuwckjD;;tW#-PVUV^K80M{lP>WImxie#yRfWyagTdi?&h5*`rx{HtC?$?309 zvzvp=5PLa_&+IMp?Z^U+Za3JOUS2R-4JsVdt@Y(12m-1P2iw0w!`s>o^`&wsPWksRx2mI~9=EZwpHQF6^3ASJfRx4eP}n56=VWJ8m7V>@@(3$1$7cx|W8KJ!T^ zwX?S)6F?tQi!!&hZyXeAbvHkx%v@RBRa(4S-dAFXQ8J^H#U)lzv8%?yn@)w|_k6)t zmU-{wr19;D3stCRkTT6?Yv@4zc2E5~8XZNr<~A-F#r&Qr)5mzFm+GB}kLx#gt77GnEwnM9s$e(g zr1Myd13doLdZhYObc2YT-?L{8hd|v{{|Hwx32(sffH;*MUDN}n^XgtJK*g~*>`g%# zxPtIyo{oB<0<*`!Hhr=Ys$-OwcVLZbGAv4vF5ehf(BvIp9H-R8>D-h`Z=zGmoYZ0e zj*y>qnz5f*OcvZ0__)r8sAd|<(1D7(uGK(MvNdzrD?%a_Q@5;JiOk#$|WE45epDdOa!P*l+z< z+hvML<#cBqlj`#>ps$N;2vJ%yaN(TW$`EzD@2ITl#cdg3G<9Tv-CG7rE9w~C+E@ii zsuE{`sgFD8dfp%#5(-^%o+7B|2pJKLp%Wo~FD8!q>37QV$m3)m?j89xXv&rmv!Bkn zR4F>6Ap7GJPck=fdHkI)xPLpu(iJj9m3ExV^82o$ORetg)?Y!@+X-HfGZklwHg@0s zG2!!~CWtP*$K3?#6^sw~F3my%-S5!p2Tkw~J@8E2fz4zs(b zlCGO***QFv9b@tj$r&GtmiskJ_0l1p<#u(|bO&}mW837)NZxSDGV65UrniWOd=1j? zu(kN%@cIY3ZF<+G@=J~Pbi?9KH5B1arsNGP0;`8^yu>l@ZhpmCW3rCy1T6QQa5E2N zH$!TH%6lt-gq(a4>d;Jjz+u2C zIZx$PspK`f1JMk>`c}9`B**PB2ropvGLKsVDG>h>d}w{=>bnMf_JNGv^(%-3?UdC2 zv}R>5UmjPJzk9b)l28Z|#-h)UPW4lp6Q(?TNW29DVmKC0-x#Q=vryl{GhBO=1VvHk zZo$<{TR?};iW7TWD9;2rF_7+OJ7G_E**-=**cQDji=C1lIz;0ar667+h+^;b#2xa<>ZYGhKu5l z5aGz*`GW^l7D0-E%ibK}=8tZ((S>{RK@_ET$!RfGBVoY1{HLl+<5C@MSH5zK5l1_| zPOn#QlTrw`b;`0t4o+or+0FUq@$-5SC6xL|EoY!+Hk#7B6y$URWgNUa!7-cH;dQmc zF5W~KIdY3q7mW6ka^8`}D(3RymOX02Accts-rrKyi?X6cGLK3${hV4Hjhq`Y%^GE1jJyg!~jd3FNfi%D6J2#cGppJHvQH^k+A zJ;dsgrHBjd8`~x6{(dvVw=kQPRS0vL27R21KZ*5_sA_fSY)z{e*xJskn_~hRbsL5Eq;hk+uiG7@73xk`HJteJ<%`|U z)@pI&;k=<@M@tpeF8So2tz-=BvZI7u2Y@v|qVL{{-+yD2*@Q zPO+XXpHef?PA14~a!__5IsEdE&xUGQ7LvvS^`0$|wvtUrN5_ROdU2flDsZ^%2l@MB zfvYru(1&Ucdb4{;aZE?FdywAm)Y6kDj(1gJ+d4&kI**??_=}b8{$kayE%0|76e52i zCRqVo{Yp&Ie?&p+v--|uqX3Q4xjovkyKO0dTP=JZbjCu(0td|^DcYq)u#Y?iqBUT0 zIpxu=ymTldn}XRP%{Km%C(a?4;uU#2h-@%8sC@B!2^pdCx!6@3){=9Vb}#Lhzt*lI z$`S}Q?W3Q9l`^%3nV6Z2d@^pnppH0L@^3P75d(_;X4x7wZ86+U-7Hq^WGam%-`!yF zOZ4=TA%1^PPcO$j4RoK2p91{p3?ZA)Sge!WulO(z(oX(HP&z9=oVsue+C52#1dSwT zs%fH3UUrNO$-l->+#PQ~+8=l6&!2x)BoZM~^wX<1_L|+T-gk()1zIn2&AK?j^zyUr zQ}32FfsU)rm$ZLOcM7}eSdP;x^(EYQ)xG1{;om=tfbd8B-Wfn|ynh{35LPd604@fF zcDcr8-3ru#(!oiw3iWq!>qU_wF(m0F5ltz3C(}Slh3T$l_v@idw+bnWc5k&^uKJ%> zwRW?Ds-q z4*iF01#<+^O7Jl-Tmii9z{T_ISK53EHq^@k4#_oqo_}fgO2sP5i4NWmn1)?W9;x*% zc9#FaaXfs9ocTPbP`t+NJ9ix1zE3o(fO_T8ffGTEYB27BM#$;BYs{`45yp__LSKH{ zne%IJic709J*L{9o;UyP-4I}WLF9EZO{>UG4jNSu@zjl5x6B|pFG9jXp1TBOKu8S^ zQc<0wWjy1%QYuSKT`Q(HHI?&lgY80J+q??)(dqitk_ejL?Z}G@QLCt@tMefmc=&jQ zd-visCQV0cU+u(|{&PUfHp!0jBL7uPWVb=D@PbO0%pzr3Z(!!J;$P1D?u!a3{YjS} zd;{`N1JOtY>MQ@B^O;0+;&B32I3f2rAqIpu^XP=EEZ=kQOa9<%u8QwYX5fWf^I(x0 zs~5bjI-+M_um@?w3<^4-&HArLim{)&NGEs0#VpktE-5!xUpe~n_x3R_F=y~lyN0j?@s{QC755z~B|cKtW$8 z2}c2X;%c@ZuW;urE?PnBPMbktSi~9ro4fZ(gIPQWiqk^vIi-bx-t8OE@pc+x;qfar zLn5VYHIh+R*VGgrz(bQwzHG*mCnO-&rf`Wqw)=)fD3d;OIN+ztUhOxBIU*k6{~?n! zIP)KR#MDN~ExflK!by>Y9K(!jZ{dM|U=AMJa?M(yk~gf};S32r!SmA~{yF<}2y^tO zkSa_4KZI0XVwcHC{6G#C_Wlh~ut2V4?}c>g+X2b{xQ*SP*5qP?==N&8xp`22Jw(D2toFY89=T4ynFz+^SGEpA~?OzNPL=_Pqnx8m?DYjcaoUv#(=T<%0%xG@pq zS(Z`KwlQ=iH{dA3ESabgAnlfRR}|Uy>skPtkI1C&LqGlPSC?98Mt+tWFH;#`(Dp4G z?LNZ;jd6Mj&#sNRwMqZxb$!V%QJ>{l?Of0UK0mB@1z5dhNP@}uP7h0!_rQ4tZzi&jG)A3snS-tBK5wnM#t4^4So zQc^{(82lMli#NGnIg>x26I~5N{Qt&Q5SKhCKoX|(p)?JDjQx?Tu}{|oX>vG~)xkjN zPN+&u`g?A^fq{#h%ku+$nd=lD{bf&N5ls{5JEC&Ud$MP_o(-E`M4dvsOBJX!nQ@-X zMuRIPEoJ^TD?PsoQ82ztx;s(B*DLX{k5FpqmQQeB46`0}XPsyQajt$AC{IA;$HvV3 zi0ja9Ou~41;bnUThg?f(y8TOUOpkE5DHU>F{q5DY5~q!uT1PEgs9(S`TRe}2xbi7F zw`t%e6i%W0wfGu@D|~;UEzYp5C>LJ=Us zw#`sGyL{*>P+#Eiz>nE${uZG9iJ}rE^8zfwRtr3Vv-!0CgS9&BeaVgN$RQ5gcFw80 zSjkXc-3kI4VV90Mps%$M!qR(YUr!~dgw#z;Ozv;d`cZ9-55S1|kJA$X!LKV(n?xj_&qGcc!YyAPvhPm12N;zT4U& z(M)Eqw~Bt@qmBit2S`awQYF?#IFoip+GP1jh5Jeg!l7(s*~~C&><$StdZol z!x_=@NgQitBSMtAxcc2uB`K7ONaBMBzc@RqRBw(GNbSW>ZcA8t+}K4iB>vmCNoKf~uY+?N__ zh*9yI{z)O93lMvzl28_?LaW;)g^|eI2~~9UBJU|6VmPn#k0&Co!*~hjm5+RFIh3uw z*DPm<{K*!mWMFuRMWl#(5pZ#ubQ;%?X-?~))mMbwgRecee`Ke1#YdtHICwg}Q!9}k ze`Gr!jAXhd9y@8tOT5=Ca0g-AF9#i>5g_k*d}#Mrj>*Q(9h3i(2>NIFbrF--R917_ zcQ3zUQTW9UVyV%&&SKeJ$+-oI*n_BjBL2mif?Q8CADU+E%`AnWPzn-g&L|ux*(yKG zZ@mzlZ;|3EO#nsN6-cKdGm56q9i$P+Z6SLJNY9FCAGDMPPYBFD%s^nv3fiC?W|-_A zK2q`|MTKczsrCC=j=gN0(-JV#iJh@M-9JCw20s9S$|swAqB zUB=W8n;%*IbhJFTmTRHf#6)#*LdWets`Fw$C`dBD2v@CO^a(rON50Mx z>C7kYGm3G43%u8r{6UvgjfB!HDSp`C$YBQslY;0*>~Zlz=kbrL%yd>T7?d?!Y^{b;WH@4D9O^YuNTye#EW&7<;BVm zYsR{$B**0&`Ws|~Sy>^1c!U4}zcJ<=gH$(*6FJ@w#@|4z1VexMp%WLb=RMf~T%u)Y z3TQoN!^g*G3Me~SK*Cy^skReMoSn-s4nYfsms#Uf;ii)ib=gD0{@{*y*ZsTtAj6*J ztA!pWAa{)R+;&Dd+WXThXq8lD4s1e#F^J6P!xz$Wn%W>+F{pY_@0vrOgL^OkX+UCE zp-Z7OTww#9^V)Ye-wLMySBd02J?_t_BZoO*#nyfVQ3aA?N*EB~gr|BT#_)QncC_&n1K0Rfxn+GQJtFH_NQIPCabq}03F=^>U<}wR`#8DYg zJ**wJsM9Kc|Kcytk|pLR>a-*F>KH+-z$~x`bD;+=g|(_<)Lshw^0-Jso_h&Dwcvw3 z@QmIk=66$}uuDgE3Aal}Ea$j;r!pT^Z=`~D>u7a!MtoRIi>Z?8mXhC>pWkvs_NB_b zl6LCMGI<=qqx3UH%ObkJ#*JM7XpK~}VbAF2__azG@*fn)z_pJbuWJFH{X^#6ZYY

Custom memory allocation functions

typedef void* (*ZS
 
 
Advanced Streaming compression functions
ZSTD_CStream* ZSTD_createCStream_advanced(ZSTD_customMem customMem);
 ZSTD_CStream* ZSTD_initStaticCStream(void* workspace, size_t workspaceSize);    /**< same as ZSTD_initStaticCCtx() */
-size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize);   /**< pledgedSrcSize must be correct, a size of 0 means unknown.  for a frame size of 0 use initCStream_advanced */
+size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize);   /**< pledgedSrcSize must be correct. If it is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN. Note that, for compatibility with older programs, a size of 0 is interepreted as "unknown". But it may change in some future version to mean "empty". */
 size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t dictSize, int compressionLevel); /**< creates of an internal CDict (incompatible with static CCtx), except if dict == NULL or dictSize < 8, in which case no dict is used. Note: dict is loaded with ZSTD_dm_auto (treated as a full zstd dictionary if it begins with ZSTD_MAGIC_DICTIONARY, else as raw content) and ZSTD_dlm_byCopy.*/
 size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs, const void* dict, size_t dictSize,
-                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize is optional and can be 0 (meaning unknown). note: if the contentSizeFlag is set, pledgedSrcSize == 0 means the source size is actually 0. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
+                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize : If srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
 size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict);  /**< note : cdict will just be referenced, and must outlive compression session */
 size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, const ZSTD_CDict* cdict, ZSTD_frameParameters fParams, unsigned long long pledgedSrcSize);  /**< same as ZSTD_initCStream_usingCDict(), with control over frame parameters */
 


@@ -642,9 +642,11 @@ 
Advanced Streaming compression functions
ZSTD_CStream
 
  start a new compression job, using same parameters from previous job.
   This is typically useful to skip dictionary loading stage, since it will re-use it in-place..
   Note that zcs must be init at least once before using ZSTD_resetCStream().
-  pledgedSrcSize==0 means "srcSize unknown".
+  If pledgedSrcSize is not known at reset time, use macro ZSTD_CONTENTSIZE_UNKNOWN.
   If pledgedSrcSize > 0, its value must be correct, as it will be written in header, and controlled at the end.
-  @return : 0, or an error code (which can be tested using ZSTD_isError()) 
+  For the time being, pledgedSrcSize==0 is interpreted as "srcSize unknown" for compatibility with older programs,
+  but it may change to mean "empty" in some future version, so prefer using macro ZSTD_CONTENTSIZE_UNKNOWN.
+ @return : 0, or an error code (which can be tested using ZSTD_isError()) 
 


 
 
Advanced Streaming decompression functions
ZSTD_DStream* ZSTD_createDStream_advanced(ZSTD_customMem customMem);
@@ -693,10 +695,10 @@ 
Advanced Streaming decompression functions
ZSTD_DStre
 
 
Buffer-less streaming compression functions
size_t ZSTD_compressBegin(ZSTD_CCtx* cctx, int compressionLevel);
 size_t ZSTD_compressBegin_usingDict(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, int compressionLevel);
-size_t ZSTD_compressBegin_advanced(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, ZSTD_parameters params, unsigned long long pledgedSrcSize); /**< pledgedSrcSize is optional and can be 0 (meaning unknown). note: if the contentSizeFlag is set, pledgedSrcSize == 0 means the source size is actually 0 */
+size_t ZSTD_compressBegin_advanced(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, ZSTD_parameters params, unsigned long long pledgedSrcSize); /**< pledgedSrcSize : If srcSize is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN */
 size_t ZSTD_compressBegin_usingCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict); /**< note: fails if cdict==NULL */
-size_t ZSTD_compressBegin_usingCDict_advanced(ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict, ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize);   /* compression parameters are already set within cdict. pledgedSrcSize=0 means null-size */
-size_t ZSTD_copyCCtx(ZSTD_CCtx* cctx, const ZSTD_CCtx* preparedCCtx, unsigned long long pledgedSrcSize); /**<  note: if pledgedSrcSize can be 0, indicating unknown size.  if it is non-zero, it must be accurate.  for 0 size frames, use compressBegin_advanced */
+size_t ZSTD_compressBegin_usingCDict_advanced(ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict, ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize);   /* compression parameters are already set within cdict. pledgedSrcSize must be correct. If srcSize is not known, use macro ZSTD_CONTENTSIZE_UNKNOWN */
+size_t ZSTD_copyCCtx(ZSTD_CCtx* cctx, const ZSTD_CCtx* preparedCCtx, unsigned long long pledgedSrcSize); /**<  note: if pledgedSrcSize is not known, use ZSTD_CONTENTSIZE_UNKNOWN */
 


 
Buffer-less streaming decompression (synchronous mode)
   A ZSTD_DCtx object is required to track streaming operations.
@@ -848,11 +850,11 @@ 
Buffer-less streaming decompression functions
typedef
                               * Special: value 0 means "do not change strategy". */
 
     /* frame parameters */
-    ZSTD_p_contentSizeFlag=200, /* Content size is written into frame header _whenever known_ (default:1)
-                              * note that content size must be known at the beginning,
-                              * it is sent using ZSTD_CCtx_setPledgedSrcSize() */
+    ZSTD_p_contentSizeFlag=200, /* Content size will be written into frame header _whenever known_ (default:1)
+                              * Content size must be known at the beginning of compression,
+                              * it is provided using ZSTD_CCtx_setPledgedSrcSize() */
     ZSTD_p_checksumFlag,     /* A 32-bits checksum of content is written at end of frame (default:0) */
-    ZSTD_p_dictIDFlag,       /* When applicable, dictID of dictionary is provided in frame header (default:1) */
+    ZSTD_p_dictIDFlag,       /* When applicable, dictionary's ID is written into frame header (default:1) */
 
     /* multi-threading parameters */
     ZSTD_p_nbThreads=400,    /* Select how many threads a compression job can spawn (default:1)
@@ -913,7 +915,7 @@ 
Buffer-less streaming decompression functions
typedef
  @result : 0, or an error code (which can be tested with ZSTD_isError()).
   Note 1 : 0 means zero, empty.
            In order to mean "unknown content size", pass constant ZSTD_CONTENTSIZE_UNKNOWN.
-           Note that ZSTD_CONTENTSIZE_UNKNOWN is default value for new compression jobs.
+           ZSTD_CONTENTSIZE_UNKNOWN is default value for any new compression job.
   Note 2 : If all data is provided and consumed in a single round,
            this value is overriden by srcSize instead. 
 


diff --git a/lib/zstd.h b/lib/zstd.h
index 2866866be04..e9c4ff9d4f8 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -731,10 +731,10 @@ ZSTDLIB_API unsigned ZSTD_getDictID_fromFrame(const void* src, size_t srcSize);
 /*=====   Advanced Streaming compression functions  =====*/
 ZSTDLIB_API ZSTD_CStream* ZSTD_createCStream_advanced(ZSTD_customMem customMem);
 ZSTDLIB_API ZSTD_CStream* ZSTD_initStaticCStream(void* workspace, size_t workspaceSize);    /**< same as ZSTD_initStaticCCtx() */
-ZSTDLIB_API size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize);   /**< pledgedSrcSize must be correct, a size of 0 means unknown.  for a frame size of 0 use initCStream_advanced */
+ZSTDLIB_API size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize);   /**< pledgedSrcSize must be correct. If it is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN. Note that, for compatibility with older programs, a size of 0 is interepreted as "unknown". But it may change in some future version to mean "empty". */
 ZSTDLIB_API size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t dictSize, int compressionLevel); /**< creates of an internal CDict (incompatible with static CCtx), except if dict == NULL or dictSize < 8, in which case no dict is used. Note: dict is loaded with ZSTD_dm_auto (treated as a full zstd dictionary if it begins with ZSTD_MAGIC_DICTIONARY, else as raw content) and ZSTD_dlm_byCopy.*/
 ZSTDLIB_API size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs, const void* dict, size_t dictSize,
-                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize is optional: it can be 0, which means "unknown" if contentSizeFlag is set, or "unknown" if contentSizeFlag is not set. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
+                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize : If srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
 ZSTDLIB_API size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict);  /**< note : cdict will just be referenced, and must outlive compression session */
 ZSTDLIB_API size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, const ZSTD_CDict* cdict, ZSTD_frameParameters fParams, unsigned long long pledgedSrcSize);  /**< same as ZSTD_initCStream_usingCDict(), with control over frame parameters */
 
@@ -742,9 +742,11 @@ ZSTDLIB_API size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, const
  *  start a new compression job, using same parameters from previous job.
  *  This is typically useful to skip dictionary loading stage, since it will re-use it in-place..
  *  Note that zcs must be init at least once before using ZSTD_resetCStream().
- *  pledgedSrcSize==0 means "srcSize unknown".
+ *  If pledgedSrcSize is not known at reset time, use macro ZSTD_CONTENTSIZE_UNKNOWN.
  *  If pledgedSrcSize > 0, its value must be correct, as it will be written in header, and controlled at the end.
- *  @return : 0, or an error code (which can be tested using ZSTD_isError()) */
+ *  For the time being, pledgedSrcSize==0 is interpreted as "srcSize unknown" for compatibility with older programs,
+ *  but it may change to mean "empty" in some future version, so prefer using macro ZSTD_CONTENTSIZE_UNKNOWN.
+ * @return : 0, or an error code (which can be tested using ZSTD_isError()) */
 ZSTDLIB_API size_t ZSTD_resetCStream(ZSTD_CStream* zcs, unsigned long long pledgedSrcSize);
 
 
@@ -800,10 +802,10 @@ ZSTDLIB_API size_t ZSTD_resetDStream(ZSTD_DStream* zds);  /**< re-use decompress
 /*=====   Buffer-less streaming compression functions  =====*/
 ZSTDLIB_API size_t ZSTD_compressBegin(ZSTD_CCtx* cctx, int compressionLevel);
 ZSTDLIB_API size_t ZSTD_compressBegin_usingDict(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, int compressionLevel);
-ZSTDLIB_API size_t ZSTD_compressBegin_advanced(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, ZSTD_parameters params, unsigned long long pledgedSrcSize); /**< pledgedSrcSize is optional and can be 0 (meaning unknown). note: if the contentSizeFlag is set, pledgedSrcSize == 0 means the source size is actually 0 */
+ZSTDLIB_API size_t ZSTD_compressBegin_advanced(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, ZSTD_parameters params, unsigned long long pledgedSrcSize); /**< pledgedSrcSize : If srcSize is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN */
 ZSTDLIB_API size_t ZSTD_compressBegin_usingCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict); /**< note: fails if cdict==NULL */
-ZSTDLIB_API size_t ZSTD_compressBegin_usingCDict_advanced(ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict, ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize);   /* compression parameters are already set within cdict. pledgedSrcSize=0 means null-size */
-ZSTDLIB_API size_t ZSTD_copyCCtx(ZSTD_CCtx* cctx, const ZSTD_CCtx* preparedCCtx, unsigned long long pledgedSrcSize); /**<  note: if pledgedSrcSize can be 0, indicating unknown size.  if it is non-zero, it must be accurate.  for 0 size frames, use compressBegin_advanced */
+ZSTDLIB_API size_t ZSTD_compressBegin_usingCDict_advanced(ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict, ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize);   /* compression parameters are already set within cdict. pledgedSrcSize must be correct. If srcSize is not known, use macro ZSTD_CONTENTSIZE_UNKNOWN */
+ZSTDLIB_API size_t ZSTD_copyCCtx(ZSTD_CCtx* cctx, const ZSTD_CCtx* preparedCCtx, unsigned long long pledgedSrcSize); /**<  note: if pledgedSrcSize is not known, use ZSTD_CONTENTSIZE_UNKNOWN */
 
 ZSTDLIB_API size_t ZSTD_compressContinue(ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
 ZSTDLIB_API size_t ZSTD_compressEnd(ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
@@ -1066,7 +1068,7 @@ ZSTDLIB_API size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param
  * @result : 0, or an error code (which can be tested with ZSTD_isError()).
  *  Note 1 : 0 means zero, empty.
  *           In order to mean "unknown content size", pass constant ZSTD_CONTENTSIZE_UNKNOWN.
- *           Note that ZSTD_CONTENTSIZE_UNKNOWN is default value for new compression jobs.
+ *           ZSTD_CONTENTSIZE_UNKNOWN is default value for any new compression job.
  *  Note 2 : If all data is provided and consumed in a single round,
  *           this value is overriden by srcSize instead. */
 ZSTDLIB_API size_t ZSTD_CCtx_setPledgedSrcSize(ZSTD_CCtx* cctx, unsigned long long pledgedSrcSize);

From fc8d293460d6767558843ac8231c249dcc704382 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Sat, 14 Oct 2017 01:21:43 -0700
Subject: [PATCH 013/145] dictionary compression use correct file size
 estimation

when determining compression parameters
to compress one file only.

For multiple files, it still "bets" that files are going to be small.

There was also a bug recently added in ZSTD_CCtx_loadDictionary_advanced()
making it incapable to use pledgedSrcSize to determine compression parameters.
---
 lib/compress/zstd_compress.c | 2 +-
 programs/fileio.c            | 9 +++++----
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 4ceba415c77..c7872e7befc 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -531,7 +531,7 @@ size_t ZSTD_CCtx_loadDictionary_advanced(
         cctx->cdict = NULL;
     } else {
         ZSTD_compressionParameters const cParams =
-                ZSTD_getCParamsFromCCtxParams(cctx->requestedParams, 0, dictSize);
+                ZSTD_getCParamsFromCCtxParams(cctx->requestedParams, cctx->pledgedSrcSizePlusOne-1, dictSize);
         cctx->cdictLocal = ZSTD_createCDict_advanced(
                                 dict, dictSize,
                                 dictLoadMethod, dictMode,
diff --git a/programs/fileio.c b/programs/fileio.c
index 01e901ed898..17ad31b32ec 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -442,7 +442,6 @@ static cRess_t FIO_createCResources(const char* dictFileName, int cLevel,
             CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_contentSizeFlag, 1) );
             CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_dictIDFlag, g_dictIDFlag) );
             CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_checksumFlag, g_checksumFlag) );
-            (void)srcSize;
             /* compression level */
             CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_compressionLevel, cLevel) );
             /* long distance matching */
@@ -468,7 +467,9 @@ static cRess_t FIO_createCResources(const char* dictFileName, int cLevel,
             DISPLAYLEVEL(5,"set nb threads = %u \n", g_nbThreads);
             CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_nbThreads, g_nbThreads) );
             /* dictionary */
+            CHECK( ZSTD_CCtx_setPledgedSrcSize(ress.cctx, srcSize) );  /* just to load dictionary with good compression parameters */
             CHECK( ZSTD_CCtx_loadDictionary(ress.cctx, dictBuffer, dictBuffSize) );
+            CHECK( ZSTD_CCtx_setPledgedSrcSize(ress.cctx, ZSTD_CONTENTSIZE_UNKNOWN) );  /* reset */
         }
 #elif defined(ZSTD_MULTITHREAD)
         {   ZSTD_parameters params = ZSTD_getParams(cLevel, srcSize, dictBuffSize);
@@ -1006,7 +1007,7 @@ int FIO_compressMultipleFilenames(const char** inFileNamesTable, unsigned nbFile
     size_t dfnSize = FNSPACE;
     char*  dstFileName = (char*)malloc(FNSPACE);
     size_t const suffixSize = suffix ? strlen(suffix) : 0;
-    U64 const srcSize = (nbFiles != 1) ? 0 : UTIL_getFileSize(inFileNamesTable[0]) ;
+    U64 const srcSize = (nbFiles != 1) ? ZSTD_CONTENTSIZE_UNKNOWN : UTIL_getFileSize(inFileNamesTable[0]) ;
     cRess_t ress = FIO_createCResources(dictFileName, compressionLevel, srcSize, comprParams);
 
     /* init */
@@ -1032,9 +1033,9 @@ int FIO_compressMultipleFilenames(const char** inFileNamesTable, unsigned nbFile
                 free(dstFileName);
                 dfnSize = ifnSize + 20;
                 dstFileName = (char*)malloc(dfnSize);
-                if (!dstFileName)
+                if (!dstFileName) {
                     EXM_THROW(30, "zstd: %s", strerror(errno));
-            }
+            }   }
             strcpy(dstFileName, inFileNamesTable[u]);
             strcat(dstFileName, suffix);
             missed_files += FIO_compressFilename_dstFile(ress, dstFileName, inFileNamesTable[u], compressionLevel);

From 7f961ba6cd592217696cc43c9a133c7ffafada17 Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Fri, 13 Oct 2017 13:23:57 -0700
Subject: [PATCH 014/145] Don't allow default tables to repeat

It isn't useful in any case to repeat default tables.
Saves a few bytes on Silesia, since we don't trigger the dictionary
heuristic.

Before: 211988480 => 73651998 bytes
After:  211988480 => 73651721 bytes
---
 lib/compress/zstd_compress.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 188b8e717a7..43e5f54617e 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -1298,7 +1298,13 @@ symbolEncodingType_e ZSTD_selectEncodingType(
     if ( isDefaultAllowed
       && ((nbSeq < MIN_SEQ_FOR_DYNAMIC_FSE) || (mostFrequent < (nbSeq >> (defaultNormLog-1)))) ) {
         DEBUGLOG(5, "Selected set_basic");
-        *repeatMode = FSE_repeat_valid;
+        /* The format allows default tables to be repeated, but it isn't useful.
+         * When using simple heuristics to select encoding type, we don't want
+         * to confuse these tables with dictionaries. When running more careful
+         * analysis, we don't need to waste time checking both repeating tables
+         * and default tables.
+         */
+        *repeatMode = FSE_repeat_none;
         return set_basic;
     }
     DEBUGLOG(5, "Selected set_compressed");

From 241c57adffe34a8267dd9526cece87b8751468d2 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Mon, 16 Oct 2017 14:01:42 -0700
Subject: [PATCH 015/145] playtest : minor cosmetic changes

---
 tests/playTests.sh | 68 +++++++++++++++++++++++-----------------------
 1 file changed, 34 insertions(+), 34 deletions(-)

diff --git a/tests/playTests.sh b/tests/playTests.sh
index 013182aeb5d..f008d899883 100755
--- a/tests/playTests.sh
+++ b/tests/playTests.sh
@@ -91,7 +91,7 @@ else
     hasMT="true"
 fi
 
-$ECHO "\n**** simple tests **** "
+$ECHO "\n===>  simple tests "
 
 ./datagen > tmp
 $ECHO "test : basic compression "
@@ -166,7 +166,7 @@ $ZSTD -f tmp && die "tmp not present : should have failed"
 test ! -f tmp.zst  # tmp.zst should not be created
 
 
-$ECHO "\n**** Advanced compression parameters **** "
+$ECHO "\n===>  Advanced compression parameters "
 $ECHO "Hello world!" | $ZSTD --zstd=windowLog=21,      - -o tmp.zst && die "wrong parameters not detected!"
 $ECHO "Hello world!" | $ZSTD --zstd=windowLo=21        - -o tmp.zst && die "wrong parameters not detected!"
 $ECHO "Hello world!" | $ZSTD --zstd=windowLog=21,slog  - -o tmp.zst && die "wrong parameters not detected!"
@@ -180,14 +180,14 @@ roundTripTest -g512K " --long --zstd=ldmhlog=20,ldmslen=64,ldmblog=1,ldmhevery=7
 roundTripTest -g512K 19
 
 
-$ECHO "\n**** Pass-Through mode **** "
+$ECHO "\n===>  Pass-Through mode "
 $ECHO "Hello world 1!" | $ZSTD -df
 $ECHO "Hello world 2!" | $ZSTD -dcf
 $ECHO "Hello world 3!" > tmp1
 $ZSTD -dcf tmp1
 
 
-$ECHO "\n**** frame concatenation **** "
+$ECHO "\n===>  frame concatenation "
 
 $ECHO "hello " > hello.tmp
 $ECHO "world!" > world.tmp
@@ -218,7 +218,7 @@ $ECHO "$ECHO foo | $ZSTD | $ZSTD -d > /dev/full"
 $ECHO foo | $ZSTD | $ZSTD -d > /dev/full && die "write error not detected!"
 
 
-$ECHO "\n**** symbolic link test **** "
+$ECHO "\n===>  symbolic link test "
 
 rm -f hello.tmp world.tmp hello.tmp.zst world.tmp.zst
 $ECHO "hello world" > hello.tmp
@@ -233,7 +233,7 @@ rm -f hello.tmp world.tmp hello.tmp.zst world.tmp.zst
 fi
 
 
-$ECHO "\n**** test sparse file support **** "
+$ECHO "\n===>  test sparse file support "
 
 ./datagen -g5M  -P100 > tmpSparse
 $ZSTD tmpSparse -c | $ZSTD -dv -o tmpSparseRegen
@@ -260,7 +260,7 @@ $DIFF tmpSparse2M tmpSparseRegenerated
 rm tmpSparse*
 
 
-$ECHO "\n**** multiple files tests **** "
+$ECHO "\n===>  multiple files tests "
 
 ./datagen -s1        > tmp1 2> $INTOVOID
 ./datagen -s2 -g100K > tmp2 2> $INTOVOID
@@ -283,7 +283,7 @@ $ECHO "compress multiple files including a missing one (notHere) : "
 $ZSTD -f tmp1 notHere tmp2 && die "missing file not detected!"
 
 
-$ECHO "\n**** dictionary tests **** "
+$ECHO "\n===>  dictionary tests "
 
 $ECHO "- test with raw dict (content only) "
 ./datagen > tmpDict
@@ -339,7 +339,7 @@ $ZSTD --train-legacy -q tmp && die "Dictionary training should fail : source is
 rm tmp*
 
 
-$ECHO "\n**** cover dictionary tests **** "
+$ECHO "\n===>  cover dictionary builder : advanced options "
 
 TESTFILE=../programs/zstdcli.c
 ./datagen > tmpDict
@@ -359,7 +359,7 @@ $ECHO "- Create dictionary with size limit"
 $ZSTD --train-cover=steps=8 *.c ../programs/*.c -o tmpDict2 --maxdict=4K
 rm tmp*
 
-$ECHO "\n**** legacy dictionary tests **** "
+$ECHO "\n===>  legacy dictionary builder "
 
 TESTFILE=../programs/zstdcli.c
 ./datagen > tmpDict
@@ -380,7 +380,7 @@ $ZSTD --train-legacy -s9 *.c ../programs/*.c -o tmpDict2 --maxdict=4K
 rm tmp*
 
 
-$ECHO "\n**** integrity tests **** "
+$ECHO "\n===>  integrity tests "
 
 $ECHO "test one file (tmp1.zst) "
 ./datagen > tmp1
@@ -405,13 +405,13 @@ $ZSTD -t tmpSplit.* && die "bad file not detected !"
 
 
 
-$ECHO "\n**** golden files tests **** "
+$ECHO "\n===>  golden files tests "
 
 $ZSTD -t -r files
 $ZSTD -c -r files | $ZSTD -t
 
 
-$ECHO "\n**** benchmark mode tests **** "
+$ECHO "\n===>  benchmark mode tests "
 
 $ECHO "bench one file"
 ./datagen > tmp1
@@ -422,7 +422,7 @@ $ECHO "with recursive and quiet modes"
 $ZSTD -rqi1b1e2 tmp1
 
 
-$ECHO "\n**** gzip compatibility tests **** "
+$ECHO "\n===>  gzip compatibility tests "
 
 GZIPMODE=1
 $ZSTD --format=gzip -V || GZIPMODE=0
@@ -445,7 +445,7 @@ else
 fi
 
 
-$ECHO "\n**** gzip frame tests **** "
+$ECHO "\n===>  gzip frame tests "
 
 if [ $GZIPMODE -eq 1 ]; then
     ./datagen > tmp
@@ -459,7 +459,7 @@ else
 fi
 
 
-$ECHO "\n**** xz compatibility tests **** "
+$ECHO "\n===>  xz compatibility tests "
 
 LZMAMODE=1
 $ZSTD --format=xz -V || LZMAMODE=0
@@ -505,7 +505,7 @@ else
 fi
 
 
-$ECHO "\n**** xz frame tests **** "
+$ECHO "\n===>  xz frame tests "
 
 if [ $LZMAMODE -eq 1 ]; then
     ./datagen > tmp
@@ -520,7 +520,7 @@ else
     $ECHO "xz mode not supported"
 fi
 
-$ECHO "\n**** lz4 compatibility tests **** "
+$ECHO "\n===>  lz4 compatibility tests "
 
 LZ4MODE=1
 $ZSTD --format=lz4 -V || LZ4MODE=0
@@ -543,7 +543,7 @@ else
 fi
 
 
-$ECHO "\n**** lz4 frame tests **** "
+$ECHO "\n===>  lz4 frame tests "
 
 if [ $LZ4MODE -eq 1 ]; then
     ./datagen > tmp
@@ -556,7 +556,7 @@ else
     $ECHO "lz4 mode not supported"
 fi
 
-$ECHO "\n**** zstd round-trip tests **** "
+$ECHO "\n===>  zstd round-trip tests "
 
 roundTripTest
 roundTripTest -g15K       # TableID==3
@@ -569,7 +569,7 @@ roundTripTest -g516K 19   # btopt
 
 fileRoundTripTest -g500K
 
-$ECHO "\n**** zstd long distance matching round-trip tests **** "
+$ECHO "\n===>  zstd long distance matching round-trip tests "
 roundTripTest -g0 "2 --long"
 roundTripTest -g1000K "1 --long"
 roundTripTest -g517K "6 --long"
@@ -580,21 +580,21 @@ fileRoundTripTest -g5M "3 --long"
 
 if [ -n "$hasMT" ]
 then
-    $ECHO "\n**** zstdmt round-trip tests **** "
+    $ECHO "\n===>  zstdmt round-trip tests "
     roundTripTest -g4M "1 -T0"
     roundTripTest -g8M "3 -T2"
     roundTripTest -g8000K "2 --threads=2"
     fileRoundTripTest -g4M "19 -T2 -B1M"
 
-    $ECHO "\n**** zstdmt long distance matching round-trip tests **** "
+    $ECHO "\n===>  zstdmt long distance matching round-trip tests "
     roundTripTest -g8M "3 --long -T2"
 else
-    $ECHO "\n**** no multithreading, skipping zstdmt tests **** "
+    $ECHO "\n===>  no multithreading, skipping zstdmt tests "
 fi
 
 rm tmp*
 
-$ECHO "\n**** zstd --list/-l single frame tests ****"
+$ECHO "\n===>  zstd --list/-l single frame tests "
 ./datagen > tmp1
 ./datagen > tmp2
 ./datagen > tmp3
@@ -604,19 +604,19 @@ $ZSTD -lv *.zst | grep "Decompressed Size:"  # check that decompressed size is p
 $ZSTD --list *.zst
 $ZSTD --list -v *.zst
 
-$ECHO "\n**** zstd --list/-l multiple frame tests ****"
+$ECHO "\n===>  zstd --list/-l multiple frame tests "
 cat tmp1.zst tmp2.zst > tmp12.zst
 cat tmp12.zst tmp3.zst > tmp123.zst
 $ZSTD -l *.zst
 $ZSTD -lv *.zst
 
-$ECHO "\n**** zstd --list/-l error detection tests ****"
+$ECHO "\n===>  zstd --list/-l error detection tests "
 ! $ZSTD -l tmp1 tmp1.zst
 ! $ZSTD --list tmp*
 ! $ZSTD -lv tmp1*
 ! $ZSTD --list -v tmp2 tmp12.zst
 
-$ECHO "\n**** zstd --list/-l test with null files ****"
+$ECHO "\n===>  zstd --list/-l test with null files "
 ./datagen -g0 > tmp5
 $ZSTD tmp5
 $ZSTD -l tmp5.zst
@@ -624,12 +624,12 @@ $ZSTD -l tmp5.zst
 $ZSTD -lv tmp5.zst
 ! $ZSTD -lv tmp5*
 
-$ECHO "\n**** zstd --list/-l test with no content size field ****"
+$ECHO "\n===>  zstd --list/-l test with no content size field "
 ./datagen -g513K | $ZSTD > tmp6.zst
 $ZSTD -l tmp6.zst
 ! $ZSTD -lv tmp6.zst | grep "Decompressed Size:"  # must NOT be present in header
 
-$ECHO "\n**** zstd --list/-l test with no checksum ****"
+$ECHO "\n===>   zstd --list/-l test with no checksum "
 $ZSTD -f --no-check tmp1
 $ZSTD -l tmp1.zst
 $ZSTD -lv tmp1.zst
@@ -637,7 +637,7 @@ $ZSTD -lv tmp1.zst
 rm tmp*
 
 
-$ECHO "\n**** zstd long distance matching tests **** "
+$ECHO "\n===>   zstd long distance matching tests "
 roundTripTest -g0 " --long"
 roundTripTest -g9M "2 --long"
 # Test parameter parsing
@@ -652,7 +652,7 @@ if [ "$1" != "--test-large-data" ]; then
     exit 0
 fi
 
-$ECHO "\n**** large files tests **** "
+$ECHO "\n===>   large files tests "
 
 roundTripTest -g270000000 1
 roundTripTest -g250000000 2
@@ -683,7 +683,7 @@ roundTripTest -g5000000000 -P99 1
 fileRoundTripTest -g4193M -P99 1
 
 
-$ECHO "\n**** zstd long, long distance matching round-trip tests **** "
+$ECHO "\n===>   zstd long, long distance matching round-trip tests "
 roundTripTest -g270000000 "1 --long"
 roundTripTest -g130000000 -P60 "5 --long"
 roundTripTest -g35000000 -P70 "8 --long"
@@ -695,7 +695,7 @@ roundTripTest -g600M -P50 "1 --long --zstd=wlog=29,clog=28"
 
 if [ -n "$hasMT" ]
 then
-    $ECHO "\n**** zstdmt long round-trip tests **** "
+    $ECHO "\n===>   zstdmt long round-trip tests "
     roundTripTest -g80000000 -P99 "19 -T2" " "
     roundTripTest -g5000000000 -P99 "1 -T2" " "
     roundTripTest -g500000000 -P97 "1 -T999" " "

From 13bfe885aa72eadbeed15eaa86f36c265fa68c7b Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Mon, 16 Oct 2017 14:06:22 -0700
Subject: [PATCH 016/145] edited ZSTD_initCStream_advanced() comment

---
 lib/compress/zstd_compress.c | 3 ++-
 lib/zstd.h                   | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index c7872e7befc..35fa99e6f80 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -2575,7 +2575,8 @@ size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict)
 }
 
 /* ZSTD_initCStream_advanced() :
- * pledgedSrcSize is optional: it can be 0, which means "unknown" if contentSizeFlag is set, or "unknown" if contentSizeFlag is not set. */
+ * pledgedSrcSize : if srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN.
+ * dict is loaded with default parameters ZSTD_dm_auto and ZSTD_dlm_byCopy. */
 size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs,
                                  const void* dict, size_t dictSize,
                                  ZSTD_parameters params, unsigned long long pledgedSrcSize)
diff --git a/lib/zstd.h b/lib/zstd.h
index e9c4ff9d4f8..4681d3f2a0e 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -734,7 +734,7 @@ ZSTDLIB_API ZSTD_CStream* ZSTD_initStaticCStream(void* workspace, size_t workspa
 ZSTDLIB_API size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize);   /**< pledgedSrcSize must be correct. If it is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN. Note that, for compatibility with older programs, a size of 0 is interepreted as "unknown". But it may change in some future version to mean "empty". */
 ZSTDLIB_API size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t dictSize, int compressionLevel); /**< creates of an internal CDict (incompatible with static CCtx), except if dict == NULL or dictSize < 8, in which case no dict is used. Note: dict is loaded with ZSTD_dm_auto (treated as a full zstd dictionary if it begins with ZSTD_MAGIC_DICTIONARY, else as raw content) and ZSTD_dlm_byCopy.*/
 ZSTDLIB_API size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs, const void* dict, size_t dictSize,
-                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize : If srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
+                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize : if srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
 ZSTDLIB_API size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict);  /**< note : cdict will just be referenced, and must outlive compression session */
 ZSTDLIB_API size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, const ZSTD_CDict* cdict, ZSTD_frameParameters fParams, unsigned long long pledgedSrcSize);  /**< same as ZSTD_initCStream_usingCDict(), with control over frame parameters */
 

From e600b5d0dab674e480e6538ad7eaffbdbb72f029 Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Mon, 16 Oct 2017 17:18:43 -0700
Subject: [PATCH 017/145] [test] Exercise all codes in dictionary tables

---
 tests/Makefile      |   2 +-
 tests/seqgen.c      | 260 ++++++++++++++++++++++++++++++++++++++++++++
 tests/seqgen.h      |  58 ++++++++++
 tests/zstreamtest.c | 131 +++++++++++++++++++---
 4 files changed, 436 insertions(+), 15 deletions(-)
 create mode 100644 tests/seqgen.c
 create mode 100644 tests/seqgen.h

diff --git a/tests/Makefile b/tests/Makefile
index 651833bbbaa..eb8bb3dd6b9 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -130,7 +130,7 @@ zbufftest-dll : $(ZSTDDIR)/common/xxhash.c $(PRGDIR)/datagen.c zbufftest.c
 	$(MAKE) -C $(ZSTDDIR) libzstd
 	$(CC) $(CPPFLAGS) $(CFLAGS) $^ $(LDFLAGS) -o $@$(EXT)
 
-ZSTREAMFILES := $(ZSTD_FILES) $(ZDICT_FILES) $(PRGDIR)/datagen.c zstreamtest.c
+ZSTREAMFILES := $(ZSTD_FILES) $(ZDICT_FILES) $(PRGDIR)/datagen.c seqgen.c zstreamtest.c
 zstreamtest : CPPFLAGS += $(MULTITHREAD_CPP)
 zstreamtest : LDFLAGS += $(MULTITHREAD_LD)
 zstreamtest : $(ZSTREAMFILES)
diff --git a/tests/seqgen.c b/tests/seqgen.c
new file mode 100644
index 00000000000..8233ecee1cf
--- /dev/null
+++ b/tests/seqgen.c
@@ -0,0 +1,260 @@
+/*
+ * Copyright (c) 2017-present, Facebook, Inc.
+ * All rights reserved.
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
+ */
+
+#include "seqgen.h"
+#include "mem.h"
+#include 
+
+#define MIN(a, b)  ((a) < (b) ? (a) : (b))
+
+static const size_t kMatchBytes = 128;
+
+#define SEQ_rotl32(x,r) ((x << r) | (x >> (32 - r)))
+static BYTE SEQ_randByte(U32* src)
+{
+    static const U32 prime1 = 2654435761U;
+    static const U32 prime2 = 2246822519U;
+    U32 rand32 = *src;
+    rand32 *= prime1;
+    rand32 ^= prime2;
+    rand32  = SEQ_rotl32(rand32, 13);
+    *src = rand32;
+    return (BYTE)(rand32 >> 5);
+}
+
+SEQ_stream SEQ_initStream(unsigned seed)
+{
+    SEQ_stream stream;
+    stream.state = 0;
+    XXH64_reset(&stream.xxh, 0);
+    stream.seed = seed;
+    return stream;
+}
+
+/* Generates a single guard byte, then match length + 1 of a different byte,
+ * then another guard byte.
+ */
+static size_t SEQ_gen_matchLength(SEQ_stream* stream, unsigned value,
+                                  SEQ_outBuffer* out)
+{
+    typedef enum {
+        ml_first_byte = 0,
+        ml_match_bytes,
+        ml_last_byte,
+    } ml_state;
+    BYTE* const ostart = (BYTE*)out->dst;
+    BYTE* const oend = ostart + out->size;
+    BYTE* op = ostart + out->pos;
+
+    switch ((ml_state)stream->state) {
+    case ml_first_byte:
+        /* Generate a single byte and pick a different byte for the match */
+        if (op >= oend) {
+            stream->bytesLeft = 1;
+            break;
+        }
+        *op = SEQ_randByte(&stream->seed) & 0xFF;
+        do {
+            stream->saved = SEQ_randByte(&stream->seed) & 0xFF;
+        } while (*op == stream->saved);
+        ++op;
+        /* State transition */
+        stream->state = ml_match_bytes;
+        stream->bytesLeft = value + 1;
+    /* fall-through */
+    case ml_match_bytes: {
+        /* Copy matchLength + 1 bytes to the output buffer */
+        size_t const setLength = MIN(stream->bytesLeft, (size_t)(oend - op));
+        if (setLength > 0) {
+            memset(op, stream->saved, setLength);
+            op += setLength;
+            stream->bytesLeft -= setLength;
+        }
+        if (stream->bytesLeft > 0)
+            break;
+        /* State transition */
+        stream->state = ml_last_byte;
+    }
+    /* fall-through */
+    case ml_last_byte:
+        /* Generate a single byte and pick a different byte for the match */
+        if (op >= oend) {
+            stream->bytesLeft = 1;
+            break;
+        }
+        do {
+            *op = SEQ_randByte(&stream->seed) & 0xFF;
+        } while (*op == stream->saved);
+        ++op;
+        /* State transition */
+    /* fall-through */
+    default:
+        stream->state = 0;
+        stream->bytesLeft = 0;
+        break;
+    }
+    XXH64_update(&stream->xxh, ostart + out->pos, (op - ostart) - out->pos);
+    out->pos = op - ostart;
+    return stream->bytesLeft;
+}
+
+/* Saves the current seed then generates kMatchBytes random bytes >= 128.
+ * Generates literal length - kMatchBytes random bytes < 128.
+ * Generates another kMatchBytes using the saved seed to generate a match.
+ * This way the match is easy to find for the compressors.
+ */
+static size_t SEQ_gen_litLength(SEQ_stream* stream, unsigned value, SEQ_outBuffer* out)
+{
+    typedef enum {
+        ll_start = 0,
+        ll_run_bytes,
+        ll_literals,
+        ll_run_match,
+    } ll_state;
+    BYTE* const ostart = (BYTE*)out->dst;
+    BYTE* const oend = ostart + out->size;
+    BYTE* op = ostart + out->pos;
+
+    switch ((ll_state)stream->state) {
+    case ll_start:
+        stream->state = ll_run_bytes;
+        stream->saved = stream->seed;
+        stream->bytesLeft = MIN(kMatchBytes, value);
+    /* fall-through */
+    case ll_run_bytes:
+        while (stream->bytesLeft > 0 && op < oend) {
+            *op++ = SEQ_randByte(&stream->seed) | 0x80;
+            --stream->bytesLeft;
+        }
+        if (stream->bytesLeft > 0)
+            break;
+        /* State transition */
+        stream->state = ll_literals;
+        stream->bytesLeft = value - MIN(kMatchBytes, value);
+    /* fall-through */
+    case ll_literals:
+        while (stream->bytesLeft > 0 && op < oend) {
+            *op++ = SEQ_randByte(&stream->seed) & 0x7F;
+            --stream->bytesLeft;
+        }
+        if (stream->bytesLeft > 0)
+            break;
+        /* State transition */
+        stream->state = ll_run_match;
+        stream->bytesLeft = MIN(kMatchBytes, value);
+    /* fall-through */
+    case ll_run_match: {
+        while (stream->bytesLeft > 0 && op < oend) {
+            *op++ = SEQ_randByte(&stream->saved) | 0x80;
+            --stream->bytesLeft;
+        }
+        if (stream->bytesLeft > 0)
+            break;
+    }
+    /* fall-through */
+    default:
+        stream->state = 0;
+        stream->bytesLeft = 0;
+        break;
+    }
+    XXH64_update(&stream->xxh, ostart + out->pos, (op - ostart) - out->pos);
+    out->pos = op - ostart;
+    return stream->bytesLeft;
+}
+
+/* Saves the current seed then generates kMatchBytes random bytes >= 128.
+ * Generates offset - kMatchBytes of zeros to get a large offset without
+ * polluting the hash tables.
+ * Generates another kMatchBytes using the saved seed to generate a with the
+ * required offset.
+ */
+static size_t SEQ_gen_offset(SEQ_stream* stream, unsigned value, SEQ_outBuffer* out)
+{
+    typedef enum {
+        of_start = 0,
+        of_run_bytes,
+        of_offset,
+        of_run_match,
+    } of_state;
+    BYTE* const ostart = (BYTE*)out->dst;
+    BYTE* const oend = ostart + out->size;
+    BYTE* op = ostart + out->pos;
+
+    switch ((of_state)stream->state) {
+    case of_start:
+        stream->state = of_run_bytes;
+        stream->saved = stream->seed;
+        stream->bytesLeft = MIN(value, kMatchBytes);
+    /* fall-through */
+    case of_run_bytes: {
+        while (stream->bytesLeft > 0 && op < oend) {
+            *op++ = SEQ_randByte(&stream->seed) | 0x80;
+            --stream->bytesLeft;
+        }
+        if (stream->bytesLeft > 0)
+            break;
+        /* State transition */
+        stream->state = of_offset;
+        stream->bytesLeft = value - MIN(value, kMatchBytes);
+    }
+    /* fall-through */
+    case of_offset: {
+        /* Copy matchLength + 1 bytes to the output buffer */
+        size_t const setLength = MIN(stream->bytesLeft, (size_t)(oend - op));
+        if (setLength > 0) {
+            memset(op, 0, setLength);
+            op += setLength;
+            stream->bytesLeft -= setLength;
+        }
+        if (stream->bytesLeft > 0)
+            break;
+        /* State transition */
+        stream->state = of_run_match;
+        stream->bytesLeft = MIN(value, kMatchBytes);
+    }
+    /* fall-through */
+    case of_run_match: {
+        while (stream->bytesLeft > 0 && op < oend) {
+            *op++ = SEQ_randByte(&stream->saved) | 0x80;
+            --stream->bytesLeft;
+        }
+        if (stream->bytesLeft > 0)
+            break;
+    }
+    /* fall-through */
+    default:
+        stream->state = 0;
+        stream->bytesLeft = 0;
+        break;
+    }
+    XXH64_update(&stream->xxh, ostart + out->pos, (op - ostart) - out->pos);
+    out->pos = op - ostart;
+    return stream->bytesLeft;
+}
+
+/* Returns the number of bytes left to generate.
+ * Must pass the same type/value until it returns 0.
+ */
+size_t SEQ_gen(SEQ_stream* stream, SEQ_gen_type type, unsigned value, SEQ_outBuffer* out)
+{
+    switch (type) {
+        case SEQ_gen_ml: return SEQ_gen_matchLength(stream, value, out);
+        case SEQ_gen_ll: return SEQ_gen_litLength(stream, value, out);
+        case SEQ_gen_of: return SEQ_gen_offset(stream, value, out);
+        case SEQ_gen_max: /* fall-through */
+        default: return 0;
+    }
+}
+
+/* Returns the xxhash of the data produced so far */
+XXH64_hash_t SEQ_digest(SEQ_stream const* stream)
+{
+    return XXH64_digest(&stream->xxh);
+}
diff --git a/tests/seqgen.h b/tests/seqgen.h
new file mode 100644
index 00000000000..72d798846aa
--- /dev/null
+++ b/tests/seqgen.h
@@ -0,0 +1,58 @@
+/*
+ * Copyright (c) 2017-present, Facebook, Inc.
+ * All rights reserved.
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
+ */
+
+#ifndef SEQGEN_H
+#define SEQGEN_H
+
+#define XXH_STATIC_LINKING_ONLY
+
+#include "xxhash.h"
+#include    /* size_t */
+
+typedef enum {
+    SEQ_gen_ml = 0,
+    SEQ_gen_ll,
+    SEQ_gen_of,
+    SEQ_gen_max /* Must be the last value */
+} SEQ_gen_type;
+
+/* Internal state, do not use */
+typedef struct {
+    XXH64_state_t xxh; /* xxh state for all the data produced so far (seed=0) */
+    unsigned seed;
+    int state; /* enum to control state machine (clean=0) */
+    unsigned saved;
+    size_t bytesLeft;
+} SEQ_stream;
+
+SEQ_stream SEQ_initStream(unsigned seed);
+
+typedef struct {
+    void* dst;
+    size_t size;
+    size_t pos;
+} SEQ_outBuffer;
+
+/* Returns non-zero until the current type/value has been generated.
+ * Must pass the same type/value until it returns 0.
+ *
+ * Recommended to pick a value in the middle of the range you want, since there
+ * may be some noise that causes actual results to be slightly different.
+ * We try to be more accurate for smaller values.
+ *
+ * NOTE: Very small values don't work well (< 6).
+ */
+size_t SEQ_gen(SEQ_stream* stream, SEQ_gen_type type, unsigned value,
+               SEQ_outBuffer* out);
+
+/* Returns the xxhash of the data produced so far */
+XXH64_hash_t SEQ_digest(SEQ_stream const* stream);
+
+#endif /* SEQGEN_H */
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 8b4c836940a..1b42bd5d8c8 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -36,6 +36,7 @@
 #include "datagen.h"      /* RDG_genBuffer */
 #define XXH_STATIC_LINKING_ONLY   /* XXH64_state_t */
 #include "xxhash.h"       /* XXH64_* */
+#include "seqgen.h"
 
 
 /*-************************************
@@ -96,15 +97,21 @@ unsigned int FUZ_rand(unsigned int* seedPtr)
     return rand32 >> 5;
 }
 
-#define CHECK_Z(f) {                                         \
-    size_t const err = f;                                    \
-    if (ZSTD_isError(err)) {                                 \
-        DISPLAY("Error => %s : %s ",                         \
-                #f, ZSTD_getErrorName(err));                 \
-        DISPLAY(" (seed %u, test nb %u)  \n", seed, testNb); \
+#define CHECK(cond, ...) {                                   \
+    if (cond) {                                              \
+        DISPLAY("Error => ");                                \
+        DISPLAY(__VA_ARGS__);                                \
+        DISPLAY(" (seed %u, test nb %u, line %u)  \n",       \
+                seed, testNb, __LINE__);                     \
         goto _output_error;                                  \
 }   }
 
+#define CHECK_Z(f) {                                         \
+    size_t const err = f;                                    \
+    CHECK(ZSTD_isError(err), "%s : %s ",                     \
+          #f, ZSTD_getErrorName(err));                       \
+}
+
 
 /*======================================================
 *   Basic Unit tests
@@ -144,6 +151,63 @@ static void FUZ_freeDictionary(buffer_t dict)
     free(dict.start);
 }
 
+/* Round trips data and updates xxh with the decompressed data produced */
+static size_t SEQ_roundTrip(ZSTD_CCtx* cctx, ZSTD_DCtx* dctx,
+                            XXH64_state_t* xxh, void* data, size_t size,
+                            ZSTD_EndDirective endOp)
+{
+    static BYTE compressed[1024];
+    static BYTE uncompressed[1024];
+
+    ZSTD_inBuffer cin = {data, size, 0};
+    size_t cret;
+
+    do {
+        ZSTD_outBuffer cout = {compressed, sizeof(compressed), 0};
+        ZSTD_inBuffer din = {compressed, 0, 0};
+        ZSTD_outBuffer dout = {uncompressed, 0, 0};
+
+        cret = ZSTD_compress_generic(cctx, &cout, &cin, endOp);
+        if (ZSTD_isError(cret))
+            return cret;
+
+        din.size = cout.pos;
+        while (din.pos < din.size || (endOp == ZSTD_e_end && cret == 0)) {
+            size_t dret;
+
+            dout.pos = 0;
+            dout.size = sizeof(uncompressed);
+            dret = ZSTD_decompressStream(dctx, &dout, &din);
+            if (ZSTD_isError(dret))
+                return dret;
+            XXH64_update(xxh, dout.dst, dout.pos);
+            if (dret == 0)
+                break;
+        }
+    } while (cin.pos < cin.size || (endOp != ZSTD_e_continue && cret != 0));
+    return 0;
+}
+
+/* Generates some data and round trips it */
+static size_t SEQ_generateRoundTrip(ZSTD_CCtx* cctx, ZSTD_DCtx* dctx,
+                                    XXH64_state_t* xxh, SEQ_stream* seq,
+                                    SEQ_gen_type type, unsigned value)
+{
+    static BYTE data[1024];
+    size_t gen;
+
+    do {
+        SEQ_outBuffer sout = {data, sizeof(data), 0};
+        size_t ret;
+        gen = SEQ_gen(seq, type, value, &sout);
+
+        ret = SEQ_roundTrip(cctx, dctx, xxh, sout.dst, sout.pos, ZSTD_e_continue);
+        if (ZSTD_isError(ret))
+            return ret;
+    } while (gen != 0);
+
+    return 0;
+}
 
 static int basicUnitTests(U32 seed, double compressibility, ZSTD_customMem customMem)
 {
@@ -618,6 +682,53 @@ static int basicUnitTests(U32 seed, double compressibility, ZSTD_customMem custo
       if (r != 0) goto _output_error; }  /* error, or some data not flushed */
     DISPLAYLEVEL(3, "OK \n");
 
+    DISPLAYLEVEL(3, "test%3i : check dictionary FSE tables can represent every code : ", testNb++);
+    {   unsigned const kMaxWindowLog = 24;
+        unsigned value;
+        ZSTD_compressionParameters cParams = ZSTD_getCParams(3, 1U << kMaxWindowLog, 1024);
+        ZSTD_CDict* cdict;
+        ZSTD_DDict* ddict;
+        SEQ_stream seq = SEQ_initStream(0x87654321);
+        SEQ_gen_type type;
+        XXH64_state_t xxh;
+
+        XXH64_reset(&xxh, 0);
+        cParams.windowLog = kMaxWindowLog;
+        cdict = ZSTD_createCDict_advanced(dictionary.start, dictionary.filled, ZSTD_dlm_byRef, ZSTD_dm_fullDict, cParams, ZSTD_defaultCMem);
+        ddict = ZSTD_createDDict(dictionary.start, dictionary.filled);
+
+        if (!cdict || !ddict) goto _output_error;
+
+        ZSTD_CCtx_reset(zc);
+        ZSTD_resetDStream(zd);
+        CHECK_Z(ZSTD_CCtx_refCDict(zc, cdict));
+        CHECK_Z(ZSTD_initDStream_usingDDict(zd, ddict));
+        CHECK_Z(ZSTD_setDStreamParameter(zd, DStream_p_maxWindowSize, 1U << kMaxWindowLog));
+        /* Test all values < 300 */
+        for (value = 0; value < 300; ++value) {
+            for (type = (SEQ_gen_type)0; type < SEQ_gen_max; ++type) {
+                CHECK_Z(SEQ_generateRoundTrip(zc, zd, &xxh, &seq, type, value));
+            }
+        }
+        /* Test values 2^8 to 2^17 */
+        for (value = (1 << 8); value < (1 << 17); value <<= 1) {
+            for (type = (SEQ_gen_type)0; type < SEQ_gen_max; ++type) {
+                CHECK_Z(SEQ_generateRoundTrip(zc, zd, &xxh, &seq, type, value));
+                CHECK_Z(SEQ_generateRoundTrip(zc, zd, &xxh, &seq, type, value + (value >> 2)));
+            }
+        }
+        /* Test offset values up to the max window log */
+        for (value = 8; value <= kMaxWindowLog; ++value) {
+            CHECK_Z(SEQ_generateRoundTrip(zc, zd, &xxh, &seq, SEQ_gen_of, (1U << value) - 1));
+        }
+
+        CHECK_Z(SEQ_roundTrip(zc, zd, &xxh, NULL, 0, ZSTD_e_end));
+        CHECK(SEQ_digest(&seq) != XXH64_digest(&xxh), "SEQ XXH64 does not match");
+
+        ZSTD_freeCDict(cdict);
+        ZSTD_freeDDict(ddict);
+    }
+    DISPLAYLEVEL(3, "OK \n");
 
     /* Overlen overwriting window data bug */
     DISPLAYLEVEL(3, "test%3i : wildcopy doesn't overwrite potential match data : ", testNb++);
@@ -708,14 +819,6 @@ static U32 FUZ_randomClampedLength(U32* seed, U32 minVal, U32 maxVal)
     return (U32)((FUZ_rand(seed) % mod) + minVal);
 }
 
-#define CHECK(cond, ...) {                                   \
-    if (cond) {                                              \
-        DISPLAY("Error => ");                                \
-        DISPLAY(__VA_ARGS__);                                \
-        DISPLAY(" (seed %u, test nb %u)  \n", seed, testNb); \
-        goto _output_error;                                  \
-}   }
-
 static int fuzzerTests(U32 seed, U32 nbTests, unsigned startTest, double compressibility, int bigTests)
 {
     U32 const maxSrcLog = bigTests ? 24 : 22;

From e2bf70ece1a623fbe4e6fdf1c4e5f03bb7011873 Mon Sep 17 00:00:00 2001
From: "W. Felix Handte" 
Date: Tue, 17 Oct 2017 16:02:39 -0400
Subject: [PATCH 018/145] Add `check` Target to Makefile

---
 Makefile | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Makefile b/Makefile
index 2805c8ad4b5..f2487c59113 100644
--- a/Makefile
+++ b/Makefile
@@ -72,6 +72,9 @@ zstdmt:
 zlibwrapper:
 	$(MAKE) -C $(ZWRAPDIR) test
 
+.PHONY: check
+check: shortest
+
 .PHONY: test shortest
 test shortest:
 	$(MAKE) -C $(PRGDIR) allVariants

From 32c9f715ae1532d4ea06ee5f811ed61c46e48d31 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 17 Oct 2017 14:07:43 -0700
Subject: [PATCH 019/145] fixed : Visual build compressing stdin with
 multi-threading enabled fails

It was multiple reasons stacked :
- Visual use a different code path, because ZSTD_NEWAPI is not defined
- fileio.c sends `0` as `pledgedSrcSize` to mean `ZSTD_CONTENTSIZE_UNKNOWN`  (fixed)
- ZSTDMT_resetCCtx() interpreted `0` as "empty" instead of "unknown" (fixed)
---
 lib/compress/zstdmt_compress.c | 5 ++++-
 lib/compress/zstdmt_compress.h | 2 +-
 programs/Makefile              | 4 ++--
 programs/fileio.c              | 6 +++---
 4 files changed, 10 insertions(+), 7 deletions(-)

diff --git a/lib/compress/zstdmt_compress.c b/lib/compress/zstdmt_compress.c
index 197a2c49e14..af7be3167b3 100644
--- a/lib/compress/zstdmt_compress.c
+++ b/lib/compress/zstdmt_compress.c
@@ -838,9 +838,12 @@ size_t ZSTDMT_initCStream_usingCDict(ZSTDMT_CCtx* mtctx,
 
 
 /* ZSTDMT_resetCStream() :
- * pledgedSrcSize is optional and can be zero == unknown */
+ * pledgedSrcSize can be zero == unknown (for the time being)
+ * prefer using ZSTD_CONTENTSIZE_UNKNOWN,
+ * as `0` might mean "empty" in the future */
 size_t ZSTDMT_resetCStream(ZSTDMT_CCtx* zcs, unsigned long long pledgedSrcSize)
 {
+    if (!pledgedSrcSize) pledgedSrcSize = ZSTD_CONTENTSIZE_UNKNOWN;
     if (zcs->params.nbThreads==1)
         return ZSTD_resetCStream(zcs->cctxPool->cctx[0], pledgedSrcSize);
     return ZSTDMT_initCStream_internal(zcs, NULL, 0, ZSTD_dm_auto, 0, zcs->params,
diff --git a/lib/compress/zstdmt_compress.h b/lib/compress/zstdmt_compress.h
index 8c59c684f1a..a7ce7c7dd63 100644
--- a/lib/compress/zstdmt_compress.h
+++ b/lib/compress/zstdmt_compress.h
@@ -50,7 +50,7 @@ ZSTDLIB_API size_t ZSTDMT_compressCCtx(ZSTDMT_CCtx* mtctx,
 /* ===   Streaming functions   === */
 
 ZSTDLIB_API size_t ZSTDMT_initCStream(ZSTDMT_CCtx* mtctx, int compressionLevel);
-ZSTDLIB_API size_t ZSTDMT_resetCStream(ZSTDMT_CCtx* mtctx, unsigned long long pledgedSrcSize);    /**< pledgedSrcSize is optional and can be zero == unknown */
+ZSTDLIB_API size_t ZSTDMT_resetCStream(ZSTDMT_CCtx* mtctx, unsigned long long pledgedSrcSize);  /**< if srcSize is not known at reset time, use ZSTD_CONTENTSIZE_UNKNOWN. Note: for compatibility with older programs, 0 means the same as ZSTD_CONTENTSIZE_UNKNOWN, but it may change in the future, to mean "empty" */
 
 ZSTDLIB_API size_t ZSTDMT_compressStream(ZSTDMT_CCtx* mtctx, ZSTD_outBuffer* output, ZSTD_inBuffer* input);
 
diff --git a/programs/Makefile b/programs/Makefile
index cbb67e9fe34..1b21281b482 100644
--- a/programs/Makefile
+++ b/programs/Makefile
@@ -37,8 +37,8 @@ endif
 
 CPPFLAGS+= -I$(ZSTDDIR) -I$(ZSTDDIR)/common -I$(ZSTDDIR)/compress \
            -I$(ZSTDDIR)/dictBuilder \
-           -DZSTD_NEWAPI \
-           -DXXH_NAMESPACE=ZSTD_   # because xxhash.o already compiled with this macro from library
+           -DXXH_NAMESPACE=ZSTD_ \
+           -DZSTD_NEWAPI
 CFLAGS  ?= -O3
 DEBUGFLAGS+=-Wall -Wextra -Wcast-qual -Wcast-align -Wshadow \
             -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement \
diff --git a/programs/fileio.c b/programs/fileio.c
index 17ad31b32ec..03e653893dc 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -817,11 +817,11 @@ static int FIO_compressFilename_internal(cRess_t ress,
     /* init */
 #ifdef ZSTD_NEWAPI
     if (fileSize!=0)  /* when src is stdin, fileSize==0, but is effectively unknown */
-        ZSTD_CCtx_setPledgedSrcSize(ress.cctx, fileSize);  /* note : fileSize==0 means "empty" */
+        ZSTD_CCtx_setPledgedSrcSize(ress.cctx, fileSize);
 #elif defined(ZSTD_MULTITHREAD)
-    CHECK( ZSTDMT_resetCStream(ress.cctx, fileSize) );   /* note : fileSize==0 means "unknown" */
+    CHECK( ZSTDMT_resetCStream(ress.cctx, fileSize ? fileSize : ZSTD_CONTENTSIZE_UNKNOWN) );
 #else
-    CHECK( ZSTD_resetCStream(ress.cctx, fileSize) );   /* note : fileSize==0 means "unknown" */
+    CHECK( ZSTD_resetCStream(ress.cctx, fileSize ? fileSize : ZSTD_CONTENTSIZE_UNKNOWN) );
 #endif
 
     /* Main compression loop */

From 18b795374a4d1e0b308fdf6e86b1ea08b4489489 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 17 Oct 2017 16:14:25 -0700
Subject: [PATCH 020/145] UTIL_getFileSize() returns UTIL_FILESIZE_UNKNOWN on
 failure

UTIL_getFileSize() used to return zero on failure.
This made it impossible to distinguish a failure from a genuine empty file.
Both cases where coalesced.

Adding UTIL_FILESIZE_UNKNOWN constant has many consequences on user code,
since in many places, the `0` was assumed to mean "error".
This is no longer the case, and the error code must be actively checked.
---
 programs/bench.c                  | 14 +++++++--
 programs/dibio.c                  |  7 +++--
 programs/fileio.c                 | 50 ++++++++++++++++++++-----------
 programs/util.h                   | 35 +++++++++++++---------
 tests/fullbench.c                 |  9 ++++--
 tests/paramgrill.c                | 10 +++++++
 zlibWrapper/examples/zwrapbench.c | 13 ++++++--
 7 files changed, 95 insertions(+), 43 deletions(-)

diff --git a/programs/bench.c b/programs/bench.c
index ec99c61cebf..efec1892aeb 100644
--- a/programs/bench.c
+++ b/programs/bench.c
@@ -551,6 +551,11 @@ static void BMK_loadFiles(void* buffer, size_t bufferSize,
             fileSizes[n] = 0;
             continue;
         }
+        if (fileSize == UTIL_FILESIZE_UNKNOWN) {
+            DISPLAYLEVEL(2, "Cannot evaluate size of %s, ignoring ... \n", fileNamesTable[n]);
+            fileSizes[n] = 0;
+            continue;
+        }
         f = fopen(fileNamesTable[n], "rb");
         if (f==NULL) EXM_THROW(10, "impossible to open file %s", fileNamesTable[n]);
         DISPLAYUPDATE(2, "Loading %s...       \r", fileNamesTable[n]);
@@ -581,11 +586,14 @@ static void BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles, co
 
     /* Load dictionary */
     if (dictFileName != NULL) {
-        U64 dictFileSize = UTIL_getFileSize(dictFileName);
-        if (dictFileSize > 64 MB) EXM_THROW(10, "dictionary file %s too large", dictFileName);
+        U64 const dictFileSize = UTIL_getFileSize(dictFileName);
+        if (dictFileSize > 64 MB)
+            EXM_THROW(10, "dictionary file %s too large", dictFileName);
         dictBufferSize = (size_t)dictFileSize;
         dictBuffer = malloc(dictBufferSize);
-        if (dictBuffer==NULL) EXM_THROW(11, "not enough memory for dictionary (%u bytes)", (U32)dictBufferSize);
+        if (dictBuffer==NULL)
+            EXM_THROW(11, "not enough memory for dictionary (%u bytes)",
+                            (U32)dictBufferSize);
         BMK_loadFiles(dictBuffer, dictBufferSize, fileSizes, &dictFileName, 1);
     }
 
diff --git a/programs/dibio.c b/programs/dibio.c
index 2cb2a42671a..dea3ec44f4f 100644
--- a/programs/dibio.c
+++ b/programs/dibio.c
@@ -117,7 +117,7 @@ static unsigned DiB_loadFiles(void* buffer, size_t* bufferSizePtr,
     for (fileIndex=0; fileIndex 2*SAMPLESIZE_MAX);
diff --git a/programs/fileio.c b/programs/fileio.c
index 03e653893dc..1b18edefa87 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -25,7 +25,7 @@
 *  Includes
 ***************************************/
 #include "platform.h"   /* Large Files support, SET_BINARY_MODE */
-#include "util.h"       /* UTIL_getFileSize */
+#include "util.h"       /* UTIL_getFileSize, UTIL_isRegularFile */
 #include       /* fprintf, fopen, fread, _fileno, stdin, stdout */
 #include      /* malloc, free */
 #include      /* strcmp, strlen */
@@ -564,7 +564,7 @@ static unsigned long long FIO_compressGzFrame(cRess_t* ress,
                 strm.avail_out = (uInt)ress->dstBufferSize;
             }
         }
-        if (!srcFileSize)
+        if (srcFileSize != UTIL_FILESIZE_UNKNOWN)
             DISPLAYUPDATE(2, "\rRead : %u MB ==> %.2f%%",
                             (U32)(inFileSize>>20),
                             (double)outFileSize/inFileSize*100)
@@ -651,7 +651,7 @@ static unsigned long long FIO_compressLzmaFrame(cRess_t* ress,
                 strm.next_out = (BYTE*)ress->dstBuffer;
                 strm.avail_out = ress->dstBufferSize;
         }   }
-        if (!srcFileSize)
+        if (srcFileSize != UTIL_FILESIZE_UNKNOWN)
             DISPLAYUPDATE(2, "\rRead : %u MB ==> %.2f%%",
                             (U32)(inFileSize>>20),
                             (double)outFileSize/inFileSize*100)
@@ -697,18 +697,17 @@ static unsigned long long FIO_compressLz4Frame(cRess_t* ress,
     prefs.frameInfo.blockSizeID = LZ4F_max4MB;
     prefs.frameInfo.contentChecksumFlag = (contentChecksum_t)g_checksumFlag;
 #if LZ4_VERSION_NUMBER >= 10600
-    prefs.frameInfo.contentSize = srcFileSize;
+    prefs.frameInfo.contentSize = (srcFileSize==UTIL_FILESIZE_UNKNOWN) ? 0 : srcFileSize;
 #endif
 
-    {
-        size_t blockSize = FIO_LZ4_GetBlockSize_FromBlockId(LZ4F_max4MB);
+    {   size_t blockSize = FIO_LZ4_GetBlockSize_FromBlockId(LZ4F_max4MB);
         size_t readSize;
         size_t headerSize = LZ4F_compressBegin(ctx, ress->dstBuffer, ress->dstBufferSize, &prefs);
         if (LZ4F_isError(headerSize))
             EXM_THROW(33, "File header generation failed : %s",
                             LZ4F_getErrorName(headerSize));
-        { size_t const sizeCheck = fwrite(ress->dstBuffer, 1, headerSize, ress->dstFile);
-          if (sizeCheck!=headerSize) EXM_THROW(34, "Write error : cannot write header"); }
+        if (fwrite(ress->dstBuffer, 1, headerSize, ress->dstFile) != headerSize)
+            EXM_THROW(34, "Write error : cannot write header");
         outFileSize += headerSize;
 
         /* Read first block */
@@ -725,7 +724,7 @@ static unsigned long long FIO_compressLz4Frame(cRess_t* ress,
                 EXM_THROW(35, "zstd: %s: lz4 compression failed : %s",
                             srcFileName, LZ4F_getErrorName(outSize));
             outFileSize += outSize;
-            if (!srcFileSize)
+            if (srcFileSize != UTIL_FILESIZE_UNKNOWN)
                 DISPLAYUPDATE(2, "\rRead : %u MB ==> %.2f%%",
                                 (U32)(inFileSize>>20),
                                 (double)outFileSize/inFileSize*100)
@@ -816,12 +815,12 @@ static int FIO_compressFilename_internal(cRess_t ress,
 
     /* init */
 #ifdef ZSTD_NEWAPI
-    if (fileSize!=0)  /* when src is stdin, fileSize==0, but is effectively unknown */
+    if (fileSize!=UTIL_FILESIZE_UNKNOWN)  /* when src is stdin, fileSize==0, but is effectively unknown */
         ZSTD_CCtx_setPledgedSrcSize(ress.cctx, fileSize);
 #elif defined(ZSTD_MULTITHREAD)
-    CHECK( ZSTDMT_resetCStream(ress.cctx, fileSize ? fileSize : ZSTD_CONTENTSIZE_UNKNOWN) );
+    CHECK( ZSTDMT_resetCStream(ress.cctx, (fileSize==UTIL_FILESIZE_UNKNOWN) ? ZSTD_CONTENTSIZE_UNKNOWN : fileSize) );
 #else
-    CHECK( ZSTD_resetCStream(ress.cctx, fileSize ? fileSize : ZSTD_CONTENTSIZE_UNKNOWN) );
+    CHECK( ZSTD_resetCStream(ress.cctx, (fileSize==UTIL_FILESIZE_UNKNOWN) ? ZSTD_CONTENTSIZE_UNKNOWN : fileSize) );
 #endif
 
     /* Main compression loop */
@@ -851,13 +850,13 @@ static int FIO_compressFilename_internal(cRess_t ress,
                 compressedfilesize += outBuff.pos;
         }   }
         if (g_nbThreads > 1) {
-            if (!fileSize)
+            if (fileSize != UTIL_FILESIZE_UNKNOWN)
                 DISPLAYUPDATE(2, "\rRead : %u MB", (U32)(readsize>>20))
             else
                 DISPLAYUPDATE(2, "\rRead : %u / %u MB",
                                     (U32)(readsize>>20), (U32)(fileSize>>20));
         } else {
-            if (!fileSize)
+            if (fileSize != UTIL_FILESIZE_UNKNOWN)
                 DISPLAYUPDATE(2, "\rRead : %u MB ==> %.2f%%",
                                 (U32)(readsize>>20),
                                 (double)compressedfilesize/readsize*100)
@@ -985,7 +984,8 @@ int FIO_compressFilename(const char* dstFileName, const char* srcFileName,
                          const char* dictFileName, int compressionLevel, ZSTD_compressionParameters* comprParams)
 {
     clock_t const start = clock();
-    U64 const srcSize = UTIL_getFileSize(srcFileName);
+    U64 const fileSize = UTIL_getFileSize(srcFileName);
+    U64 const srcSize = (fileSize == UTIL_FILESIZE_UNKNOWN) ? ZSTD_CONTENTSIZE_UNKNOWN : fileSize;
 
     cRess_t const ress = FIO_createCResources(dictFileName, compressionLevel, srcSize, comprParams);
     int const result = FIO_compressFilename_dstFile(ress, dstFileName, srcFileName, compressionLevel);
@@ -1007,7 +1007,9 @@ int FIO_compressMultipleFilenames(const char** inFileNamesTable, unsigned nbFile
     size_t dfnSize = FNSPACE;
     char*  dstFileName = (char*)malloc(FNSPACE);
     size_t const suffixSize = suffix ? strlen(suffix) : 0;
-    U64 const srcSize = (nbFiles != 1) ? ZSTD_CONTENTSIZE_UNKNOWN : UTIL_getFileSize(inFileNamesTable[0]) ;
+    U64 const firstFileSize = UTIL_getFileSize(inFileNamesTable[0]);
+    U64 const firstSrcSize = (firstFileSize == UTIL_FILESIZE_UNKNOWN) ? ZSTD_CONTENTSIZE_UNKNOWN : firstFileSize;
+    U64 const srcSize = (nbFiles != 1) ? ZSTD_CONTENTSIZE_UNKNOWN : firstSrcSize ;
     cRess_t ress = FIO_createCResources(dictFileName, compressionLevel, srcSize, comprParams);
 
     /* init */
@@ -1799,7 +1801,7 @@ typedef struct {
  *           2 for file not compressed with zstd
  *           3 for cases in which file could not be opened.
  */
-static int getFileInfo(fileInfo_t* info, const char* inFileName){
+static int getFileInfo_fileConfirmed(fileInfo_t* info, const char* inFileName){
     int detectError = 0;
     FILE* const srcFile = FIO_openSrcFile(inFileName);
     if (srcFile == NULL) {
@@ -1815,7 +1817,8 @@ static int getFileInfo(fileInfo_t* info, const char* inFileName){
         if (numBytesRead < ZSTD_frameHeaderSize_min) {
             if ( feof(srcFile)
               && (numBytesRead == 0)
-              && (info->compressedSize > 0) ) {
+              && (info->compressedSize > 0)
+              && (info->compressedSize != UTIL_FILESIZE_UNKNOWN) ) {
                 break;
             }
             else if (feof(srcFile)) {
@@ -1928,6 +1931,17 @@ static int getFileInfo(fileInfo_t* info, const char* inFileName){
     return detectError;
 }
 
+static int getFileInfo(fileInfo_t* info, const char* srcFileName)
+{
+    int const isAFile = UTIL_isRegularFile(srcFileName);
+    if (!isAFile) {
+        DISPLAY("Error : %s is not a file", srcFileName);
+        return 3;
+    }
+    return getFileInfo_fileConfirmed(info, srcFileName);
+}
+
+
 static void displayInfo(const char* inFileName, const fileInfo_t* info, int displayLevel){
     unsigned const unit = info->compressedSize < (1 MB) ? (1 KB) : (1 MB);
     const char* const unitStr = info->compressedSize < (1 MB) ? "KB" : "MB";
diff --git a/programs/util.h b/programs/util.h
index c8be5f5fb57..c5e4365d04d 100644
--- a/programs/util.h
+++ b/programs/util.h
@@ -313,33 +313,40 @@ UTIL_STATIC U32 UTIL_isLink(const char* infilename)
 }
 
 
+#define UTIL_FILESIZE_UNKNOWN  ((U64)(-1))
 UTIL_STATIC U64 UTIL_getFileSize(const char* infilename)
 {
-    int r;
+    if (!UTIL_isRegularFile(infilename)) return UTIL_FILESIZE_UNKNOWN;
+    {   int r;
 #if defined(_MSC_VER)
-    struct __stat64 statbuf;
-    r = _stat64(infilename, &statbuf);
-    if (r || !(statbuf.st_mode & S_IFREG)) return 0;   /* No good... */
+        struct __stat64 statbuf;
+        r = _stat64(infilename, &statbuf);
+        if (r || !(statbuf.st_mode & S_IFREG)) return UTIL_FILESIZE_UNKNOWN;
 #elif defined(__MINGW32__) && defined (__MSVCRT__)
-    struct _stati64 statbuf;
-    r = _stati64(infilename, &statbuf);
-    if (r || !(statbuf.st_mode & S_IFREG)) return 0;   /* No good... */
+        struct _stati64 statbuf;
+        r = _stati64(infilename, &statbuf);
+        if (r || !(statbuf.st_mode & S_IFREG)) return UTIL_FILESIZE_UNKNOWN;
 #else
-    struct stat statbuf;
-    r = stat(infilename, &statbuf);
-    if (r || !S_ISREG(statbuf.st_mode)) return 0;   /* No good... */
+        struct stat statbuf;
+        r = stat(infilename, &statbuf);
+        if (r || !S_ISREG(statbuf.st_mode)) return UTIL_FILESIZE_UNKNOWN;
 #endif
-    return (U64)statbuf.st_size;
+        return (U64)statbuf.st_size;
+    }
 }
 
 
 UTIL_STATIC U64 UTIL_getTotalFileSize(const char** fileNamesTable, unsigned nbFiles)
 {
     U64 total = 0;
+    int error = 0;
     unsigned n;
-    for (n=0; n inFileSize) benchedSize = (size_t)inFileSize;
         if (benchedSize < inFileSize)
diff --git a/tests/paramgrill.c b/tests/paramgrill.c
index 317ec461ca9..a387a44c60c 100644
--- a/tests/paramgrill.c
+++ b/tests/paramgrill.c
@@ -687,6 +687,11 @@ int benchFiles(const char** fileNamesTable, int nbFiles)
             DISPLAY( "Pb opening %s\n", inFileName);
             return 11;
         }
+        if (inFileSize == UTIL_FILESIZE_UNKNOWN) {
+            DISPLAY("Pb evaluatin size of %s \n", inFileName);
+            fclose(inFile);
+            return 11;
+        }
 
         /* Memory allocation */
         benchedSize = BMK_findMaxMem(inFileSize*3) / 3;
@@ -740,6 +745,11 @@ int optimizeForSize(const char* inFileName, U32 targetSpeed)
 
     /* Init */
     if (inFile==NULL) { DISPLAY( "Pb opening %s\n", inFileName); return 11; }
+    if (inFileSize == UTIL_FILESIZE_UNKNOWN) {
+        DISPLAY("Pb evaluatin size of %s \n", inFileName);
+        fclose(inFile);
+        return 11;
+    }
 
     /* Memory allocation & restrictions */
     if ((U64)benchedSize > inFileSize) benchedSize = (size_t)inFileSize;
diff --git a/zlibWrapper/examples/zwrapbench.c b/zlibWrapper/examples/zwrapbench.c
index 25a23f88573..9226932b5e9 100644
--- a/zlibWrapper/examples/zwrapbench.c
+++ b/zlibWrapper/examples/zwrapbench.c
@@ -684,6 +684,11 @@ static void BMK_loadFiles(void* buffer, size_t bufferSize,
             fileSizes[n] = 0;
             continue;
         }
+        if (fileSize == UTIL_FILESIZE_UNKNOWN) {
+            DISPLAYLEVEL(2, "Cannot determine size of %s ...    \n", fileNamesTable[n]);
+            fileSizes[n] = 0;
+            continue;
+        }
         f = fopen(fileNamesTable[n], "rb");
         if (f==NULL) EXM_THROW(10, "impossible to open file %s", fileNamesTable[n]);
         DISPLAYUPDATE(2, "Loading %s...       \r", fileNamesTable[n]);
@@ -714,11 +719,13 @@ static void BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles,
 
     /* Load dictionary */
     if (dictFileName != NULL) {
-        U64 dictFileSize = UTIL_getFileSize(dictFileName);
-        if (dictFileSize > 64 MB) EXM_THROW(10, "dictionary file %s too large", dictFileName);
+        U64 const dictFileSize = UTIL_getFileSize(dictFileName);
+        if (dictFileSize > 64 MB)
+            EXM_THROW(10, "dictionary file %s too large", dictFileName);
         dictBufferSize = (size_t)dictFileSize;
         dictBuffer = malloc(dictBufferSize);
-        if (dictBuffer==NULL) EXM_THROW(11, "not enough memory for dictionary (%u bytes)", (U32)dictBufferSize);
+        if (dictBuffer==NULL)
+            EXM_THROW(11, "not enough memory for dictionary (%u bytes)", (U32)dictBufferSize);
         BMK_loadFiles(dictBuffer, dictBufferSize, fileSizes, &dictFileName, 1);
     }
 

From 5891f1d952b26f75ab80c1869e7a6af249e104eb Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 17 Oct 2017 16:23:20 -0700
Subject: [PATCH 021/145] added a check to ensure 0-size is present in frame
 header

---
 tests/playTests.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/playTests.sh b/tests/playTests.sh
index f008d899883..299c2d883b1 100755
--- a/tests/playTests.sh
+++ b/tests/playTests.sh
@@ -621,7 +621,7 @@ $ECHO "\n===>  zstd --list/-l test with null files "
 $ZSTD tmp5
 $ZSTD -l tmp5.zst
 ! $ZSTD -l tmp5*
-$ZSTD -lv tmp5.zst
+$ZSTD -lv tmp5.zst | grep "Decompressed Size: 0.00 KB (0 B)"  # check that 0 size is present in header
 ! $ZSTD -lv tmp5*
 
 $ECHO "\n===>  zstd --list/-l test with no content size field "

From ca1a9ebac54b01766373e138833dc39674bad7c3 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 18 Oct 2017 11:18:27 -0700
Subject: [PATCH 022/145] fixed zlib wrapper

it was invoking ZSTD_initCStream_advanced() with pledgedSrcSize==0 and contentSizeFlag=1
which means "empty"
while the intention was to mean "unknown".

The contentSizeFlag==1 is new, it is a consequence of setting this value to 1 by default.

The solution selected here is to pass ZSTD_CONTENTSIZE_UNKNOWN to mean "unknown".
So contentSizeFlag remains set (it wasn't in previous versions).
---
 doc/zstd_manual.html           | 4 ++--
 lib/zstd.h                     | 2 +-
 zlibWrapper/zstd_zlibwrapper.c | 6 +++---
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/doc/zstd_manual.html b/doc/zstd_manual.html
index 7ef86a23dba..9697ebd57cc 100644
--- a/doc/zstd_manual.html
+++ b/doc/zstd_manual.html
@@ -518,7 +518,7 @@ 
Custom memory allocation functions
typedef void* (*ZS
 
 
ZSTD_parameters ZSTD_getParams(int compressionLevel, unsigned long long estimatedSrcSize, size_t dictSize);
 
   same as ZSTD_getCParams(), but @return a full `ZSTD_parameters` object instead of sub-component `ZSTD_compressionParameters`.
-   All fields of `ZSTD_frameParameters` are set to default (0) 
+   All fields of `ZSTD_frameParameters` are set to default : contentSize=1, checksum=0, noDictID=0 
 


 
 
size_t ZSTD_checkCParams(ZSTD_compressionParameters params);
@@ -634,7 +634,7 @@ 
Advanced Streaming compression functions
ZSTD_CStream
 size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize);   /**< pledgedSrcSize must be correct. If it is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN. Note that, for compatibility with older programs, a size of 0 is interepreted as "unknown". But it may change in some future version to mean "empty". */
 size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t dictSize, int compressionLevel); /**< creates of an internal CDict (incompatible with static CCtx), except if dict == NULL or dictSize < 8, in which case no dict is used. Note: dict is loaded with ZSTD_dm_auto (treated as a full zstd dictionary if it begins with ZSTD_MAGIC_DICTIONARY, else as raw content) and ZSTD_dlm_byCopy.*/
 size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs, const void* dict, size_t dictSize,
-                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize : If srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
+                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize : if srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
 size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict);  /**< note : cdict will just be referenced, and must outlive compression session */
 size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, const ZSTD_CDict* cdict, ZSTD_frameParameters fParams, unsigned long long pledgedSrcSize);  /**< same as ZSTD_initCStream_usingCDict(), with control over frame parameters */
 


diff --git a/lib/zstd.h b/lib/zstd.h
index 4681d3f2a0e..6b7bfd03cef 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -613,7 +613,7 @@ ZSTDLIB_API ZSTD_compressionParameters ZSTD_getCParams(int compressionLevel, uns
 
 /*! ZSTD_getParams() :
 *   same as ZSTD_getCParams(), but @return a full `ZSTD_parameters` object instead of sub-component `ZSTD_compressionParameters`.
-*   All fields of `ZSTD_frameParameters` are set to default (0) */
+*   All fields of `ZSTD_frameParameters` are set to default : contentSize=1, checksum=0, noDictID=0 */
 ZSTDLIB_API ZSTD_parameters ZSTD_getParams(int compressionLevel, unsigned long long estimatedSrcSize, size_t dictSize);
 
 /*! ZSTD_checkCParams() :
diff --git a/zlibWrapper/zstd_zlibwrapper.c b/zlibWrapper/zstd_zlibwrapper.c
index c5fbdf98e19..b3e5f36725a 100644
--- a/zlibWrapper/zstd_zlibwrapper.c
+++ b/zlibWrapper/zstd_zlibwrapper.c
@@ -270,7 +270,7 @@ ZEXTERN int ZEXPORT z_deflateSetDictionary OF((z_streamp strm,
             zwc->zbc = ZSTD_createCStream_advanced(zwc->customMem);
             if (zwc->zbc == NULL) return ZWRAPC_finishWithError(zwc, strm, 0);
         }
-        { int res = ZWRAP_initializeCStream(zwc, dictionary, dictLength, 0);
+        { int res = ZWRAP_initializeCStream(zwc, dictionary, dictLength, ZSTD_CONTENTSIZE_UNKNOWN);
           if (res != Z_OK) return ZWRAPC_finishWithError(zwc, strm, res); }
         zwc->comprState = ZWRAP_useReset;
     }
@@ -295,7 +295,7 @@ ZEXTERN int ZEXPORT z_deflate OF((z_streamp strm, int flush))
     if (zwc->zbc == NULL) {
         zwc->zbc = ZSTD_createCStream_advanced(zwc->customMem);
         if (zwc->zbc == NULL) return ZWRAPC_finishWithError(zwc, strm, 0);
-        { int const initErr = ZWRAP_initializeCStream(zwc, NULL, 0, (flush == Z_FINISH) ? strm->avail_in : 0);
+        { int const initErr = ZWRAP_initializeCStream(zwc, NULL, 0, (flush == Z_FINISH) ? strm->avail_in : ZSTD_CONTENTSIZE_UNKNOWN);
           if (initErr != Z_OK) return ZWRAPC_finishWithError(zwc, strm, initErr); }
         if (flush != Z_FINISH) zwc->comprState = ZWRAP_useReset;
     } else {
@@ -308,7 +308,7 @@ ZEXTERN int ZEXPORT z_deflate OF((z_streamp strm, int flush))
                     return ZWRAPC_finishWithError(zwc, strm, 0);
                 }
             } else {
-                int const res = ZWRAP_initializeCStream(zwc, NULL, 0, (flush == Z_FINISH) ? strm->avail_in : 0);
+                int const res = ZWRAP_initializeCStream(zwc, NULL, 0, (flush == Z_FINISH) ? strm->avail_in : ZSTD_CONTENTSIZE_UNKNOWN);
                 if (res != Z_OK) return ZWRAPC_finishWithError(zwc, strm, res);
                 if (flush != Z_FINISH) zwc->comprState = ZWRAP_useReset;
             }

From 300e1df0a38b8200155fe11a851b3ecc9bc6ddfe Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 18 Oct 2017 11:41:52 -0700
Subject: [PATCH 023/145] fixed wrong test to display compression status

---
 programs/fileio.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/programs/fileio.c b/programs/fileio.c
index 1b18edefa87..4ddb8f8e010 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -564,7 +564,7 @@ static unsigned long long FIO_compressGzFrame(cRess_t* ress,
                 strm.avail_out = (uInt)ress->dstBufferSize;
             }
         }
-        if (srcFileSize != UTIL_FILESIZE_UNKNOWN)
+        if (srcFileSize == UTIL_FILESIZE_UNKNOWN)
             DISPLAYUPDATE(2, "\rRead : %u MB ==> %.2f%%",
                             (U32)(inFileSize>>20),
                             (double)outFileSize/inFileSize*100)
@@ -651,7 +651,7 @@ static unsigned long long FIO_compressLzmaFrame(cRess_t* ress,
                 strm.next_out = (BYTE*)ress->dstBuffer;
                 strm.avail_out = ress->dstBufferSize;
         }   }
-        if (srcFileSize != UTIL_FILESIZE_UNKNOWN)
+        if (srcFileSize == UTIL_FILESIZE_UNKNOWN)
             DISPLAYUPDATE(2, "\rRead : %u MB ==> %.2f%%",
                             (U32)(inFileSize>>20),
                             (double)outFileSize/inFileSize*100)
@@ -724,7 +724,7 @@ static unsigned long long FIO_compressLz4Frame(cRess_t* ress,
                 EXM_THROW(35, "zstd: %s: lz4 compression failed : %s",
                             srcFileName, LZ4F_getErrorName(outSize));
             outFileSize += outSize;
-            if (srcFileSize != UTIL_FILESIZE_UNKNOWN)
+            if (srcFileSize == UTIL_FILESIZE_UNKNOWN)
                 DISPLAYUPDATE(2, "\rRead : %u MB ==> %.2f%%",
                                 (U32)(inFileSize>>20),
                                 (double)outFileSize/inFileSize*100)
@@ -850,18 +850,18 @@ static int FIO_compressFilename_internal(cRess_t ress,
                 compressedfilesize += outBuff.pos;
         }   }
         if (g_nbThreads > 1) {
-            if (fileSize != UTIL_FILESIZE_UNKNOWN)
+            if (fileSize == UTIL_FILESIZE_UNKNOWN)
                 DISPLAYUPDATE(2, "\rRead : %u MB", (U32)(readsize>>20))
             else
                 DISPLAYUPDATE(2, "\rRead : %u / %u MB",
                                     (U32)(readsize>>20), (U32)(fileSize>>20));
         } else {
-            if (fileSize != UTIL_FILESIZE_UNKNOWN)
+            if (fileSize == UTIL_FILESIZE_UNKNOWN)
                 DISPLAYUPDATE(2, "\rRead : %u MB ==> %.2f%%",
                                 (U32)(readsize>>20),
                                 (double)compressedfilesize/readsize*100)
             else
-            DISPLAYUPDATE(2, "\rRead : %u / %u MB ==> %.2f%%",
+                DISPLAYUPDATE(2, "\rRead : %u / %u MB ==> %.2f%%",
                                 (U32)(readsize>>20), (U32)(fileSize>>20),
                                 (double)compressedfilesize/readsize*100);
         }

From d3b9547aa4cab7421946ef4010a998de34703da8 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 18 Oct 2017 17:01:53 -0700
Subject: [PATCH 024/145] IO and bench : ZSTD_NEWAPI is the only remaining code
 path

removed the other 2 code paths (single thread, and ZSTDMT ones)
keeping only the new advanced API, for easier code coverage.

It shall also fix identified issue with Visual Studio
which doesn't have ZSTD_NEWAPI defined.
---
 programs/Makefile |   3 +-
 programs/bench.c  |  39 --------------
 programs/fileio.c | 135 +++++++++++-----------------------------------
 3 files changed, 33 insertions(+), 144 deletions(-)

diff --git a/programs/Makefile b/programs/Makefile
index 1b21281b482..e1c5f50b6c1 100644
--- a/programs/Makefile
+++ b/programs/Makefile
@@ -37,8 +37,7 @@ endif
 
 CPPFLAGS+= -I$(ZSTDDIR) -I$(ZSTDDIR)/common -I$(ZSTDDIR)/compress \
            -I$(ZSTDDIR)/dictBuilder \
-           -DXXH_NAMESPACE=ZSTD_ \
-           -DZSTD_NEWAPI
+           -DXXH_NAMESPACE=ZSTD_
 CFLAGS  ?= -O3
 DEBUGFLAGS+=-Wall -Wextra -Wcast-qual -Wcast-align -Wshadow \
             -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement \
diff --git a/programs/bench.c b/programs/bench.c
index efec1892aeb..5b67d5c7629 100644
--- a/programs/bench.c
+++ b/programs/bench.c
@@ -287,7 +287,6 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                     U64 const clockLoop = g_nbSeconds ? TIMELOOP_MICROSEC : 1;
                     U32 nbLoops = 0;
                     ZSTD_CDict* cdict = NULL;
-#ifdef ZSTD_NEWAPI
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_nbThreads, g_nbThreads);
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_compressionLevel, cLevel);
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_enableLongDistanceMatching, g_ldmFlag);
@@ -306,25 +305,10 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_targetLength, comprParams->targetLength);
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_compressionStrategy, comprParams->strategy);
                     ZSTD_CCtx_loadDictionary(ctx, dictBuffer, dictBufferSize);
-#else
-                    size_t const avgSize = MIN(blockSize, (srcSize / nbFiles));
-                    ZSTD_parameters zparams = ZSTD_getParams(cLevel, avgSize, dictBufferSize);
-                    ZSTD_customMem const cmem = { NULL, NULL, NULL };
-                    if (comprParams->windowLog) zparams.cParams.windowLog = comprParams->windowLog;
-                    if (comprParams->chainLog) zparams.cParams.chainLog = comprParams->chainLog;
-                    if (comprParams->hashLog) zparams.cParams.hashLog = comprParams->hashLog;
-                    if (comprParams->searchLog) zparams.cParams.searchLog = comprParams->searchLog;
-                    if (comprParams->searchLength) zparams.cParams.searchLength = comprParams->searchLength;
-                    if (comprParams->targetLength) zparams.cParams.targetLength = comprParams->targetLength;
-                    if (comprParams->strategy) zparams.cParams.strategy = comprParams->strategy;
-                    cdict = ZSTD_createCDict_advanced(dictBuffer, dictBufferSize, ZSTD_dlm_byRef, ZSTD_dm_auto, zparams.cParams, cmem);
-                    if (cdict==NULL) EXM_THROW(1, "ZSTD_createCDict_advanced() allocation failure");
-#endif
                     do {
                         U32 blockNb;
                         for (blockNb=0; blockNbwindowLog) );
-            CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_chainLog, comprParams->chainLog) );
-            CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_hashLog, comprParams->hashLog) );
-            CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_searchLog, comprParams->searchLog) );
-            CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_minMatch, comprParams->searchLength) );
-            CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_targetLength, comprParams->targetLength) );
-            CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_compressionStrategy, (U32)comprParams->strategy) );
-            /* multi-threading */
-            DISPLAYLEVEL(5,"set nb threads = %u \n", g_nbThreads);
-            CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_nbThreads, g_nbThreads) );
-            /* dictionary */
-            CHECK( ZSTD_CCtx_setPledgedSrcSize(ress.cctx, srcSize) );  /* just to load dictionary with good compression parameters */
-            CHECK( ZSTD_CCtx_loadDictionary(ress.cctx, dictBuffer, dictBuffSize) );
-            CHECK( ZSTD_CCtx_setPledgedSrcSize(ress.cctx, ZSTD_CONTENTSIZE_UNKNOWN) );  /* reset */
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_contentSizeFlag, 1) );  /* always enable content size, when available (note: supposed to be default anyway) */
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_dictIDFlag, g_dictIDFlag) );
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_checksumFlag, g_checksumFlag) );
+        /* compression level */
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_compressionLevel, cLevel) );
+        /* long distance matching */
+        CHECK( ZSTD_CCtx_setParameter(
+                      ress.cctx, ZSTD_p_enableLongDistanceMatching, g_ldmFlag) );
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_ldmHashLog, g_ldmHashLog) );
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_ldmMinMatch, g_ldmMinMatch) );
+        if (g_ldmBucketSizeLog != FIO_LDM_PARAM_NOTSET) {
+            CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_ldmBucketSizeLog, g_ldmBucketSizeLog) );
         }
-#elif defined(ZSTD_MULTITHREAD)
-        {   ZSTD_parameters params = ZSTD_getParams(cLevel, srcSize, dictBuffSize);
-            params.fParams.checksumFlag = g_checksumFlag;
-            params.fParams.noDictIDFlag = !g_dictIDFlag;
-            if (comprParams->windowLog) params.cParams.windowLog = comprParams->windowLog;
-            if (comprParams->chainLog) params.cParams.chainLog = comprParams->chainLog;
-            if (comprParams->hashLog) params.cParams.hashLog = comprParams->hashLog;
-            if (comprParams->searchLog) params.cParams.searchLog = comprParams->searchLog;
-            if (comprParams->searchLength) params.cParams.searchLength = comprParams->searchLength;
-            if (comprParams->targetLength) params.cParams.targetLength = comprParams->targetLength;
-            if (comprParams->strategy) params.cParams.strategy = (ZSTD_strategy) comprParams->strategy;
-            CHECK( ZSTDMT_initCStream_advanced(ress.cctx, dictBuffer, dictBuffSize, params, srcSize) );
-            ZSTDMT_setMTCtxParameter(ress.cctx, ZSTDMT_p_sectionSize, g_blockSize);
+        if (g_ldmHashEveryLog != FIO_LDM_PARAM_NOTSET) {
+            CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_ldmHashEveryLog, g_ldmHashEveryLog) );
         }
-#else
-        {   ZSTD_parameters params = ZSTD_getParams(cLevel, srcSize, dictBuffSize);
-            params.fParams.checksumFlag = g_checksumFlag;
-            params.fParams.noDictIDFlag = !g_dictIDFlag;
-            if (comprParams->windowLog) params.cParams.windowLog = comprParams->windowLog;
-            if (comprParams->chainLog) params.cParams.chainLog = comprParams->chainLog;
-            if (comprParams->hashLog) params.cParams.hashLog = comprParams->hashLog;
-            if (comprParams->searchLog) params.cParams.searchLog = comprParams->searchLog;
-            if (comprParams->searchLength) params.cParams.searchLength = comprParams->searchLength;
-            if (comprParams->targetLength) params.cParams.targetLength = comprParams->targetLength;
-            if (comprParams->strategy) params.cParams.strategy = (ZSTD_strategy) comprParams->strategy;
-            CHECK( ZSTD_initCStream_advanced(ress.cctx, dictBuffer, dictBuffSize, params, srcSize) );
-        }
-#endif
+        /* compression parameters */
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_windowLog, comprParams->windowLog) );
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_chainLog, comprParams->chainLog) );
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_hashLog, comprParams->hashLog) );
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_searchLog, comprParams->searchLog) );
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_minMatch, comprParams->searchLength) );
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_targetLength, comprParams->targetLength) );
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_compressionStrategy, (U32)comprParams->strategy) );
+        /* multi-threading */
+        DISPLAYLEVEL(5,"set nb threads = %u \n", g_nbThreads);
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_nbThreads, g_nbThreads) );
+        /* dictionary */
+        CHECK( ZSTD_CCtx_setPledgedSrcSize(ress.cctx, srcSize) );  /* just to load dictionary with good compression parameters */
+        CHECK( ZSTD_CCtx_loadDictionary(ress.cctx, dictBuffer, dictBuffSize) );
+        CHECK( ZSTD_CCtx_setPledgedSrcSize(ress.cctx, ZSTD_CONTENTSIZE_UNKNOWN) );  /* reset */
+
         free(dictBuffer);
     }
 
@@ -509,11 +460,7 @@ static void FIO_freeCResources(cRess_t ress)
 {
     free(ress.srcBuffer);
     free(ress.dstBuffer);
-#if !defined(ZSTD_NEWAPI) && defined(ZSTD_MULTITHREAD)
-    ZSTDMT_freeCCtx(ress.cctx);
-#else
     ZSTD_freeCStream(ress.cctx);   /* never fails */
-#endif
 }
 
 
@@ -814,14 +761,8 @@ static int FIO_compressFilename_internal(cRess_t ress,
     }
 
     /* init */
-#ifdef ZSTD_NEWAPI
-    if (fileSize!=UTIL_FILESIZE_UNKNOWN)  /* when src is stdin, fileSize==0, but is effectively unknown */
+    if (fileSize != UTIL_FILESIZE_UNKNOWN)
         ZSTD_CCtx_setPledgedSrcSize(ress.cctx, fileSize);
-#elif defined(ZSTD_MULTITHREAD)
-    CHECK( ZSTDMT_resetCStream(ress.cctx, (fileSize==UTIL_FILESIZE_UNKNOWN) ? ZSTD_CONTENTSIZE_UNKNOWN : fileSize) );
-#else
-    CHECK( ZSTD_resetCStream(ress.cctx, (fileSize==UTIL_FILESIZE_UNKNOWN) ? ZSTD_CONTENTSIZE_UNKNOWN : fileSize) );
-#endif
 
     /* Main compression loop */
     while (1) {
@@ -833,14 +774,8 @@ static int FIO_compressFilename_internal(cRess_t ress,
 
         while (inBuff.pos != inBuff.size) {
             ZSTD_outBuffer outBuff = { ress.dstBuffer, ress.dstBufferSize, 0 };
-#ifdef ZSTD_NEWAPI
             CHECK( ZSTD_compress_generic(ress.cctx,
                         &outBuff, &inBuff, ZSTD_e_continue) );
-#elif defined(ZSTD_MULTITHREAD)
-            CHECK( ZSTDMT_compressStream(ress.cctx, &outBuff, &inBuff) );
-#else
-            CHECK( ZSTD_compressStream(ress.cctx, &outBuff, &inBuff) );
-#endif
 
             /* Write compressed stream */
             if (outBuff.pos) {
@@ -871,21 +806,15 @@ static int FIO_compressFilename_internal(cRess_t ress,
     {   size_t result = 1;
         while (result != 0) {
             ZSTD_outBuffer outBuff = { ress.dstBuffer, ress.dstBufferSize, 0 };
-#ifdef ZSTD_NEWAPI
             ZSTD_inBuffer inBuff = { NULL, 0, 0 };
             result = ZSTD_compress_generic(ress.cctx,
                         &outBuff, &inBuff, ZSTD_e_end);
-#elif defined(ZSTD_MULTITHREAD)
-            result = ZSTDMT_endStream(ress.cctx, &outBuff);
-#else
-            result = ZSTD_endStream(ress.cctx, &outBuff);
-#endif
             if (ZSTD_isError(result)) {
                 EXM_THROW(26, "Compression error during frame end : %s",
                             ZSTD_getErrorName(result));
             }
             {   size_t const sizeCheck = fwrite(ress.dstBuffer, 1, outBuff.pos, dstFile);
-                if (sizeCheck!=outBuff.pos)
+                if (sizeCheck != outBuff.pos)
                     EXM_THROW(27, "Write error : cannot write frame end into %s", dstFileName);
             }
             compressedfilesize += outBuff.pos;

From eac42534fe4f631606c2d604d5db0b6613cf01cd Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Thu, 19 Oct 2017 11:56:14 -0700
Subject: [PATCH 025/145] bench: fixed Visual warning regarding struct
 initialization

also :
removed dependency on zstdmt_compress.h
removed several unused macros
fileio : small code refactoring to reduce some variable scope
---
 programs/bench.c  |  51 ++++++++--------
 programs/fileio.c | 144 +++++++++++++++++++++-------------------------
 2 files changed, 92 insertions(+), 103 deletions(-)

diff --git a/programs/bench.c b/programs/bench.c
index 5b67d5c7629..8ecae9ebd3f 100644
--- a/programs/bench.c
+++ b/programs/bench.c
@@ -41,7 +41,6 @@
 #include "zstd.h"
 #include "datagen.h"     /* RDG_genBuffer */
 #include "xxhash.h"
-#include "zstdmt_compress.h"
 
 
 /* *************************************
@@ -189,7 +188,6 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
     size_t const maxCompressedSize = ZSTD_compressBound(srcSize) + (maxNbBlocks * 1024);   /* add some room for safety */
     void* const compressedBuffer = malloc(maxCompressedSize);
     void* resultBuffer = malloc(srcSize);
-    ZSTDMT_CCtx* const mtctx = ZSTDMT_createCCtx(g_nbThreads);
     ZSTD_CCtx* const ctx = ZSTD_createCCtx();
     ZSTD_DCtx* const dctx = ZSTD_createDCtx();
     size_t const loadedCompressedSize = srcSize;
@@ -286,7 +284,6 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                 if (!cCompleted) {   /* still some time to do compression tests */
                     U64 const clockLoop = g_nbSeconds ? TIMELOOP_MICROSEC : 1;
                     U32 nbLoops = 0;
-                    ZSTD_CDict* cdict = NULL;
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_nbThreads, g_nbThreads);
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_compressionLevel, cLevel);
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_enableLongDistanceMatching, g_ldmFlag);
@@ -308,27 +305,31 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                     do {
                         U32 blockNb;
                         for (blockNb=0; blockNb= maxTime);
+                    {   U64 const loopDuration = UTIL_clockSpanMicro(clockStart);
+                        if (loopDuration < fastestC*nbLoops)
+                            fastestC = loopDuration / nbLoops;
+                        totalCTime += loopDuration;
+                        cCompleted = (totalCTime >= maxTime);  /* end compression tests */
                 }   }
 
                 cSize = 0;
@@ -336,15 +337,15 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                 ratio = (double)srcSize / (double)cSize;
                 markNb = (markNb+1) % NB_MARKS;
                 DISPLAYLEVEL(2, "%2s-%-17.17s :%10u ->%10u (%5.3f),%6.1f MB/s\r",
-                        marks[markNb], displayName, (U32)srcSize, (U32)cSize, ratio,
-                        (double)srcSize / fastestC );
+                        marks[markNb], displayName, (U32)srcSize, (U32)cSize,
+                        ratio, (double)srcSize / fastestC );
             } else {   /* g_decodeOnly */
                 memcpy(compressedBuffer, srcBuffer, loadedCompressedSize);
             }
 
 #if 0       /* disable decompression test */
             dCompleted=1;
-            (void)totalDTime; (void)fastestD; (void)crcOrig;   /*  unused when decompression disabled */
+            (void)totalDTime; (void)fastestD; (void)crcOrig;   /* unused when decompression disabled */
 #else
             /* Decompression */
             if (!dCompleted) memset(resultBuffer, 0xD6, srcSize);  /* warm result buffer */
@@ -374,9 +375,10 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                     nbLoops++;
                 } while (UTIL_clockSpanMicro(clockStart) < clockLoop);
                 ZSTD_freeDDict(ddict);
-                {   U64 const clockSpanMicro = UTIL_clockSpanMicro(clockStart);
-                    if (clockSpanMicro < fastestD*nbLoops) fastestD = clockSpanMicro / nbLoops;
-                    totalDTime += clockSpanMicro;
+                {   U64 const loopDuration = UTIL_clockSpanMicro(clockStart);
+                    if (loopDuration < fastestD*nbLoops)
+                        fastestD = loopDuration / nbLoops;
+                    totalDTime += loopDuration;
                     dCompleted = (totalDTime >= maxTime);
             }   }
 
@@ -439,7 +441,6 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
     free(blockTable);
     free(compressedBuffer);
     free(resultBuffer);
-    ZSTDMT_freeCCtx(mtctx);
     ZSTD_freeCCtx(ctx);
     ZSTD_freeDCtx(dctx);
     return 0;
diff --git a/programs/fileio.c b/programs/fileio.c
index c8eab06791e..7efa3c65604 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -42,9 +42,6 @@
 #include "fileio.h"
 #define ZSTD_STATIC_LINKING_ONLY   /* ZSTD_magicNumber, ZSTD_frameHeaderSize_max */
 #include "zstd.h"
-#ifdef ZSTD_MULTITHREAD
-#  include "zstdmt_compress.h"
-#endif
 #if defined(ZSTD_GZCOMPRESS) || defined(ZSTD_GZDECOMPRESS)
 #  include 
 #  if !defined(z_const)
@@ -70,18 +67,6 @@
 #define MB *(1<<20)
 #define GB *(1U<<30)
 
-#define _1BIT  0x01
-#define _2BITS 0x03
-#define _3BITS 0x07
-#define _4BITS 0x0F
-#define _6BITS 0x3F
-#define _8BITS 0xFF
-
-#define BLOCKSIZE      (128 KB)
-#define ROLLBUFFERSIZE (BLOCKSIZE*8*64)
-
-#define FIO_FRAMEHEADERSIZE  5    /* as a define, because needed to allocated table on stack */
-
 #define DICTSIZE_MAX (32 MB)   /* protection against large input (attack scenario) */
 
 #define FNSPACE 30
@@ -122,14 +107,14 @@ static clock_t g_time = 0;
 #  define ZSTD_DEBUG 0
 #endif
 #define DEBUGLOG(l,...) if (l<=ZSTD_DEBUG) DISPLAY(__VA_ARGS__);
-#define EXM_THROW(error, ...)                                              \
-{                                                                          \
-    DISPLAYLEVEL(1, "zstd: ");                                             \
-    DEBUGLOG(1, "Error defined at %s, line %i : \n", __FILE__, __LINE__);  \
-    DISPLAYLEVEL(1, "error %i : ", error);                                 \
-    DISPLAYLEVEL(1, __VA_ARGS__);                                          \
-    DISPLAYLEVEL(1, " \n");                                                \
-    exit(error);                                                           \
+#define EXM_THROW(error, ...)                                             \
+{                                                                         \
+    DISPLAYLEVEL(1, "zstd: ");                                            \
+    DEBUGLOG(1, "Error defined at %s, line %i : \n", __FILE__, __LINE__); \
+    DISPLAYLEVEL(1, "error %i : ", error);                                \
+    DISPLAYLEVEL(1, __VA_ARGS__);                                         \
+    DISPLAYLEVEL(1, " \n");                                               \
+    exit(error);                                                          \
 }
 
 #define CHECK(f) {                                   \
@@ -273,86 +258,88 @@ static int FIO_remove(const char* path)
 }
 
 /** FIO_openSrcFile() :
- * condition : `dstFileName` must be non-NULL.
- * @result : FILE* to `dstFileName`, or NULL if it fails */
+ *  condition : `srcFileName` must be non-NULL.
+ * @result : FILE* to `srcFileName`, or NULL if it fails */
 static FILE* FIO_openSrcFile(const char* srcFileName)
 {
-    FILE* f;
-
+    assert(srcFileName != NULL);
     if (!strcmp (srcFileName, stdinmark)) {
         DISPLAYLEVEL(4,"Using stdin for input\n");
-        f = stdin;
         SET_BINARY_MODE(stdin);
-    } else {
-        if (!UTIL_isRegularFile(srcFileName)) {
-            DISPLAYLEVEL(1, "zstd: %s is not a regular file -- ignored \n",
-                            srcFileName);
-            return NULL;
-        }
-        f = fopen(srcFileName, "rb");
-        if ( f==NULL )
-            DISPLAYLEVEL(1, "zstd: %s: %s \n", srcFileName, strerror(errno));
+        return stdin;
+    }
+
+    if (!UTIL_isRegularFile(srcFileName)) {
+        DISPLAYLEVEL(1, "zstd: %s is not a regular file -- ignored \n",
+                        srcFileName);
+        return NULL;
     }
 
-    return f;
+    {   FILE* const f = fopen(srcFileName, "rb");
+        if (f == NULL)
+            DISPLAYLEVEL(1, "zstd: %s: %s \n", srcFileName, strerror(errno));
+        return f;
+    }
 }
 
 /** FIO_openDstFile() :
- * condition : `dstFileName` must be non-NULL.
+ *  condition : `dstFileName` must be non-NULL.
  * @result : FILE* to `dstFileName`, or NULL if it fails */
 static FILE* FIO_openDstFile(const char* dstFileName)
 {
-    FILE* f;
-
+    assert(dstFileName != NULL);
     if (!strcmp (dstFileName, stdoutmark)) {
         DISPLAYLEVEL(4,"Using stdout for output\n");
-        f = stdout;
         SET_BINARY_MODE(stdout);
         if (g_sparseFileSupport==1) {
             g_sparseFileSupport = 0;
             DISPLAYLEVEL(4, "Sparse File Support is automatically disabled on stdout ; try --sparse \n");
         }
-    } else {
-        if (g_sparseFileSupport == 1) {
-            g_sparseFileSupport = ZSTD_SPARSE_DEFAULT;
-        }
-        if (strcmp (dstFileName, nulmark)) {
-            /* Check if destination file already exists */
-            f = fopen( dstFileName, "rb" );
-            if (f != 0) {  /* dst file exists, prompt for overwrite authorization */
-                fclose(f);
-                if (!g_overwrite) {
-                    if (g_displayLevel <= 1) {
-                        /* No interaction possible */
-                        DISPLAY("zstd: %s already exists; not overwritten  \n",
-                                dstFileName);
+        return stdout;
+    }
+
+    if (g_sparseFileSupport == 1) {
+        g_sparseFileSupport = ZSTD_SPARSE_DEFAULT;
+    }
+
+    if (strcmp (dstFileName, nulmark)) {  /* not /dev/null */
+        /* Check if destination file already exists */
+        FILE* const fCheck = fopen( dstFileName, "rb" );
+        if (fCheck != NULL) {  /* dst file exists, authorization prompt */
+            fclose(fCheck);
+            if (!g_overwrite) {
+                if (g_displayLevel <= 1) {
+                    /* No interaction possible */
+                    DISPLAY("zstd: %s already exists; not overwritten  \n",
+                            dstFileName);
+                    return NULL;
+                }
+                DISPLAY("zstd: %s already exists; overwrite (y/N) ? ",
+                        dstFileName);
+                {   int ch = getchar();
+                    if ((ch!='Y') && (ch!='y')) {
+                        DISPLAY("    not overwritten  \n");
                         return NULL;
                     }
-                    DISPLAY("zstd: %s already exists; do you wish to overwrite (y/N) ? ",
-                            dstFileName);
-                    {   int ch = getchar();
-                        if ((ch!='Y') && (ch!='y')) {
-                            DISPLAY("    not overwritten  \n");
-                            return NULL;
-                        }
-                        /* flush rest of input line */
-                        while ((ch!=EOF) && (ch!='\n')) ch = getchar();
-                }   }
-                /* need to unlink */
-                FIO_remove(dstFileName);
-        }   }
-        f = fopen( dstFileName, "wb" );
-        if (f==NULL) DISPLAYLEVEL(1, "zstd: %s: %s\n", dstFileName, strerror(errno));
-    }
+                    /* flush rest of input line */
+                    while ((ch!=EOF) && (ch!='\n')) ch = getchar();
+            }   }
+            /* need to unlink */
+            FIO_remove(dstFileName);
+    }   }
 
-    return f;
+    {   FILE* const f = fopen( dstFileName, "wb" );
+        if (f == NULL)
+            DISPLAYLEVEL(1, "zstd: %s: %s\n", dstFileName, strerror(errno));
+        return f;
+    }
 }
 
 
 /*! FIO_createDictBuffer() :
  *  creates a buffer, pointed by `*bufferPtr`,
  *  loads `filename` content into it, up to DICTSIZE_MAX bytes.
- *  @return : loaded size
+ * @return : loaded size
  *  if fileName==NULL, returns 0 and a NULL pointer
  */
 static size_t FIO_createDictBuffer(void** bufferPtr, const char* fileName)
@@ -360,12 +347,13 @@ static size_t FIO_createDictBuffer(void** bufferPtr, const char* fileName)
     FILE* fileHandle;
     U64 fileSize;
 
+    assert(bufferPtr != NULL);
     *bufferPtr = NULL;
     if (fileName == NULL) return 0;
 
     DISPLAYLEVEL(4,"Loading %s as dictionary \n", fileName);
     fileHandle = fopen(fileName, "rb");
-    if (fileHandle==0) EXM_THROW(31, "%s: %s", fileName, strerror(errno));
+    if (fileHandle==NULL) EXM_THROW(31, "%s: %s", fileName, strerror(errno));
     fileSize = UTIL_getFileSize(fileName);
     if (fileSize > DICTSIZE_MAX) {
         EXM_THROW(32, "Dictionary file %s is too large (> %u MB)",
@@ -412,13 +400,13 @@ static cRess_t FIO_createCResources(const char* dictFileName, int cLevel,
     if (!ress.srcBuffer || !ress.dstBuffer)
         EXM_THROW(31, "allocation error : not enough memory");
 
-    /* dictionary */
+    /* Advances parameters, including dictionary */
     {   void* dictBuffer;
         size_t const dictBuffSize = FIO_createDictBuffer(&dictBuffer, dictFileName);   /* works with dictFileName==NULL */
         if (dictFileName && (dictBuffer==NULL))
             EXM_THROW(32, "allocation error : can't create dictBuffer");
 
-        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_contentSizeFlag, 1) );  /* always enable content size, when available (note: supposed to be default anyway) */
+        CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_contentSizeFlag, 1) );  /* always enable content size when available (note: supposed to be default) */
         CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_dictIDFlag, g_dictIDFlag) );
         CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_checksumFlag, g_checksumFlag) );
         /* compression level */
@@ -446,7 +434,7 @@ static cRess_t FIO_createCResources(const char* dictFileName, int cLevel,
         DISPLAYLEVEL(5,"set nb threads = %u \n", g_nbThreads);
         CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_nbThreads, g_nbThreads) );
         /* dictionary */
-        CHECK( ZSTD_CCtx_setPledgedSrcSize(ress.cctx, srcSize) );  /* just to load dictionary with good compression parameters */
+        CHECK( ZSTD_CCtx_setPledgedSrcSize(ress.cctx, srcSize) );  /* just for dictionary loading, using good compression parameters */
         CHECK( ZSTD_CCtx_loadDictionary(ress.cctx, dictBuffer, dictBuffSize) );
         CHECK( ZSTD_CCtx_setPledgedSrcSize(ress.cctx, ZSTD_CONTENTSIZE_UNKNOWN) );  /* reset */
 

From 91535d71ec58b9ef9090c5b4ed39a31243a5ced4 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Thu, 19 Oct 2017 12:09:34 -0700
Subject: [PATCH 026/145] fixed missing zstdmt_compress.h dependency

we lose a warning message :
when a job size is chosen < minimum job size for multithreading,
it is automatically resized to minimum size.

If this information is really useful, it should be present in zstd.h now.
---
 programs/fileio.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/programs/fileio.c b/programs/fileio.c
index 7efa3c65604..a8e33994cd6 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -203,10 +203,6 @@ static U32 g_blockSize = 0;
 void FIO_setBlockSize(unsigned blockSize) {
     if (blockSize && g_nbThreads==1)
         DISPLAYLEVEL(2, "Setting block size is useless in single-thread mode \n");
-#ifdef ZSTD_MULTITHREAD
-    if (blockSize-1 < ZSTDMT_SECTION_SIZE_MIN-1)   /* intentional underflow */
-        DISPLAYLEVEL(2, "Note : minimum block size is %u KB \n", (ZSTDMT_SECTION_SIZE_MIN>>10));
-#endif
     g_blockSize = blockSize;
 }
 #define FIO_OVERLAP_LOG_NOTSET 9999

From 97dccbbb2b9905d491c65ccab79d53e3b0a05187 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Thu, 19 Oct 2017 14:05:23 -0700
Subject: [PATCH 027/145] fixed zbufftest

preserve "pledgedSrcSize=0" means "unknown" in init_advanced()
---
 lib/deprecated/zbuff_compress.c | 1 +
 tests/zbufftest.c               | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/deprecated/zbuff_compress.c b/lib/deprecated/zbuff_compress.c
index 8adbaec260d..f39c60d89f6 100644
--- a/lib/deprecated/zbuff_compress.c
+++ b/lib/deprecated/zbuff_compress.c
@@ -72,6 +72,7 @@ size_t ZBUFF_compressInit_advanced(ZBUFF_CCtx* zbc,
                                    const void* dict, size_t dictSize,
                                    ZSTD_parameters params, unsigned long long pledgedSrcSize)
 {
+    if (pledgedSrcSize==0) pledgedSrcSize = ZSTD_CONTENTSIZE_UNKNOWN;  /* preserve "0 == unknown" behavior */
     return ZSTD_initCStream_advanced(zbc, dict, dictSize, params, pledgedSrcSize);
 }
 
diff --git a/tests/zbufftest.c b/tests/zbufftest.c
index e64c886a5e7..ce5e5518a5e 100644
--- a/tests/zbufftest.c
+++ b/tests/zbufftest.c
@@ -357,7 +357,7 @@ static int fuzzerTests(U32 seed, U32 nbTests, unsigned startTest, double compres
             {   ZSTD_parameters params = ZSTD_getParams(cLevel, 0, dictSize);
                 params.fParams.checksumFlag = FUZ_rand(&lseed) & 1;
                 params.fParams.noDictIDFlag = FUZ_rand(&lseed) & 1;
-                {   size_t const initError = ZBUFF_compressInit_advanced(zc, dict, dictSize, params, 0);
+                {   size_t const initError = ZBUFF_compressInit_advanced(zc, dict, dictSize, params, ZSTD_CONTENTSIZE_UNKNOWN);
                     CHECK (ZBUFF_isError(initError),"init error : %s", ZBUFF_getErrorName(initError));
         }   }   }
 

From b495140f67cb638beaa9dd0120fe7954861d0bdf Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Wed, 25 Oct 2017 12:47:57 -0700
Subject: [PATCH 028/145] Update BUCK files

* Correct XXH namespace (Fixes #901)
* Multithreading always enabled
* GZIP/LZ4/LZMA always enabled
* Legacy support always fully enabled
---
 lib/BUCK         | 41 +++++++++++++++++++++++++++++++----------
 programs/BUCK    | 41 +++++++++++------------------------------
 zlibWrapper/BUCK |  2 +-
 3 files changed, 43 insertions(+), 41 deletions(-)

diff --git a/lib/BUCK b/lib/BUCK
index 6812c1b1e54..9e01468f4dd 100644
--- a/lib/BUCK
+++ b/lib/BUCK
@@ -15,15 +15,9 @@ cxx_library(
     header_namespace='',
     visibility=['PUBLIC'],
     exported_headers=subdir_glob([
-        ('compress', 'zstdmt_compress.h'),
+        ('compress', 'zstd*.h'),
     ]),
-    headers=subdir_glob([
-        ('compress', 'zstd_opt.h'),
-    ]),
-    srcs=[
-        'compress/zstd_compress.c',
-        'compress/zstdmt_compress.c',
-    ],
+    srcs=glob(['compress/zstd*.c']),
     deps=[':common'],
 )
 
@@ -31,7 +25,7 @@ cxx_library(
     name='decompress',
     header_namespace='',
     visibility=['PUBLIC'],
-    srcs=['decompress/zstd_decompress.c'],
+    srcs=glob(['decompress/zstd*.c']),
     deps=[
         ':common',
         ':legacy',
@@ -58,6 +52,9 @@ cxx_library(
     ]),
     srcs=glob(['legacy/*.c']),
     deps=[':common'],
+    exported_preprocessor_flags=[
+        '-DZSTD_LEGACY_SUPPORT=4',
+    ],
 )
 
 cxx_library(
@@ -74,6 +71,15 @@ cxx_library(
     deps=[':common'],
 )
 
+cxx_library(
+    name='compiler',
+    header_namespace='',
+    visibility=['PUBLIC'],
+    exported_headers=subdir_glob([
+        ('common', 'compiler.h'),
+    ]),
+)
+
 cxx_library(
     name='bitstream',
     header_namespace='',
@@ -100,6 +106,7 @@ cxx_library(
     ],
     deps=[
         ':bitstream',
+        ':compiler',
         ':errors',
         ':mem',
     ],
@@ -133,7 +140,10 @@ cxx_library(
         ('common', 'pool.h'),
     ]),
     srcs=['common/pool.c'],
-    deps=[':threading'],
+    deps=[
+        ':threading',
+        ':zstd_common',
+    ],
 )
 
 cxx_library(
@@ -144,6 +154,12 @@ cxx_library(
         ('common', 'threading.h'),
     ]),
     srcs=['common/threading.c'],
+    exported_preprocessor_flags=[
+        '-DZSTD_MULTITHREAD',
+    ],
+    exported_linker_flags=[
+        '-pthread',
+    ],
 )
 
 cxx_library(
@@ -154,6 +170,9 @@ cxx_library(
         ('common', 'xxhash.h'),
     ]),
     srcs=['common/xxhash.c'],
+    exported_preprocessor_flags=[
+        '-DXXH_NAMESPACE=ZSTD_',
+    ],
 )
 
 cxx_library(
@@ -166,6 +185,7 @@ cxx_library(
     ]),
     srcs=['common/zstd_common.c'],
     deps=[
+        ':compiler',
         ':errors',
         ':mem',
     ],
@@ -175,6 +195,7 @@ cxx_library(
     name='common',
     deps=[
         ':bitstream',
+        ':compiler',
         ':entropy',
         ':errors',
         ':mem',
diff --git a/programs/BUCK b/programs/BUCK
index 06940304299..d2aa6373ff4 100644
--- a/programs/BUCK
+++ b/programs/BUCK
@@ -10,38 +10,19 @@ cxx_binary(
         '//lib:mem',
         '//lib:xxhash',
     ],
-)
-
-cxx_binary(
-    name='zstdmt',
-    headers=glob(['*.h'], excludes=['datagen.h', 'platform.h', 'util.h']),
-    srcs=glob(['*.c'], excludes=['datagen.c']),
-    deps=[
-        ':datagen',
-        ':util',
-        '//lib:zstd',
-        '//lib:zdict',
-        '//lib:mem',
-        '//lib:xxhash',
+    preprocessor_flags=[
+        '-DZSTD_GZCOMPRESS',
+        '-DZSTD_GZDECOMPRESS',
+        '-DZSTD_LZMACOMPRESS',
+        '-DZSTD_LZMADECOMPRES',
+        '-DZSTD_LZ4COMPRESS',
+        '-DZSTD_LZ4DECOMPRES',
     ],
-    preprocessor_flags=['-DZSTD_MULTITHREAD'],
-    linker_flags=['-lpthread'],
-)
-
-cxx_binary(
-    name='gzstd',
-    headers=glob(['*.h'], excludes=['datagen.h', 'platform.h', 'util.h']),
-    srcs=glob(['*.c'], excludes=['datagen.c']),
-    deps=[
-        ':datagen',
-        ':util',
-        '//lib:zstd',
-        '//lib:zdict',
-        '//lib:mem',
-        '//lib:xxhash',
+    linker_flags=[
+        '-lz',
+        '-llzma',
+        '-llz4',
     ],
-    preprocessor_flags=['-DZSTD_GZDECOMPRESS'],
-    linker_flags=['-lz'],
 )
 
 cxx_library(
diff --git a/zlibWrapper/BUCK b/zlibWrapper/BUCK
index a3b74ac3f9f..a0710e1cf4c 100644
--- a/zlibWrapper/BUCK
+++ b/zlibWrapper/BUCK
@@ -12,7 +12,7 @@ cxx_library(
     deps=[
         '//lib:zstd',
         '//lib:zstd_common',
-    ]
+    ],
 )
 
 cxx_binary(

From 86b8134cadadc7dcfb7f9e43baa54212ea7c58c3 Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Wed, 25 Oct 2017 17:10:57 -0700
Subject: [PATCH 029/145] [libzstd] Fix parameter selection for empty input

ZSTD_compress() and friends would treat an empty input as an unknown size
when selecting parameters. Thus, they would drastically overallocate the
context. Tell ZSTD_getParams() that the source size is 1 when it is empty.
---
 lib/compress/zstd_compress.c | 2 +-
 tests/fuzzer.c               | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 635cc665c85..21b84d1046d 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -2213,7 +2213,7 @@ size_t ZSTD_compress_advanced_internal(
 size_t ZSTD_compress_usingDict(ZSTD_CCtx* ctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize,
                                const void* dict, size_t dictSize, int compressionLevel)
 {
-    ZSTD_parameters params = ZSTD_getParams(compressionLevel, srcSize, dict ? dictSize : 0);
+    ZSTD_parameters params = ZSTD_getParams(compressionLevel, srcSize ? srcSize : 1, dict ? dictSize : 0);
     params.fParams.contentSizeFlag = 1;
     return ZSTD_compress_internal(ctx, dst, dstCapacity, src, srcSize, dict, dictSize, params);
 }
diff --git a/tests/fuzzer.c b/tests/fuzzer.c
index 76df77af901..e40f1997826 100644
--- a/tests/fuzzer.c
+++ b/tests/fuzzer.c
@@ -359,6 +359,14 @@ static int basicUnitTests(U32 seed, double compressibility)
       if (ZSTD_getErrorCode(r) != ZSTD_error_srcSize_wrong) goto _output_error; }
     DISPLAYLEVEL(4, "OK \n");
 
+    DISPLAYLEVEL(4, "test%di : check CCtx size after compressing empty input : ", testNb++);
+    {   ZSTD_CCtx* cctx = ZSTD_createCCtx();
+        size_t const r = ZSTD_compressCCtx(cctx, compressedBuffer, compressedBufferSize, NULL, 0, 19);
+        if (ZSTD_isError(r)) goto _output_error;
+        if (ZSTD_sizeof_CCtx(cctx) > (1U << 20)) goto _output_error;
+        ZSTD_freeCCtx(cctx);
+    }
+    DISPLAYLEVEL(4, "OK \n");
 
     /* Static CCtx tests */
 #define STATIC_CCTX_LEVEL 3

From 61e5a1adfc1645bf44e7c69a41c372b595202b22 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 31 Oct 2017 17:43:24 -0700
Subject: [PATCH 030/145] removed direct call to malloc() from pool.c

---
 lib/common/pool.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/lib/common/pool.c b/lib/common/pool.c
index 1b0fe1035d2..98b109e72a8 100644
--- a/lib/common/pool.c
+++ b/lib/common/pool.c
@@ -11,7 +11,6 @@
 
 /* ======   Dependencies   ======= */
 #include   /* size_t */
-#include   /* malloc, calloc, free */
 #include "pool.h"
 
 /* ======   Compiler specifics   ====== */
@@ -115,7 +114,7 @@ POOL_ctx* POOL_create_advanced(size_t numThreads, size_t queueSize, ZSTD_customM
      * and full queues.
      */
     ctx->queueSize = queueSize + 1;
-    ctx->queue = (POOL_job*) malloc(ctx->queueSize * sizeof(POOL_job));
+    ctx->queue = (POOL_job*)ZSTD_malloc(ctx->queueSize * sizeof(POOL_job), customMem);
     ctx->queueHead = 0;
     ctx->queueTail = 0;
     ctx->numThreadsBusy = 0;

From 1fc4f593da3cc810ae1744550196dacfa36d628d Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Wed, 1 Nov 2017 12:49:51 -0700
Subject: [PATCH 031/145] Allow skippable frames of any size

---
 lib/decompress/zstd_decompress.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/lib/decompress/zstd_decompress.c b/lib/decompress/zstd_decompress.c
index 96fc6090896..caeafc255fe 100644
--- a/lib/decompress/zstd_decompress.c
+++ b/lib/decompress/zstd_decompress.c
@@ -2555,17 +2555,21 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
             /* fall-through */
         case zdss_load:
             {   size_t const neededInSize = ZSTD_nextSrcSizeToDecompress(zds);
-                size_t const toLoad = neededInSize - zds->inPos;   /* should always be <= remaining space within inBuff */
+                size_t const toLoad = neededInSize - zds->inPos;
+                int const isSkipFrame = ZSTD_isSkipFrame(zds);
                 size_t loadedSize;
-                if (toLoad > zds->inBuffSize - zds->inPos) return ERROR(corruption_detected);   /* should never happen */
-                loadedSize = ZSTD_limitCopy(zds->inBuff + zds->inPos, toLoad, ip, iend-ip);
+                if (isSkipFrame) {
+                    loadedSize = MIN(toLoad, (size_t)(iend-ip));
+                } else {
+                    if (toLoad > zds->inBuffSize - zds->inPos) return ERROR(corruption_detected);   /* should never happen */
+                    loadedSize = ZSTD_limitCopy(zds->inBuff + zds->inPos, toLoad, ip, iend-ip);
+                }
                 ip += loadedSize;
                 zds->inPos += loadedSize;
                 if (loadedSize < toLoad) { someMoreWork = 0; break; }   /* not enough input, wait for more */
 
                 /* decode loaded input */
-                {  const int isSkipFrame = ZSTD_isSkipFrame(zds);
-                   size_t const decodedSize = ZSTD_decompressContinue(zds,
+                {   size_t const decodedSize = ZSTD_decompressContinue(zds,
                         zds->outBuff + zds->outStart, zds->outBuffSize - zds->outStart,
                         zds->inBuff, neededInSize);
                     if (ZSTD_isError(decodedSize)) return decodedSize;

From e19b0822bc08e0427daa4b1a7c366e51a3efb809 Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Wed, 1 Nov 2017 13:10:03 -0700
Subject: [PATCH 032/145] Test large skippable frames

---
 tests/fuzzer.c      | 2 +-
 tests/zstreamtest.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/fuzzer.c b/tests/fuzzer.c
index e40f1997826..be9dc3e062e 100644
--- a/tests/fuzzer.c
+++ b/tests/fuzzer.c
@@ -525,7 +525,7 @@ static int basicUnitTests(U32 seed, double compressibility)
             off += r;
             if (i == segs/2) {
                 /* insert skippable frame */
-                const U32 skipLen = 128 KB;
+                const U32 skipLen = 129 KB;
                 MEM_writeLE32((BYTE*)compressedBuffer + off, ZSTD_MAGIC_SKIPPABLE_START);
                 MEM_writeLE32((BYTE*)compressedBuffer + off + 4, skipLen);
                 off += skipLen + ZSTD_skippableHeaderSize;
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 6f1f6df1856..53dbaf3bace 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -213,7 +213,7 @@ static int basicUnitTests(U32 seed, double compressibility, ZSTD_customMem custo
 {
     size_t const CNBufferSize = COMPRESSIBLE_NOISE_LENGTH;
     void* CNBuffer = malloc(CNBufferSize);
-    size_t const skippableFrameSize = 11;
+    size_t const skippableFrameSize = 200 KB;
     size_t const compressedBufferSize = (8 + skippableFrameSize) + ZSTD_compressBound(COMPRESSIBLE_NOISE_LENGTH);
     void* compressedBuffer = malloc(compressedBufferSize);
     size_t const decodedBufferSize = CNBufferSize;

From 428e8b3bf49ed9d36e732acf634d6150d0e2b302 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 1 Nov 2017 13:15:23 -0700
Subject: [PATCH 033/145] fix : ZSTD_compress_generic(,,,ZSTD_e_end)
 automatically sets pledgedSrcSize

as per documentation, on ZSTD_setPledgedSrcSize() :
> If all data is provided and consumed in a single round,
> this value (pledgedSrcSize) is overriden by srcSize instead.

This wasn't applied before compression level is transformed into compression parameters.
As a consequence, small input missed compression parameters adaptation.

It seems to work fine now : compression was compared with ZSTD_compress_advanced(),
results were the same.
---
 lib/compress/zstd_compress.c |  1 +
 programs/bench.c             | 10 ++++++++++
 2 files changed, 11 insertions(+)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 21b84d1046d..d50f35116ab 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -2814,6 +2814,7 @@ size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
     if (cctx->streamStage == zcss_init) {
         ZSTD_prefixDict const prefixDict = cctx->prefixDict;
         ZSTD_CCtx_params params = cctx->requestedParams;
+        if (endOp == ZSTD_e_end) cctx->pledgedSrcSizePlusOne = input->size + 1;
         params.cParams = ZSTD_getCParamsFromCCtxParams(
                 cctx->requestedParams, cctx->pledgedSrcSizePlusOne-1, 0 /*dictSize*/);
         memset(&cctx->prefixDict, 0, sizeof(cctx->prefixDict));  /* single usage */
diff --git a/programs/bench.c b/programs/bench.c
index 8ecae9ebd3f..66c406abd0e 100644
--- a/programs/bench.c
+++ b/programs/bench.c
@@ -286,6 +286,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                     U32 nbLoops = 0;
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_nbThreads, g_nbThreads);
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_compressionLevel, cLevel);
+                    //ZSTD_CCtx_setPledgedSrcSize(ctx, blockTable[0].srcSize);  /* estimation, only to help cLevel conversion before applying advanced compression parameters */
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_enableLongDistanceMatching, g_ldmFlag);
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_ldmMinMatch, g_ldmMinMatch);
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_ldmHashLog, g_ldmHashLog);
@@ -305,6 +306,14 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                     do {
                         U32 blockNb;
                         for (blockNb=0; blockNb
Date: Wed, 1 Nov 2017 16:57:48 -0700
Subject: [PATCH 034/145] minor refactor

added some traces and assert
related to hunting a potential ubsan error in 32-bits more
(it ends up being a compiler-side issue : https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82802).

Modified one pointer arithmetic expression for a more conformant way.
---
 lib/compress/zstd_compress.c   | 15 +++++++++-----
 lib/compress/zstdmt_compress.c | 36 ++++++++++++++++++++++------------
 tests/zstreamtest.c            |  3 ++-
 3 files changed, 36 insertions(+), 18 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index d50f35116ab..808418a6082 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -1468,14 +1468,18 @@ MEM_STATIC size_t ZSTD_compressSequences_internal(seqStore_t* seqStorePtr,
                 entropy, cParams->strategy, op, dstCapacity, literals, litSize);
         if (ZSTD_isError(cSize))
           return cSize;
+        assert(cSize <= dstCapacity);
         op += cSize;
     }
 
     /* Sequences Header */
-    if ((oend-op) < 3 /*max nbSeq Size*/ + 1 /*seqHead */) return ERROR(dstSize_tooSmall);
-    if (nbSeq < 0x7F) *op++ = (BYTE)nbSeq;
-    else if (nbSeq < LONGNBSEQ) op[0] = (BYTE)((nbSeq>>8) + 0x80), op[1] = (BYTE)nbSeq, op+=2;
-    else op[0]=0xFF, MEM_writeLE16(op+1, (U16)(nbSeq - LONGNBSEQ)), op+=3;
+    if ((oend-op) < 3 /*max nbSeq Size*/ + 1 /*seqHead*/) return ERROR(dstSize_tooSmall);
+    if (nbSeq < 0x7F)
+        *op++ = (BYTE)nbSeq;
+    else if (nbSeq < LONGNBSEQ)
+        op[0] = (BYTE)((nbSeq>>8) + 0x80), op[1] = (BYTE)nbSeq, op+=2;
+    else
+        op[0]=0xFF, MEM_writeLE16(op+1, (U16)(nbSeq - LONGNBSEQ)), op+=3;
     if (nbSeq==0) return op - ostart;
 
     /* seqHead : flags for FSE encoding type */
@@ -1606,6 +1610,7 @@ static void ZSTD_storeLastLiterals(seqStore_t* seqStorePtr,
 
 static size_t ZSTD_compressBlock_internal(ZSTD_CCtx* zc, void* dst, size_t dstCapacity, const void* src, size_t srcSize)
 {
+    DEBUGLOG(5, "ZSTD_compressBlock_internal : dstCapacity = %u", (U32)dstCapacity);
     if (srcSize < MIN_CBLOCK_SIZE+ZSTD_blockHeaderSize+1)
         return 0;   /* don't even attempt compression below a certain srcSize */
     ZSTD_resetSeqStore(&(zc->seqStore));
@@ -1867,7 +1872,7 @@ static size_t ZSTD_loadDictionaryContent(ZSTD_CCtx* zc, const void* src, size_t
     zc->lowLimit = zc->dictLimit;
     zc->dictLimit = (U32)(zc->nextSrc - zc->base);
     zc->dictBase = zc->base;
-    zc->base += ip - zc->nextSrc;
+    zc->base = ip - zc->dictLimit;
     zc->nextToUpdate = zc->dictLimit;
     zc->loadedDictEnd = zc->appliedParams.forceWindow ? 0 : (U32)(iend - zc->base);
 
diff --git a/lib/compress/zstdmt_compress.c b/lib/compress/zstdmt_compress.c
index af7be3167b3..a855745be69 100644
--- a/lib/compress/zstdmt_compress.c
+++ b/lib/compress/zstdmt_compress.c
@@ -142,7 +142,10 @@ static size_t ZSTDMT_sizeof_bufferPool(ZSTDMT_bufferPool* bufPool)
 
 static void ZSTDMT_setBufferSize(ZSTDMT_bufferPool* bufPool, size_t bSize)
 {
+    ZSTD_pthread_mutex_lock(&bufPool->poolMutex);
+    DEBUGLOG(4, "ZSTDMT_setBufferSize: bSize = %u", (U32)bSize);
     bufPool->bufferSize = bSize;
+    ZSTD_pthread_mutex_unlock(&bufPool->poolMutex);
 }
 
 /** ZSTDMT_getBuffer() :
@@ -150,28 +153,31 @@ static void ZSTDMT_setBufferSize(ZSTDMT_bufferPool* bufPool, size_t bSize)
 static buffer_t ZSTDMT_getBuffer(ZSTDMT_bufferPool* bufPool)
 {
     size_t const bSize = bufPool->bufferSize;
-    DEBUGLOG(5, "ZSTDMT_getBuffer");
+    DEBUGLOG(5, "ZSTDMT_getBuffer: bSize = %u", (U32)bufPool->bufferSize);
     ZSTD_pthread_mutex_lock(&bufPool->poolMutex);
     if (bufPool->nbBuffers) {   /* try to use an existing buffer */
         buffer_t const buf = bufPool->bTable[--(bufPool->nbBuffers)];
         size_t const availBufferSize = buf.size;
         bufPool->bTable[bufPool->nbBuffers] = g_nullBuffer;
-        if ((availBufferSize >= bSize) & (availBufferSize <= 10*bSize)) {
+        if ((availBufferSize >= bSize) & ((availBufferSize>>3) <= bSize)) {
             /* large enough, but not too much */
+            DEBUGLOG(5, "ZSTDMT_getBuffer: provide buffer %u of size %u",
+                        bufPool->nbBuffers, (U32)buf.size);
             ZSTD_pthread_mutex_unlock(&bufPool->poolMutex);
             return buf;
         }
         /* size conditions not respected : scratch this buffer, create new one */
-        DEBUGLOG(5, "existing buffer does not meet size conditions => freeing");
+        DEBUGLOG(5, "ZSTDMT_getBuffer: existing buffer does not meet size conditions => freeing");
         ZSTD_free(buf.start, bufPool->cMem);
     }
     ZSTD_pthread_mutex_unlock(&bufPool->poolMutex);
     /* create new buffer */
-    DEBUGLOG(5, "create a new buffer");
+    DEBUGLOG(5, "ZSTDMT_getBuffer: create a new buffer");
     {   buffer_t buffer;
         void* const start = ZSTD_malloc(bSize, bufPool->cMem);
         buffer.start = start;   /* note : start can be NULL if malloc fails ! */
         buffer.size = (start==NULL) ? 0 : bSize;
+        DEBUGLOG(5, "ZSTDMT_getBuffer: created buffer of size %u", (U32)bSize);
         return buffer;
     }
 }
@@ -184,12 +190,14 @@ static void ZSTDMT_releaseBuffer(ZSTDMT_bufferPool* bufPool, buffer_t buf)
     ZSTD_pthread_mutex_lock(&bufPool->poolMutex);
     if (bufPool->nbBuffers < bufPool->totalBuffers) {
         bufPool->bTable[bufPool->nbBuffers++] = buf;  /* stored for later use */
+        DEBUGLOG(5, "ZSTDMT_releaseBuffer: stored buffer of size %u in slot %u",
+                    (U32)buf.size, (U32)(bufPool->nbBuffers-1));
         ZSTD_pthread_mutex_unlock(&bufPool->poolMutex);
         return;
     }
     ZSTD_pthread_mutex_unlock(&bufPool->poolMutex);
     /* Reached bufferPool capacity (should not happen) */
-    DEBUGLOG(5, "buffer pool capacity reached => freeing ");
+    DEBUGLOG(5, "ZSTDMT_releaseBuffer: pool capacity reached => freeing ");
     ZSTD_free(buf.start, bufPool->cMem);
 }
 
@@ -324,11 +332,10 @@ typedef struct {
 void ZSTDMT_compressChunk(void* jobDescription)
 {
     ZSTDMT_jobDescription* const job = (ZSTDMT_jobDescription*)jobDescription;
-    ZSTD_CCtx* cctx = ZSTDMT_getCCtx(job->cctxPool);
+    ZSTD_CCtx* const cctx = ZSTDMT_getCCtx(job->cctxPool);
     const void* const src = (const char*)job->srcStart + job->dictSize;
     buffer_t dstBuff = job->dstBuff;
-    DEBUGLOG(5, "ZSTDMT_compressChunk")
-    DEBUGLOG(5, "job (first:%u) (last:%u) : dictSize %u, srcSize %u",
+    DEBUGLOG(5, "ZSTDMT_compressChunk: job (first:%u) (last:%u) : dictSize %u, srcSize %u",
                  job->firstChunk, job->lastChunk, (U32)job->dictSize, (U32)job->srcSize);
 
     if (cctx==NULL) {
@@ -343,11 +350,12 @@ void ZSTDMT_compressChunk(void* jobDescription)
             goto _endJob;
         }
         job->dstBuff = dstBuff;
+        DEBUGLOG(5, "ZSTDMT_compressChunk: allocated dstBuff of size %u", (U32)dstBuff.size);
     }
 
     if (job->cdict) {
         size_t const initError = ZSTD_compressBegin_usingCDict_advanced(cctx, job->cdict, job->params.fParams, job->fullFrameSize);
-        DEBUGLOG(4, "ZSTDMT_compressChunk, using CDict");
+        DEBUGLOG(4, "ZSTDMT_compressChunk: init using CDict");
         assert(job->firstChunk); /* should only happen for first segment */
         if (ZSTD_isError(initError)) { job->cSize = initError; goto _endJob; }
     } else {  /* srcStart points at reloaded section */
@@ -367,7 +375,7 @@ void ZSTDMT_compressChunk(void* jobDescription)
         ZSTD_invalidateRepCodes(cctx);
     }
 
-    DEBUGLOG(5, "Compressing : ");
+    DEBUGLOG(5, "Compressing into dstBuff of size %u", (U32)dstBuff.size);
     DEBUG_PRINTHEX(6, job->srcStart, 12);
     job->cSize = (job->lastChunk) ?
                  ZSTD_compressEnd     (cctx, dstBuff.start, dstBuff.size, src, job->srcSize) :
@@ -769,6 +777,7 @@ size_t ZSTDMT_initCStream_internal(
                                          dict, dictSize, cdict,
                                          singleThreadParams, pledgedSrcSize);
     }
+    DEBUGLOG(4, "multi - threading mode");
 
     if (zcs->allJobsCompleted == 0) {   /* previous compression not correctly finished */
         ZSTDMT_waitForAllJobsCompleted(zcs);
@@ -796,9 +805,10 @@ size_t ZSTDMT_initCStream_internal(
     DEBUGLOG(4, "overlap Size : %u KB", (U32)(zcs->targetDictSize>>10));
     zcs->targetSectionSize = params.jobSize ? params.jobSize : (size_t)1 << (params.cParams.windowLog + 2);
     zcs->targetSectionSize = MAX(ZSTDMT_SECTION_SIZE_MIN, zcs->targetSectionSize);
-    zcs->targetSectionSize = MAX(zcs->targetDictSize, zcs->targetSectionSize);
+    zcs->targetSectionSize = MAX(zcs->targetDictSize, zcs->targetSectionSize);   /* one job size must be at least overlap */
     DEBUGLOG(4, "Section Size : %u KB", (U32)(zcs->targetSectionSize>>10));
     zcs->inBuffSize = zcs->targetDictSize + zcs->targetSectionSize;
+    DEBUGLOG(4, "inBuff Size : %u KB", (U32)(zcs->inBuffSize>>10));
     ZSTDMT_setBufferSize(zcs->bufPool, MAX(zcs->inBuffSize, ZSTD_compressBound(zcs->targetSectionSize)) );
     zcs->inBuff.buffer = g_nullBuffer;
     zcs->dictSize = 0;
@@ -863,7 +873,8 @@ static size_t ZSTDMT_createCompressionJob(ZSTDMT_CCtx* zcs, size_t srcSize, unsi
 {
     unsigned const jobID = zcs->nextJobID & zcs->jobIDMask;
 
-    DEBUGLOG(4, "preparing job %u to compress %u bytes with %u preload ",
+    DEBUGLOG(5, "ZSTDMT_createCompressionJob");
+    DEBUGLOG(5, "preparing job %u to compress %u bytes with %u preload ",
                 zcs->nextJobID, (U32)srcSize, (U32)zcs->dictSize);
     zcs->jobs[jobID].src = zcs->inBuff.buffer;
     zcs->jobs[jobID].srcStart = zcs->inBuff.buffer.start;
@@ -994,6 +1005,7 @@ size_t ZSTDMT_compressStream_generic(ZSTDMT_CCtx* mtctx,
 {
     size_t const newJobThreshold = mtctx->dictSize + mtctx->targetSectionSize;
     unsigned forwardInputProgress = 0;
+    DEBUGLOG(5, "ZSTDMT_compressStream_generic");
     assert(output->pos <= output->size);
     assert(input->pos  <= input->size);
     if ((mtctx->frameEnded) && (endOp==ZSTD_e_continue)) {
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 6f1f6df1856..84dcc6fbff2 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -951,7 +951,7 @@ static int fuzzerTests(U32 seed, U32 nbTests, unsigned startTest, double compres
             for (n=0, cSize=0, totalTestSize=0 ; totalTestSize < maxTestSize ; n++) {
                 /* compress random chunks into randomly sized dst buffers */
                 {   size_t const randomSrcSize = FUZ_randomLength(&lseed, maxSampleLog);
-                    size_t const srcSize = MIN (maxTestSize-totalTestSize, randomSrcSize);
+                    size_t const srcSize = MIN(maxTestSize-totalTestSize, randomSrcSize);
                     size_t const srcStart = FUZ_rand(&lseed) % (srcBufferSize - srcSize);
                     size_t const randomDstSize = FUZ_randomLength(&lseed, maxSampleLog);
                     size_t const dstBuffSize = MIN(cBufferSize - cSize, randomDstSize);
@@ -1564,6 +1564,7 @@ static int fuzzerTests_newAPI(U32 seed, U32 nbTests, unsigned startTest, double
                 size_t const srcStart = FUZ_rand(&lseed) % (srcBufferSize - srcSize);
                 size_t const randomDstSize = FUZ_randomLength(&lseed, maxSampleLog+1);
                 size_t const dstBuffSize = MIN(cBufferSize - cSize, randomDstSize);
+                DISPLAYLEVEL(2, "dstBuffSize : %u \n", (U32)dstBuffSize);
                 ZSTD_EndDirective const flush = (FUZ_rand(&lseed) & 15) ? ZSTD_e_continue : ZSTD_e_flush;
                 ZSTD_inBuffer inBuff = { srcBuffer+srcStart, srcSize, 0 };
                 outBuff.size = outBuff.pos + dstBuffSize;

From 6f1dfa8adf0cc3a8038266ce8926534d08ab7d71 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 1 Nov 2017 17:01:45 -0700
Subject: [PATCH 035/145] removed line with `//` comment

this is for a different topic
(better parameter adaptation for small files + dictionary and/or custome parameters)
---
 programs/bench.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/programs/bench.c b/programs/bench.c
index 66c406abd0e..5093f110d90 100644
--- a/programs/bench.c
+++ b/programs/bench.c
@@ -286,7 +286,6 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                     U32 nbLoops = 0;
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_nbThreads, g_nbThreads);
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_compressionLevel, cLevel);
-                    //ZSTD_CCtx_setPledgedSrcSize(ctx, blockTable[0].srcSize);  /* estimation, only to help cLevel conversion before applying advanced compression parameters */
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_enableLongDistanceMatching, g_ldmFlag);
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_ldmMinMatch, g_ldmMinMatch);
                     ZSTD_CCtx_setParameter(ctx, ZSTD_p_ldmHashLog, g_ldmHashLog);

From 3b59a010397f5da74aabdf8eca7c8f9c6765b6b5 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 1 Nov 2017 17:36:31 -0700
Subject: [PATCH 036/145] fixed zstreamtest declaration after statement

---
 tests/zstreamtest.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 84dcc6fbff2..6dc9cada165 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -1564,7 +1564,6 @@ static int fuzzerTests_newAPI(U32 seed, U32 nbTests, unsigned startTest, double
                 size_t const srcStart = FUZ_rand(&lseed) % (srcBufferSize - srcSize);
                 size_t const randomDstSize = FUZ_randomLength(&lseed, maxSampleLog+1);
                 size_t const dstBuffSize = MIN(cBufferSize - cSize, randomDstSize);
-                DISPLAYLEVEL(2, "dstBuffSize : %u \n", (U32)dstBuffSize);
                 ZSTD_EndDirective const flush = (FUZ_rand(&lseed) & 15) ? ZSTD_e_continue : ZSTD_e_flush;
                 ZSTD_inBuffer inBuff = { srcBuffer+srcStart, srcSize, 0 };
                 outBuff.size = outBuff.pos + dstBuffSize;

From 8b6aecf2cb6671c943a31a069bbed1e8ffea5064 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 7 Nov 2017 15:27:06 -0800
Subject: [PATCH 037/145] moved a few structures from `zstd_internal.h` to
 `zstd_compress.h`

which is a more precise scope
---
 lib/common/zstd_internal.h     | 155 +++------------------------------
 lib/compress/zstd_compress.h   | 146 ++++++++++++++++++++++++++++++-
 lib/compress/zstd_opt.c        |  15 ++--
 lib/compress/zstdmt_compress.c |   2 +-
 4 files changed, 165 insertions(+), 153 deletions(-)

diff --git a/lib/common/zstd_internal.h b/lib/common/zstd_internal.h
index e91cd20baa7..0fb8e49da19 100644
--- a/lib/common/zstd_internal.h
+++ b/lib/common/zstd_internal.h
@@ -11,6 +11,10 @@
 #ifndef ZSTD_CCOMMON_H_MODULE
 #define ZSTD_CCOMMON_H_MODULE
 
+/* this module contains definitions which must be identical
+ * across compression, decompression and dictBuilder.
+ * It also contains a few functions useful to at least 2 of them
+ * and which benefit from being inlined */
 
 /*-*************************************
 *  Dependencies
@@ -191,17 +195,14 @@ MEM_STATIC void ZSTD_wildcopy_e(void* dst, const void* src, void* dstEnd)   /* s
 
 
 /*-*******************************************
-*  Private interfaces
+*  Private declarations
 *********************************************/
-typedef struct ZSTD_stats_s ZSTD_stats_t;
-
 typedef struct seqDef_s {
     U32 offset;
     U16 litLength;
     U16 matchLength;
 } seqDef;
 
-
 typedef struct {
     seqDef* sequencesStart;
     seqDef* sequences;
@@ -216,100 +217,8 @@ typedef struct {
     U32   repToConfirm[ZSTD_REP_NUM];
 } seqStore_t;
 
-typedef struct {
-    U32 off;
-    U32 len;
-} ZSTD_match_t;
-
-typedef struct {
-    U32 price;
-    U32 off;
-    U32 mlen;
-    U32 litlen;
-    U32 rep[ZSTD_REP_NUM];
-} ZSTD_optimal_t;
-
-typedef struct {
-    U32* litFreq;
-    U32* litLengthFreq;
-    U32* matchLengthFreq;
-    U32* offCodeFreq;
-    ZSTD_match_t* matchTable;
-    ZSTD_optimal_t* priceTable;
-
-    U32  matchLengthSum;
-    U32  matchSum;
-    U32  litLengthSum;
-    U32  litSum;
-    U32  offCodeSum;
-    U32  log2matchLengthSum;
-    U32  log2matchSum;
-    U32  log2litLengthSum;
-    U32  log2litSum;
-    U32  log2offCodeSum;
-    U32  factor;
-    U32  staticPrices;
-    U32  cachedPrice;
-    U32  cachedLitLength;
-    const BYTE* cachedLiterals;
-} optState_t;
-
-typedef struct {
-    U32 offset;
-    U32 checksum;
-} ldmEntry_t;
-
-typedef struct {
-    ldmEntry_t* hashTable;
-    BYTE* bucketOffsets;    /* Next position in bucket to insert entry */
-    U64 hashPower;          /* Used to compute the rolling hash.
-                             * Depends on ldmParams.minMatchLength */
-} ldmState_t;
-
-typedef struct {
-    U32 enableLdm;          /* 1 if enable long distance matching */
-    U32 hashLog;            /* Log size of hashTable */
-    U32 bucketSizeLog;      /* Log bucket size for collision resolution, at most 8 */
-    U32 minMatchLength;     /* Minimum match length */
-    U32 hashEveryLog;       /* Log number of entries to skip */
-} ldmParams_t;
-
-typedef struct {
-    U32 hufCTable[HUF_CTABLE_SIZE_U32(255)];
-    FSE_CTable offcodeCTable[FSE_CTABLE_SIZE_U32(OffFSELog, MaxOff)];
-    FSE_CTable matchlengthCTable[FSE_CTABLE_SIZE_U32(MLFSELog, MaxML)];
-    FSE_CTable litlengthCTable[FSE_CTABLE_SIZE_U32(LLFSELog, MaxLL)];
-    U32 workspace[HUF_WORKSPACE_SIZE_U32];
-    HUF_repeat hufCTable_repeatMode;
-    FSE_repeat offcode_repeatMode;
-    FSE_repeat matchlength_repeatMode;
-    FSE_repeat litlength_repeatMode;
-} ZSTD_entropyCTables_t;
-
-struct ZSTD_CCtx_params_s {
-    ZSTD_format_e format;
-    ZSTD_compressionParameters cParams;
-    ZSTD_frameParameters fParams;
-
-    int compressionLevel;
-    U32 forceWindow;           /* force back-references to respect limit of
-                                * 1<


 
-
Helper functions
#define ZSTD_COMPRESSBOUND(srcSize)   ((srcSize) + ((srcSize)>>8) + (((srcSize) < 128 KB) ? ((128 KB - (srcSize)) >> 11) /* margin, from 64 to 0 */ : 0))  /* this formula ensures that bound(A) + bound(B) <= bound(A+B) as long as A and B >= 128 KB */
+
Helper functions
#define ZSTD_COMPRESSBOUND(srcSize)   ((srcSize) + ((srcSize)>>8) + (((srcSize) < (128<<10)) ? (((128<<10) - (srcSize)) >> 11) /* margin, from 64 to 0 */ : 0))  /* this formula ensures that bound(A) + bound(B) <= bound(A+B) as long as A and B >= 128 KB */
 size_t      ZSTD_compressBound(size_t srcSize); /*!< maximum compressed size in worst case scenario */
 unsigned    ZSTD_isError(size_t code);          /*!< tells if a `size_t` function result is an error code */
 const char* ZSTD_getErrorName(size_t code);     /*!< provides readable string from an error code */
@@ -906,7 +906,8 @@ 
Buffer-less streaming decompression functions
typedef
 
size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, unsigned value);
 
  Set one compression parameter, selected by enum ZSTD_cParameter.
   Note : when `value` is an enum, cast it to unsigned for proper type checking.
-  @result : 0, or an error code (which can be tested with ZSTD_isError()). 
+  @result : informational value (typically, the one being set, possibly corrected),
+            or an error code (which can be tested with ZSTD_isError()). 
 


 
 
size_t ZSTD_CCtx_setPledgedSrcSize(ZSTD_CCtx* cctx, unsigned long long pledgedSrcSize);
diff --git a/lib/compress/zstd_opt.c b/lib/compress/zstd_opt.c
index b9a2cf85f58..1defcb8fe14 100644
--- a/lib/compress/zstd_opt.c
+++ b/lib/compress/zstd_opt.c
@@ -606,7 +606,8 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
                         } else {
                             if (optLevel==0) break;  /* gets ~+10% speed for about -0.01 ratio loss */
                         }
-        }   }   }   }
+            }   }   }
+        }  /* for (cur = 1; cur <= last_pos; cur++) */
 
         best_mlen = opt[last_pos].mlen;
         best_off = opt[last_pos].off;
@@ -661,7 +662,7 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
                 anchor = ip;
         }   }
         ZSTD_setLog2Prices(optStatePtr);
-    }   /* for (cur=0; cur < last_pos; ) */
+    }   while (ip < ilimit)
 
     /* Save reps for next block */
     { int i; for (i=0; irepToConfirm[i] = rep[i]; }

From f8d5c478afe143ffff63ac67c3ade593974efa27 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 21 Nov 2017 10:36:14 -0800
Subject: [PATCH 075/145] fixed comment, reported by @gyscos

---
 lib/compress/zstd_opt.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/compress/zstd_opt.c b/lib/compress/zstd_opt.c
index 1defcb8fe14..66a7da19c8e 100644
--- a/lib/compress/zstd_opt.c
+++ b/lib/compress/zstd_opt.c
@@ -662,7 +662,7 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
                 anchor = ip;
         }   }
         ZSTD_setLog2Prices(optStatePtr);
-    }   while (ip < ilimit)
+    }   /* while (ip < ilimit) */
 
     /* Save reps for next block */
     { int i; for (i=0; irepToConfirm[i] = rep[i]; }

From 31a191b17895d115b35fa4c254a9b99b67bbe58a Mon Sep 17 00:00:00 2001
From: Scott Baker 
Date: Wed, 22 Nov 2017 20:29:46 -0800
Subject: [PATCH 076/145] Include information about the benchmark
 output/methodology

Addresses #930
---
 programs/zstd.1.md | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/programs/zstd.1.md b/programs/zstd.1.md
index eea68548982..9e4c5d30555 100644
--- a/programs/zstd.1.md
+++ b/programs/zstd.1.md
@@ -258,6 +258,9 @@ BENCHMARK
 * `--priority=rt`:
     set process priority to real-time
 
+**Output Format:** CompressionLevel#Filename : IntputSize -> OutputSize (CompressionRatio), CompressionSpeed, DecompressionSpeed
+
+**Methodology:** For both compression and decompression speed, the entire input is compressed/decompressed in-memory to measure speed. A run lasts at least 1 sec, so when files are small, they are compressed/decompressed several times per run, in order to improve measurement accuracy.
 
 ADVANCED COMPRESSION OPTIONS
 ----------------------------

From c857ee850ad6bd830b44fcf54692478e1206da53 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 24 Nov 2017 16:44:28 -0800
Subject: [PATCH 077/145] minor update

---
 programs/platform.h | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/programs/platform.h b/programs/platform.h
index a4d7850fd71..81dee23f917 100644
--- a/programs/platform.h
+++ b/programs/platform.h
@@ -21,10 +21,10 @@ extern "C" {
 *  Compiler Options
 ****************************************/
 #if defined(_MSC_VER)
-#  define _CRT_SECURE_NO_WARNINGS   /* Disable Visual Studio warning messages for fopen, strncpy, strerror */
-#  define _CRT_SECURE_NO_DEPRECATE  /* VS2005 - must be declared before  and  */
-#  if (_MSC_VER <= 1800)            /* (1800 = Visual Studio 2013) */
-#    define snprintf sprintf_s      /* snprintf unsupported by Visual <= 2013 */
+#  define _CRT_SECURE_NO_WARNINGS    /* Disable Visual Studio warning messages for fopen, strncpy, strerror */
+#  if (_MSC_VER <= 1800)             /* 1800 == Visual Studio 2013 */
+#    define _CRT_SECURE_NO_DEPRECATE /* VS2005 - must be declared before  and  */
+#    define snprintf sprintf_s       /* snprintf unsupported by Visual <= 2013 */
 #  endif
 #endif
 
@@ -117,7 +117,7 @@ static __inline int IS_CONSOLE(FILE* stdStream)
 
 
 /******************************
-*  OS-specific Includes
+*  OS-specific IO behaviors
 ******************************/
 #if defined(MSDOS) || defined(OS2) || defined(WIN32) || defined(_WIN32)
 #  include    /* _O_BINARY */
@@ -125,7 +125,7 @@ static __inline int IS_CONSOLE(FILE* stdStream)
 #  if !defined(__DJGPP__)
 #    include  /* DeviceIoControl, HANDLE, FSCTL_SET_SPARSE */
 #    include  /* FSCTL_SET_SPARSE */
-#    define SET_BINARY_MODE(file) { int unused=_setmode(_fileno(file), _O_BINARY); (void)unused; }
+#    define SET_BINARY_MODE(file) { int const unused=_setmode(_fileno(file), _O_BINARY); (void)unused; }
 #    define SET_SPARSE_FILE_MODE(file) { DWORD dw; DeviceIoControl((HANDLE) _get_osfhandle(_fileno(file)), FSCTL_SET_SPARSE, 0, 0, 0, 0, &dw, 0); }
 #  else
 #    define SET_BINARY_MODE(file) setmode(fileno(file), O_BINARY)

From 2fd765498a750f009404feedd26f8e8f60f83a1e Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 24 Nov 2017 17:20:54 -0800
Subject: [PATCH 078/145] updated man page

following patch #931 by @scottchiefbaker
---
 programs/zstd.1 | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/programs/zstd.1 b/programs/zstd.1
index 8187c740350..eb53e2c3309 100644
--- a/programs/zstd.1
+++ b/programs/zstd.1
@@ -1,5 +1,5 @@
 .
-.TH "ZSTD" "1" "September 2017" "zstd 1.3.1" "User Commands"
+.TH "ZSTD" "1" "November 2017" "zstd 1.3.2" "User Commands"
 .
 .SH "NAME"
 \fBzstd\fR \- zstd, zstdmt, unzstd, zstdcat \- Compress or decompress \.zst files
@@ -60,11 +60,11 @@ In most places where an integer argument is expected, an optional suffix is supp
 .
 .TP
 \fBKiB\fR
-Multiply the integer by 1,024 (2\e \fBKi\fR, \fBK\fR, and \fBKB\fR are accepted as synonyms for \fBKiB\fR\.
+Multiply the integer by 1,024 (2^10)\. \fBKi\fR, \fBK\fR, and \fBKB\fR are accepted as synonyms for \fBKiB\fR\.
 .
 .TP
 \fBMiB\fR
-Multiply the integer by 1,048,576 (2\e \fBMi\fR, \fBM\fR, and \fBMB\fR are accepted as synonyms for \fBMiB\fR\.
+Multiply the integer by 1,048,576 (2^20)\. \fBMi\fR, \fBM\fR, and \fBMB\fR are accepted as synonyms for \fBMiB\fR\.
 .
 .SS "Operation mode"
 If multiple operation mode options are given, the last one takes effect\.
@@ -261,6 +261,12 @@ cut file(s) into independent blocks of size # (default: no block)
 \fB\-\-priority=rt\fR
 set process priority to real\-time
 .
+.P
+\fBOutput Format:\fR CompressionLevel#Filename : IntputSize \-> OutputSize (CompressionRatio), CompressionSpeed, DecompressionSpeed
+.
+.P
+\fBMethodology:\fR For both compression and decompression speed, the entire input is compressed/decompressed in\-memory to measure speed\. A run lasts at least 1 sec, so when files are small, they are compressed/decompressed several times per run, in order to improve measurement accuracy\.
+.
 .SH "ADVANCED COMPRESSION OPTIONS"
 .
 .SS "\-\-zstd[=options]:"

From bd88f633acda0de4f9eaef6b0914c3b569a34724 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Mon, 27 Nov 2017 12:15:23 -0800
Subject: [PATCH 079/145] zstreamtest : in `-T#s`, s considered a suffix
 meaning "seconds"

avoid unintentionnally triggering `seedset`,
so that seed gets automatically determined when not set.
---
 doc/zstd_manual.html | 62 +++++++++++++++++++++++---------------------
 lib/zstd.h           | 17 +++++++-----
 tests/zstreamtest.c  | 31 +++++++++++-----------
 3 files changed, 58 insertions(+), 52 deletions(-)

diff --git a/doc/zstd_manual.html b/doc/zstd_manual.html
index e50f4666120..872dbf6377d 100644
--- a/doc/zstd_manual.html
+++ b/doc/zstd_manual.html
@@ -19,16 +19,17 @@ 
zstd 1.3.2 Manual

 
  • Streaming decompression - HowTo
    • 
 
  • START OF ADVANCED AND EXPERIMENTAL FUNCTIONS
    • 
 
  • Advanced types
    • 
-
  • Frame size functions
    • 
-
  • Context memory usage
    • 
-
  • Advanced compression functions
    • 
-
  • Advanced decompression functions
    • 
-
  • Advanced streaming functions
    • 
-
  • Buffer-less and synchronous inner streaming functions
    • 
-
  • Buffer-less streaming compression (synchronous mode)
    • 
-
  • Buffer-less streaming decompression (synchronous mode)
    • 
-
  • New advanced API (experimental)
    • 
-
  • Block level API
    • 
+
  • Custom memory allocation functions
    • 
+
  • Frame size functions
    • 
+
  • Context memory usage
    • 
+
  • Advanced compression functions
    • 
+
  • Advanced decompression functions
    • 
+
  • Advanced streaming functions
    • 
+
  • Buffer-less and synchronous inner streaming functions
    • 
+
  • Buffer-less streaming compression (synchronous mode)
    • 
+
  • Buffer-less streaming decompression (synchronous mode)
    • 
+
  • New advanced API (experimental)
    • 
+
  • Block level API
    • 
 
 
    
 
    Introduction
    @@ -346,17 +347,15 @@ 
    Streaming decompression functions
    size_t ZSTD_initDSt
     ZSTD_frameParameters fParams;
 } ZSTD_parameters;
 

    
-
    Custom memory allocation functions
    typedef void* (*ZSTD_allocFunction) (void* opaque, size_t size);
-typedef void  (*ZSTD_freeFunction) (void* opaque, void* address);
-typedef struct { ZSTD_allocFunction customAlloc; ZSTD_freeFunction customFree; void* opaque; } ZSTD_customMem;
-/* use this constant to defer to stdlib's functions */
-static const ZSTD_customMem ZSTD_defaultCMem = { NULL, NULL, NULL };
-

    
-
    Frame size functions
    
+
    Custom memory allocation functions
    
+
+
    typedef struct { ZSTD_allocFunction customAlloc; ZSTD_freeFunction customFree; void* opaque; } ZSTD_customMem;
+

    
+
    Frame size functions
    
 
 
    size_t ZSTD_findFrameCompressedSize(const void* src, size_t srcSize);
 
      `src` should point to the start of a ZSTD encoded frame or skippable frame
-  `srcSize` must be at least as large as the frame
+  `srcSize` must be >= first frame size
   @return : the compressed size of the first frame starting at `src`,
             suitable to pass to `ZSTD_decompress` or similar,
             or an error code if input is invalid 
@@ -391,7 +390,7 @@ 
    Custom memory allocation functions
    typedef void* (*ZS
    @return : size of the Frame Header 
 

    
 
-
    Context memory usage
    
+
    Context memory usage
    
 
 
    size_t ZSTD_sizeof_CCtx(const ZSTD_CCtx* cctx);
 size_t ZSTD_sizeof_DCtx(const ZSTD_DCtx* dctx);
@@ -450,7 +449,7 @@ 
    Custom memory allocation functions
    typedef void* (*ZS
  
 

    
 
-
    Advanced compression functions
    
+
    Advanced compression functions
    
 
 
    ZSTD_CCtx* ZSTD_createCCtx_advanced(ZSTD_customMem customMem);
 
      Create a ZSTD compression context using external alloc and free functions 
@@ -462,7 +461,8 @@ 
    Custom memory allocation functions
    typedef void* (*ZS
              It must outlive context usage.
   workspaceSize: Use ZSTD_estimateCCtxSize() or ZSTD_estimateCStreamSize()
                  to determine how large workspace must be to support scenario.
- @return : pointer to ZSTD_CCtx*, or NULL if error (size too small)
+ @return : pointer to ZSTD_CCtx* (same address as workspace, but different type),
+           or NULL if error (typically size too small)
   Note : zstd will never resize nor malloc() when using a static cctx.
          If it needs more memory than available, it will simply error out.
   Note 2 : there is no corresponding "free" function.
@@ -505,7 +505,8 @@ 
    Custom memory allocation functions
    typedef void* (*ZS
                  to determine how large workspace must be.
   cParams : use ZSTD_getCParams() to transform a compression level
             into its relevants cParams.
- @return : pointer to ZSTD_CDict*, or NULL if error (size too small)
+ @return : pointer to ZSTD_CDict* (same address as workspace, but different type),
+           or NULL if error (typically, size too small).
   Note : there is no corresponding "free" function.
          Since workspace was allocated externally, it must be freed externally.
  
@@ -545,7 +546,7 @@ 
    Custom memory allocation functions
    typedef void* (*ZS
 
       Same as ZSTD_compress_usingCDict(), with fine-tune control over frame parameters 
 

    
 
-
    Advanced decompression functions
    
+
    Advanced decompression functions
    
 
 
    unsigned ZSTD_isFrame(const void* buffer, size_t size);
 
      Tells if the content of `buffer` starts with a valid Frame Identifier.
@@ -564,7 +565,8 @@ 
    Custom memory allocation functions
    typedef void* (*ZS
              It must outlive context usage.
   workspaceSize: Use ZSTD_estimateDCtxSize() or ZSTD_estimateDStreamSize()
                  to determine how large workspace must be to support scenario.
- @return : pointer to ZSTD_DCtx*, or NULL if error (size too small)
+ @return : pointer to ZSTD_DCtx* (same address as workspace, but different type),
+           or NULL if error (typically size too small)
   Note : zstd will never resize nor malloc() when using a static dctx.
          If it needs more memory than available, it will simply error out.
   Note 2 : static dctx is incompatible with legacy support
@@ -627,7 +629,7 @@ 
    Custom memory allocation functions
    typedef void* (*ZS
   When identifying the exact failure cause, it's possible to use ZSTD_getFrameHeader(), which will provide a more precise error code. 
 

    
 
-
    Advanced streaming functions
    
+
    Advanced streaming functions
    
 
 
    Advanced Streaming compression functions
    ZSTD_CStream* ZSTD_createCStream_advanced(ZSTD_customMem customMem);
 ZSTD_CStream* ZSTD_initStaticCStream(void* workspace, size_t workspaceSize);    /**< same as ZSTD_initStaticCCtx() */
@@ -657,14 +659,14 @@ 
    Advanced Streaming decompression functions
    ZSTD_DStre
 size_t ZSTD_initDStream_usingDDict(ZSTD_DStream* zds, const ZSTD_DDict* ddict);  /**< note : ddict is referenced, it must outlive decompression session */
 size_t ZSTD_resetDStream(ZSTD_DStream* zds);  /**< re-use decompression parameters from previous init; saves dictionary loading */
 

    
-
    Buffer-less and synchronous inner streaming functions
    +
    Buffer-less and synchronous inner streaming functions
       This is an advanced API, giving full control over buffer management, for users which need direct control over memory.
   But it's also a complex one, with several restrictions, documented below.
   Prefer normal streaming API for an easier experience.
  
 
    
 
-
    Buffer-less streaming compression (synchronous mode)
    +
    Buffer-less streaming compression (synchronous mode)
       A ZSTD_CCtx object is required to track streaming operations.
   Use ZSTD_createCCtx() / ZSTD_freeCCtx() to manage resource.
   ZSTD_CCtx object can be re-used multiple times within successive compression operations.
@@ -700,7 +702,7 @@ 
    Buffer-less streaming compression functions
    size_t ZS
 size_t ZSTD_compressBegin_usingCDict_advanced(ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict, ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize);   /* compression parameters are already set within cdict. pledgedSrcSize must be correct. If srcSize is not known, use macro ZSTD_CONTENTSIZE_UNKNOWN */
 size_t ZSTD_copyCCtx(ZSTD_CCtx* cctx, const ZSTD_CCtx* preparedCCtx, unsigned long long pledgedSrcSize); /**<  note: if pledgedSrcSize is not known, use ZSTD_CONTENTSIZE_UNKNOWN */
 

    
-
    Buffer-less streaming decompression (synchronous mode)
    +
    Buffer-less streaming decompression (synchronous mode)
       A ZSTD_DCtx object is required to track streaming operations.
   Use ZSTD_createDCtx() / ZSTD_freeDCtx() to manage it.
   A ZSTD_DCtx object can be re-used multiple times.
@@ -786,7 +788,7 @@ 
    Buffer-less streaming decompression functions
    typedef
 

    
 
    typedef enum { ZSTDnit_frameHeader, ZSTDnit_blockHeader, ZSTDnit_block, ZSTDnit_lastBlock, ZSTDnit_checksum, ZSTDnit_skippableFrame } ZSTD_nextInputType_e;
 

    
-
    New advanced API (experimental)
    
+
    New advanced API (experimental)
    
 
 
    typedef enum {
     /* Question : should we have a format ZSTD_f_auto ?
@@ -1169,7 +1171,7 @@ 
    Advanced parameters for decompression API
    <
  
 

    
 
-
    Block level API
    
+
    Block level API
    
 
 
        Frame metadata cost is typically ~18 bytes, which can be non-negligible for very small blocks (< 100 bytes).
     User will have to take in charge required information to regenerate data, such as compressed and content sizes.
diff --git a/lib/zstd.h b/lib/zstd.h
index 919d1e2e0c8..fe37cec8bdf 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -432,12 +432,12 @@ typedef struct {
 
 typedef struct ZSTD_CCtx_params_s ZSTD_CCtx_params;
 
-/*= Custom memory allocation functions */
+/*--- Custom memory allocation functions ---*/
 typedef void* (*ZSTD_allocFunction) (void* opaque, size_t size);
 typedef void  (*ZSTD_freeFunction) (void* opaque, void* address);
 typedef struct { ZSTD_allocFunction customAlloc; ZSTD_freeFunction customFree; void* opaque; } ZSTD_customMem;
 /* use this constant to defer to stdlib's functions */
-static const ZSTD_customMem ZSTD_defaultCMem = { NULL, NULL, NULL };
+static ZSTD_customMem const ZSTD_defaultCMem = { NULL, NULL, NULL };
 
 
 /***************************************
@@ -446,7 +446,7 @@ static const ZSTD_customMem ZSTD_defaultCMem = { NULL, NULL, NULL };
 
 /*! ZSTD_findFrameCompressedSize() :
  *  `src` should point to the start of a ZSTD encoded frame or skippable frame
- *  `srcSize` must be at least as large as the frame
+ *  `srcSize` must be >= first frame size
  *  @return : the compressed size of the first frame starting at `src`,
  *            suitable to pass to `ZSTD_decompress` or similar,
  *            or an error code if input is invalid */
@@ -557,7 +557,8 @@ ZSTDLIB_API ZSTD_CCtx* ZSTD_createCCtx_advanced(ZSTD_customMem customMem);
  *             It must outlive context usage.
  *  workspaceSize: Use ZSTD_estimateCCtxSize() or ZSTD_estimateCStreamSize()
  *                 to determine how large workspace must be to support scenario.
- * @return : pointer to ZSTD_CCtx*, or NULL if error (size too small)
+ * @return : pointer to ZSTD_CCtx* (same address as workspace, but different type),
+ *           or NULL if error (typically size too small)
  *  Note : zstd will never resize nor malloc() when using a static cctx.
  *         If it needs more memory than available, it will simply error out.
  *  Note 2 : there is no corresponding "free" function.
@@ -587,7 +588,7 @@ ZSTDLIB_API ZSTD_CDict* ZSTD_createCDict_advanced(const void* dict, size_t dictS
                                                   ZSTD_compressionParameters cParams,
                                                   ZSTD_customMem customMem);
 
-/*! ZSTD_initStaticCDict_advanced() :
+/*! ZSTD_initStaticCDict() :
  *  Generate a digested dictionary in provided memory area.
  *  workspace: The memory area to emplace the dictionary into.
  *             Provided pointer must 8-bytes aligned.
@@ -596,7 +597,8 @@ ZSTDLIB_API ZSTD_CDict* ZSTD_createCDict_advanced(const void* dict, size_t dictS
  *                 to determine how large workspace must be.
  *  cParams : use ZSTD_getCParams() to transform a compression level
  *            into its relevants cParams.
- * @return : pointer to ZSTD_CDict*, or NULL if error (size too small)
+ * @return : pointer to ZSTD_CDict* (same address as workspace, but different type),
+ *           or NULL if error (typically, size too small).
  *  Note : there is no corresponding "free" function.
  *         Since workspace was allocated externally, it must be freed externally.
  */
@@ -660,7 +662,8 @@ ZSTDLIB_API ZSTD_DCtx* ZSTD_createDCtx_advanced(ZSTD_customMem customMem);
  *             It must outlive context usage.
  *  workspaceSize: Use ZSTD_estimateDCtxSize() or ZSTD_estimateDStreamSize()
  *                 to determine how large workspace must be to support scenario.
- * @return : pointer to ZSTD_DCtx*, or NULL if error (size too small)
+ * @return : pointer to ZSTD_DCtx* (same address as workspace, but different type),
+ *           or NULL if error (typically size too small)
  *  Note : zstd will never resize nor malloc() when using a static dctx.
  *         If it needs more memory than available, it will simply error out.
  *  Note 2 : static dctx is incompatible with legacy support
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 4cdd6235bf5..b8c437a4197 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -209,7 +209,7 @@ static size_t SEQ_generateRoundTrip(ZSTD_CCtx* cctx, ZSTD_DCtx* dctx,
     return 0;
 }
 
-static int basicUnitTests(U32 seed, double compressibility, ZSTD_customMem customMem)
+static int basicUnitTests(U32 seed, double compressibility)
 {
     size_t const CNBufferSize = COMPRESSIBLE_NOISE_LENGTH;
     void* CNBuffer = malloc(CNBufferSize);
@@ -221,8 +221,8 @@ static int basicUnitTests(U32 seed, double compressibility, ZSTD_customMem custo
     size_t cSize;
     int testResult = 0;
     U32 testNb = 1;
-    ZSTD_CStream* zc = ZSTD_createCStream_advanced(customMem);
-    ZSTD_DStream* zd = ZSTD_createDStream_advanced(customMem);
+    ZSTD_CStream* zc = ZSTD_createCStream();
+    ZSTD_DStream* zd = ZSTD_createDStream();
     ZSTDMT_CCtx* mtctx = ZSTDMT_createCCtx(2);
 
     ZSTD_inBuffer  inBuff, inBuff2;
@@ -431,7 +431,7 @@ static int basicUnitTests(U32 seed, double compressibility, ZSTD_customMem custo
     /* Complex context re-use scenario */
     DISPLAYLEVEL(3, "test%3i : context re-use : ", testNb++);
     ZSTD_freeCStream(zc);
-    zc = ZSTD_createCStream_advanced(customMem);
+    zc = ZSTD_createCStream();
     if (zc==NULL) goto _output_error;   /* memory allocation issue */
     /* use 1 */
     {   size_t const inSize = 513;
@@ -548,7 +548,7 @@ static int basicUnitTests(U32 seed, double compressibility, ZSTD_customMem custo
     DISPLAYLEVEL(3, "test%3i : ZSTD_initCStream_usingCDict_advanced with masked dictID : ", testNb++);
     {   ZSTD_compressionParameters const cParams = ZSTD_getCParams(1, CNBufferSize, dictionary.filled);
         ZSTD_frameParameters const fParams = { 1 /* contentSize */, 1 /* checksum */, 1 /* noDictID */};
-        ZSTD_CDict* const cdict = ZSTD_createCDict_advanced(dictionary.start, dictionary.filled, ZSTD_dlm_byRef, ZSTD_dm_auto, cParams, customMem);
+        ZSTD_CDict* const cdict = ZSTD_createCDict_advanced(dictionary.start, dictionary.filled, ZSTD_dlm_byRef, ZSTD_dm_auto, cParams, ZSTD_defaultCMem);
         size_t const initError = ZSTD_initCStream_usingCDict_advanced(zc, cdict, fParams, CNBufferSize);
         if (ZSTD_isError(initError)) goto _output_error;
         cSize = 0;
@@ -1702,24 +1702,23 @@ typedef enum { simple_api, mt_api, advanced_api } e_api;
 
 int main(int argc, const char** argv)
 {
-    U32 seed=0;
-    int seedset=0;
-    int argNb;
+    U32 seed = 0;
+    int seedset = 0;
     int nbTests = nbTestsDefault;
     int testNb = 0;
     int proba = FUZ_COMPRESSIBILITY_DEFAULT;
-    int result=0;
+    int result = 0;
     int mainPause = 0;
     int bigTests = (sizeof(size_t) == 8);
     e_api selected_api = simple_api;
     const char* const programName = argv[0];
-    ZSTD_customMem const customNULL = ZSTD_defaultCMem;
     U32 useOpaqueAPI = 0;
+    int argNb;
 
     /* Check command line */
     for(argNb=1; argNb='0') && (*argument<='9')) {
                         seed *= 10;
                         seed += *argument - '0';
@@ -1827,7 +1828,7 @@ int main(int argc, const char** argv)
     if (nbTests<=0) nbTests=1;
 
     if (testNb==0) {
-        result = basicUnitTests(0, ((double)proba) / 100, customNULL);  /* constant seed for predictability */
+        result = basicUnitTests(0, ((double)proba) / 100);  /* constant seed for predictability */
     }
 
     if (!result) {

From 743b23878eaa5c71c019a70327d52d95c6016b62 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Mon, 27 Nov 2017 13:47:35 -0800
Subject: [PATCH 080/145] install: changed variable MANDIR into MAN1DIR

MANDIR still exists, and is now the parent of MAN1DIR
---
 programs/Makefile | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/programs/Makefile b/programs/Makefile
index e1c5f50b6c1..37788db3918 100644
--- a/programs/Makefile
+++ b/programs/Makefile
@@ -65,6 +65,7 @@ endif
 else
 endif
 
+# Sort files in alphabetical order for reproducible builds
 ZSTDLIB_FILES := $(sort $(wildcard $(ZSTD_FILES)) $(wildcard $(ZSTDLEGACY_FILES)) $(wildcard $(ZDICT_FILES)))
 
 # Define *.exe as extension for Windows systems
@@ -262,9 +263,10 @@ mandir      ?= $(datarootdir)/man
 man1dir     ?= $(mandir)/man1
 
 ifneq (,$(filter $(shell uname),OpenBSD FreeBSD NetBSD DragonFly SunOS))
-MANDIR  ?= $(PREFIX)/man/man1
+MANDIR  ?= $(PREFIX)/man
+MAN1DIR ?= $(MANDIR)/man1
 else
-MANDIR  ?= $(man1dir)
+MAN1DIR ?= $(man1dir)
 endif
 
 ifneq (,$(filter $(shell uname),SunOS))
@@ -281,7 +283,7 @@ INSTALL_MAN     ?= $(INSTALL_DATA)
 .PHONY: install
 install: zstd
 	@echo Installing binaries
-	@$(INSTALL) -d -m 755 $(DESTDIR)$(BINDIR)/ $(DESTDIR)$(MANDIR)/
+	@$(INSTALL) -d -m 755 $(DESTDIR)$(BINDIR)/ $(DESTDIR)$(MAN1DIR)/
 	@$(INSTALL_PROGRAM) zstd $(DESTDIR)$(BINDIR)/zstd
 	@ln -sf zstd $(DESTDIR)$(BINDIR)/zstdcat
 	@ln -sf zstd $(DESTDIR)$(BINDIR)/unzstd
@@ -289,9 +291,9 @@ install: zstd
 	@$(INSTALL_SCRIPT) zstdless $(DESTDIR)$(BINDIR)/zstdless
 	@$(INSTALL_SCRIPT) zstdgrep $(DESTDIR)$(BINDIR)/zstdgrep
 	@echo Installing man pages
-	@$(INSTALL_MAN) zstd.1 $(DESTDIR)$(MANDIR)/zstd.1
-	@ln -sf zstd.1 $(DESTDIR)$(MANDIR)/zstdcat.1
-	@ln -sf zstd.1 $(DESTDIR)$(MANDIR)/unzstd.1
+	@$(INSTALL_MAN) zstd.1 $(DESTDIR)$(MAN1DIR)/zstd.1
+	@ln -sf zstd.1 $(DESTDIR)$(MAN1DIR)/zstdcat.1
+	@ln -sf zstd.1 $(DESTDIR)$(MAN1DIR)/unzstd.1
 	@echo zstd installation completed
 
 .PHONY: uninstall
@@ -301,9 +303,9 @@ uninstall:
 	@$(RM) $(DESTDIR)$(BINDIR)/zstdcat
 	@$(RM) $(DESTDIR)$(BINDIR)/unzstd
 	@$(RM) $(DESTDIR)$(BINDIR)/zstd
-	@$(RM) $(DESTDIR)$(MANDIR)/zstdcat.1
-	@$(RM) $(DESTDIR)$(MANDIR)/unzstd.1
-	@$(RM) $(DESTDIR)$(MANDIR)/zstd.1
+	@$(RM) $(DESTDIR)$(MAN1DIR)/zstdcat.1
+	@$(RM) $(DESTDIR)$(MAN1DIR)/unzstd.1
+	@$(RM) $(DESTDIR)$(MAN1DIR)/zstd.1
 	@echo zstd programs successfully uninstalled
 
 endif

From e9d1987fd7bbeb669fe23a6f396159b4579c9d21 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Mon, 27 Nov 2017 17:01:51 -0800
Subject: [PATCH 081/145] btopt: minor speed optimization

matchPrice is always right at beginning
---
 lib/compress/zstd_opt.c | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/lib/compress/zstd_opt.c b/lib/compress/zstd_opt.c
index 66a7da19c8e..459fe65713c 100644
--- a/lib/compress/zstd_opt.c
+++ b/lib/compress/zstd_opt.c
@@ -523,14 +523,11 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
                     U32 const offset = matches[matchNb].off;
                     U32 const end = matches[matchNb].len;
                     repcodes_t const repHistory = ZSTD_updateRep(rep, offset, ll0);
-                    while (pos <= end) {
+                    for ( ; pos <= end ; pos++) {
                         U32 const matchPrice = ZSTD_getPrice(optStatePtr, litlen, anchor, offset, pos, optLevel);
-                        if (pos > last_pos || matchPrice < opt[pos].price) {
-                            DEBUGLOG(7, "rPos:%u => set initial price : %u",
-                                        pos, matchPrice);
-                            SET_PRICE(pos, pos, offset, litlen, matchPrice, repHistory);   /* note : macro modifies last_pos */
-                        }
-                        pos++;
+                        DEBUGLOG(7, "rPos:%u => set initial price : %u",
+                                    pos, matchPrice);
+                        SET_PRICE(pos, pos, offset, litlen, matchPrice, repHistory);   /* note : macro modifies last_pos */
             }   }   }
         }
 

From eee87cd6f20522d34db862ec9ef000572c26164a Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Mon, 27 Nov 2017 17:13:59 -0800
Subject: [PATCH 082/145] btopt: minor refactor : removed one SET_PRICE() macro
 invocation

direct assignment makes operation cleaner.
Also allows some (very minor) optimization (non-measurable)
---
 lib/compress/zstd_opt.c | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

diff --git a/lib/compress/zstd_opt.c b/lib/compress/zstd_opt.c
index 459fe65713c..c1ebea4d99f 100644
--- a/lib/compress/zstd_opt.c
+++ b/lib/compress/zstd_opt.c
@@ -517,18 +517,28 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
             }   }
 
             /* set prices for first matches starting position == 0 */
-            {   U32 pos = minMatch;
+            {   U32 pos;
                 U32 matchNb;
+                for (pos = 0; pos < minMatch; pos++) {
+                    opt[pos].mlen = 1;
+                    opt[pos].price = ZSTD_MAX_PRICE;
+                }
                 for (matchNb = 0; matchNb < nbMatches; matchNb++) {
                     U32 const offset = matches[matchNb].off;
                     U32 const end = matches[matchNb].len;
                     repcodes_t const repHistory = ZSTD_updateRep(rep, offset, ll0);
-                    for ( ; pos <= end ; pos++) {
+                    for ( ; pos <= end ; pos++ ) {
                         U32 const matchPrice = ZSTD_getPrice(optStatePtr, litlen, anchor, offset, pos, optLevel);
                         DEBUGLOG(7, "rPos:%u => set initial price : %u",
                                     pos, matchPrice);
-                        SET_PRICE(pos, pos, offset, litlen, matchPrice, repHistory);   /* note : macro modifies last_pos */
-            }   }   }
+                        opt[pos].mlen = pos;
+                        opt[pos].off = offset;
+                        opt[pos].litlen = litlen;
+                        opt[pos].price = matchPrice;
+                        memcpy(opt[pos].rep, &repHistory, sizeof(repHistory));
+                }   }
+                last_pos = pos-1;
+            }
         }
 
         /* check further positions */

From 62c746dcf90c9f69f404df08533795d72083361d Mon Sep 17 00:00:00 2001
From: "W. Felix Handte" 
Date: Tue, 28 Nov 2017 12:06:48 -0500
Subject: [PATCH 083/145] Add Test on LZ4 Format Input Buffer Overrun

---
 tests/Makefile | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/tests/Makefile b/tests/Makefile
index 11d55a75d4d..f944fd5a867 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -374,9 +374,9 @@ test-pool: poolTests
 test-lz4: ZSTD = LD_LIBRARY_PATH=/usr/local/lib $(PRGDIR)/zstd
 test-lz4: ZSTD_LZ4 = LD_LIBRARY_PATH=/usr/local/lib ./lz4
 test-lz4: ZSTD_UNLZ4 = LD_LIBRARY_PATH=/usr/local/lib ./unlz4
-test-lz4: zstd decodecorpus
-	ln -s $(PRGDIR)/zstd lz4
-	ln -s $(PRGDIR)/zstd unlz4
+test-lz4: zstd decodecorpus datagen
+	[ -f lz4 ] || ln -s $(PRGDIR)/zstd lz4
+	[ -f unlz4 ] || ln -s $(PRGDIR)/zstd unlz4
 
 	./decodecorpus -ptmp
 	# lz4 -> zstd
@@ -402,6 +402,8 @@ test-lz4: zstd decodecorpus
 	$(ZSTD) -d | \
 	cmp - tmp
 
+	./datagen -g384KB | $(ZSTD) --format=lz4 | $(ZSTD) -d > /dev/null
+
 	rm tmp lz4 unlz4
 
 endif

From baff9dd15eb65eaf54f0dee9ee5fadc9ce16d6ab Mon Sep 17 00:00:00 2001
From: "W. Felix Handte" 
Date: Tue, 17 Oct 2017 01:19:29 -0400
Subject: [PATCH 084/145] Fix LZ4 Compression Buffer Overflow

Fixes issue where, when `zstd --format=lz4` is fed an input larger than 128KB,
the read overruns the input buffer. This changes Zstd to use LZ4 with chained
64KB blocks. This is technically a breaking change in that some third party
LZ4 implementations may not support linked blocks. However, progress should not
be allowed to be stopped by such petty concerns as backwards compatibility!
---
 programs/fileio.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/programs/fileio.c b/programs/fileio.c
index 8bee6086cc1..eb004f91604 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -616,11 +616,16 @@ static unsigned long long FIO_compressLzmaFrame(cRess_t* ress,
 #endif
 
 #ifdef ZSTD_LZ4COMPRESS
+#if LZ4_VERSION_NUMBER <= 10600
+#define LZ4F_blockLinked blockLinked
+#define LZ4F_max64KB max64KB
+#endif
 static int FIO_LZ4_GetBlockSize_FromBlockId (int id) { return (1 << (8 + (2 * id))); }
 static unsigned long long FIO_compressLz4Frame(cRess_t* ress,
                             const char* srcFileName, U64 const srcFileSize,
                             int compressionLevel, U64* readsize)
 {
+    const size_t blockSize = FIO_LZ4_GetBlockSize_FromBlockId(LZ4F_max64KB);
     unsigned long long inFileSize = 0, outFileSize = 0;
 
     LZ4F_preferences_t prefs;
@@ -632,21 +637,19 @@ static unsigned long long FIO_compressLz4Frame(cRess_t* ress,
 
     memset(&prefs, 0, sizeof(prefs));
 
-#if LZ4_VERSION_NUMBER <= 10600
-#define LZ4F_blockIndependent blockIndependent
-#define LZ4F_max4MB max4MB
-#endif
+    assert(blockSize <= ress->srcBufferSize);
+    assert(LZ4F_compressBound(blockSize) <= ress->dstBufferSize);
 
     prefs.autoFlush = 1;
     prefs.compressionLevel = compressionLevel;
-    prefs.frameInfo.blockMode = LZ4F_blockIndependent; /* stick to defaults for lz4 cli */
-    prefs.frameInfo.blockSizeID = LZ4F_max4MB;
+    prefs.frameInfo.blockMode = LZ4F_blockLinked;
+    prefs.frameInfo.blockSizeID = LZ4F_max64KB;
     prefs.frameInfo.contentChecksumFlag = (contentChecksum_t)g_checksumFlag;
 #if LZ4_VERSION_NUMBER >= 10600
     prefs.frameInfo.contentSize = (srcFileSize==UTIL_FILESIZE_UNKNOWN) ? 0 : srcFileSize;
 #endif
 
-    {   size_t blockSize = FIO_LZ4_GetBlockSize_FromBlockId(LZ4F_max4MB);
+    {
         size_t readSize;
         size_t headerSize = LZ4F_compressBegin(ctx, ress->dstBuffer, ress->dstBufferSize, &prefs);
         if (LZ4F_isError(headerSize))

From 03f30d9dcbb8c20fc239d4a10b5df377ea8dc532 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 28 Nov 2017 12:14:46 -0800
Subject: [PATCH 085/145] separate rawLiterals, fullLiterals and match costs

removed one SET_PRICE() macro invocation
---
 lib/compress/zstd_opt.c | 120 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 110 insertions(+), 10 deletions(-)

diff --git a/lib/compress/zstd_opt.c b/lib/compress/zstd_opt.c
index c1ebea4d99f..83696f64d75 100644
--- a/lib/compress/zstd_opt.c
+++ b/lib/compress/zstd_opt.c
@@ -96,6 +96,55 @@ static void ZSTD_rescaleFreqs(optState_t* optPtr, const BYTE* src, size_t srcSiz
 }
 
 
+static U32 ZSTD_rawLiteralsCost(const optState_t* const optPtr,
+                                 const BYTE* const literals, U32 const litLength)
+{
+    if (optPtr->staticPrices) return (litLength*6);  /* 6 bit per literal - no statistic used */
+    if (litLength == 0) return 0;
+
+    /* literals */
+    {   U32 u;
+        U32 cost = litLength * optPtr->log2litSum;
+        for (u=0; u < litLength; u++)
+            cost -= ZSTD_highbit32(optPtr->litFreq[literals[u]]+1);
+        return cost;
+    }
+}
+
+static U32 ZSTD_litLengthPrice(const optState_t* const optPtr, U32 const litLength)
+{
+    if (optPtr->staticPrices) return ZSTD_highbit32((U32)litLength+1);
+
+    /* literal Length */
+    {   U32 const llCode = ZSTD_LLcode(litLength);
+        U32 const price = LL_bits[llCode] + optPtr->log2litLengthSum - ZSTD_highbit32(optPtr->litLengthFreq[llCode]+1);
+        return price;
+    }
+}
+
+static U32 ZSTD_fullLiteralsCost(const optState_t* const optPtr,
+                                 const BYTE* const literals, U32 const litLength)
+{
+    return ZSTD_rawLiteralsCost(optPtr, literals, litLength) + ZSTD_litLengthPrice(optPtr, litLength);
+}
+
+static U32 ZSTD_updateRawLiteralsCost(const optState_t* const optPtr,
+                            U32 const previousCost,
+                            const BYTE* const literals, U32 const litLength)
+{
+    return previousCost + ZSTD_rawLiteralsCost(optPtr, literals, litLength);
+}
+
+static U32 ZSTD_newLiteralsCost(const optState_t* const optPtr,
+                                U32 const previousCost, BYTE const newLiteral)
+{
+    return previousCost
+         + optPtr->log2litSum - ZSTD_highbit32(optPtr->litFreq[newLiteral]+1);
+}
+
+
+
+
 static U32 ZSTD_getLiteralPrice(optState_t* const optPtr,
                                 U32 const litLength, const BYTE* const literals)
 {
@@ -139,10 +188,10 @@ static U32 ZSTD_getLiteralPrice(optState_t* const optPtr,
 }
 
 
-/* ZSTD_getPrice() :
+/* ZSTD_getMatchPrice() :
  * optLevel: when <2, favors small offset for decompression speed (improved cache efficiency) */
-FORCE_INLINE_TEMPLATE U32 ZSTD_getPrice(optState_t* optPtr,
-                                    U32 const litLength, const BYTE* const literals, U32 const offset, U32 const matchLength,
+FORCE_INLINE_TEMPLATE U32 ZSTD_getMatchPrice(optState_t* optPtr,
+                                    U32 const offset, U32 const matchLength,
                                     int const optLevel)
 {
     U32 price;
@@ -151,7 +200,7 @@ FORCE_INLINE_TEMPLATE U32 ZSTD_getPrice(optState_t* optPtr,
     assert(matchLength >= MINMATCH);
 
     if (optPtr->staticPrices)  /* fixed scheme, do not use statistics */
-        return ZSTD_getLiteralPrice(optPtr, litLength, literals) + ZSTD_highbit32((U32)mlBase+1) + 16 + offCode;
+        return ZSTD_highbit32((U32)mlBase+1) + 16 + offCode;
 
     price = offCode + optPtr->log2offCodeSum - ZSTD_highbit32(optPtr->offCodeFreq[offCode]+1);
     if ((optLevel<2) /*static*/ && offCode >= 20) price += (offCode-19)*2; /* handicap for long distance offsets, favor decompression speed */
@@ -161,7 +210,18 @@ FORCE_INLINE_TEMPLATE U32 ZSTD_getPrice(optState_t* optPtr,
         price += ML_bits[mlCode] + optPtr->log2matchLengthSum - ZSTD_highbit32(optPtr->matchLengthFreq[mlCode]+1);
     }
 
-    price += ZSTD_getLiteralPrice(optPtr, litLength, literals) + optPtr->factor;
+    price += + optPtr->factor;
+    DEBUGLOG(8, "ZSTD_getMatchPrice(ml:%u) = %u", matchLength, price);
+    return price;
+}
+
+/* ZSTD_getPrice() :
+ * optLevel: when <2, favors small offset for decompression speed (improved cache efficiency) */
+FORCE_INLINE_TEMPLATE U32 ZSTD_getPrice(optState_t* optPtr,
+                                    U32 const litLength, const BYTE* const literals, U32 const offset, U32 const matchLength,
+                                    int const optLevel)
+{
+    U32 const price = ZSTD_getLiteralPrice(optPtr, litLength, literals) + ZSTD_getMatchPrice(optPtr, offset, matchLength, optLevel);
     DEBUGLOG(8, "ZSTD_getPrice(ll:%u, ml:%u) = %u", litLength, matchLength, price);
     return price;
 }
@@ -454,6 +514,39 @@ repcodes_t ZSTD_updateRep(U32 const rep[3], U32 const offset, U32 const ll0)
         memcpy(opt[pos].rep, &rep_, sizeof(rep_));            \
     }
 
+typedef struct {
+    const BYTE* anchor;
+    U32 litlen;
+    U32 rawLitCost;
+} cachedLiteralPrice_t;
+
+static U32 ZSTD_updateCachedLPrice(const optState_t* const optStatePtr,
+                            cachedLiteralPrice_t* const cachedLitPrice,
+                            const BYTE* const anchor, U32 const litlen)
+{
+    U32 startCost;
+    U32 remainingLength;
+    const BYTE* startPosition;
+
+    if (anchor == cachedLitPrice->anchor) {
+        startCost = cachedLitPrice->rawLitCost;
+        startPosition = anchor + cachedLitPrice->litlen;
+        assert(litlen >= cachedLitPrice->litlen);
+        remainingLength = litlen - cachedLitPrice->litlen;
+    } else {
+        startCost = 0;
+        startPosition = anchor;
+        remainingLength = litlen;
+    }
+
+    {   U32 const rawLitCost = startCost + ZSTD_rawLiteralsCost(optStatePtr, startPosition, remainingLength);
+        cachedLitPrice->anchor = anchor;
+        cachedLitPrice->litlen = litlen;
+        cachedLitPrice->rawLitCost = rawLitCost;
+        return rawLitCost + ZSTD_litLengthPrice(optStatePtr, litlen);
+    }
+}
+
 FORCE_INLINE_TEMPLATE
 size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
                                       const void* src, size_t srcSize,
@@ -476,6 +569,7 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
 
     ZSTD_optimal_t* const opt = optStatePtr->priceTable;
     ZSTD_match_t* const matches = optStatePtr->matchTable;
+    cachedLiteralPrice_t cachedLitPrice;
     U32 rep[ZSTD_REP_NUM];
 
     /* init */
@@ -484,6 +578,7 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
     ZSTD_rescaleFreqs(optStatePtr, (const BYTE*)src, srcSize);
     ip += (ip==prefixStart);
     { int i; for (i=0; irep[i]; }
+    memset(&cachedLitPrice, 0, sizeof(cachedLitPrice));
 
     /* Match Loop */
     while (ip < ilimit) {
@@ -517,7 +612,8 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
             }   }
 
             /* set prices for first matches starting position == 0 */
-            {   U32 pos;
+            {   U32 const literalsPrice = ZSTD_updateCachedLPrice(optStatePtr, &cachedLitPrice, anchor, litlen);
+                U32 pos;
                 U32 matchNb;
                 for (pos = 0; pos < minMatch; pos++) {
                     opt[pos].mlen = 1;
@@ -528,7 +624,7 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
                     U32 const end = matches[matchNb].len;
                     repcodes_t const repHistory = ZSTD_updateRep(rep, offset, ll0);
                     for ( ; pos <= end ; pos++ ) {
-                        U32 const matchPrice = ZSTD_getPrice(optStatePtr, litlen, anchor, offset, pos, optLevel);
+                        U32 const matchPrice = literalsPrice + ZSTD_getMatchPrice(optStatePtr, offset, pos, optLevel);
                         DEBUGLOG(7, "rPos:%u => set initial price : %u",
                                     pos, matchPrice);
                         opt[pos].mlen = pos;
@@ -550,14 +646,18 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
             {   U32 const litlen = (opt[cur-1].mlen == 1) ? opt[cur-1].litlen + 1 : 1;
                 U32 price;
                 if (cur > litlen) {
-                    price = opt[cur - litlen].price + ZSTD_getLiteralPrice(optStatePtr, litlen, inr-litlen);
+                    price = opt[cur - litlen].price + ZSTD_fullLiteralsCost(optStatePtr, inr-litlen, litlen);
                 } else {
-                    price = ZSTD_getLiteralPrice(optStatePtr, litlen, anchor);
+                    price = ZSTD_updateCachedLPrice(optStatePtr, &cachedLitPrice, anchor, litlen);
                 }
                 if (price <= opt[cur].price) {
                     DEBUGLOG(7, "rPos:%u : better price (%u<%u) using literal",
                                 cur, price, opt[cur].price);
-                    SET_PRICE(cur, 1/*mlen*/, 0/*offset*/, litlen, price, opt[cur-1].rep);
+                    opt[cur].mlen = 1;
+                    opt[cur].off = 0;
+                    opt[cur].litlen = litlen;
+                    opt[cur].price = price;
+                    memcpy(opt[cur].rep, opt[cur-1].rep, sizeof(opt[cur].rep));
             }   }
 
             /* last match must start at a minimum distance of 8 from oend */

From b71405dc51cd6fb05ea2417ac6ed0516ef24d2f0 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 28 Nov 2017 12:32:24 -0800
Subject: [PATCH 086/145] removed a bunch of code related to cached literal
 price

optState was used both to evaluate price
and to cache cost of previously calculated literals.
This created a strong dependency, forcing parser to request cost in a strict order.
This limitation is forbids future parser with skipping capabilities.

After this patch, caching literals price still exists,
but is now explicit, in a stack structure.
---
 lib/compress/zstd_compress_internal.h |   3 -
 lib/compress/zstd_opt.c               | 115 +++++---------------------
 2 files changed, 22 insertions(+), 96 deletions(-)

diff --git a/lib/compress/zstd_compress_internal.h b/lib/compress/zstd_compress_internal.h
index 4fd55c43883..e33ec4086b9 100644
--- a/lib/compress/zstd_compress_internal.h
+++ b/lib/compress/zstd_compress_internal.h
@@ -93,9 +93,6 @@ typedef struct {
     U32  factor;                 /* fixed cost added when calculating ZSTD_getPrice() (but why ? seems to favor less sequences) */
     /* end : updated by ZSTD_setLog2Prices */
     U32  staticPrices;           /* prices follow a pre-defined cost structure, statistics are irrelevant */
-    U32  cachedPrice;
-    U32  cachedLitLength;
-    const BYTE* cachedLiterals;
 } optState_t;
 
 typedef struct {
diff --git a/lib/compress/zstd_opt.c b/lib/compress/zstd_opt.c
index 83696f64d75..fe52fed5899 100644
--- a/lib/compress/zstd_opt.c
+++ b/lib/compress/zstd_opt.c
@@ -32,10 +32,9 @@ static void ZSTD_setLog2Prices(optState_t* optPtr)
 }
 
 
-static void ZSTD_rescaleFreqs(optState_t* optPtr, const BYTE* src, size_t srcSize)
+static void ZSTD_rescaleFreqs(optState_t* const optPtr,
+                              const BYTE* const src, size_t const srcSize)
 {
-    optPtr->cachedLiterals = NULL;
-    optPtr->cachedPrice = optPtr->cachedLitLength = 0;
     optPtr->staticPrices = 0;
 
     if (optPtr->litLengthSum == 0) {  /* first init */
@@ -97,7 +96,7 @@ static void ZSTD_rescaleFreqs(optState_t* optPtr, const BYTE* src, size_t srcSiz
 
 
 static U32 ZSTD_rawLiteralsCost(const optState_t* const optPtr,
-                                 const BYTE* const literals, U32 const litLength)
+                                const BYTE* const literals, U32 const litLength)
 {
     if (optPtr->staticPrices) return (litLength*6);  /* 6 bit per literal - no statistic used */
     if (litLength == 0) return 0;
@@ -128,69 +127,10 @@ static U32 ZSTD_fullLiteralsCost(const optState_t* const optPtr,
     return ZSTD_rawLiteralsCost(optPtr, literals, litLength) + ZSTD_litLengthPrice(optPtr, litLength);
 }
 
-static U32 ZSTD_updateRawLiteralsCost(const optState_t* const optPtr,
-                            U32 const previousCost,
-                            const BYTE* const literals, U32 const litLength)
-{
-    return previousCost + ZSTD_rawLiteralsCost(optPtr, literals, litLength);
-}
-
-static U32 ZSTD_newLiteralsCost(const optState_t* const optPtr,
-                                U32 const previousCost, BYTE const newLiteral)
-{
-    return previousCost
-         + optPtr->log2litSum - ZSTD_highbit32(optPtr->litFreq[newLiteral]+1);
-}
-
-
-
-
-static U32 ZSTD_getLiteralPrice(optState_t* const optPtr,
-                                U32 const litLength, const BYTE* const literals)
-{
-    U32 price;
-
-    if (optPtr->staticPrices)
-        return ZSTD_highbit32((U32)litLength+1) + (litLength*6);  /* 6 bit per literal - no statistic used */
-
-    if (litLength == 0)
-        return optPtr->log2litLengthSum - ZSTD_highbit32(optPtr->litLengthFreq[0]+1);
-
-    /* literals */
-    if (optPtr->cachedLiterals == literals) {
-        U32 u;
-        U32 const additional = litLength - optPtr->cachedLitLength;
-        const BYTE* const literals2 = optPtr->cachedLiterals + optPtr->cachedLitLength;
-        price = optPtr->cachedPrice + additional * optPtr->log2litSum;
-        for (u=0; u < additional; u++)
-            price -= ZSTD_highbit32(optPtr->litFreq[literals2[u]]+1);
-        optPtr->cachedPrice = price;
-        optPtr->cachedLitLength = litLength;
-    } else {
-        U32 u;
-        price = litLength * optPtr->log2litSum;
-        for (u=0; u < litLength; u++)
-            price -= ZSTD_highbit32(optPtr->litFreq[literals[u]]+1);
-
-        if (litLength >= 12) {
-            optPtr->cachedLiterals = literals;
-            optPtr->cachedPrice = price;
-            optPtr->cachedLitLength = litLength;
-        }
-    }
-
-    /* literal Length */
-    {   U32 const llCode = ZSTD_LLcode(litLength);
-        price += LL_bits[llCode] + optPtr->log2litLengthSum - ZSTD_highbit32(optPtr->litLengthFreq[llCode]+1);
-    }
-
-    return price;
-}
-
 
 /* ZSTD_getMatchPrice() :
  * optLevel: when <2, favors small offset for decompression speed (improved cache efficiency) */
-FORCE_INLINE_TEMPLATE U32 ZSTD_getMatchPrice(optState_t* optPtr,
+FORCE_INLINE_TEMPLATE U32 ZSTD_getMatchPrice(const optState_t* const optPtr,
                                     U32 const offset, U32 const matchLength,
                                     int const optLevel)
 {
@@ -210,24 +150,14 @@ FORCE_INLINE_TEMPLATE U32 ZSTD_getMatchPrice(optState_t* optPtr,
         price += ML_bits[mlCode] + optPtr->log2matchLengthSum - ZSTD_highbit32(optPtr->matchLengthFreq[mlCode]+1);
     }
 
-    price += + optPtr->factor;
+    price += optPtr->factor;
     DEBUGLOG(8, "ZSTD_getMatchPrice(ml:%u) = %u", matchLength, price);
     return price;
 }
 
-/* ZSTD_getPrice() :
- * optLevel: when <2, favors small offset for decompression speed (improved cache efficiency) */
-FORCE_INLINE_TEMPLATE U32 ZSTD_getPrice(optState_t* optPtr,
-                                    U32 const litLength, const BYTE* const literals, U32 const offset, U32 const matchLength,
-                                    int const optLevel)
-{
-    U32 const price = ZSTD_getLiteralPrice(optPtr, litLength, literals) + ZSTD_getMatchPrice(optPtr, offset, matchLength, optLevel);
-    DEBUGLOG(8, "ZSTD_getPrice(ll:%u, ml:%u) = %u", litLength, matchLength, price);
-    return price;
-}
-
-
-static void ZSTD_updateStats(optState_t* optPtr, U32 litLength, const BYTE* literals, U32 offsetCode, U32 matchLength)
+static void ZSTD_updateStats(optState_t* const optPtr,
+                            U32 litLength, const BYTE* literals,
+                            U32 offsetCode, U32 matchLength)
 {
     /* literals */
     {   U32 u;
@@ -275,13 +205,13 @@ MEM_STATIC U32 ZSTD_readMINMATCH(const void* memPtr, U32 length)
 
 /* Update hashTable3 up to ip (excluded)
    Assumption : always within prefix (i.e. not within extDict) */
-static U32 ZSTD_insertAndFindFirstIndexHash3 (ZSTD_CCtx* zc, const BYTE* ip)
+static U32 ZSTD_insertAndFindFirstIndexHash3 (ZSTD_CCtx* const cctx, const BYTE* const ip)
 {
-    U32* const hashTable3  = zc->hashTable3;
-    U32 const hashLog3  = zc->hashLog3;
-    const BYTE* const base = zc->base;
-    U32 idx = zc->nextToUpdate3;
-    U32 const target = zc->nextToUpdate3 = (U32)(ip - base);
+    U32* const hashTable3  = cctx->hashTable3;
+    U32 const hashLog3  = cctx->hashLog3;
+    const BYTE* const base = cctx->base;
+    U32 idx = cctx->nextToUpdate3;
+    U32 const target = cctx->nextToUpdate3 = (U32)(ip - base);
     size_t const hash3 = ZSTD_hash3Ptr(ip, hashLog3);
 
     while(idx < target) {
@@ -298,11 +228,11 @@ static U32 ZSTD_insertAndFindFirstIndexHash3 (ZSTD_CCtx* zc, const BYTE* ip)
 ***************************************/
 FORCE_INLINE_TEMPLATE
 U32 ZSTD_insertBtAndGetAllMatches (
-                        ZSTD_CCtx* zc,
-                        const BYTE* const ip, const BYTE* const iLimit, const int extDict,
-                        U32 nbCompares, U32 const mls, U32 const sufficient_len,
-                        U32 rep[ZSTD_REP_NUM], U32 const ll0,
-                        ZSTD_match_t* matches, const U32 lengthToBeat)
+                    ZSTD_CCtx* zc,
+                    const BYTE* const ip, const BYTE* const iLimit, int const extDict,
+                    U32 nbCompares, U32 const mls, U32 const sufficient_len,
+                    U32 rep[ZSTD_REP_NUM], U32 const ll0,
+                    ZSTD_match_t* matches, const U32 lengthToBeat)
 {
     const BYTE* const base = zc->base;
     U32 const current = (U32)(ip-base);
@@ -671,12 +601,11 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
 
             {   U32 const ll0 = (opt[cur].mlen != 1);
                 U32 const litlen = (opt[cur].mlen == 1) ? opt[cur].litlen : 0;
-                U32 const basePrice = (cur > litlen) ? opt[cur-litlen].price : 0;
-                const BYTE* const baseLiterals = ip + cur - litlen;
+                U32 const previousPrice = (cur > litlen) ? opt[cur-litlen].price : 0;
+                U32 const basePrice = previousPrice + ZSTD_fullLiteralsCost(optStatePtr, inr-litlen, litlen);
                 U32 const nbMatches = ZSTD_BtGetAllMatches(ctx, inr, iend, extDict, maxSearches, mls, sufficient_len, opt[cur].rep, ll0, matches, minMatch);
                 U32 matchNb;
                 if (!nbMatches) continue;
-                assert(baseLiterals >= prefixStart);
 
                 {   U32 const maxML = matches[nbMatches-1].len;
                     DEBUGLOG(7, "rPos:%u, found %u matches, of maxLength=%u",
@@ -704,7 +633,7 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
 
                     for (mlen = lastML; mlen >= startML; mlen--) {
                         U32 const pos = cur + mlen;
-                        U32 const price = basePrice + ZSTD_getPrice(optStatePtr, litlen, baseLiterals, offset, mlen, optLevel);
+                        U32 const price = basePrice + ZSTD_getMatchPrice(optStatePtr, offset, mlen, optLevel);
 
                         if ((pos > last_pos) || (price < opt[pos].price)) {
                             DEBUGLOG(7, "rPos:%u => new better price (%u<%u)",

From 0a0a21293436a5f19e3fc79b1e4d69dc3acabc1b Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 28 Nov 2017 14:07:03 -0800
Subject: [PATCH 087/145] zstd_opt: changed cost formula

There was a flaw in the formula
which compared literal cost with match cost :
at a given position,
a non-null literal suite is going to be part of next sequence,
while if position ends a previous match, to immediately start another match,
next sequence will have a litlength of zero.
A litlength of zero has a non-null cost.
It follows that literals cost should be compared to match cost + litlength==0.

Not doing so gave a structural advantage to matches, which would be selected more often.
I believe that's what led to the creation of the strange heuristic which added a complex cost to matches.
The heuristic was actually compensating.
It was probably created through multiple trials, settling for best outcome on a given scenario (I suspect silesia.tar).
The problem with this heuristic is that it's hard to understand,
and unfortunately, any future change in the parser would impact the way it should be calculated and its effects.

The "proper" formula makes it possible to remove this heuristic.

Now, the problem is : in a head to head comparison, it's sometimes better, sometimes worse.
Note that all differences are small (< 0.01 ratio).
In general, the newer formula is better for smaller files (for example, calgary.tar and enwik7).
I suspect that's because starting statistics are pretty poor (another area of improvement).
However, for silesia.tar specifically, it's worse at level 22 (while being better at level 17, so even compression level has an impact ...).

It's a pity that zstd -22 gets worse on silesia.tar.
That being said, I like that the new code gets rid of strange variables,
which were introducing complexity for any future evolution (faster variants being in mind).
Therefore, in spite of this detrimental side effect, I tend to be in favor of it.
---
 lib/common/zstd_internal.h            |  6 +-
 lib/compress/zstd_compress_internal.h |  4 +-
 lib/compress/zstd_opt.c               | 92 +++++++++++++++++++--------
 programs/bench.c                      |  2 +-
 4 files changed, 71 insertions(+), 33 deletions(-)

diff --git a/lib/common/zstd_internal.h b/lib/common/zstd_internal.h
index 01d343da666..2f812482550 100644
--- a/lib/common/zstd_internal.h
+++ b/lib/common/zstd_internal.h
@@ -253,16 +253,14 @@ MEM_STATIC U32 ZSTD_highbit32(U32 val)   /* compress, dictBuilder, decodeCorpus
 #   elif defined(__GNUC__) && (__GNUC__ >= 3)   /* GCC Intrinsic */
         return 31 - __builtin_clz(val);
 #   else   /* Software version */
-        static const int DeBruijnClz[32] = { 0, 9, 1, 10, 13, 21, 2, 29, 11, 14, 16, 18, 22, 25, 3, 30, 8, 12, 20, 28, 15, 17, 24, 7, 19, 27, 23, 6, 26, 5, 4, 31 };
+        static const U32 DeBruijnClz[32] = { 0, 9, 1, 10, 13, 21, 2, 29, 11, 14, 16, 18, 22, 25, 3, 30, 8, 12, 20, 28, 15, 17, 24, 7, 19, 27, 23, 6, 26, 5, 4, 31 };
         U32 v = val;
-        int r;
         v |= v >> 1;
         v |= v >> 2;
         v |= v >> 4;
         v |= v >> 8;
         v |= v >> 16;
-        r = DeBruijnClz[(U32)(v * 0x07C4ACDDU) >> 27];
-        return r;
+        return DeBruijnClz[(v * 0x07C4ACDDU) >> 27];
 #   endif
     }
 }
diff --git a/lib/compress/zstd_compress_internal.h b/lib/compress/zstd_compress_internal.h
index e33ec4086b9..1308541f404 100644
--- a/lib/compress/zstd_compress_internal.h
+++ b/lib/compress/zstd_compress_internal.h
@@ -64,7 +64,7 @@ typedef struct {
 } ZSTD_match_t;
 
 typedef struct {
-    U32 price;
+    int price;
     U32 off;
     U32 mlen;
     U32 litlen;
@@ -83,14 +83,12 @@ typedef struct {
     U32  litSum;                 /* nb of literals */
     U32  litLengthSum;           /* nb of litLength codes */
     U32  matchLengthSum;         /* nb of matchLength codes */
-    U32  matchSum;               /* a strange argument used in calculating `factor` */
     U32  offCodeSum;             /* nb of offset codes */
     /* begin updated by ZSTD_setLog2Prices */
     U32  log2litSum;             /* pow2 to compare log2(litfreq) to */
     U32  log2litLengthSum;       /* pow2 to compare log2(llfreq) to */
     U32  log2matchLengthSum;     /* pow2 to compare log2(mlfreq) to */
     U32  log2offCodeSum;         /* pow2 to compare log2(offreq) to */
-    U32  factor;                 /* fixed cost added when calculating ZSTD_getPrice() (but why ? seems to favor less sequences) */
     /* end : updated by ZSTD_setLog2Prices */
     U32  staticPrices;           /* prices follow a pre-defined cost structure, statistics are irrelevant */
 } optState_t;
diff --git a/lib/compress/zstd_opt.c b/lib/compress/zstd_opt.c
index fe52fed5899..6218d368246 100644
--- a/lib/compress/zstd_opt.c
+++ b/lib/compress/zstd_opt.c
@@ -27,8 +27,6 @@ static void ZSTD_setLog2Prices(optState_t* optPtr)
     optPtr->log2litLengthSum = ZSTD_highbit32(optPtr->litLengthSum+1);
     optPtr->log2matchLengthSum = ZSTD_highbit32(optPtr->matchLengthSum+1);
     optPtr->log2offCodeSum = ZSTD_highbit32(optPtr->offCodeSum+1);
-    optPtr->factor = 1 + ((optPtr->litSum>>5) / optPtr->litLengthSum)
-                       + ((optPtr->litSum<<1) / (optPtr->litSum + optPtr->matchSum));  /* => {0,1}, == (optPtr->matchSum <= optPtr->litSum) */
 }
 
 
@@ -58,7 +56,6 @@ static void ZSTD_rescaleFreqs(optState_t* const optPtr,
         for (u=0; u<=MaxML; u++)
             optPtr->matchLengthFreq[u] = 1;
         optPtr->matchLengthSum = MaxML+1;
-        optPtr->matchSum = (ZSTD_LITFREQ_ADD << Litbits);
         for (u=0; u<=MaxOff; u++)
             optPtr->offCodeFreq[u] = 1;
         optPtr->offCodeSum = (MaxOff+1);
@@ -77,13 +74,10 @@ static void ZSTD_rescaleFreqs(optState_t* const optPtr,
             optPtr->litLengthSum += optPtr->litLengthFreq[u];
         }
         optPtr->matchLengthSum = 0;
-        optPtr->matchSum = 0;
         for (u=0; u<=MaxML; u++) {
             optPtr->matchLengthFreq[u] = 1 + (optPtr->matchLengthFreq[u]>>ZSTD_FREQ_DIV);
             optPtr->matchLengthSum += optPtr->matchLengthFreq[u];
-            optPtr->matchSum += optPtr->matchLengthFreq[u] * (u + 3);
         }
-        optPtr->matchSum *= ZSTD_LITFREQ_ADD;
         optPtr->offCodeSum = 0;
         for (u=0; u<=MaxOff; u++) {
             optPtr->offCodeFreq[u] = 1 + (optPtr->offCodeFreq[u]>>ZSTD_FREQ_DIV);
@@ -95,8 +89,8 @@ static void ZSTD_rescaleFreqs(optState_t* const optPtr,
 }
 
 
-static U32 ZSTD_rawLiteralsCost(const optState_t* const optPtr,
-                                const BYTE* const literals, U32 const litLength)
+static U32 ZSTD_rawLiteralsCost(const BYTE* const literals, U32 const litLength,
+                                const optState_t* const optPtr)
 {
     if (optPtr->staticPrices) return (litLength*6);  /* 6 bit per literal - no statistic used */
     if (litLength == 0) return 0;
@@ -110,7 +104,7 @@ static U32 ZSTD_rawLiteralsCost(const optState_t* const optPtr,
     }
 }
 
-static U32 ZSTD_litLengthPrice(const optState_t* const optPtr, U32 const litLength)
+static U32 ZSTD_litLengthPrice(U32 const litLength, const optState_t* const optPtr)
 {
     if (optPtr->staticPrices) return ZSTD_highbit32((U32)litLength+1);
 
@@ -121,17 +115,44 @@ static U32 ZSTD_litLengthPrice(const optState_t* const optPtr, U32 const litLeng
     }
 }
 
-static U32 ZSTD_fullLiteralsCost(const optState_t* const optPtr,
-                                 const BYTE* const literals, U32 const litLength)
+static U32 ZSTD_fullLiteralsCost(const BYTE* const literals, U32 const litLength,
+                                 const optState_t* const optPtr)
 {
-    return ZSTD_rawLiteralsCost(optPtr, literals, litLength) + ZSTD_litLengthPrice(optPtr, litLength);
+    return ZSTD_rawLiteralsCost(literals, litLength, optPtr)
+         + ZSTD_litLengthPrice(litLength, optPtr);
+}
+
+static int ZSTD_litLengthContribution(U32 const litLength, const optState_t* const optPtr)
+{
+    if (optPtr->staticPrices) return ZSTD_highbit32(litLength+1);
+
+    /* literal Length */
+    {   U32 const llCode = ZSTD_LLcode(litLength);
+        int const contribution = LL_bits[llCode]
+                        + ZSTD_highbit32(optPtr->litLengthFreq[0]+1)
+                        - ZSTD_highbit32(optPtr->litLengthFreq[llCode]+1);
+#if 1
+        return contribution;
+#else
+        return MAX(0, contribution); /* sometimes better, sometimes not ... */
+#endif
+    }
+}
+
+static int ZSTD_literalsContribution(const BYTE* const literals, U32 const litLength,
+                                     const optState_t* const optPtr)
+{
+    int const contribution = ZSTD_rawLiteralsCost(literals, litLength, optPtr)
+                           + ZSTD_litLengthContribution(litLength, optPtr);
+    return contribution;
 }
 
 
 /* ZSTD_getMatchPrice() :
  * optLevel: when <2, favors small offset for decompression speed (improved cache efficiency) */
-FORCE_INLINE_TEMPLATE U32 ZSTD_getMatchPrice(const optState_t* const optPtr,
+FORCE_INLINE_TEMPLATE U32 ZSTD_getMatchPrice(
                                     U32 const offset, U32 const matchLength,
+                                    const optState_t* const optPtr,
                                     int const optLevel)
 {
     U32 price;
@@ -150,7 +171,7 @@ FORCE_INLINE_TEMPLATE U32 ZSTD_getMatchPrice(const optState_t* const optPtr,
         price += ML_bits[mlCode] + optPtr->log2matchLengthSum - ZSTD_highbit32(optPtr->matchLengthFreq[mlCode]+1);
     }
 
-    price += optPtr->factor;
+    //price += optPtr->factor;
     DEBUGLOG(8, "ZSTD_getMatchPrice(ml:%u) = %u", matchLength, price);
     return price;
 }
@@ -450,9 +471,10 @@ typedef struct {
     U32 rawLitCost;
 } cachedLiteralPrice_t;
 
-static U32 ZSTD_updateCachedLPrice(const optState_t* const optStatePtr,
+static U32 ZSTD_rawLiteralsCost_cached(
                             cachedLiteralPrice_t* const cachedLitPrice,
-                            const BYTE* const anchor, U32 const litlen)
+                            const BYTE* const anchor, U32 const litlen,
+                            const optState_t* const optStatePtr)
 {
     U32 startCost;
     U32 remainingLength;
@@ -469,14 +491,33 @@ static U32 ZSTD_updateCachedLPrice(const optState_t* const optStatePtr,
         remainingLength = litlen;
     }
 
-    {   U32 const rawLitCost = startCost + ZSTD_rawLiteralsCost(optStatePtr, startPosition, remainingLength);
+    {   U32 const rawLitCost = startCost + ZSTD_rawLiteralsCost(startPosition, remainingLength, optStatePtr);
         cachedLitPrice->anchor = anchor;
         cachedLitPrice->litlen = litlen;
         cachedLitPrice->rawLitCost = rawLitCost;
-        return rawLitCost + ZSTD_litLengthPrice(optStatePtr, litlen);
+        return rawLitCost;
     }
 }
 
+static U32 ZSTD_fullLiteralsCost_cached(
+                            cachedLiteralPrice_t* const cachedLitPrice,
+                            const BYTE* const anchor, U32 const litlen,
+                            const optState_t* const optStatePtr)
+{
+    return ZSTD_rawLiteralsCost_cached(cachedLitPrice, anchor, litlen, optStatePtr)
+         + ZSTD_litLengthPrice(litlen, optStatePtr);
+}
+
+static int ZSTD_literalsContribution_cached(
+                            cachedLiteralPrice_t* const cachedLitPrice,
+                            const BYTE* const anchor, U32 const litlen,
+                            const optState_t* const optStatePtr)
+{
+    int const contribution = ZSTD_rawLiteralsCost_cached(cachedLitPrice, anchor, litlen, optStatePtr)
+                           + ZSTD_litLengthContribution(litlen, optStatePtr);
+    return contribution;
+}
+
 FORCE_INLINE_TEMPLATE
 size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
                                       const void* src, size_t srcSize,
@@ -542,7 +583,7 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
             }   }
 
             /* set prices for first matches starting position == 0 */
-            {   U32 const literalsPrice = ZSTD_updateCachedLPrice(optStatePtr, &cachedLitPrice, anchor, litlen);
+            {   U32 const literalsPrice = ZSTD_fullLiteralsCost_cached(&cachedLitPrice, anchor, litlen, optStatePtr);
                 U32 pos;
                 U32 matchNb;
                 for (pos = 0; pos < minMatch; pos++) {
@@ -554,7 +595,7 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
                     U32 const end = matches[matchNb].len;
                     repcodes_t const repHistory = ZSTD_updateRep(rep, offset, ll0);
                     for ( ; pos <= end ; pos++ ) {
-                        U32 const matchPrice = literalsPrice + ZSTD_getMatchPrice(optStatePtr, offset, pos, optLevel);
+                        U32 const matchPrice = literalsPrice + ZSTD_getMatchPrice(offset, pos, optStatePtr, optLevel);
                         DEBUGLOG(7, "rPos:%u => set initial price : %u",
                                     pos, matchPrice);
                         opt[pos].mlen = pos;
@@ -574,12 +615,13 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
 
             /* Fix current position with one literal if cheaper */
             {   U32 const litlen = (opt[cur-1].mlen == 1) ? opt[cur-1].litlen + 1 : 1;
-                U32 price;
+                int price;  /* note : contribution can be negative */
                 if (cur > litlen) {
-                    price = opt[cur - litlen].price + ZSTD_fullLiteralsCost(optStatePtr, inr-litlen, litlen);
+                    price = opt[cur - litlen].price + ZSTD_literalsContribution(inr-litlen, litlen, optStatePtr);
                 } else {
-                    price = ZSTD_updateCachedLPrice(optStatePtr, &cachedLitPrice, anchor, litlen);
+                    price = ZSTD_literalsContribution_cached(&cachedLitPrice, anchor, litlen, optStatePtr);
                 }
+                assert(price < 1000000000); /* overflow check */
                 if (price <= opt[cur].price) {
                     DEBUGLOG(7, "rPos:%u : better price (%u<%u) using literal",
                                 cur, price, opt[cur].price);
@@ -602,7 +644,7 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
             {   U32 const ll0 = (opt[cur].mlen != 1);
                 U32 const litlen = (opt[cur].mlen == 1) ? opt[cur].litlen : 0;
                 U32 const previousPrice = (cur > litlen) ? opt[cur-litlen].price : 0;
-                U32 const basePrice = previousPrice + ZSTD_fullLiteralsCost(optStatePtr, inr-litlen, litlen);
+                U32 const basePrice = previousPrice + ZSTD_fullLiteralsCost(inr-litlen, litlen, optStatePtr);
                 U32 const nbMatches = ZSTD_BtGetAllMatches(ctx, inr, iend, extDict, maxSearches, mls, sufficient_len, opt[cur].rep, ll0, matches, minMatch);
                 U32 matchNb;
                 if (!nbMatches) continue;
@@ -633,7 +675,7 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
 
                     for (mlen = lastML; mlen >= startML; mlen--) {
                         U32 const pos = cur + mlen;
-                        U32 const price = basePrice + ZSTD_getMatchPrice(optStatePtr, offset, mlen, optLevel);
+                        int const price = basePrice + ZSTD_getMatchPrice(offset, mlen, optStatePtr, optLevel);
 
                         if ((pos > last_pos) || (price < opt[pos].price)) {
                             DEBUGLOG(7, "rPos:%u => new better price (%u<%u)",
diff --git a/programs/bench.c b/programs/bench.c
index 1df8aeb0d01..2f30c65ac03 100644
--- a/programs/bench.c
+++ b/programs/bench.c
@@ -191,7 +191,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                         const char* displayName, int cLevel,
                         const size_t* fileSizes, U32 nbFiles,
                         const void* dictBuffer, size_t dictBufferSize,
-                        const ZSTD_compressionParameters* comprParams)
+                        const ZSTD_compressionParameters* const comprParams)
 {
     size_t const blockSize = ((g_blockSize>=32 && !g_decodeOnly) ? g_blockSize : srcSize) + (!srcSize) /* avoid div by 0 */ ;
     U32 const maxNbBlocks = (U32) ((srcSize + (blockSize-1)) / blockSize) + nbFiles;

From f5747afe0fffa5fb1c993c5366d6a6c7f73783a0 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 28 Nov 2017 15:02:48 -0800
Subject: [PATCH 088/145] add integer overflow trapping to debug builds

doc: https://gcc.gnu.org/onlinedocs/gcc-4.0.2/gcc/Code-Gen-Options.html
---
 tests/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/Makefile b/tests/Makefile
index 11d55a75d4d..89a109c78f2 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -32,7 +32,7 @@ CFLAGS     += -Wall -Wextra -Wcast-qual -Wcast-align -Wshadow                 \
               -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement \
               -Wstrict-prototypes -Wundef -Wformat-security                   \
               -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings      \
-              -Wredundant-decls
+              -Wredundant-decls -ftrapv
 CFLAGS     += $(DEBUGFLAGS) $(MOREFLAGS)
 FLAGS       = $(CPPFLAGS) $(CFLAGS) $(LDFLAGS)
 

From bc42bc3b1dcfb0d9bcffa4e4710ac0dd42a3ec27 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 28 Nov 2017 16:08:43 -0800
Subject: [PATCH 089/145] removed one invocation of SET_PRICE() macro

---
 lib/compress/zstd_opt.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/lib/compress/zstd_opt.c b/lib/compress/zstd_opt.c
index 6218d368246..d44b5be5899 100644
--- a/lib/compress/zstd_opt.c
+++ b/lib/compress/zstd_opt.c
@@ -680,7 +680,12 @@ size_t ZSTD_compressBlock_opt_generic(ZSTD_CCtx* ctx,
                         if ((pos > last_pos) || (price < opt[pos].price)) {
                             DEBUGLOG(7, "rPos:%u => new better price (%u<%u)",
                                         pos, price, opt[pos].price);
-                            SET_PRICE(pos, mlen, offset, litlen, price, repHistory);  /* note : macro modifies last_pos */
+                            while (last_pos < pos) { opt[last_pos+1].price = ZSTD_MAX_PRICE; last_pos++; }
+                            opt[pos].mlen = mlen;
+                            opt[pos].off = offset;
+                            opt[pos].litlen = litlen;
+                            opt[pos].price = price;
+                            memcpy(opt[pos].rep, &repHistory, sizeof(repHistory));
                         } else {
                             if (optLevel==0) break;  /* gets ~+10% speed for about -0.01 ratio loss */
                         }

From f98ee994c4569de52d60b70bc1e2dae75e0c9b90 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 29 Nov 2017 15:19:00 -0800
Subject: [PATCH 090/145] zstd_opt: added comments, as requested by @terrelln

---
 lib/compress/zstd_opt.c | 38 +++++++++++++++++++++++---------------
 1 file changed, 23 insertions(+), 15 deletions(-)

diff --git a/lib/compress/zstd_opt.c b/lib/compress/zstd_opt.c
index d44b5be5899..7171ff5373f 100644
--- a/lib/compress/zstd_opt.c
+++ b/lib/compress/zstd_opt.c
@@ -89,6 +89,9 @@ static void ZSTD_rescaleFreqs(optState_t* const optPtr,
 }
 
 
+/* ZSTD_rawLiteralsCost() :
+ * cost of literals (only) in given segment (which length can be null)
+ * does not include cost of literalLength symbol */
 static U32 ZSTD_rawLiteralsCost(const BYTE* const literals, U32 const litLength,
                                 const optState_t* const optPtr)
 {
@@ -104,6 +107,8 @@ static U32 ZSTD_rawLiteralsCost(const BYTE* const literals, U32 const litLength,
     }
 }
 
+/* ZSTD_litLengthPrice() :
+ * cost of literalLength symbol */
 static U32 ZSTD_litLengthPrice(U32 const litLength, const optState_t* const optPtr)
 {
     if (optPtr->staticPrices) return ZSTD_highbit32((U32)litLength+1);
@@ -115,6 +120,9 @@ static U32 ZSTD_litLengthPrice(U32 const litLength, const optState_t* const optP
     }
 }
 
+/* ZSTD_litLengthPrice() :
+ * cost of the literal part of a sequence,
+ * including literals themselves, and literalLength symbol */
 static U32 ZSTD_fullLiteralsCost(const BYTE* const literals, U32 const litLength,
                                  const optState_t* const optPtr)
 {
@@ -122,6 +130,10 @@ static U32 ZSTD_fullLiteralsCost(const BYTE* const literals, U32 const litLength
          + ZSTD_litLengthPrice(litLength, optPtr);
 }
 
+/* ZSTD_litLengthContribution() :
+ * @return ( cost(litlength) - cost(0) )
+ * this value can then be added to rawLiteralsCost()
+ * to provide a cost which is directly comparable to a match ending at same position */
 static int ZSTD_litLengthContribution(U32 const litLength, const optState_t* const optPtr)
 {
     if (optPtr->staticPrices) return ZSTD_highbit32(litLength+1);
@@ -139,6 +151,10 @@ static int ZSTD_litLengthContribution(U32 const litLength, const optState_t* con
     }
 }
 
+/* ZSTD_literalsContribution() :
+ * creates a fake cost for the literals part of a sequence
+ * which can be compared to the ending cost of a match
+ * should a new match start at this position */
 static int ZSTD_literalsContribution(const BYTE* const literals, U32 const litLength,
                                      const optState_t* const optPtr)
 {
@@ -147,8 +163,9 @@ static int ZSTD_literalsContribution(const BYTE* const literals, U32 const litLe
     return contribution;
 }
 
-
 /* ZSTD_getMatchPrice() :
+ * Provides the cost of the match part (offset + matchLength) of a sequence
+ * Must be combined with ZSTD_fullLiteralsCost() to get the full cost of a sequence.
  * optLevel: when <2, favors small offset for decompression speed (improved cache efficiency) */
 FORCE_INLINE_TEMPLATE U32 ZSTD_getMatchPrice(
                                     U32 const offset, U32 const matchLength,
@@ -171,14 +188,13 @@ FORCE_INLINE_TEMPLATE U32 ZSTD_getMatchPrice(
         price += ML_bits[mlCode] + optPtr->log2matchLengthSum - ZSTD_highbit32(optPtr->matchLengthFreq[mlCode]+1);
     }
 
-    //price += optPtr->factor;
     DEBUGLOG(8, "ZSTD_getMatchPrice(ml:%u) = %u", matchLength, price);
     return price;
 }
 
 static void ZSTD_updateStats(optState_t* const optPtr,
-                            U32 litLength, const BYTE* literals,
-                            U32 offsetCode, U32 matchLength)
+                             U32 litLength, const BYTE* literals,
+                             U32 offsetCode, U32 matchLength)
 {
     /* literals */
     {   U32 u;
@@ -209,7 +225,9 @@ static void ZSTD_updateStats(optState_t* const optPtr,
 }
 
 
-/* function safe only for comparisons */
+/* ZSTD_readMINMATCH() :
+ * function safe only for comparisons
+ * assumption : memPtr must be at least 4 bytes before end of buffer */
 MEM_STATIC U32 ZSTD_readMINMATCH(const void* memPtr, U32 length)
 {
     switch (length)
@@ -454,16 +472,6 @@ repcodes_t ZSTD_updateRep(U32 const rep[3], U32 const offset, U32 const ll0)
     return newReps;
 }
 
-/* update opt[pos] and last_pos */
-#define SET_PRICE(pos, mlen_, offset_, litlen_, price_, rep_) \
-    {                                                         \
-        while (last_pos < pos)  { opt[last_pos+1].price = ZSTD_MAX_PRICE; last_pos++; } \
-        opt[pos].mlen = mlen_;                                \
-        opt[pos].off = offset_;                               \
-        opt[pos].litlen = litlen_;                            \
-        opt[pos].price = price_;                              \
-        memcpy(opt[pos].rep, &rep_, sizeof(rep_));            \
-    }
 
 typedef struct {
     const BYTE* anchor;

From 998a93b784798a6cab9d4a536a207f863ea3bcf3 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 29 Nov 2017 16:13:05 -0800
Subject: [PATCH 091/145] simplified ZSTD_CCtx_setParametersUsingCCtxParams()

Any ZSTD_CCtx_setParameter() shall just write the requested parameter, without further action.
Any action shall be taken at parameter application only (during init).
It makes it possible to just copy CCtxParams from external container to internal state,
and get rid of the more complex code which was trying to compensate for missing actions.
---
 lib/compress/zstd_compress.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 858eaca301a..e6e0c021081 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -271,11 +271,11 @@ size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, unsigned v
     case ZSTD_p_forceMaxWindow :  /* Force back-references to remain < windowSize,
                                    * even when referencing into Dictionary content
                                    * default : 0 when using a CDict, 1 when using a Prefix */
-        cctx->loadedDictEnd = 0;
+        cctx->loadedDictEnd = 0;  /* ? */
         return ZSTD_CCtxParam_setParameter(&cctx->requestedParams, param, value);
 
     case ZSTD_p_nbThreads:
-        if (value > 1 && cctx->staticSize) {
+        if ((value > 1) && cctx->staticSize) {
             return ERROR(parameter_unsupported);  /* MT not compatible with static alloc */
         }
         return ZSTD_CCtxParam_setParameter(&cctx->requestedParams, param, value);
@@ -465,14 +465,17 @@ size_t ZSTD_CCtxParam_setParameter(
  * The multithreading parameters jobSize and overlapSizeLog are set only if
  * nbThreads > 1.
  *
- * Pledged srcSize is treated as unknown.
+ * pledgedSrcSize is considered unknown
  */
 size_t ZSTD_CCtx_setParametersUsingCCtxParams(
-        ZSTD_CCtx* cctx, const ZSTD_CCtx_params* params)
+        ZSTD_CCtx* const cctx, const ZSTD_CCtx_params* const params)
 {
     if (cctx->streamStage != zcss_init) return ERROR(stage_wrong);
     if (cctx->cdict) return ERROR(stage_wrong);
 
+#if 1
+    cctx->requestedParams = *params;
+#else
     /* Assume the compression and frame parameters are validated */
     cctx->requestedParams.cParams = params->cParams;
     cctx->requestedParams.fParams = params->fParams;
@@ -492,6 +495,7 @@ size_t ZSTD_CCtx_setParametersUsingCCtxParams(
 
     /* Copy long distance matching parameters */
     cctx->requestedParams.ldmParams = params->ldmParams;
+#endif
 
     /* customMem is used only for create/free params and can be ignored */
     return 0;
@@ -2858,6 +2862,7 @@ size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
                 if (cctx->mtctx == NULL) return ERROR(memory_allocation);
             }
             DEBUGLOG(4, "call ZSTDMT_initCStream_internal as nbThreads=%u", params.nbThreads);
+            DEBUGLOG(2, "params.windowLog = %u", params.cParams.windowLog);
             CHECK_F( ZSTDMT_initCStream_internal(
                         cctx->mtctx,
                         prefixDict.dict, prefixDict.dictSize, ZSTD_dm_rawContent,

From d3c59edac9e0bf544b7414e5442448f7a67b55ff Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 29 Nov 2017 16:42:20 -0800
Subject: [PATCH 092/145] removed long-range-mode tests from `zstreamtest
 --no-big-tests`

---
 lib/compress/zstd_compress.c   |  3 ++-
 lib/compress/zstdmt_compress.c |  2 +-
 tests/zstreamtest.c            | 33 +++++++++++++++++++--------------
 3 files changed, 22 insertions(+), 16 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index e6e0c021081..dd12b171b51 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -241,6 +241,7 @@ static ZSTD_CCtx_params ZSTD_assignParamsToCCtxParams(
 
 size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, unsigned value)
 {
+    DEBUGLOG(4, "ZSTD_CCtx_setParameter");
     if (cctx->streamStage != zcss_init) return ERROR(stage_wrong);
 
     switch(param)
@@ -320,6 +321,7 @@ size_t ZSTD_CCtxParam_setParameter(
         return CCtxParams->compressionLevel;
 
     case ZSTD_p_windowLog :
+        DEBUGLOG(4, "setting windowLog=%u", value);
         if (value) {  /* 0 : does not change current windowLog */
             CLAMPCHECK(value, ZSTD_WINDOWLOG_MIN, ZSTD_WINDOWLOG_MAX);
             ZSTD_cLevelToCCtxParams(CCtxParams);
@@ -2862,7 +2864,6 @@ size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
                 if (cctx->mtctx == NULL) return ERROR(memory_allocation);
             }
             DEBUGLOG(4, "call ZSTDMT_initCStream_internal as nbThreads=%u", params.nbThreads);
-            DEBUGLOG(2, "params.windowLog = %u", params.cParams.windowLog);
             CHECK_F( ZSTDMT_initCStream_internal(
                         cctx->mtctx,
                         prefixDict.dict, prefixDict.dictSize, ZSTD_dm_rawContent,
diff --git a/lib/compress/zstdmt_compress.c b/lib/compress/zstdmt_compress.c
index 2926d3b487f..ce813dcccaa 100644
--- a/lib/compress/zstdmt_compress.c
+++ b/lib/compress/zstdmt_compress.c
@@ -140,7 +140,7 @@ static size_t ZSTDMT_sizeof_bufferPool(ZSTDMT_bufferPool* bufPool)
     return poolSize + totalBufferSize;
 }
 
-static void ZSTDMT_setBufferSize(ZSTDMT_bufferPool* bufPool, size_t bSize)
+static void ZSTDMT_setBufferSize(ZSTDMT_bufferPool* const bufPool, size_t const bSize)
 {
     ZSTD_pthread_mutex_lock(&bufPool->poolMutex);
     DEBUGLOG(4, "ZSTDMT_setBufferSize: bSize = %u", (U32)bSize);
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index b8c437a4197..0a09cfa1527 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -1486,43 +1486,47 @@ static int fuzzerTests_newAPI(U32 seed, U32 nbTests, unsigned startTest, double
                 cParams = ZSTD_adjustCParams(cParams, 0, 0);
 
                 if (FUZ_rand(&lseed) & 1) {
+                    DISPLAYLEVEL(5, "t%u: windowLog : %u \n", testNb, cParams.windowLog);
                     CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_windowLog, cParams.windowLog, useOpaqueAPI) );
                     assert(cParams.windowLog >= ZSTD_WINDOWLOG_MIN);   /* guaranteed by ZSTD_adjustCParams() */
                     windowLogMalus = (cParams.windowLog - ZSTD_WINDOWLOG_MIN) / 5;
-                    DISPLAYLEVEL(5, "t%u: windowLog : %u \n", testNb, cParams.windowLog);
                 }
                 if (FUZ_rand(&lseed) & 1) {
-                    CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_hashLog, cParams.hashLog, useOpaqueAPI) );
                     DISPLAYLEVEL(5, "t%u: hashLog : %u \n", testNb, cParams.hashLog);
+                    CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_hashLog, cParams.hashLog, useOpaqueAPI) );
                 }
                 if (FUZ_rand(&lseed) & 1) {
-                    CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_chainLog, cParams.chainLog, useOpaqueAPI) );
                     DISPLAYLEVEL(5, "t%u: chainLog : %u \n", testNb, cParams.chainLog);
+                    CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_chainLog, cParams.chainLog, useOpaqueAPI) );
                 }
                 if (FUZ_rand(&lseed) & 1) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_searchLog, cParams.searchLog, useOpaqueAPI) );
                 if (FUZ_rand(&lseed) & 1) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_minMatch, cParams.searchLength, useOpaqueAPI) );
                 if (FUZ_rand(&lseed) & 1) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_targetLength, cParams.targetLength, useOpaqueAPI) );
 
                 /* mess with long distance matching parameters */
-                if (FUZ_rand(&lseed) & 1) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_enableLongDistanceMatching, FUZ_rand(&lseed) & 63, useOpaqueAPI) );
-                if (FUZ_rand(&lseed) & 3) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_ldmHashLog, FUZ_randomClampedLength(&lseed, ZSTD_HASHLOG_MIN, 23), useOpaqueAPI) );
-                if (FUZ_rand(&lseed) & 3) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_ldmMinMatch, FUZ_randomClampedLength(&lseed, ZSTD_LDM_MINMATCH_MIN, ZSTD_LDM_MINMATCH_MAX), useOpaqueAPI) );
-                if (FUZ_rand(&lseed) & 3) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_ldmBucketSizeLog, FUZ_randomClampedLength(&lseed, 0, ZSTD_LDM_BUCKETSIZELOG_MAX), useOpaqueAPI) );
-                if (FUZ_rand(&lseed) & 3) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_ldmHashEveryLog, FUZ_randomClampedLength(&lseed, 0, ZSTD_WINDOWLOG_MAX - ZSTD_HASHLOG_MIN), useOpaqueAPI) );
+                if (bigTests) {
+                    if (FUZ_rand(&lseed) & 1) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_enableLongDistanceMatching, FUZ_rand(&lseed) & 63, useOpaqueAPI) );
+                    if (FUZ_rand(&lseed) & 3) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_ldmHashLog, FUZ_randomClampedLength(&lseed, ZSTD_HASHLOG_MIN, 23), useOpaqueAPI) );
+                    if (FUZ_rand(&lseed) & 3) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_ldmMinMatch, FUZ_randomClampedLength(&lseed, ZSTD_LDM_MINMATCH_MIN, ZSTD_LDM_MINMATCH_MAX), useOpaqueAPI) );
+                    if (FUZ_rand(&lseed) & 3) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_ldmBucketSizeLog, FUZ_randomClampedLength(&lseed, 0, ZSTD_LDM_BUCKETSIZELOG_MAX), useOpaqueAPI) );
+                    if (FUZ_rand(&lseed) & 3) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_ldmHashEveryLog, FUZ_randomClampedLength(&lseed, 0, ZSTD_WINDOWLOG_MAX - ZSTD_HASHLOG_MIN), useOpaqueAPI) );
+                }
 
                 /* mess with frame parameters */
                 if (FUZ_rand(&lseed) & 1) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_checksumFlag, FUZ_rand(&lseed) & 1, useOpaqueAPI) );
                 if (FUZ_rand(&lseed) & 1) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_dictIDFlag, FUZ_rand(&lseed) & 1, useOpaqueAPI) );
                 if (FUZ_rand(&lseed) & 1) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_contentSizeFlag, FUZ_rand(&lseed) & 1, useOpaqueAPI) );
-                if (FUZ_rand(&lseed) & 1) CHECK_Z( ZSTD_CCtx_setPledgedSrcSize(zc, pledgedSrcSize) );
-                DISPLAYLEVEL(5, "t%u: pledgedSrcSize : %u \n", testNb, (U32)pledgedSrcSize);
+                if (FUZ_rand(&lseed) & 1) {
+                    DISPLAYLEVEL(5, "t%u: pledgedSrcSize : %u \n", testNb, (U32)pledgedSrcSize);
+                    CHECK_Z( ZSTD_CCtx_setPledgedSrcSize(zc, pledgedSrcSize) );
+                }
 
                 /* multi-threading parameters */
                 {   U32 const nbThreadsCandidate = (FUZ_rand(&lseed) & 4) + 1;
                     U32 const nbThreadsAdjusted = (windowLogMalus < nbThreadsCandidate) ? nbThreadsCandidate - windowLogMalus : 1;
                     U32 const nbThreads = MIN(nbThreadsAdjusted, nbThreadsMax);
-                    CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_nbThreads, nbThreads, useOpaqueAPI) );
                     DISPLAYLEVEL(5, "t%u: nbThreads : %u \n", testNb, nbThreads);
+                    CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_nbThreads, nbThreads, useOpaqueAPI) );
                     if (nbThreads > 1) {
                         U32 const jobLog = FUZ_rand(&lseed) % (testLog+1);
                         CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_overlapSizeLog, FUZ_rand(&lseed) % 10, useOpaqueAPI) );
@@ -1530,10 +1534,11 @@ static int fuzzerTests_newAPI(U32 seed, U32 nbTests, unsigned startTest, double
                     }
                 }
 
-                if (FUZ_rand(&lseed) & 1) CHECK_Z (setCCtxParameter(zc, cctxParams, ZSTD_p_forceMaxWindow, FUZ_rand(&lseed) & 1, useOpaqueAPI) );
+                if (FUZ_rand(&lseed) & 1) CHECK_Z( setCCtxParameter(zc, cctxParams, ZSTD_p_forceMaxWindow, FUZ_rand(&lseed) & 1, useOpaqueAPI) );
 
                 /* Apply parameters */
                 if (useOpaqueAPI) {
+                    DISPLAYLEVEL(6," t%u: applying CCtxParams \n", testNb);
                     CHECK_Z (ZSTD_CCtx_setParametersUsingCCtxParams(zc, cctxParams) );
                 }
 
@@ -1573,8 +1578,8 @@ static int fuzzerTests_newAPI(U32 seed, U32 nbTests, unsigned startTest, double
                 outBuff.size = outBuff.pos + dstBuffSize;
 
                 CHECK_Z( ZSTD_compress_generic(zc, &outBuff, &inBuff, flush) );
-                DISPLAYLEVEL(6, "compress consumed %u bytes (total : %u) \n",
-                    (U32)inBuff.pos, (U32)(totalTestSize + inBuff.pos));
+                DISPLAYLEVEL(6, "t%u: compress consumed %u bytes (total : %u) \n",
+                    testNb, (U32)inBuff.pos, (U32)(totalTestSize + inBuff.pos));
 
                 XXH64_update(&xxhState, srcBuffer+srcStart, inBuff.pos);
                 memcpy(copyBuffer+totalTestSize, srcBuffer+srcStart, inBuff.pos);

From 78290874a5e91e9d9c95ffea67592caa8425ba50 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 29 Nov 2017 17:01:14 -0800
Subject: [PATCH 093/145] fixed Visual warning on minor interface discrepancy

---
 lib/compress/zstd_compress.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index dd12b171b51..bc2d8a576bf 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -470,7 +470,7 @@ size_t ZSTD_CCtxParam_setParameter(
  * pledgedSrcSize is considered unknown
  */
 size_t ZSTD_CCtx_setParametersUsingCCtxParams(
-        ZSTD_CCtx* const cctx, const ZSTD_CCtx_params* const params)
+        ZSTD_CCtx* cctx, const ZSTD_CCtx_params* params)
 {
     if (cctx->streamStage != zcss_init) return ERROR(stage_wrong);
     if (cctx->cdict) return ERROR(stage_wrong);

From 9a2f6f477ba8cefa0040bb0323203aadde1e7c32 Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Wed, 29 Nov 2017 19:11:12 -0800
Subject: [PATCH 094/145] Use util.h for timing

---
 programs/bench.c     | 15 +++++++-------
 programs/dibio.c     | 16 +++++++--------
 programs/fileio.c    | 12 ++++++-----
 programs/util.h      |  4 ++++
 tests/decodecorpus.c | 21 +++++++++-----------
 tests/fuzzer.c       | 28 ++++++++++++--------------
 tests/paramgrill.c   | 47 +++++++++++++++++++++-----------------------
 tests/zbufftest.c    | 31 +++++++++++++----------------
 tests/zstreamtest.c  | 41 ++++++++++++++++++--------------------
 9 files changed, 103 insertions(+), 112 deletions(-)

diff --git a/programs/bench.c b/programs/bench.c
index 1df8aeb0d01..384b982f400 100644
--- a/programs/bench.c
+++ b/programs/bench.c
@@ -34,7 +34,6 @@
 #include       /* malloc, free */
 #include       /* memset */
 #include        /* fprintf, fopen */
-#include         /* clock_t, clock, CLOCKS_PER_SEC */
 
 #include "mem.h"
 #define ZSTD_STATIC_LINKING_ONLY
@@ -72,12 +71,14 @@ static U32 g_compressibilityDefault = 50;
 #define DISPLAYLEVEL(l, ...) if (g_displayLevel>=l) { DISPLAY(__VA_ARGS__); }
 static int g_displayLevel = 2;   /* 0 : no display;   1: errors;   2 : + result + interaction + warnings;   3 : + progression;   4 : + information */
 
-#define DISPLAYUPDATE(l, ...) if (g_displayLevel>=l) { \
-            if ((clock() - g_time > refreshRate) || (g_displayLevel>=4)) \
-            { g_time = clock(); DISPLAY(__VA_ARGS__); \
-            if (g_displayLevel>=4) fflush(stderr); } }
-static const clock_t refreshRate = CLOCKS_PER_SEC * 15 / 100;
-static clock_t g_time = 0;
+#define SEC_TO_MICRO 1000000
+static const U64 g_refreshRate = SEC_TO_MICRO / 6;
+static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
+
+#define DISPLAYUPDATE(l, ...) { if (g_displayLevel>=l) { \
+            if ((UTIL_clockSpanMicro(g_displayClock) > g_refreshRate) || (g_displayLevel>=4)) \
+            { g_displayClock = UTIL_getTime(); DISPLAY(__VA_ARGS__); \
+            if (g_displayLevel>=4) fflush(stderr); } } }
 
 
 /* *************************************
diff --git a/programs/dibio.c b/programs/dibio.c
index dea3ec44f4f..9cc5264d27a 100644
--- a/programs/dibio.c
+++ b/programs/dibio.c
@@ -26,7 +26,6 @@
 #include          /* malloc, free */
 #include          /* memset */
 #include           /* fprintf, fopen, ftello64 */
-#include            /* clock_t, clock, CLOCKS_PER_SEC */
 #include           /* errno */
 
 #include "mem.h"            /* read */
@@ -55,15 +54,14 @@ static const size_t g_maxMemory = (sizeof(size_t) == 4) ? (2 GB - 64 MB) : ((siz
 #define DISPLAY(...)         fprintf(stderr, __VA_ARGS__)
 #define DISPLAYLEVEL(l, ...) if (displayLevel>=l) { DISPLAY(__VA_ARGS__); }
 
-#define DISPLAYUPDATE(l, ...) if (displayLevel>=l) { \
-            if ((DIB_clockSpan(g_time) > refreshRate) || (displayLevel>=4)) \
-            { g_time = clock(); DISPLAY(__VA_ARGS__); \
-            if (displayLevel>=4) fflush(stderr); } }
-static const clock_t refreshRate = CLOCKS_PER_SEC * 2 / 10;
-static clock_t g_time = 0;
-
-static clock_t DIB_clockSpan(clock_t nPrevious) { return clock() - nPrevious; }
+#define SEC_TO_MICRO 1000000
+static const U64 g_refreshRate = SEC_TO_MICRO / 6;
+static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
 
+#define DISPLAYUPDATE(l, ...) { if (displayLevel>=l) { \
+            if ((UTIL_clockSpanMicro(g_displayClock) > g_refreshRate) || (displayLevel>=4)) \
+            { g_displayClock = UTIL_getTime(); DISPLAY(__VA_ARGS__); \
+            if (displayLevel>=4) fflush(stderr); } } }
 
 /*-*************************************
 *  Exceptions
diff --git a/programs/fileio.c b/programs/fileio.c
index eb004f91604..0ea1ad1521d 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -29,7 +29,6 @@
 #include       /* fprintf, fopen, fread, _fileno, stdin, stdout */
 #include      /* malloc, free */
 #include      /* strcmp, strlen */
-#include        /* clock */
 #include       /* errno */
 
 #if defined (_MSC_VER)
@@ -40,6 +39,7 @@
 #include "bitstream.h"
 #include "mem.h"
 #include "fileio.h"
+#include "util.h"
 #define ZSTD_STATIC_LINKING_ONLY   /* ZSTD_magicNumber, ZSTD_frameHeaderSize_max */
 #include "zstd.h"
 #if defined(ZSTD_GZCOMPRESS) || defined(ZSTD_GZDECOMPRESS)
@@ -81,12 +81,14 @@
 static int g_displayLevel = 2;   /* 0 : no display;  1: errors;  2: + result + interaction + warnings;  3: + progression;  4: + information */
 void FIO_setNotificationLevel(unsigned level) { g_displayLevel=level; }
 
+#define SEC_TO_MICRO 1000000
+static const U64 g_refreshRate = SEC_TO_MICRO / 6;
+static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
+
 #define DISPLAYUPDATE(l, ...) { if (g_displayLevel>=l) { \
-            if ((clock() - g_time > refreshRate) || (g_displayLevel>=4)) \
-            { g_time = clock(); DISPLAY(__VA_ARGS__); \
+            if ((UTIL_clockSpanMicro(g_displayClock) > g_refreshRate) || (g_displayLevel>=4)) \
+            { g_displayClock = UTIL_getTime(); DISPLAY(__VA_ARGS__); \
             if (g_displayLevel>=4) fflush(stderr); } } }
-static const clock_t refreshRate = CLOCKS_PER_SEC * 15 / 100;
-static clock_t g_time = 0;
 
 #undef MIN  /* in case it would be already defined */
 #define MIN(a,b)    ((a) < (b) ? (a) : (b))
diff --git a/programs/util.h b/programs/util.h
index e44d7459ddf..41155694edd 100644
--- a/programs/util.h
+++ b/programs/util.h
@@ -118,6 +118,7 @@ static int g_utilDisplayLevel;
 *  Time functions
 ******************************************/
 #if defined(_WIN32)   /* Windows */
+    #define UTIL_TIME_INITIALIZER { { 0, 0 } }
     typedef LARGE_INTEGER UTIL_time_t;
     UTIL_STATIC UTIL_time_t UTIL_getTime(void) { UTIL_time_t x; QueryPerformanceCounter(&x); return x; }
     UTIL_STATIC U64 UTIL_getSpanTimeMicro(UTIL_time_t clockStart, UTIL_time_t clockEnd)
@@ -144,6 +145,7 @@ static int g_utilDisplayLevel;
     }
 #elif defined(__APPLE__) && defined(__MACH__)
     #include 
+    #define UTIL_TIME_INITIALIZER 0
     typedef U64 UTIL_time_t;
     UTIL_STATIC UTIL_time_t UTIL_getTime(void) { return mach_absolute_time(); }
     UTIL_STATIC U64 UTIL_getSpanTimeMicro(UTIL_time_t clockStart, UTIL_time_t clockEnd)
@@ -168,6 +170,7 @@ static int g_utilDisplayLevel;
     }
 #elif (PLATFORM_POSIX_VERSION >= 200112L)
     #include 
+    #define UTIL_TIME_INITIALIZER { 0, 0 }
     typedef struct timespec UTIL_freq_t;
     typedef struct timespec UTIL_time_t;
     UTIL_STATIC UTIL_time_t UTIL_getTime(void)
@@ -207,6 +210,7 @@ static int g_utilDisplayLevel;
     }
 #else   /* relies on standard C (note : clock_t measurements can be wrong when using multi-threading) */
     typedef clock_t UTIL_time_t;
+    #define UTIL_TIME_INITIALIZER 0
     UTIL_STATIC UTIL_time_t UTIL_getTime(void) { return clock(); }
     UTIL_STATIC U64 UTIL_getSpanTimeMicro(UTIL_time_t clockStart, UTIL_time_t clockEnd) { return 1000000ULL * (clockEnd - clockStart) / CLOCKS_PER_SEC; }
     UTIL_STATIC U64 UTIL_getSpanTimeNano(UTIL_time_t clockStart, UTIL_time_t clockEnd) { return 1000000000ULL * (clockEnd - clockStart) / CLOCKS_PER_SEC; }
diff --git a/tests/decodecorpus.c b/tests/decodecorpus.c
index e697e519cfc..7d7b64664f8 100644
--- a/tests/decodecorpus.c
+++ b/tests/decodecorpus.c
@@ -14,8 +14,8 @@
 #include 
 #include 
 #include 
-#include 
 
+#include "util.h"
 #include "zstd.h"
 #include "zstd_internal.h"
 #include "mem.h"
@@ -49,20 +49,17 @@ static U32 g_displayLevel = 2;
 
 #define DISPLAYUPDATE(...)                                                     \
     do {                                                                       \
-        if ((clockSpan(g_displayClock) > g_refreshRate) ||                     \
+        if ((UTIL_clockSpanMicro(g_displayClock) > g_refreshRate) ||           \
             (g_displayLevel >= 4)) {                                           \
-            g_displayClock = clock();                                          \
+            g_displayClock = UTIL_getTime();                                   \
             DISPLAY(__VA_ARGS__);                                              \
             if (g_displayLevel >= 4) fflush(stderr);                           \
         }                                                                      \
     } while (0)
-static const clock_t g_refreshRate = CLOCKS_PER_SEC / 6;
-static clock_t g_displayClock = 0;
 
-static clock_t clockSpan(clock_t cStart)
-{
-    return clock() - cStart;   /* works even when overflow; max span ~ 30mn */
-}
+#define SEC_TO_MICRO 1000000
+static const U64 g_refreshRate = SEC_TO_MICRO / 6;
+static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
 
 #define CHECKERR(code)                                                         \
     do {                                                                       \
@@ -1531,14 +1528,14 @@ static int runTestMode(U32 seed, unsigned numFiles, unsigned const testDurationS
 {
     unsigned fnum;
 
-    clock_t const startClock = clock();
-    clock_t const maxClockSpan = testDurationS * CLOCKS_PER_SEC;
+    UTIL_time_t const startClock = UTIL_getTime();
+    U64 const maxClockSpan = testDurationS * SEC_TO_MICRO;
 
     if (numFiles == 0 && !testDurationS) numFiles = 1;
 
     DISPLAY("seed: %u\n", seed);
 
-    for (fnum = 0; fnum < numFiles || clockSpan(startClock) < maxClockSpan; fnum++) {
+    for (fnum = 0; fnum < numFiles || UTIL_clockSpanMicro(startClock) < maxClockSpan; fnum++) {
         if (fnum < numFiles)
             DISPLAYUPDATE("\r%u/%u        ", fnum, numFiles);
         else
diff --git a/tests/fuzzer.c b/tests/fuzzer.c
index be9dc3e062e..980c361768f 100644
--- a/tests/fuzzer.c
+++ b/tests/fuzzer.c
@@ -25,7 +25,6 @@
 #include        /* free */
 #include         /* fgets, sscanf */
 #include        /* strcmp */
-#include          /* clock_t */
 #define ZSTD_STATIC_LINKING_ONLY  /* ZSTD_compressContinue, ZSTD_compressBlock */
 #include "zstd.h"         /* ZSTD_VERSION_STRING */
 #include "zstd_errors.h"  /* ZSTD_getErrorCode */
@@ -36,6 +35,7 @@
 #include "mem.h"
 #define XXH_STATIC_LINKING_ONLY
 #include "xxhash.h"       /* XXH64 */
+#include "util.h"
 
 
 /*-************************************
@@ -56,25 +56,23 @@ static const U32 nbTestsDefault = 30000;
 #define DISPLAYLEVEL(l, ...)  if (g_displayLevel>=l) { DISPLAY(__VA_ARGS__); }
 static U32 g_displayLevel = 2;
 
-#define DISPLAYUPDATE(l, ...) if (g_displayLevel>=l) { \
-            if ((FUZ_clockSpan(g_displayClock) > g_refreshRate) || (g_displayLevel>=4)) \
-            { g_displayClock = clock(); DISPLAY(__VA_ARGS__); \
-            if (g_displayLevel>=4) fflush(stdout); } }
-static const clock_t g_refreshRate = CLOCKS_PER_SEC / 6;
-static clock_t g_displayClock = 0;
+#define SEC_TO_MICRO 1000000
+static const U64 g_refreshRate = SEC_TO_MICRO / 6;
+static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
 
+#define DISPLAYUPDATE(l, ...) if (g_displayLevel>=l) { \
+            if ((UTIL_clockSpanMicro(g_displayClock) > g_refreshRate) || (g_displayLevel>=4)) \
+            { g_displayClock = UTIL_getTime(); DISPLAY(__VA_ARGS__); \
+            if (g_displayLevel>=4) fflush(stderr); } }
 
 /*-*******************************************************
 *  Fuzzer functions
 *********************************************************/
+#undef MIN
+#undef MAX
 #define MIN(a,b) ((a)<(b)?(a):(b))
 #define MAX(a,b) ((a)>(b)?(a):(b))
 
-static clock_t FUZ_clockSpan(clock_t cStart)
-{
-    return clock() - cStart;   /* works even when overflow; max span ~ 30mn */
-}
-
 #define FUZ_rotl32(x,r) ((x << r) | (x >> (32 - r)))
 static unsigned FUZ_rand(unsigned* src)
 {
@@ -1208,8 +1206,8 @@ static int fuzzerTests(U32 seed, U32 nbTests, unsigned startTest, U32 const maxD
     U32 result = 0;
     U32 testNb = 0;
     U32 coreSeed = seed, lseed = 0;
-    clock_t const startClock = clock();
-    clock_t const maxClockSpan = maxDurationS * CLOCKS_PER_SEC;
+    UTIL_time_t const startClock = UTIL_getTime();
+    U64 const maxClockSpan = maxDurationS * SEC_TO_MICRO;
     int const cLevelLimiter = bigTests ? 3 : 2;
 
     /* allocation */
@@ -1234,7 +1232,7 @@ static int fuzzerTests(U32 seed, U32 nbTests, unsigned startTest, U32 const maxD
     for (testNb=1; testNb < startTest; testNb++) FUZ_rand(&coreSeed);
 
     /* main test loop */
-    for ( ; (testNb <= nbTests) || (FUZ_clockSpan(startClock) < maxClockSpan); testNb++ ) {
+    for ( ; (testNb <= nbTests) || (UTIL_clockSpanMicro(startClock) < maxClockSpan); testNb++ ) {
         size_t sampleSize, maxTestSize, totalTestSize;
         size_t cSize, totalCSize, totalGenSize;
         U64 crcOrig;
diff --git a/tests/paramgrill.c b/tests/paramgrill.c
index 081a284e42f..e258932dabb 100644
--- a/tests/paramgrill.c
+++ b/tests/paramgrill.c
@@ -17,13 +17,14 @@
 #include      /* fprintf, fopen, ftello64 */
 #include     /* strcmp */
 #include       /* log */
-#include       /* clock_t */
+#include 
 
 #include "mem.h"
 #define ZSTD_STATIC_LINKING_ONLY   /* ZSTD_parameters, ZSTD_estimateCCtxSize */
 #include "zstd.h"
 #include "datagen.h"
 #include "xxhash.h"
+#include "util.h"
 
 
 /*-************************************
@@ -38,8 +39,9 @@
 #define MB *(1<<20)
 #define GB *(1ULL<<30)
 
+#define SEC_TO_MICRO 1000000
 #define NBLOOPS    2
-#define TIMELOOP  (2 * CLOCKS_PER_SEC)
+#define TIMELOOP  (2 * SEC_TO_MICRO)
 
 #define NB_LEVELS_TRACKED 30
 
@@ -49,8 +51,8 @@ static const size_t maxMemory = (sizeof(size_t)==4)  ?  (2 GB - 64 MB) : (size_t
 static const size_t sampleSize = 10000000;
 
 static const double g_grillDuration_s = 90000;   /* about 24 hours */
-static const clock_t g_maxParamTime = 15 * CLOCKS_PER_SEC;
-static const clock_t g_maxVariationTime = 60 * CLOCKS_PER_SEC;
+static const U64 g_maxParamTime = 15 * SEC_TO_MICRO;
+static const U64 g_maxVariationTime = 60 * SEC_TO_MICRO;
 static const int g_maxNbVariations = 64;
 
 
@@ -88,13 +90,9 @@ void BMK_SetNbIterations(int nbLoops)
 *  Private functions
 *********************************************************/
 
-/* works even if overflow ; max span ~ 30 mn */
-static clock_t BMK_clockSpan(clock_t cStart) { return clock() - cStart; }
-
 /* accuracy in seconds only, span can be multiple years */
 static double BMK_timeSpan(time_t tStart) { return difftime(time(NULL), tStart); }
 
-
 static size_t BMK_findMaxMem(U64 requiredMem)
 {
     size_t const step = 64 MB;
@@ -221,7 +219,7 @@ static size_t BMK_benchParam(BMK_result_t* resultPtr,
         size_t cSize = 0;
         double fastestC = 100000000., fastestD = 100000000.;
         double ratio = 0.;
-        clock_t const benchStart = clock();
+        UTIL_time_t const benchStart = UTIL_getTime();
 
         DISPLAY("\r%79s\r", "");
         memset(¶ms, 0, sizeof(params));
@@ -229,9 +227,10 @@ static size_t BMK_benchParam(BMK_result_t* resultPtr,
         for (loopNb = 1; loopNb <= g_nbIterations; loopNb++) {
             int nbLoops;
             U32 blockNb;
-            clock_t roundStart, roundClock;
+            UTIL_time_t roundStart;
+            U64 roundClock;
 
-            { clock_t const benchTime = BMK_clockSpan(benchStart);
+            { U64 const benchTime = UTIL_clockSpanMicro(benchStart);
               if (benchTime > g_maxParamTime) break; }
 
             /* Compression */
@@ -239,10 +238,9 @@ static size_t BMK_benchParam(BMK_result_t* resultPtr,
             memset(compressedBuffer, 0xE5, maxCompressedSize);
 
             nbLoops = 0;
-            roundStart = clock();
-            while (clock() == roundStart);
-            roundStart = clock();
-            while (BMK_clockSpan(roundStart) < TIMELOOP) {
+            UTIL_waitForNextTick();
+            roundStart = UTIL_getTime();
+            while (UTIL_clockSpanMicro(roundStart) < TIMELOOP) {
                 for (blockNb=0; blockNb", loopNb, name, (U32)srcSize);
             DISPLAY(" %9u (%4.3f),%7.1f MB/s", (U32)cSize, ratio, (double)srcSize / fastestC / 1000000.);
@@ -269,17 +267,16 @@ static size_t BMK_benchParam(BMK_result_t* resultPtr,
             memset(resultBuffer, 0xD6, srcSize);
 
             nbLoops = 0;
-            roundStart = clock();
-            while (clock() == roundStart);
-            roundStart = clock();
-            for ( ; BMK_clockSpan(roundStart) < TIMELOOP; nbLoops++) {
+            UTIL_waitForNextTick();
+            roundStart = UTIL_getTime();
+            for ( ; UTIL_clockSpanMicro(roundStart) < TIMELOOP; nbLoops++) {
                 for (blockNb=0; blockNb ", loopNb, name, (U32)srcSize);
             DISPLAY("%9u (%4.3f),%7.1f MB/s, ", (U32)cSize, ratio, (double)srcSize / fastestC / 1000000.);
@@ -521,9 +518,9 @@ static void playAround(FILE* f, winnerInfo_t* winners,
                        ZSTD_CCtx* ctx)
 {
     int nbVariations = 0;
-    clock_t const clockStart = clock();
+    UTIL_time_t const clockStart = UTIL_getTime();
 
-    while (BMK_clockSpan(clockStart) < g_maxVariationTime) {
+    while (UTIL_clockSpanMicro(clockStart) < g_maxVariationTime) {
         ZSTD_compressionParameters p = params;
 
         if (nbVariations++ > g_maxNbVariations) break;
diff --git a/tests/zbufftest.c b/tests/zbufftest.c
index ce5e5518a5e..f770c8395d2 100644
--- a/tests/zbufftest.c
+++ b/tests/zbufftest.c
@@ -24,7 +24,6 @@
 **************************************/
 #include        /* free */
 #include         /* fgets, sscanf */
-#include          /* clock_t, clock() */
 #include        /* strcmp */
 #include "mem.h"
 #define ZSTD_STATIC_LINKING_ONLY   /* ZSTD_maxCLevel */
@@ -34,6 +33,7 @@
 #include "datagen.h"      /* RDG_genBuffer */
 #define XXH_STATIC_LINKING_ONLY
 #include "xxhash.h"       /* XXH64_* */
+#include "util.h"
 
 
 /*-************************************
@@ -58,26 +58,25 @@ static const U32 prime2 = 2246822519U;
 #define DISPLAYLEVEL(l, ...)  if (g_displayLevel>=l) { DISPLAY(__VA_ARGS__); }
 static U32 g_displayLevel = 2;
 
+#define SEC_TO_MICRO 1000000
+static const U64 g_refreshRate = SEC_TO_MICRO / 6;
+static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
+
 #define DISPLAYUPDATE(l, ...) if (g_displayLevel>=l) { \
-            if ((FUZ_GetClockSpan(g_displayClock) > g_refreshRate) || (g_displayLevel>=4)) \
-            { g_displayClock = clock(); DISPLAY(__VA_ARGS__); \
+            if ((UTIL_clockSpanMicro(g_displayClock) > g_refreshRate) || (g_displayLevel>=4)) \
+            { g_displayClock = UTIL_getTime(); DISPLAY(__VA_ARGS__); \
             if (g_displayLevel>=4) fflush(stderr); } }
-static const clock_t g_refreshRate = CLOCKS_PER_SEC * 15 / 100;
-static clock_t g_displayClock = 0;
 
-static clock_t g_clockTime = 0;
+static U64 g_clockTime = 0;
 
 
 /*-*******************************************************
 *  Fuzzer functions
 *********************************************************/
+#undef MIN
+#undef MAX
+#define MIN(a,b) ((a)<(b)?(a):(b))
 #define MAX(a,b) ((a)>(b)?(a):(b))
-
-static clock_t FUZ_GetClockSpan(clock_t clockStart)
-{
-    return clock() - clockStart;  /* works even when overflow. Max span ~ 30 mn */
-}
-
 /*! FUZ_rand() :
     @return : a 27 bits random value, from a 32-bits `seed`.
     `seed` is also modified */
@@ -256,8 +255,6 @@ static size_t FUZ_randomLength(U32* seed, U32 maxLog)
     return FUZ_rLogLength(seed, logLength);
 }
 
-#define MIN(a,b)   ( (a) < (b) ? (a) : (b) )
-
 #define CHECK(cond, ...) if (cond) { DISPLAY("Error => "); DISPLAY(__VA_ARGS__); \
                          DISPLAY(" (seed %u, test nb %u)  \n", seed, testNb); goto _output_error; }
 
@@ -278,7 +275,7 @@ static int fuzzerTests(U32 seed, U32 nbTests, unsigned startTest, double compres
     U32 coreSeed = seed;
     ZBUFF_CCtx* zc;
     ZBUFF_DCtx* zd;
-    clock_t startClock = clock();
+    UTIL_time_t startClock = UTIL_getTime();
 
     /* allocations */
     zc = ZBUFF_createCCtx();
@@ -308,7 +305,7 @@ static int fuzzerTests(U32 seed, U32 nbTests, unsigned startTest, double compres
         FUZ_rand(&coreSeed);
 
     /* test loop */
-    for ( ; (testNb <= nbTests) || (FUZ_GetClockSpan(startClock) < g_clockTime) ; testNb++ ) {
+    for ( ; (testNb <= nbTests) || (UTIL_clockSpanMicro(startClock) < g_clockTime) ; testNb++ ) {
         U32 lseed;
         const BYTE* srcBuffer;
         const BYTE* dict;
@@ -548,7 +545,7 @@ int main(int argc, const char** argv)
                     }
                     if (*argument=='m') g_clockTime *=60, argument++;
                     if (*argument=='n') argument++;
-                    g_clockTime *= CLOCKS_PER_SEC;
+                    g_clockTime *= SEC_TO_MICRO;
                     break;
 
                 case 's':
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index b8c437a4197..aad9cba71de 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -24,7 +24,6 @@
 **************************************/
 #include        /* free */
 #include         /* fgets, sscanf */
-#include          /* clock_t, clock() */
 #include        /* strcmp */
 #include        /* assert */
 #include "mem.h"
@@ -37,6 +36,7 @@
 #define XXH_STATIC_LINKING_ONLY   /* XXH64_state_t */
 #include "xxhash.h"       /* XXH64_* */
 #include "seqgen.h"
+#include "util.h"
 
 
 /*-************************************
@@ -62,26 +62,25 @@ static const U32 prime32 = 2654435761U;
                                   if (g_displayLevel>=4) fflush(stderr); }
 static U32 g_displayLevel = 2;
 
+#define SEC_TO_MICRO 1000000
+static const U64 g_refreshRate = SEC_TO_MICRO / 6;
+static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
+
 #define DISPLAYUPDATE(l, ...) if (g_displayLevel>=l) { \
-            if ((FUZ_GetClockSpan(g_displayClock) > g_refreshRate) || (g_displayLevel>=4)) \
-            { g_displayClock = clock(); DISPLAY(__VA_ARGS__); \
-              if (g_displayLevel>=4) fflush(stderr); } }
-static const clock_t g_refreshRate = CLOCKS_PER_SEC / 6;
-static clock_t g_displayClock = 0;
+            if ((UTIL_clockSpanMicro(g_displayClock) > g_refreshRate) || (g_displayLevel>=4)) \
+            { g_displayClock = UTIL_getTime(); DISPLAY(__VA_ARGS__); \
+            if (g_displayLevel>=4) fflush(stderr); } }
 
-static clock_t g_clockTime = 0;
+static U64 g_clockTime = 0;
 
 
 /*-*******************************************************
 *  Fuzzer functions
 *********************************************************/
+#undef MIN
+#undef MAX
+#define MIN(a,b) ((a)<(b)?(a):(b))
 #define MAX(a,b) ((a)>(b)?(a):(b))
-
-static clock_t FUZ_GetClockSpan(clock_t clockStart)
-{
-    return clock() - clockStart;  /* works even when overflow. Max span ~ 30 mn */
-}
-
 /*! FUZ_rand() :
     @return : a 27 bits random value, from a 32-bits `seed`.
     `seed` is also modified */
@@ -815,8 +814,6 @@ static size_t FUZ_randomLength(U32* seed, U32 maxLog)
     return FUZ_rLogLength(seed, logLength);
 }
 
-#define MIN(a,b)   ( (a) < (b) ? (a) : (b) )
-
 /* Return value in range minVal <= v <= maxVal */
 static U32 FUZ_randomClampedLength(U32* seed, U32 minVal, U32 maxVal)
 {
@@ -842,7 +839,7 @@ static int fuzzerTests(U32 seed, U32 nbTests, unsigned startTest, double compres
     ZSTD_CStream* zc = ZSTD_createCStream();   /* will be re-created sometimes */
     ZSTD_DStream* zd = ZSTD_createDStream();   /* will be re-created sometimes */
     ZSTD_DStream* const zd_noise = ZSTD_createDStream();
-    clock_t const startClock = clock();
+    UTIL_time_t const startClock = UTIL_getTime();
     const BYTE* dict = NULL;  /* can keep same dict on 2 consecutive tests */
     size_t dictSize = 0;
     U32 oldTestLog = 0;
@@ -872,7 +869,7 @@ static int fuzzerTests(U32 seed, U32 nbTests, unsigned startTest, double compres
         FUZ_rand(&coreSeed);
 
     /* test loop */
-    for ( ; (testNb <= nbTests) || (FUZ_GetClockSpan(startClock) < g_clockTime) ; testNb++ ) {
+    for ( ; (testNb <= nbTests) || (UTIL_clockSpanMicro(startClock) < g_clockTime) ; testNb++ ) {
         U32 lseed;
         const BYTE* srcBuffer;
         size_t totalTestSize, totalGenSize, cSize;
@@ -1092,7 +1089,7 @@ static int fuzzerTests_MT(U32 seed, U32 nbTests, unsigned startTest, double comp
     ZSTDMT_CCtx* zc = ZSTDMT_createCCtx(nbThreads);   /* will be reset sometimes */
     ZSTD_DStream* zd = ZSTD_createDStream();   /* will be reset sometimes */
     ZSTD_DStream* const zd_noise = ZSTD_createDStream();
-    clock_t const startClock = clock();
+    UTIL_time_t const startClock = UTIL_getTime();
     const BYTE* dict=NULL;   /* can keep same dict on 2 consecutive tests */
     size_t dictSize = 0;
     U32 oldTestLog = 0;
@@ -1123,7 +1120,7 @@ static int fuzzerTests_MT(U32 seed, U32 nbTests, unsigned startTest, double comp
         FUZ_rand(&coreSeed);
 
     /* test loop */
-    for ( ; (testNb <= nbTests) || (FUZ_GetClockSpan(startClock) < g_clockTime) ; testNb++ ) {
+    for ( ; (testNb <= nbTests) || (UTIL_clockSpanMicro(startClock) < g_clockTime) ; testNb++ ) {
         U32 lseed;
         const BYTE* srcBuffer;
         size_t totalTestSize, totalGenSize, cSize;
@@ -1364,7 +1361,7 @@ static int fuzzerTests_newAPI(U32 seed, U32 nbTests, unsigned startTest, double
     ZSTD_CCtx* zc = ZSTD_createCCtx();   /* will be reset sometimes */
     ZSTD_DStream* zd = ZSTD_createDStream();   /* will be reset sometimes */
     ZSTD_DStream* const zd_noise = ZSTD_createDStream();
-    clock_t const startClock = clock();
+    UTIL_time_t const startClock = UTIL_getTime();
     const BYTE* dict = NULL;   /* can keep same dict on 2 consecutive tests */
     size_t dictSize = 0;
     U32 oldTestLog = 0;
@@ -1397,7 +1394,7 @@ static int fuzzerTests_newAPI(U32 seed, U32 nbTests, unsigned startTest, double
         FUZ_rand(&coreSeed);
 
     /* test loop */
-    for ( ; (testNb <= nbTests) || (FUZ_GetClockSpan(startClock) < g_clockTime) ; testNb++ ) {
+    for ( ; (testNb <= nbTests) || (UTIL_clockSpanMicro(startClock) < g_clockTime) ; testNb++ ) {
         U32 lseed;
         const BYTE* srcBuffer;
         size_t totalTestSize, totalGenSize, cSize;
@@ -1772,7 +1769,7 @@ int main(int argc, const char** argv)
                         g_clockTime *=60, argument++;
                         if (*argument=='n') argument++; /* -T1mn == -T60 */
                     } else if (*argument=='s') argument++; /* -T10s == -T10 */
-                    g_clockTime *= CLOCKS_PER_SEC;
+                    g_clockTime *= SEC_TO_MICRO;
                     break;
 
                 case 's':   /* manually select seed */

From dab8cfa3c73be56ad3424f0299f1bb3740ac104d Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Thu, 30 Nov 2017 19:40:53 -0800
Subject: [PATCH 095/145] Combine definitions of SEC_TO_MICRO

---
 programs/bench.c     | 1 -
 programs/dibio.c     | 1 -
 programs/fileio.c    | 1 -
 programs/util.h      | 1 +
 tests/decodecorpus.c | 1 -
 tests/fuzzer.c       | 1 -
 tests/paramgrill.c   | 1 -
 tests/zbufftest.c    | 1 -
 tests/zstreamtest.c  | 1 -
 9 files changed, 1 insertion(+), 8 deletions(-)

diff --git a/programs/bench.c b/programs/bench.c
index 384b982f400..63b98e53220 100644
--- a/programs/bench.c
+++ b/programs/bench.c
@@ -71,7 +71,6 @@ static U32 g_compressibilityDefault = 50;
 #define DISPLAYLEVEL(l, ...) if (g_displayLevel>=l) { DISPLAY(__VA_ARGS__); }
 static int g_displayLevel = 2;   /* 0 : no display;   1: errors;   2 : + result + interaction + warnings;   3 : + progression;   4 : + information */
 
-#define SEC_TO_MICRO 1000000
 static const U64 g_refreshRate = SEC_TO_MICRO / 6;
 static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
 
diff --git a/programs/dibio.c b/programs/dibio.c
index 9cc5264d27a..112259ddcd0 100644
--- a/programs/dibio.c
+++ b/programs/dibio.c
@@ -54,7 +54,6 @@ static const size_t g_maxMemory = (sizeof(size_t) == 4) ? (2 GB - 64 MB) : ((siz
 #define DISPLAY(...)         fprintf(stderr, __VA_ARGS__)
 #define DISPLAYLEVEL(l, ...) if (displayLevel>=l) { DISPLAY(__VA_ARGS__); }
 
-#define SEC_TO_MICRO 1000000
 static const U64 g_refreshRate = SEC_TO_MICRO / 6;
 static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
 
diff --git a/programs/fileio.c b/programs/fileio.c
index 0ea1ad1521d..c9b6b04e1a1 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -81,7 +81,6 @@
 static int g_displayLevel = 2;   /* 0 : no display;  1: errors;  2: + result + interaction + warnings;  3: + progression;  4: + information */
 void FIO_setNotificationLevel(unsigned level) { g_displayLevel=level; }
 
-#define SEC_TO_MICRO 1000000
 static const U64 g_refreshRate = SEC_TO_MICRO / 6;
 static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
 
diff --git a/programs/util.h b/programs/util.h
index 41155694edd..37098f2f725 100644
--- a/programs/util.h
+++ b/programs/util.h
@@ -216,6 +216,7 @@ static int g_utilDisplayLevel;
     UTIL_STATIC U64 UTIL_getSpanTimeNano(UTIL_time_t clockStart, UTIL_time_t clockEnd) { return 1000000000ULL * (clockEnd - clockStart) / CLOCKS_PER_SEC; }
 #endif
 
+#define SEC_TO_MICRO 1000000
 
 /* returns time span in microseconds */
 UTIL_STATIC U64 UTIL_clockSpanMicro( UTIL_time_t clockStart )
diff --git a/tests/decodecorpus.c b/tests/decodecorpus.c
index 7d7b64664f8..407653119dd 100644
--- a/tests/decodecorpus.c
+++ b/tests/decodecorpus.c
@@ -57,7 +57,6 @@ static U32 g_displayLevel = 2;
         }                                                                      \
     } while (0)
 
-#define SEC_TO_MICRO 1000000
 static const U64 g_refreshRate = SEC_TO_MICRO / 6;
 static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
 
diff --git a/tests/fuzzer.c b/tests/fuzzer.c
index 980c361768f..46671a15cd4 100644
--- a/tests/fuzzer.c
+++ b/tests/fuzzer.c
@@ -56,7 +56,6 @@ static const U32 nbTestsDefault = 30000;
 #define DISPLAYLEVEL(l, ...)  if (g_displayLevel>=l) { DISPLAY(__VA_ARGS__); }
 static U32 g_displayLevel = 2;
 
-#define SEC_TO_MICRO 1000000
 static const U64 g_refreshRate = SEC_TO_MICRO / 6;
 static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
 
diff --git a/tests/paramgrill.c b/tests/paramgrill.c
index e258932dabb..ae14aa07481 100644
--- a/tests/paramgrill.c
+++ b/tests/paramgrill.c
@@ -39,7 +39,6 @@
 #define MB *(1<<20)
 #define GB *(1ULL<<30)
 
-#define SEC_TO_MICRO 1000000
 #define NBLOOPS    2
 #define TIMELOOP  (2 * SEC_TO_MICRO)
 
diff --git a/tests/zbufftest.c b/tests/zbufftest.c
index f770c8395d2..9b6f7bad6dc 100644
--- a/tests/zbufftest.c
+++ b/tests/zbufftest.c
@@ -58,7 +58,6 @@ static const U32 prime2 = 2246822519U;
 #define DISPLAYLEVEL(l, ...)  if (g_displayLevel>=l) { DISPLAY(__VA_ARGS__); }
 static U32 g_displayLevel = 2;
 
-#define SEC_TO_MICRO 1000000
 static const U64 g_refreshRate = SEC_TO_MICRO / 6;
 static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
 
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index aad9cba71de..207b2e01575 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -62,7 +62,6 @@ static const U32 prime32 = 2654435761U;
                                   if (g_displayLevel>=4) fflush(stderr); }
 static U32 g_displayLevel = 2;
 
-#define SEC_TO_MICRO 1000000
 static const U64 g_refreshRate = SEC_TO_MICRO / 6;
 static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
 

From 453fb5e830356559b8a54a5655219a874a39509e Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 1 Dec 2017 10:30:53 -0800
Subject: [PATCH 096/145] removed -ftrapv from tests/ debug flags

-ftrapv is apparently buggy for gcc.
versions >= 5 are supposed to work better,
but even then, some complaints say it's still flaky when optimizations are enabled.
I even saw a post saying it only works if one creates its own signal handler,
which would make this flag no longer transparent.

on clang, it seems to work correctly.
But we would need to add a method to selectively add flags depending on compiler.
That's too much troubles for the intended benefit
(just catch integer overflows, which we can also do using ubsan).
---
 tests/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/Makefile b/tests/Makefile
index 8cc953ec84a..f944fd5a867 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -32,7 +32,7 @@ CFLAGS     += -Wall -Wextra -Wcast-qual -Wcast-align -Wshadow                 \
               -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement \
               -Wstrict-prototypes -Wundef -Wformat-security                   \
               -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings      \
-              -Wredundant-decls -ftrapv
+              -Wredundant-decls
 CFLAGS     += $(DEBUGFLAGS) $(MOREFLAGS)
 FLAGS       = $(CPPFLAGS) $(CFLAGS) $(LDFLAGS)
 

From 4413ed27e450d7154ffbfa7c17b9313c2fd466fa Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 1 Dec 2017 16:27:24 -0800
Subject: [PATCH 097/145] circleci : make libzstd-nomt had its own test

---
 circle.yml | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/circle.yml b/circle.yml
index b488c1a7b8a..7e042d1041e 100644
--- a/circle.yml
+++ b/circle.yml
@@ -9,7 +9,7 @@ dependencies:
 test:
   override:
     - ? |
-        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then cc -v; make all   && make clean && make -C lib libzstd-nomt && make clean; fi &&
+        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then cc -v; make all   && make clean; fi &&
         if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make gnu90build   && make clean; fi
       :
         parallel: true
@@ -58,6 +58,10 @@ test:
         if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then true; fi # Could add another test here
       :
         parallel: true
+    - ? |
+        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make -C lib libzstd-nomt && make clean; fi
+      :
+        parallel: true
 
   post:
     - echo Circle CI tests finished

From 519cf3b400b6f87578938ceff3762dc4d85ad07c Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 1 Dec 2017 16:31:40 -0800
Subject: [PATCH 098/145] circleci: add CFLAGS=-O0 environment variable

for faster compilation
---
 circle.yml | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/circle.yml b/circle.yml
index 7e042d1041e..e7f17979cad 100644
--- a/circle.yml
+++ b/circle.yml
@@ -1,3 +1,7 @@
+machine:
+  environment:
+    CFLAGS: -O0   # for faster compilation, will preserve warnings
+
 dependencies:
   override:
     - sudo dpkg --add-architecture i386

From 363355448b5032d062673cb5faf56a742571d190 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 1 Dec 2017 17:02:55 -0800
Subject: [PATCH 099/145] circleci: removed one test node

---
 circle.yml | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/circle.yml b/circle.yml
index e7f17979cad..cc494905cc8 100644
--- a/circle.yml
+++ b/circle.yml
@@ -59,11 +59,7 @@ test:
         parallel: true
     - ? |
         if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make -j regressiontest && make clean; fi &&
-        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then true; fi # Could add another test here
-      :
-        parallel: true
-    - ? |
-        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make -C lib libzstd-nomt && make clean; fi
+        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make -C lib libzstd-nomt && make clean; fi
       :
         parallel: true
 

From 30eeb248a8e85dc60030003dee2677af8c6709c6 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 1 Dec 2017 17:22:04 -0800
Subject: [PATCH 100/145] added environment variable through circleci dashboard

check if it works better
---
 circle.yml | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/circle.yml b/circle.yml
index cc494905cc8..478775b7031 100644
--- a/circle.yml
+++ b/circle.yml
@@ -1,7 +1,3 @@
-machine:
-  environment:
-    CFLAGS: -O0   # for faster compilation, will preserve warnings
-
 dependencies:
   override:
     - sudo dpkg --add-architecture i386

From 74991c2764f32482815569a8efe4ee1e61bdfa71 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 1 Dec 2017 17:28:24 -0800
Subject: [PATCH 101/145] circleci: removed gccX/clang tests

does no longer need to install these large packages
---
 circle.yml | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)

diff --git a/circle.yml b/circle.yml
index 478775b7031..85061faa6cd 100644
--- a/circle.yml
+++ b/circle.yml
@@ -3,7 +3,6 @@ dependencies:
     - sudo dpkg --add-architecture i386
     - sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test; sudo apt-get -y -qq update
     - sudo apt-get -y install gcc-powerpc-linux-gnu gcc-arm-linux-gnueabi libc6-dev-armel-cross gcc-aarch64-linux-gnu libc6-dev-arm64-cross
-    - sudo apt-get -y install libstdc++-7-dev clang gcc g++ gcc-5 gcc-6 gcc-7 zlib1g-dev liblzma-dev
     - sudo apt-get -y install linux-libc-dev:i386 libc6-dev-i386
 
 test:
@@ -25,12 +24,7 @@ test:
         parallel: true
     - ? |
         if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make cxxtest      && make clean; fi &&
-        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make gcc5build    && make clean; fi
-      :
-        parallel: true
-    - ? |
-        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make gcc6build    && make clean; fi &&
-        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make clangbuild   && make clean; fi
+        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make ppc64build   && make clean; fi
       :
         parallel: true
     - ? |
@@ -43,11 +37,6 @@ test:
         if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make ppcbuild     && make clean; fi
       :
         parallel: true
-    - ? |
-        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make ppc64build   && make clean; fi &&
-        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make gcc7build    && make clean; fi
-      :
-        parallel: true
     - ? |
         if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make shortest     && make clean; fi &&
         if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make -C tests test-legacy test-longmatch test-symbols && make clean; fi

From c6f5bf071cd3e246b3a4dae990489d3294e499eb Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 1 Dec 2017 17:35:19 -0800
Subject: [PATCH 102/145] fixed powerpc test

misspelled CFLAGS
---
 Makefile   | 2 +-
 circle.yml | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 9c265705271..c8f0ed4d4d4 100644
--- a/Makefile
+++ b/Makefile
@@ -161,7 +161,7 @@ aarch64build: clean
 	CC=aarch64-linux-gnu-gcc CFLAGS="-Werror" $(MAKE) allzstd
 
 ppcbuild: clean
-	CC=powerpc-linux-gnu-gcc CLAGS="-m32 -Wno-attributes -Werror" $(MAKE) allzstd
+	CC=powerpc-linux-gnu-gcc CFLAGS="-m32 -Wno-attributes -Werror" $(MAKE) allzstd
 
 ppc64build: clean
 	CC=powerpc-linux-gnu-gcc CFLAGS="-m64 -Werror" $(MAKE) allzstd
diff --git a/circle.yml b/circle.yml
index 85061faa6cd..3b9faccedee 100644
--- a/circle.yml
+++ b/circle.yml
@@ -8,7 +8,7 @@ dependencies:
 test:
   override:
     - ? |
-        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then cc -v; make all   && make clean; fi &&
+        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then cc -v; CFLAGS="-O0 -Werror" make all && make clean; fi &&
         if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make gnu90build   && make clean; fi
       :
         parallel: true

From 71f012e5bfbebe91ca89d9fcc5700a26f0dadc87 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 1 Dec 2017 17:42:46 -0800
Subject: [PATCH 103/145] zstdcli: fixed minor warning when bench module not
 enabled

one variable defined but not used
---
 programs/zstdcli.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/programs/zstdcli.c b/programs/zstdcli.c
index 91af49e2484..8b4186207bd 100644
--- a/programs/zstdcli.c
+++ b/programs/zstdcli.c
@@ -773,8 +773,9 @@ int main(int argCount, const char* argv[])
             BMK_setLdmHashEveryLog(g_ldmHashEveryLog);
         }
         BMK_benchFiles(filenameTable, filenameIdx, dictFileName, cLevel, cLevelLast, &compressionParams);
+#else
+        (void)bench_nbSeconds; (void)blockSize; (void)setRealTimePrio; (void)separateFiles;
 #endif
-        (void)bench_nbSeconds; (void)blockSize; (void)setRealTimePrio;
         goto _end;
     }
 

From 6127592391bfda25154a5167954eda6b38e6efde Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 1 Dec 2017 17:56:16 -0800
Subject: [PATCH 104/145] circleci : removed cmaketest

strange error on pthread detection
(seems a bug in cmake detection, since
`make zstd` works perfectly with `-pthread` on same conf)
---
 circle.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/circle.yml b/circle.yml
index 3b9faccedee..07f2990ddee 100644
--- a/circle.yml
+++ b/circle.yml
@@ -19,7 +19,7 @@ test:
         parallel: true
     - ? |
         if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make c11build     && make clean; fi &&
-        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make cmakebuild   && make clean; fi
+        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make -C lib libzstd-nomt && make clean; fi
       :
         parallel: true
     - ? |
@@ -44,7 +44,7 @@ test:
         parallel: true
     - ? |
         if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make -j regressiontest && make clean; fi &&
-        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make -C lib libzstd-nomt && make clean; fi
+        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then true; fi   # can add one more test here
       :
         parallel: true
 

From 272c14233032eb02b79f8d2e69364a9e90ed7ae7 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 1 Dec 2017 18:11:34 -0800
Subject: [PATCH 105/145] circleci : balance some tests

try to group them by "equivalent" duration
since next job will not start as long as previous one is not completed on both VM
---
 circle.yml | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/circle.yml b/circle.yml
index 07f2990ddee..9b87ea565e9 100644
--- a/circle.yml
+++ b/circle.yml
@@ -19,11 +19,6 @@ test:
         parallel: true
     - ? |
         if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make c11build     && make clean; fi &&
-        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make -C lib libzstd-nomt && make clean; fi
-      :
-        parallel: true
-    - ? |
-        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make cxxtest      && make clean; fi &&
         if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make ppc64build   && make clean; fi
       :
         parallel: true
@@ -42,6 +37,11 @@ test:
         if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make -C tests test-legacy test-longmatch test-symbols && make clean; fi
       :
         parallel: true
+    - ? |
+        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make cxxtest      && make clean; fi &&
+        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make -C lib libzstd-nomt && make clean; fi
+      :
+        parallel: true
     - ? |
         if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make -j regressiontest && make clean; fi &&
         if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then true; fi   # can add one more test here

From 6f1377301f7a33f39c9bbd9354730141f627733e Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 1 Dec 2017 18:34:22 -0800
Subject: [PATCH 106/145] circleci: removed -m32 tests

installing dependencies costs ~2mn, making it the longest test
(test itself was < 15sec)
---
 circle.yml | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/circle.yml b/circle.yml
index 9b87ea565e9..ed50d59e5d9 100644
--- a/circle.yml
+++ b/circle.yml
@@ -3,7 +3,6 @@ dependencies:
     - sudo dpkg --add-architecture i386
     - sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test; sudo apt-get -y -qq update
     - sudo apt-get -y install gcc-powerpc-linux-gnu gcc-arm-linux-gnueabi libc6-dev-armel-cross gcc-aarch64-linux-gnu libc6-dev-arm64-cross
-    - sudo apt-get -y install linux-libc-dev:i386 libc6-dev-i386
 
 test:
   override:
@@ -23,13 +22,13 @@ test:
       :
         parallel: true
     - ? |
-        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make m32build     && make clean; fi &&
-        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make armbuild     && make clean; fi
+        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make aarch64build && make clean; fi &&
+        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make ppcbuild     && make clean; fi
       :
         parallel: true
     - ? |
-        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make aarch64build && make clean; fi &&
-        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make ppcbuild     && make clean; fi
+        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make -j regressiontest && make clean; fi &&
+        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make armbuild     && make clean; fi
       :
         parallel: true
     - ? |
@@ -42,11 +41,6 @@ test:
         if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make -C lib libzstd-nomt && make clean; fi
       :
         parallel: true
-    - ? |
-        if [[ "$CIRCLE_NODE_INDEX" == "0" ]]                                    ; then make -j regressiontest && make clean; fi &&
-        if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then true; fi   # can add one more test here
-      :
-        parallel: true
 
   post:
     - echo Circle CI tests finished

From 5e1f34b7e4c407f7f733c09c435141d18de2dc18 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 1 Dec 2017 21:17:09 -0800
Subject: [PATCH 107/145] setParameter : no side-effect on setting a
 compression parameter

last such side-effect was modifying cctx->loadedDictEnd on setting forceWindow.
It is no a useless operation, so it's removed.
No side-effect left when setting a compression parameter.
---
 lib/compress/zstd_compress.c   | 37 ++++------------------------------
 lib/compress/zstdmt_compress.c |  2 +-
 2 files changed, 5 insertions(+), 34 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index bc2d8a576bf..8d493852ffa 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -270,9 +270,8 @@ size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, unsigned v
         return ZSTD_CCtxParam_setParameter(&cctx->requestedParams, param, value);
 
     case ZSTD_p_forceMaxWindow :  /* Force back-references to remain < windowSize,
-                                   * even when referencing into Dictionary content
+                                   * even when referencing into Dictionary content.
                                    * default : 0 when using a CDict, 1 when using a Prefix */
-        cctx->loadedDictEnd = 0;  /* ? */
         return ZSTD_CCtxParam_setParameter(&cctx->requestedParams, param, value);
 
     case ZSTD_p_nbThreads:
@@ -461,13 +460,9 @@ size_t ZSTD_CCtxParam_setParameter(
     }
 }
 
-/**
- * This function should be updated whenever ZSTD_CCtx_params is updated.
- * Parameters are copied manually before the dictionary is loaded.
- * The multithreading parameters jobSize and overlapSizeLog are set only if
- * nbThreads > 1.
- *
- * pledgedSrcSize is considered unknown
+/** ZSTD_CCtx_setParametersUsingCCtxParams() :
+ *  just applies `params` into `cctx`
+ *  no action is performed, parameters are merely stored.
  */
 size_t ZSTD_CCtx_setParametersUsingCCtxParams(
         ZSTD_CCtx* cctx, const ZSTD_CCtx_params* params)
@@ -475,31 +470,8 @@ size_t ZSTD_CCtx_setParametersUsingCCtxParams(
     if (cctx->streamStage != zcss_init) return ERROR(stage_wrong);
     if (cctx->cdict) return ERROR(stage_wrong);
 
-#if 1
     cctx->requestedParams = *params;
-#else
-    /* Assume the compression and frame parameters are validated */
-    cctx->requestedParams.cParams = params->cParams;
-    cctx->requestedParams.fParams = params->fParams;
-    cctx->requestedParams.compressionLevel = params->compressionLevel;
-
-    /* Set force window explicitly since it sets cctx->loadedDictEnd */
-    CHECK_F( ZSTD_CCtx_setParameter(
-                   cctx, ZSTD_p_forceMaxWindow, params->forceWindow) );
-
-    /* Set multithreading parameters explicitly */
-    CHECK_F( ZSTD_CCtx_setParameter(cctx, ZSTD_p_nbThreads, params->nbThreads) );
-    if (params->nbThreads > 1) {
-        CHECK_F( ZSTD_CCtx_setParameter(cctx, ZSTD_p_jobSize, params->jobSize) );
-        CHECK_F( ZSTD_CCtx_setParameter(
-                    cctx, ZSTD_p_overlapSizeLog, params->overlapSizeLog) );
-    }
-
-    /* Copy long distance matching parameters */
-    cctx->requestedParams.ldmParams = params->ldmParams;
-#endif
 
-    /* customMem is used only for create/free params and can be ignored */
     return 0;
 }
 
@@ -2097,7 +2069,6 @@ size_t ZSTD_compressBegin_advanced_internal(
                                     unsigned long long pledgedSrcSize)
 {
     DEBUGLOG(4, "ZSTD_compressBegin_advanced_internal");
-    DEBUGLOG(4, "contentSizeFlag : %u", params.fParams.contentSizeFlag);
     /* compression parameters verification and optimization */
     CHECK_F( ZSTD_checkCParams(params.cParams) );
     return ZSTD_compressBegin_internal(cctx, dict, dictSize, dictMode, NULL,
diff --git a/lib/compress/zstdmt_compress.c b/lib/compress/zstdmt_compress.c
index ce813dcccaa..6b6466fe56d 100644
--- a/lib/compress/zstdmt_compress.c
+++ b/lib/compress/zstdmt_compress.c
@@ -359,7 +359,7 @@ void ZSTDMT_compressChunk(void* jobDescription)
         assert(job->firstChunk); /* should only happen for first segment */
         if (ZSTD_isError(initError)) { job->cSize = initError; goto _endJob; }
     } else {  /* srcStart points at reloaded section */
-        ZSTD_CCtx_params jobParams = job->params;
+        ZSTD_CCtx_params jobParams = job->params;   /* do not modify job->params ! copy it, modify the copy */
         size_t const forceWindowError = ZSTD_CCtxParam_setParameter(&jobParams, ZSTD_p_forceMaxWindow, !job->firstChunk);
         U64 const pledgedSrcSize = job->firstChunk ? job->fullFrameSize : ZSTD_CONTENTSIZE_UNKNOWN;
         /* load dictionary in "content-only" mode (no header analysis) */

From 55faa5492d5ba593746868991556e593134e6b57 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Mon, 4 Dec 2017 11:26:59 -0800
Subject: [PATCH 108/145] fileio: fixed LZ4F invocation from assert()

---
 programs/fileio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/programs/fileio.c b/programs/fileio.c
index c9b6b04e1a1..101c99b1ae8 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -639,7 +639,6 @@ static unsigned long long FIO_compressLz4Frame(cRess_t* ress,
     memset(&prefs, 0, sizeof(prefs));
 
     assert(blockSize <= ress->srcBufferSize);
-    assert(LZ4F_compressBound(blockSize) <= ress->dstBufferSize);
 
     prefs.autoFlush = 1;
     prefs.compressionLevel = compressionLevel;
@@ -649,6 +648,7 @@ static unsigned long long FIO_compressLz4Frame(cRess_t* ress,
 #if LZ4_VERSION_NUMBER >= 10600
     prefs.frameInfo.contentSize = (srcFileSize==UTIL_FILESIZE_UNKNOWN) ? 0 : srcFileSize;
 #endif
+    assert(LZ4F_compressBound(blockSize, &prefs) <= ress->dstBufferSize);
 
     {
         size_t readSize;

From e46194bbf95aa11b81f3cc1be83f13d3f888421d Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Mon, 4 Dec 2017 15:57:01 -0800
Subject: [PATCH 109/145] fix #911 : changed detection macro for
 clock_gettime()

The new macro might be a bit too restrictive.
Systems which do not support new test will simply default to 's `clock_t clock()`,
suffering lesser benchmark accuracy.
Should it matter, the detection macro will have to be upgraded.
---
 programs/util.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/programs/util.h b/programs/util.h
index 37098f2f725..f510bc5e0e8 100644
--- a/programs/util.h
+++ b/programs/util.h
@@ -168,7 +168,7 @@ static int g_utilDisplayLevel;
         }
         return ((clockEnd - clockStart) * (U64)rate.numer) / ((U64)rate.denom);
     }
-#elif (PLATFORM_POSIX_VERSION >= 200112L)
+#elif defined __UCLIBC__ || ((__GLIBC__ == 2 && __GLIBC_MINOR__ >= 17) || __GLIBC__ > 2)
     #include 
     #define UTIL_TIME_INITIALIZER { 0, 0 }
     typedef struct timespec UTIL_freq_t;

From 00974692380e79ea702e99214c6e7acf672c058e Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Mon, 4 Dec 2017 16:02:42 -0800
Subject: [PATCH 110/145] removed a few redundant #include

---
 programs/util.h | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/programs/util.h b/programs/util.h
index f510bc5e0e8..38dad2ab45c 100644
--- a/programs/util.h
+++ b/programs/util.h
@@ -34,7 +34,7 @@ extern "C" {
 #  include      /* chown, stat */
 #  include       /* utime */
 #endif
-#include          /* time */
+#include          /* clock_t, clock, CLOCKS_PER_SEC, nanosleep */
 #include 
 #include "mem.h"          /* U32, U64 */
 
@@ -64,7 +64,6 @@ extern "C" {
 #elif PLATFORM_POSIX_VERSION >= 0 /* Unix-like operating system */
 #  include 
 #  include  /* setpriority */
-#  include          /* clock_t, nanosleep, clock, CLOCKS_PER_SEC */
 #  if defined(PRIO_PROCESS)
 #    define SET_REALTIME_PRIORITY setpriority(PRIO_PROCESS, 0, -20)
 #  else
@@ -169,7 +168,6 @@ static int g_utilDisplayLevel;
         return ((clockEnd - clockStart) * (U64)rate.numer) / ((U64)rate.denom);
     }
 #elif defined __UCLIBC__ || ((__GLIBC__ == 2 && __GLIBC_MINOR__ >= 17) || __GLIBC__ > 2)
-    #include 
     #define UTIL_TIME_INITIALIZER { 0, 0 }
     typedef struct timespec UTIL_freq_t;
     typedef struct timespec UTIL_time_t;

From 31293330d0cd3295646b208f5fa05cfa1ad8914c Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Mon, 4 Dec 2017 16:31:59 -0800
Subject: [PATCH 111/145] It's still necessary to check PLATFORM_POSIX_VERSION
 for clock_gettime()

glibc/uclibc is not enough
---
 programs/util.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/programs/util.h b/programs/util.h
index 38dad2ab45c..af1fa7fcae7 100644
--- a/programs/util.h
+++ b/programs/util.h
@@ -167,7 +167,7 @@ static int g_utilDisplayLevel;
         }
         return ((clockEnd - clockStart) * (U64)rate.numer) / ((U64)rate.denom);
     }
-#elif defined __UCLIBC__ || ((__GLIBC__ == 2 && __GLIBC_MINOR__ >= 17) || __GLIBC__ > 2)
+#elif (PLATFORM_POSIX_VERSION >= 200112L) && (defined __UCLIBC__ || ((__GLIBC__ == 2 && __GLIBC_MINOR__ >= 17) || __GLIBC__ > 2))
     #define UTIL_TIME_INITIALIZER { 0, 0 }
     typedef struct timespec UTIL_freq_t;
     typedef struct timespec UTIL_time_t;

From c173dbd6e766bf894b1e846c82ab7a00fb0e7481 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Mon, 4 Dec 2017 17:57:42 -0800
Subject: [PATCH 112/145] no longer supported starting C++17

---
 contrib/long_distance_matching/ldm.c  |  3 +-
 lib/common/bitstream.h                |  2 +-
 lib/compress/zstd_compress_internal.h |  2 +-
 lib/dictBuilder/zdict.c               |  2 +-
 lib/legacy/zstd_v01.c                 |  2 +-
 lib/legacy/zstd_v02.c                 | 75 +-------------------------
 lib/legacy/zstd_v03.c                 | 75 +-------------------------
 lib/legacy/zstd_v04.c                 | 73 +------------------------
 lib/legacy/zstd_v05.c                 | 74 +------------------------
 lib/legacy/zstd_v06.c                 | 78 +--------------------------
 lib/legacy/zstd_v07.c                 | 78 +--------------------------
 11 files changed, 11 insertions(+), 453 deletions(-)

diff --git a/contrib/long_distance_matching/ldm.c b/contrib/long_distance_matching/ldm.c
index 4dccd0bfac6..37b188b72ad 100644
--- a/contrib/long_distance_matching/ldm.c
+++ b/contrib/long_distance_matching/ldm.c
@@ -141,7 +141,7 @@ static LDM_hashEntry *getBucket(const LDM_hashTable *table, const hash_t hash) {
   return table->entries + (hash << HASH_BUCKET_SIZE_LOG);
 }
 
-static unsigned ZSTD_NbCommonBytes (register size_t val) {
+static unsigned ZSTD_NbCommonBytes (size_t val) {
   if (MEM_isLittleEndian()) {
     if (MEM_64bits()) {
 #    if defined(_MSC_VER) && defined(_WIN64)
@@ -854,4 +854,3 @@ void LDM_outputConfiguration(void) {
   printf("HASH_CHAR_OFFSET: %d\n", HASH_CHAR_OFFSET);
   printf("=====================\n");
 }
-
diff --git a/lib/common/bitstream.h b/lib/common/bitstream.h
index 2094823fe24..fcf3843079a 100644
--- a/lib/common/bitstream.h
+++ b/lib/common/bitstream.h
@@ -167,7 +167,7 @@ MEM_STATIC size_t BIT_readBitsFast(BIT_DStream_t* bitD, unsigned nbBits);
 /*-**************************************************************
 *  Internal functions
 ****************************************************************/
-MEM_STATIC unsigned BIT_highbit32 (register U32 val)
+MEM_STATIC unsigned BIT_highbit32 (U32 val)
 {
     assert(val != 0);
     {
diff --git a/lib/compress/zstd_compress_internal.h b/lib/compress/zstd_compress_internal.h
index 1308541f404..278f2427c59 100644
--- a/lib/compress/zstd_compress_internal.h
+++ b/lib/compress/zstd_compress_internal.h
@@ -268,7 +268,7 @@ MEM_STATIC void ZSTD_storeSeq(seqStore_t* seqStorePtr, size_t litLength, const v
 /*-*************************************
 *  Match length counter
 ***************************************/
-static unsigned ZSTD_NbCommonBytes (register size_t val)
+static unsigned ZSTD_NbCommonBytes (size_t val)
 {
     if (MEM_isLittleEndian()) {
         if (MEM_64bits()) {
diff --git a/lib/dictBuilder/zdict.c b/lib/dictBuilder/zdict.c
index 1bb8b0683d0..e8f454045fc 100644
--- a/lib/dictBuilder/zdict.c
+++ b/lib/dictBuilder/zdict.c
@@ -103,7 +103,7 @@ unsigned ZDICT_getDictID(const void* dictBuffer, size_t dictSize)
 /*-********************************************************
 *  Dictionary training functions
 **********************************************************/
-static unsigned ZDICT_NbCommonBytes (register size_t val)
+static unsigned ZDICT_NbCommonBytes (size_t val)
 {
     if (MEM_isLittleEndian()) {
         if (MEM_64bits()) {
diff --git a/lib/legacy/zstd_v01.c b/lib/legacy/zstd_v01.c
index 70003cbedff..ae1cb2ce5aa 100644
--- a/lib/legacy/zstd_v01.c
+++ b/lib/legacy/zstd_v01.c
@@ -339,7 +339,7 @@ typedef U32 DTable_max_t[FSE_DTABLE_SIZE_U32(FSE_MAX_TABLELOG)];
 /****************************************************************
 *  Internal functions
 ****************************************************************/
-FORCE_INLINE unsigned FSE_highbit32 (register U32 val)
+FORCE_INLINE unsigned FSE_highbit32 (U32 val)
 {
 #   if defined(_MSC_VER)   /* Visual */
     unsigned long r;
diff --git a/lib/legacy/zstd_v02.c b/lib/legacy/zstd_v02.c
index b935a4d1800..8bc0eceeda8 100644
--- a/lib/legacy/zstd_v02.c
+++ b/lib/legacy/zstd_v02.c
@@ -330,18 +330,6 @@ MEM_STATIC BIT_DStream_status BIT_reloadDStream(BIT_DStream_t* bitD);
 MEM_STATIC unsigned BIT_endOfDStream(const BIT_DStream_t* bitD);
 
 
-/*
-* Start by invoking BIT_initDStream().
-* A chunk of the bitStream is then stored into a local register.
-* Local register size is 64-bits on 64-bits systems, 32-bits on 32-bits systems (size_t).
-* You can then retrieve bitFields stored into the local register, **in reverse order**.
-* Local register is manually filled from memory by the BIT_reloadDStream() method.
-* A reload guarantee a minimum of ((8*sizeof(size_t))-7) bits when its result is BIT_DStream_unfinished.
-* Otherwise, it can be less than that, so proceed accordingly.
-* Checking if DStream has reached its end can be performed with BIT_endOfDStream()
-*/
-
-
 /******************************************
 *  unsafe API
 ******************************************/
@@ -353,7 +341,7 @@ MEM_STATIC size_t BIT_readBitsFast(BIT_DStream_t* bitD, unsigned nbBits);
 /****************************************************************
 *  Helper functions
 ****************************************************************/
-MEM_STATIC unsigned BIT_highbit32 (register U32 val)
+MEM_STATIC unsigned BIT_highbit32 (U32 val)
 {
 #   if defined(_MSC_VER)   /* Visual */
     unsigned long r=0;
@@ -427,13 +415,6 @@ MEM_STATIC size_t BIT_initDStream(BIT_DStream_t* bitD, const void* srcBuffer, si
     return srcSize;
 }
 
-/*!BIT_lookBits
- * Provides next n bits from local register
- * local register is not modified (bits are still present for next read/look)
- * On 32-bits, maxNbBits==25
- * On 64-bits, maxNbBits==57
- * @return : value extracted
- */
 MEM_STATIC size_t BIT_lookBits(BIT_DStream_t* bitD, U32 nbBits)
 {
     const U32 bitMask = sizeof(bitD->bitContainer)*8 - 1;
@@ -453,11 +434,6 @@ MEM_STATIC void BIT_skipBits(BIT_DStream_t* bitD, U32 nbBits)
     bitD->bitsConsumed += nbBits;
 }
 
-/*!BIT_readBits
- * Read next n bits from local register.
- * pay attention to not read more than nbBits contained into local register.
- * @return : extracted value.
- */
 MEM_STATIC size_t BIT_readBits(BIT_DStream_t* bitD, U32 nbBits)
 {
     size_t value = BIT_lookBits(bitD, nbBits);
@@ -695,55 +671,6 @@ static unsigned char FSE_decodeSymbol(FSE_DState_t* DStatePtr, BIT_DStream_t* bi
 
 static unsigned FSE_endOfDState(const FSE_DState_t* DStatePtr);
 
-/*
-Let's now decompose FSE_decompress_usingDTable() into its unitary components.
-You will decode FSE-encoded symbols from the bitStream,
-and also any other bitFields you put in, **in reverse order**.
-
-You will need a few variables to track your bitStream. They are :
-
-BIT_DStream_t DStream;    // Stream context
-FSE_DState_t  DState;     // State context. Multiple ones are possible
-FSE_DTable*   DTablePtr;  // Decoding table, provided by FSE_buildDTable()
-
-The first thing to do is to init the bitStream.
-    errorCode = BIT_initDStream(&DStream, srcBuffer, srcSize);
-
-You should then retrieve your initial state(s)
-(in reverse flushing order if you have several ones) :
-    errorCode = FSE_initDState(&DState, &DStream, DTablePtr);
-
-You can then decode your data, symbol after symbol.
-For information the maximum number of bits read by FSE_decodeSymbol() is 'tableLog'.
-Keep in mind that symbols are decoded in reverse order, like a LIFO stack (last in, first out).
-    unsigned char symbol = FSE_decodeSymbol(&DState, &DStream);
-
-You can retrieve any bitfield you eventually stored into the bitStream (in reverse order)
-Note : maximum allowed nbBits is 25, for 32-bits compatibility
-    size_t bitField = BIT_readBits(&DStream, nbBits);
-
-All above operations only read from local register (which size depends on size_t).
-Refueling the register from memory is manually performed by the reload method.
-    endSignal = FSE_reloadDStream(&DStream);
-
-BIT_reloadDStream() result tells if there is still some more data to read from DStream.
-BIT_DStream_unfinished : there is still some data left into the DStream.
-BIT_DStream_endOfBuffer : Dstream reached end of buffer. Its container may no longer be completely filled.
-BIT_DStream_completed : Dstream reached its exact end, corresponding in general to decompression completed.
-BIT_DStream_tooFar : Dstream went too far. Decompression result is corrupted.
-
-When reaching end of buffer (BIT_DStream_endOfBuffer), progress slowly, notably if you decode multiple symbols per loop,
-to properly detect the exact end of stream.
-After each decoded symbol, check if DStream is fully consumed using this simple test :
-    BIT_reloadDStream(&DStream) >= BIT_DStream_completed
-
-When it's done, verify decompression is fully completed, by checking both DStream and the relevant states.
-Checking if DStream has reached its end is performed by :
-    BIT_endOfDStream(&DStream);
-Check also the states. There might be some symbols left there, if some high probability ones (>50%) are possible.
-    FSE_endOfDState(&DState);
-*/
-
 
 /******************************************
 *  FSE unsafe API
diff --git a/lib/legacy/zstd_v03.c b/lib/legacy/zstd_v03.c
index 35370dd0ca6..54445af577e 100644
--- a/lib/legacy/zstd_v03.c
+++ b/lib/legacy/zstd_v03.c
@@ -332,17 +332,6 @@ MEM_STATIC BIT_DStream_status BIT_reloadDStream(BIT_DStream_t* bitD);
 MEM_STATIC unsigned BIT_endOfDStream(const BIT_DStream_t* bitD);
 
 
-/*
-* Start by invoking BIT_initDStream().
-* A chunk of the bitStream is then stored into a local register.
-* Local register size is 64-bits on 64-bits systems, 32-bits on 32-bits systems (size_t).
-* You can then retrieve bitFields stored into the local register, **in reverse order**.
-* Local register is manually filled from memory by the BIT_reloadDStream() method.
-* A reload guarantee a minimum of ((8*sizeof(size_t))-7) bits when its result is BIT_DStream_unfinished.
-* Otherwise, it can be less than that, so proceed accordingly.
-* Checking if DStream has reached its end can be performed with BIT_endOfDStream()
-*/
-
 
 /******************************************
 *  unsafe API
@@ -355,7 +344,7 @@ MEM_STATIC size_t BIT_readBitsFast(BIT_DStream_t* bitD, unsigned nbBits);
 /****************************************************************
 *  Helper functions
 ****************************************************************/
-MEM_STATIC unsigned BIT_highbit32 (register U32 val)
+MEM_STATIC unsigned BIT_highbit32 (U32 val)
 {
 #   if defined(_MSC_VER)   /* Visual */
     unsigned long r=0;
@@ -428,14 +417,6 @@ MEM_STATIC size_t BIT_initDStream(BIT_DStream_t* bitD, const void* srcBuffer, si
 
     return srcSize;
 }
-
-/*!BIT_lookBits
- * Provides next n bits from local register
- * local register is not modified (bits are still present for next read/look)
- * On 32-bits, maxNbBits==25
- * On 64-bits, maxNbBits==57
- * @return : value extracted
- */
 MEM_STATIC size_t BIT_lookBits(BIT_DStream_t* bitD, U32 nbBits)
 {
     const U32 bitMask = sizeof(bitD->bitContainer)*8 - 1;
@@ -455,11 +436,6 @@ MEM_STATIC void BIT_skipBits(BIT_DStream_t* bitD, U32 nbBits)
     bitD->bitsConsumed += nbBits;
 }
 
-/*!BIT_readBits
- * Read next n bits from local register.
- * pay attention to not read more than nbBits contained into local register.
- * @return : extracted value.
- */
 MEM_STATIC size_t BIT_readBits(BIT_DStream_t* bitD, U32 nbBits)
 {
     size_t value = BIT_lookBits(bitD, nbBits);
@@ -697,55 +673,6 @@ static unsigned char FSE_decodeSymbol(FSE_DState_t* DStatePtr, BIT_DStream_t* bi
 
 static unsigned FSE_endOfDState(const FSE_DState_t* DStatePtr);
 
-/*
-Let's now decompose FSE_decompress_usingDTable() into its unitary components.
-You will decode FSE-encoded symbols from the bitStream,
-and also any other bitFields you put in, **in reverse order**.
-
-You will need a few variables to track your bitStream. They are :
-
-BIT_DStream_t DStream;    // Stream context
-FSE_DState_t  DState;     // State context. Multiple ones are possible
-FSE_DTable*   DTablePtr;  // Decoding table, provided by FSE_buildDTable()
-
-The first thing to do is to init the bitStream.
-    errorCode = BIT_initDStream(&DStream, srcBuffer, srcSize);
-
-You should then retrieve your initial state(s)
-(in reverse flushing order if you have several ones) :
-    errorCode = FSE_initDState(&DState, &DStream, DTablePtr);
-
-You can then decode your data, symbol after symbol.
-For information the maximum number of bits read by FSE_decodeSymbol() is 'tableLog'.
-Keep in mind that symbols are decoded in reverse order, like a LIFO stack (last in, first out).
-    unsigned char symbol = FSE_decodeSymbol(&DState, &DStream);
-
-You can retrieve any bitfield you eventually stored into the bitStream (in reverse order)
-Note : maximum allowed nbBits is 25, for 32-bits compatibility
-    size_t bitField = BIT_readBits(&DStream, nbBits);
-
-All above operations only read from local register (which size depends on size_t).
-Refueling the register from memory is manually performed by the reload method.
-    endSignal = FSE_reloadDStream(&DStream);
-
-BIT_reloadDStream() result tells if there is still some more data to read from DStream.
-BIT_DStream_unfinished : there is still some data left into the DStream.
-BIT_DStream_endOfBuffer : Dstream reached end of buffer. Its container may no longer be completely filled.
-BIT_DStream_completed : Dstream reached its exact end, corresponding in general to decompression completed.
-BIT_DStream_tooFar : Dstream went too far. Decompression result is corrupted.
-
-When reaching end of buffer (BIT_DStream_endOfBuffer), progress slowly, notably if you decode multiple symbols per loop,
-to properly detect the exact end of stream.
-After each decoded symbol, check if DStream is fully consumed using this simple test :
-    BIT_reloadDStream(&DStream) >= BIT_DStream_completed
-
-When it's done, verify decompression is fully completed, by checking both DStream and the relevant states.
-Checking if DStream has reached its end is performed by :
-    BIT_endOfDStream(&DStream);
-Check also the states. There might be some symbols left there, if some high probability ones (>50%) are possible.
-    FSE_endOfDState(&DState);
-*/
-
 
 /******************************************
 *  FSE unsafe API
diff --git a/lib/legacy/zstd_v04.c b/lib/legacy/zstd_v04.c
index 1b5f6f3b0cd..1ff7c95e068 100644
--- a/lib/legacy/zstd_v04.c
+++ b/lib/legacy/zstd_v04.c
@@ -738,16 +738,6 @@ MEM_STATIC BIT_DStream_status BIT_reloadDStream(BIT_DStream_t* bitD);
 MEM_STATIC unsigned BIT_endOfDStream(const BIT_DStream_t* bitD);
 
 
-/*
-* Start by invoking BIT_initDStream().
-* A chunk of the bitStream is then stored into a local register.
-* Local register size is 64-bits on 64-bits systems, 32-bits on 32-bits systems (size_t).
-* You can then retrieve bitFields stored into the local register, **in reverse order**.
-* Local register is manually filled from memory by the BIT_reloadDStream() method.
-* A reload guarantee a minimum of ((8*sizeof(size_t))-7) bits when its result is BIT_DStream_unfinished.
-* Otherwise, it can be less than that, so proceed accordingly.
-* Checking if DStream has reached its end can be performed with BIT_endOfDStream()
-*/
 
 
 /******************************************
@@ -761,7 +751,7 @@ MEM_STATIC size_t BIT_readBitsFast(BIT_DStream_t* bitD, unsigned nbBits);
 /****************************************************************
 *  Helper functions
 ****************************************************************/
-MEM_STATIC unsigned BIT_highbit32 (register U32 val)
+MEM_STATIC unsigned BIT_highbit32 (U32 val)
 {
 #   if defined(_MSC_VER)   /* Visual */
     unsigned long r=0;
@@ -834,13 +824,6 @@ MEM_STATIC size_t BIT_initDStream(BIT_DStream_t* bitD, const void* srcBuffer, si
     return srcSize;
 }
 
-/*!BIT_lookBits
- * Provides next n bits from local register
- * local register is not modified (bits are still present for next read/look)
- * On 32-bits, maxNbBits==25
- * On 64-bits, maxNbBits==57
- * @return : value extracted
- */
 MEM_STATIC size_t BIT_lookBits(BIT_DStream_t* bitD, U32 nbBits)
 {
     const U32 bitMask = sizeof(bitD->bitContainer)*8 - 1;
@@ -860,11 +843,6 @@ MEM_STATIC void BIT_skipBits(BIT_DStream_t* bitD, U32 nbBits)
     bitD->bitsConsumed += nbBits;
 }
 
-/*!BIT_readBits
- * Read next n bits from local register.
- * pay attention to not read more than nbBits contained into local register.
- * @return : extracted value.
- */
 MEM_STATIC size_t BIT_readBits(BIT_DStream_t* bitD, U32 nbBits)
 {
     size_t value = BIT_lookBits(bitD, nbBits);
@@ -1011,55 +989,6 @@ static unsigned char FSE_decodeSymbol(FSE_DState_t* DStatePtr, BIT_DStream_t* bi
 
 static unsigned FSE_endOfDState(const FSE_DState_t* DStatePtr);
 
-/*!
-Let's now decompose FSE_decompress_usingDTable() into its unitary components.
-You will decode FSE-encoded symbols from the bitStream,
-and also any other bitFields you put in, **in reverse order**.
-
-You will need a few variables to track your bitStream. They are :
-
-BIT_DStream_t DStream;    // Stream context
-FSE_DState_t  DState;     // State context. Multiple ones are possible
-FSE_DTable*   DTablePtr;  // Decoding table, provided by FSE_buildDTable()
-
-The first thing to do is to init the bitStream.
-    errorCode = BIT_initDStream(&DStream, srcBuffer, srcSize);
-
-You should then retrieve your initial state(s)
-(in reverse flushing order if you have several ones) :
-    errorCode = FSE_initDState(&DState, &DStream, DTablePtr);
-
-You can then decode your data, symbol after symbol.
-For information the maximum number of bits read by FSE_decodeSymbol() is 'tableLog'.
-Keep in mind that symbols are decoded in reverse order, like a LIFO stack (last in, first out).
-    unsigned char symbol = FSE_decodeSymbol(&DState, &DStream);
-
-You can retrieve any bitfield you eventually stored into the bitStream (in reverse order)
-Note : maximum allowed nbBits is 25, for 32-bits compatibility
-    size_t bitField = BIT_readBits(&DStream, nbBits);
-
-All above operations only read from local register (which size depends on size_t).
-Refueling the register from memory is manually performed by the reload method.
-    endSignal = FSE_reloadDStream(&DStream);
-
-BIT_reloadDStream() result tells if there is still some more data to read from DStream.
-BIT_DStream_unfinished : there is still some data left into the DStream.
-BIT_DStream_endOfBuffer : Dstream reached end of buffer. Its container may no longer be completely filled.
-BIT_DStream_completed : Dstream reached its exact end, corresponding in general to decompression completed.
-BIT_DStream_tooFar : Dstream went too far. Decompression result is corrupted.
-
-When reaching end of buffer (BIT_DStream_endOfBuffer), progress slowly, notably if you decode multiple symbols per loop,
-to properly detect the exact end of stream.
-After each decoded symbol, check if DStream is fully consumed using this simple test :
-    BIT_reloadDStream(&DStream) >= BIT_DStream_completed
-
-When it's done, verify decompression is fully completed, by checking both DStream and the relevant states.
-Checking if DStream has reached its end is performed by :
-    BIT_endOfDStream(&DStream);
-Check also the states. There might be some symbols left there, if some high probability ones (>50%) are possible.
-    FSE_endOfDState(&DState);
-*/
-
 
 /* *****************************************
 *  FSE unsafe API
diff --git a/lib/legacy/zstd_v05.c b/lib/legacy/zstd_v05.c
index 23188f504d3..a5e1b1ffc8c 100644
--- a/lib/legacy/zstd_v05.c
+++ b/lib/legacy/zstd_v05.c
@@ -736,18 +736,6 @@ MEM_STATIC BITv05_DStream_status BITv05_reloadDStream(BITv05_DStream_t* bitD);
 MEM_STATIC unsigned BITv05_endOfDStream(const BITv05_DStream_t* bitD);
 
 
-/*!
-* Start by invoking BITv05_initDStream().
-* A chunk of the bitStream is then stored into a local register.
-* Local register size is 64-bits on 64-bits systems, 32-bits on 32-bits systems (size_t).
-* You can then retrieve bitFields stored into the local register, **in reverse order**.
-* Local register is explicitly reloaded from memory by the BITv05_reloadDStream() method.
-* A reload guarantee a minimum of ((8*sizeof(size_t))-7) bits when its result is BITv05_DStream_unfinished.
-* Otherwise, it can be less than that, so proceed accordingly.
-* Checking if DStream has reached its end can be performed with BITv05_endOfDStream()
-*/
-
-
 /*-****************************************
 *  unsafe API
 ******************************************/
@@ -759,7 +747,7 @@ MEM_STATIC size_t BITv05_readBitsFast(BITv05_DStream_t* bitD, unsigned nbBits);
 /*-**************************************************************
 *  Helper functions
 ****************************************************************/
-MEM_STATIC unsigned BITv05_highbit32 (register U32 val)
+MEM_STATIC unsigned BITv05_highbit32 (U32 val)
 {
 #   if defined(_MSC_VER)   /* Visual */
     unsigned long r=0;
@@ -829,13 +817,6 @@ MEM_STATIC size_t BITv05_initDStream(BITv05_DStream_t* bitD, const void* srcBuff
     return srcSize;
 }
 
-/*!BITv05_lookBits
- * Provides next n bits from local register
- * local register is not modified (bits are still present for next read/look)
- * On 32-bits, maxNbBits==25
- * On 64-bits, maxNbBits==57
- * @return : value extracted
- */
 MEM_STATIC size_t BITv05_lookBits(BITv05_DStream_t* bitD, U32 nbBits)
 {
     const U32 bitMask = sizeof(bitD->bitContainer)*8 - 1;
@@ -855,11 +836,6 @@ MEM_STATIC void BITv05_skipBits(BITv05_DStream_t* bitD, U32 nbBits)
     bitD->bitsConsumed += nbBits;
 }
 
-/*!BITv05_readBits
- * Read next n bits from local register.
- * pay attention to not read more than nbBits contained into local register.
- * @return : extracted value.
- */
 MEM_STATIC size_t BITv05_readBits(BITv05_DStream_t* bitD, U32 nbBits)
 {
     size_t value = BITv05_lookBits(bitD, nbBits);
@@ -995,54 +971,6 @@ static unsigned char FSEv05_decodeSymbol(FSEv05_DState_t* DStatePtr, BITv05_DStr
 
 static unsigned FSEv05_endOfDState(const FSEv05_DState_t* DStatePtr);
 
-/*!
-Let's now decompose FSEv05_decompress_usingDTable() into its unitary components.
-You will decode FSEv05-encoded symbols from the bitStream,
-and also any other bitFields you put in, **in reverse order**.
-
-You will need a few variables to track your bitStream. They are :
-
-BITv05_DStream_t DStream;    // Stream context
-FSEv05_DState_t  DState;     // State context. Multiple ones are possible
-FSEv05_DTable*   DTablePtr;  // Decoding table, provided by FSEv05_buildDTable()
-
-The first thing to do is to init the bitStream.
-    errorCode = BITv05_initDStream(&DStream, srcBuffer, srcSize);
-
-You should then retrieve your initial state(s)
-(in reverse flushing order if you have several ones) :
-    errorCode = FSEv05_initDState(&DState, &DStream, DTablePtr);
-
-You can then decode your data, symbol after symbol.
-For information the maximum number of bits read by FSEv05_decodeSymbol() is 'tableLog'.
-Keep in mind that symbols are decoded in reverse order, like a LIFO stack (last in, first out).
-    unsigned char symbol = FSEv05_decodeSymbol(&DState, &DStream);
-
-You can retrieve any bitfield you eventually stored into the bitStream (in reverse order)
-Note : maximum allowed nbBits is 25, for 32-bits compatibility
-    size_t bitField = BITv05_readBits(&DStream, nbBits);
-
-All above operations only read from local register (which size depends on size_t).
-Refueling the register from memory is manually performed by the reload method.
-    endSignal = FSEv05_reloadDStream(&DStream);
-
-BITv05_reloadDStream() result tells if there is still some more data to read from DStream.
-BITv05_DStream_unfinished : there is still some data left into the DStream.
-BITv05_DStream_endOfBuffer : Dstream reached end of buffer. Its container may no longer be completely filled.
-BITv05_DStream_completed : Dstream reached its exact end, corresponding in general to decompression completed.
-BITv05_DStream_tooFar : Dstream went too far. Decompression result is corrupted.
-
-When reaching end of buffer (BITv05_DStream_endOfBuffer), progress slowly, notably if you decode multiple symbols per loop,
-to properly detect the exact end of stream.
-After each decoded symbol, check if DStream is fully consumed using this simple test :
-    BITv05_reloadDStream(&DStream) >= BITv05_DStream_completed
-
-When it's done, verify decompression is fully completed, by checking both DStream and the relevant states.
-Checking if DStream has reached its end is performed by :
-    BITv05_endOfDStream(&DStream);
-Check also the states. There might be some symbols left there, if some high probability ones (>50%) are possible.
-    FSEv05_endOfDState(&DState);
-*/
 
 
 /* *****************************************
diff --git a/lib/legacy/zstd_v06.c b/lib/legacy/zstd_v06.c
index 62683f994a9..19116f10814 100644
--- a/lib/legacy/zstd_v06.c
+++ b/lib/legacy/zstd_v06.c
@@ -839,16 +839,6 @@ MEM_STATIC BITv06_DStream_status BITv06_reloadDStream(BITv06_DStream_t* bitD);
 MEM_STATIC unsigned BITv06_endOfDStream(const BITv06_DStream_t* bitD);
 
 
-/* Start by invoking BITv06_initDStream().
-*  A chunk of the bitStream is then stored into a local register.
-*  Local register size is 64-bits on 64-bits systems, 32-bits on 32-bits systems (size_t).
-*  You can then retrieve bitFields stored into the local register, **in reverse order**.
-*  Local register is explicitly reloaded from memory by the BITv06_reloadDStream() method.
-*  A reload guarantee a minimum of ((8*sizeof(bitD->bitContainer))-7) bits when its result is BITv06_DStream_unfinished.
-*  Otherwise, it can be less than that, so proceed accordingly.
-*  Checking if DStream has reached its end can be performed with BITv06_endOfDStream().
-*/
-
 
 /*-****************************************
 *  unsafe API
@@ -861,7 +851,7 @@ MEM_STATIC size_t BITv06_readBitsFast(BITv06_DStream_t* bitD, unsigned nbBits);
 /*-**************************************************************
 *  Internal functions
 ****************************************************************/
-MEM_STATIC unsigned BITv06_highbit32 (register U32 val)
+MEM_STATIC unsigned BITv06_highbit32 ( U32 val)
 {
 #   if defined(_MSC_VER)   /* Visual */
     unsigned long r=0;
@@ -929,13 +919,6 @@ MEM_STATIC size_t BITv06_initDStream(BITv06_DStream_t* bitD, const void* srcBuff
 }
 
 
-/*! BITv06_lookBits() :
- *  Provides next n bits from local register.
- *  local register is not modified.
- *  On 32-bits, maxNbBits==24.
- *  On 64-bits, maxNbBits==56.
- *  @return : value extracted
- */
  MEM_STATIC size_t BITv06_lookBits(const BITv06_DStream_t* bitD, U32 nbBits)
 {
     U32 const bitMask = sizeof(bitD->bitContainer)*8 - 1;
@@ -955,11 +938,6 @@ MEM_STATIC void BITv06_skipBits(BITv06_DStream_t* bitD, U32 nbBits)
     bitD->bitsConsumed += nbBits;
 }
 
-/*! BITv06_readBits() :
- *  Read (consume) next n bits from local register and update.
- *  Pay attention to not read more than nbBits contained into local register.
- *  @return : extracted value.
- */
 MEM_STATIC size_t BITv06_readBits(BITv06_DStream_t* bitD, U32 nbBits)
 {
     size_t const value = BITv06_lookBits(bitD, nbBits);
@@ -976,11 +954,6 @@ MEM_STATIC size_t BITv06_readBitsFast(BITv06_DStream_t* bitD, U32 nbBits)
     return value;
 }
 
-/*! BITv06_reloadDStream() :
-*   Refill `BITv06_DStream_t` from src buffer previously defined (see BITv06_initDStream() ).
-*   This function is safe, it guarantees it will not read beyond src buffer.
-*   @return : status of `BITv06_DStream_t` internal register.
-              if status == unfinished, internal register is filled with >= (sizeof(bitD->bitContainer)*8 - 7) bits */
 MEM_STATIC BITv06_DStream_status BITv06_reloadDStream(BITv06_DStream_t* bitD)
 {
     if (bitD->bitsConsumed > (sizeof(bitD->bitContainer)*8))  /* should never happen */
@@ -1103,55 +1076,6 @@ static void     FSEv06_initDState(FSEv06_DState_t* DStatePtr, BITv06_DStream_t*
 
 static unsigned char FSEv06_decodeSymbol(FSEv06_DState_t* DStatePtr, BITv06_DStream_t* bitD);
 
-/*!
-Let's now decompose FSEv06_decompress_usingDTable() into its unitary components.
-You will decode FSE-encoded symbols from the bitStream,
-and also any other bitFields you put in, **in reverse order**.
-
-You will need a few variables to track your bitStream. They are :
-
-BITv06_DStream_t DStream;    // Stream context
-FSEv06_DState_t  DState;     // State context. Multiple ones are possible
-FSEv06_DTable*   DTablePtr;  // Decoding table, provided by FSEv06_buildDTable()
-
-The first thing to do is to init the bitStream.
-    errorCode = BITv06_initDStream(&DStream, srcBuffer, srcSize);
-
-You should then retrieve your initial state(s)
-(in reverse flushing order if you have several ones) :
-    errorCode = FSEv06_initDState(&DState, &DStream, DTablePtr);
-
-You can then decode your data, symbol after symbol.
-For information the maximum number of bits read by FSEv06_decodeSymbol() is 'tableLog'.
-Keep in mind that symbols are decoded in reverse order, like a LIFO stack (last in, first out).
-    unsigned char symbol = FSEv06_decodeSymbol(&DState, &DStream);
-
-You can retrieve any bitfield you eventually stored into the bitStream (in reverse order)
-Note : maximum allowed nbBits is 25, for 32-bits compatibility
-    size_t bitField = BITv06_readBits(&DStream, nbBits);
-
-All above operations only read from local register (which size depends on size_t).
-Refueling the register from memory is manually performed by the reload method.
-    endSignal = FSEv06_reloadDStream(&DStream);
-
-BITv06_reloadDStream() result tells if there is still some more data to read from DStream.
-BITv06_DStream_unfinished : there is still some data left into the DStream.
-BITv06_DStream_endOfBuffer : Dstream reached end of buffer. Its container may no longer be completely filled.
-BITv06_DStream_completed : Dstream reached its exact end, corresponding in general to decompression completed.
-BITv06_DStream_tooFar : Dstream went too far. Decompression result is corrupted.
-
-When reaching end of buffer (BITv06_DStream_endOfBuffer), progress slowly, notably if you decode multiple symbols per loop,
-to properly detect the exact end of stream.
-After each decoded symbol, check if DStream is fully consumed using this simple test :
-    BITv06_reloadDStream(&DStream) >= BITv06_DStream_completed
-
-When it's done, verify decompression is fully completed, by checking both DStream and the relevant states.
-Checking if DStream has reached its end is performed by :
-    BITv06_endOfDStream(&DStream);
-Check also the states. There might be some symbols left there, if some high probability ones (>50%) are possible.
-    FSEv06_endOfDState(&DState);
-*/
-
 
 /* *****************************************
 *  FSE unsafe API
diff --git a/lib/legacy/zstd_v07.c b/lib/legacy/zstd_v07.c
index aad9b1f659f..114ce005b3a 100644
--- a/lib/legacy/zstd_v07.c
+++ b/lib/legacy/zstd_v07.c
@@ -511,16 +511,6 @@ MEM_STATIC BITv07_DStream_status BITv07_reloadDStream(BITv07_DStream_t* bitD);
 MEM_STATIC unsigned BITv07_endOfDStream(const BITv07_DStream_t* bitD);
 
 
-/* Start by invoking BITv07_initDStream().
-*  A chunk of the bitStream is then stored into a local register.
-*  Local register size is 64-bits on 64-bits systems, 32-bits on 32-bits systems (size_t).
-*  You can then retrieve bitFields stored into the local register, **in reverse order**.
-*  Local register is explicitly reloaded from memory by the BITv07_reloadDStream() method.
-*  A reload guarantee a minimum of ((8*sizeof(bitD->bitContainer))-7) bits when its result is BITv07_DStream_unfinished.
-*  Otherwise, it can be less than that, so proceed accordingly.
-*  Checking if DStream has reached its end can be performed with BITv07_endOfDStream().
-*/
-
 
 /*-****************************************
 *  unsafe API
@@ -533,7 +523,7 @@ MEM_STATIC size_t BITv07_readBitsFast(BITv07_DStream_t* bitD, unsigned nbBits);
 /*-**************************************************************
 *  Internal functions
 ****************************************************************/
-MEM_STATIC unsigned BITv07_highbit32 (register U32 val)
+MEM_STATIC unsigned BITv07_highbit32 (U32 val)
 {
 #   if defined(_MSC_VER)   /* Visual */
     unsigned long r=0;
@@ -599,13 +589,6 @@ MEM_STATIC size_t BITv07_initDStream(BITv07_DStream_t* bitD, const void* srcBuff
 }
 
 
-/*! BITv07_lookBits() :
- *  Provides next n bits from local register.
- *  local register is not modified.
- *  On 32-bits, maxNbBits==24.
- *  On 64-bits, maxNbBits==56.
- *  @return : value extracted
- */
  MEM_STATIC size_t BITv07_lookBits(const BITv07_DStream_t* bitD, U32 nbBits)
 {
     U32 const bitMask = sizeof(bitD->bitContainer)*8 - 1;
@@ -625,11 +608,6 @@ MEM_STATIC void BITv07_skipBits(BITv07_DStream_t* bitD, U32 nbBits)
     bitD->bitsConsumed += nbBits;
 }
 
-/*! BITv07_readBits() :
- *  Read (consume) next n bits from local register and update.
- *  Pay attention to not read more than nbBits contained into local register.
- *  @return : extracted value.
- */
 MEM_STATIC size_t BITv07_readBits(BITv07_DStream_t* bitD, U32 nbBits)
 {
     size_t const value = BITv07_lookBits(bitD, nbBits);
@@ -646,11 +624,6 @@ MEM_STATIC size_t BITv07_readBitsFast(BITv07_DStream_t* bitD, U32 nbBits)
     return value;
 }
 
-/*! BITv07_reloadDStream() :
-*   Refill `BITv07_DStream_t` from src buffer previously defined (see BITv07_initDStream() ).
-*   This function is safe, it guarantees it will not read beyond src buffer.
-*   @return : status of `BITv07_DStream_t` internal register.
-              if status == unfinished, internal register is filled with >= (sizeof(bitD->bitContainer)*8 - 7) bits */
 MEM_STATIC BITv07_DStream_status BITv07_reloadDStream(BITv07_DStream_t* bitD)
 {
     if (bitD->bitsConsumed > (sizeof(bitD->bitContainer)*8))  /* should not happen => corruption detected */
@@ -874,55 +847,6 @@ static void     FSEv07_initDState(FSEv07_DState_t* DStatePtr, BITv07_DStream_t*
 static unsigned char FSEv07_decodeSymbol(FSEv07_DState_t* DStatePtr, BITv07_DStream_t* bitD);
 
 
-/**<
-Let's now decompose FSEv07_decompress_usingDTable() into its unitary components.
-You will decode FSE-encoded symbols from the bitStream,
-and also any other bitFields you put in, **in reverse order**.
-
-You will need a few variables to track your bitStream. They are :
-
-BITv07_DStream_t DStream;    // Stream context
-FSEv07_DState_t  DState;     // State context. Multiple ones are possible
-FSEv07_DTable*   DTablePtr;  // Decoding table, provided by FSEv07_buildDTable()
-
-The first thing to do is to init the bitStream.
-    errorCode = BITv07_initDStream(&DStream, srcBuffer, srcSize);
-
-You should then retrieve your initial state(s)
-(in reverse flushing order if you have several ones) :
-    errorCode = FSEv07_initDState(&DState, &DStream, DTablePtr);
-
-You can then decode your data, symbol after symbol.
-For information the maximum number of bits read by FSEv07_decodeSymbol() is 'tableLog'.
-Keep in mind that symbols are decoded in reverse order, like a LIFO stack (last in, first out).
-    unsigned char symbol = FSEv07_decodeSymbol(&DState, &DStream);
-
-You can retrieve any bitfield you eventually stored into the bitStream (in reverse order)
-Note : maximum allowed nbBits is 25, for 32-bits compatibility
-    size_t bitField = BITv07_readBits(&DStream, nbBits);
-
-All above operations only read from local register (which size depends on size_t).
-Refueling the register from memory is manually performed by the reload method.
-    endSignal = FSEv07_reloadDStream(&DStream);
-
-BITv07_reloadDStream() result tells if there is still some more data to read from DStream.
-BITv07_DStream_unfinished : there is still some data left into the DStream.
-BITv07_DStream_endOfBuffer : Dstream reached end of buffer. Its container may no longer be completely filled.
-BITv07_DStream_completed : Dstream reached its exact end, corresponding in general to decompression completed.
-BITv07_DStream_tooFar : Dstream went too far. Decompression result is corrupted.
-
-When reaching end of buffer (BITv07_DStream_endOfBuffer), progress slowly, notably if you decode multiple symbols per loop,
-to properly detect the exact end of stream.
-After each decoded symbol, check if DStream is fully consumed using this simple test :
-    BITv07_reloadDStream(&DStream) >= BITv07_DStream_completed
-
-When it's done, verify decompression is fully completed, by checking both DStream and the relevant states.
-Checking if DStream has reached its end is performed by :
-    BITv07_endOfDStream(&DStream);
-Check also the states. There might be some symbols left there, if some high probability ones (>50%) are possible.
-    FSEv07_endOfDState(&DState);
-*/
-
 
 /* *****************************************
 *  FSE unsafe API

From 3aa2b27a890e9aa4c64b9e52632a6ef4c1418041 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Thu, 7 Dec 2017 02:52:50 -0500
Subject: [PATCH 113/145] fix #942 : streaming interface does not compress
 after ZSTD_initCStream()

While the final result is still, technically, a frame,
the resulting frame expands initial data instead of compressing it.
This is because the streaming API creates a tiny 1-byte buffer for input,
because it believes input is empty (0-bytes),
because in the past, 0 used to mean "unknown" instead.

This patch fixes the issue.
Todo : add a test which traps the issue.
---
 lib/compress/zstd_compress.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 8d493852ffa..69b0d64026b 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -2466,6 +2466,7 @@ size_t ZSTD_compress_usingCDict(ZSTD_CCtx* cctx,
 
 ZSTD_CStream* ZSTD_createCStream(void)
 {
+    DEBUGLOG(3, "ZSTD_createCStream");
     return ZSTD_createCStream_advanced(ZSTD_defaultCMem);
 }
 
@@ -2633,7 +2634,8 @@ size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigne
 
 size_t ZSTD_initCStream(ZSTD_CStream* zcs, int compressionLevel)
 {
-    return ZSTD_initCStream_srcSize(zcs, compressionLevel, 0);
+    DEBUGLOG(4, "ZSTD_initCStream");
+    return ZSTD_initCStream_srcSize(zcs, compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN);
 }
 
 /*======   Compression   ======*/

From 21f7672d43549e1f87b879fb8a14ac1a4fabc266 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Thu, 7 Dec 2017 03:06:01 -0500
Subject: [PATCH 114/145] zstreamtest : add test which detects when
 ZSTD_initCStream() inflate data

as in #942, when ZSTD_initCStream() believes data is empty, instead of unknown.
---
 tests/zstreamtest.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 920e988b054..e08a96d8fc9 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -245,13 +245,28 @@ static int basicUnitTests(U32 seed, double compressibility)
     }
     dictID = ZDICT_getDictID(dictionary.start, dictionary.filled);
 
+    /* Basic compression test */
+    DISPLAYLEVEL(3, "test%3i : compress %u bytes : ", testNb++, COMPRESSIBLE_NOISE_LENGTH);
+    CHECK_Z( ZSTD_initCStream(zc, 1 /* cLevel */) );
+    outBuff.dst = (char*)(compressedBuffer);
+    outBuff.size = compressedBufferSize;
+    outBuff.pos = 0;
+    inBuff.src = CNBuffer;
+    inBuff.size = CNBufferSize;
+    inBuff.pos = 0;
+    CHECK_Z( ZSTD_compressStream(zc, &outBuff, &inBuff) );
+    if (inBuff.pos != inBuff.size) goto _output_error;   /* entire input should be consumed */
+    { size_t const r = ZSTD_endStream(zc, &outBuff);
+      if (r != 0) goto _output_error; }  /* error, or some data not flushed */
+    DISPLAYLEVEL(3, "OK (%u bytes)\n", (U32)outBuff.pos);
+
     /* generate skippable frame */
     MEM_writeLE32(compressedBuffer, ZSTD_MAGIC_SKIPPABLE_START);
     MEM_writeLE32(((char*)compressedBuffer)+4, (U32)skippableFrameSize);
     cSize = skippableFrameSize + 8;
 
-    /* Basic compression test */
-    DISPLAYLEVEL(3, "test%3i : compress %u bytes : ", testNb++, COMPRESSIBLE_NOISE_LENGTH);
+    /* Basic compression test using dict */
+    DISPLAYLEVEL(3, "test%3i : skipframe + compress %u bytes : ", testNb++, COMPRESSIBLE_NOISE_LENGTH);
     CHECK_Z( ZSTD_initCStream_usingDict(zc, CNBuffer, dictSize, 1 /* cLevel */) );
     outBuff.dst = (char*)(compressedBuffer)+cSize;
     assert(compressedBufferSize > cSize);

From c029ee1f0bc464754c899deb2f5f0a2f16f895e4 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Thu, 7 Dec 2017 17:13:10 -0500
Subject: [PATCH 115/145] ZSTD_initCStream_srcSize() considers "0" to mean
 "unknown"

to not break existing programs relying on this behavior.
Might be changed to mean "empty" in the future.
---
 lib/compress/zstd_compress.c | 7 +++----
 lib/zstd.h                   | 2 +-
 2 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 69b0d64026b..20b09599870 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -2623,12 +2623,11 @@ size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t di
     return ZSTD_initCStream_internal(zcs, dict, dictSize, NULL, cctxParams, ZSTD_CONTENTSIZE_UNKNOWN);
 }
 
-size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize)
+size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pss)
 {
-    ZSTD_CCtx_params cctxParams;
+    U64 const pledgedSrcSize = (pss==0) ? ZSTD_CONTENTSIZE_UNKNOWN : pss;  /* temporary : 0 interpreted as "unknown" during transition period. Users willing to specify "unknown" **must** use ZSTD_CONTENTSIZE_UNKNOWN. `0` will be interpreted as "empty" in the future */
     ZSTD_parameters const params = ZSTD_getParams(compressionLevel, pledgedSrcSize, 0);
-    cctxParams = ZSTD_assignParamsToCCtxParams(zcs->requestedParams, params);
-    cctxParams.fParams.contentSizeFlag = (pledgedSrcSize>0);
+    ZSTD_CCtx_params const cctxParams = ZSTD_assignParamsToCCtxParams(zcs->requestedParams, params);
     return ZSTD_initCStream_internal(zcs, NULL, 0, NULL, cctxParams, pledgedSrcSize);
 }
 
diff --git a/lib/zstd.h b/lib/zstd.h
index fe37cec8bdf..76d75e2a059 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -734,7 +734,7 @@ ZSTDLIB_API unsigned ZSTD_getDictID_fromFrame(const void* src, size_t srcSize);
 /*=====   Advanced Streaming compression functions  =====*/
 ZSTDLIB_API ZSTD_CStream* ZSTD_createCStream_advanced(ZSTD_customMem customMem);
 ZSTDLIB_API ZSTD_CStream* ZSTD_initStaticCStream(void* workspace, size_t workspaceSize);    /**< same as ZSTD_initStaticCCtx() */
-ZSTDLIB_API size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize);   /**< pledgedSrcSize must be correct. If it is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN. Note that, for compatibility with older programs, a size of 0 is interepreted as "unknown". But it may change in some future version to mean "empty". */
+ZSTDLIB_API size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize);   /**< pledgedSrcSize must be correct. If it is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN. Note that, for compatibility with older programs, "0" also disables frame content size field. It may be enabled in the future. */
 ZSTDLIB_API size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t dictSize, int compressionLevel); /**< creates of an internal CDict (incompatible with static CCtx), except if dict == NULL or dictSize < 8, in which case no dict is used. Note: dict is loaded with ZSTD_dm_auto (treated as a full zstd dictionary if it begins with ZSTD_MAGIC_DICTIONARY, else as raw content) and ZSTD_dlm_byCopy.*/
 ZSTDLIB_API size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs, const void* dict, size_t dictSize,
                                              ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize : if srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */

From dfc697e967ad736060dc9ec695a7db0d9a805fe3 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 8 Dec 2017 12:16:49 -0500
Subject: [PATCH 116/145] comment clarification

---
 lib/compress/zstd_compress.c | 8 ++++----
 lib/zstd.h                   | 4 ++--
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 20b09599870..3f39875f5bd 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -2599,7 +2599,8 @@ size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict)
 }
 
 /* ZSTD_initCStream_advanced() :
- * pledgedSrcSize : if srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN.
+ * pledgedSrcSize must be correct.
+ * if srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN.
  * dict is loaded with default parameters ZSTD_dm_auto and ZSTD_dlm_byCopy. */
 size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs,
                                  const void* dict, size_t dictSize,
@@ -2610,9 +2611,8 @@ size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs,
     DEBUGLOG(4, "ZSTD_initCStream_advanced: pledgedSrcSize=%u, flag=%u",
             (U32)pledgedSrcSize, params.fParams.contentSizeFlag);
     CHECK_F( ZSTD_checkCParams(params.cParams) );
-    if ((pledgedSrcSize==0) && (params.fParams.contentSizeFlag==0))
-        pledgedSrcSize = ZSTD_CONTENTSIZE_UNKNOWN;
-    return ZSTD_initCStream_internal(zcs, dict, dictSize, NULL, cctxParams, pledgedSrcSize);
+    if ((pledgedSrcSize==0) && (params.fParams.contentSizeFlag==0)) pledgedSrcSize = ZSTD_CONTENTSIZE_UNKNOWN;  /* for compatibility with older programs relying on this behavior. Users should now specify ZSTD_CONTENTSIZE_UNKNOWN. This line will be removed in the future. */
+    return ZSTD_initCStream_internal(zcs, dict, dictSize, NULL /*cdict*/, cctxParams, pledgedSrcSize);
 }
 
 size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t dictSize, int compressionLevel)
diff --git a/lib/zstd.h b/lib/zstd.h
index 76d75e2a059..15c467535a4 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -737,9 +737,9 @@ ZSTDLIB_API ZSTD_CStream* ZSTD_initStaticCStream(void* workspace, size_t workspa
 ZSTDLIB_API size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize);   /**< pledgedSrcSize must be correct. If it is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN. Note that, for compatibility with older programs, "0" also disables frame content size field. It may be enabled in the future. */
 ZSTDLIB_API size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t dictSize, int compressionLevel); /**< creates of an internal CDict (incompatible with static CCtx), except if dict == NULL or dictSize < 8, in which case no dict is used. Note: dict is loaded with ZSTD_dm_auto (treated as a full zstd dictionary if it begins with ZSTD_MAGIC_DICTIONARY, else as raw content) and ZSTD_dlm_byCopy.*/
 ZSTDLIB_API size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs, const void* dict, size_t dictSize,
-                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize : if srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
+                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize must be correct. If srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
 ZSTDLIB_API size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict);  /**< note : cdict will just be referenced, and must outlive compression session */
-ZSTDLIB_API size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, const ZSTD_CDict* cdict, ZSTD_frameParameters fParams, unsigned long long pledgedSrcSize);  /**< same as ZSTD_initCStream_usingCDict(), with control over frame parameters */
+ZSTDLIB_API size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, const ZSTD_CDict* cdict, ZSTD_frameParameters fParams, unsigned long long pledgedSrcSize);  /**< same as ZSTD_initCStream_usingCDict(), with control over frame parameters. pledgedSrcSize must be correct. If srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. */
 
 /*! ZSTD_resetCStream() :
  *  start a new compression job, using same parameters from previous job.

From 5098d1fbe2a0b3d0ddb66c0880e3004feac65f3e Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 12 Dec 2017 12:48:31 -0800
Subject: [PATCH 117/145] Created a test case which reliably reproduces bug
 #944

in zstreamtest.
---
 lib/compress/zstd_compress.c     | 11 ++---
 lib/compress/zstdmt_compress.c   | 30 ++++++------
 lib/decompress/zstd_decompress.c | 19 +++++---
 lib/zstd.h                       | 19 +++++---
 tests/zstreamtest.c              | 81 +++++++++++++++++++++++++++-----
 5 files changed, 114 insertions(+), 46 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 3f39875f5bd..1072918885f 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -477,7 +477,7 @@ size_t ZSTD_CCtx_setParametersUsingCCtxParams(
 
 ZSTDLIB_API size_t ZSTD_CCtx_setPledgedSrcSize(ZSTD_CCtx* cctx, unsigned long long pledgedSrcSize)
 {
-    DEBUGLOG(4, " setting pledgedSrcSize to %u", (U32)pledgedSrcSize);
+    DEBUGLOG(4, "ZSTD_CCtx_setPledgedSrcSize to %u bytes", (U32)pledgedSrcSize);
     if (cctx->streamStage != zcss_init) return ERROR(stage_wrong);
     cctx->pledgedSrcSizePlusOne = pledgedSrcSize+1;
     return 0;
@@ -489,7 +489,7 @@ size_t ZSTD_CCtx_loadDictionary_advanced(
 {
     if (cctx->streamStage != zcss_init) return ERROR(stage_wrong);
     if (cctx->staticSize) return ERROR(memory_allocation);  /* no malloc for static CCtx */
-    DEBUGLOG(4, "load dictionary of size %u", (U32)dictSize);
+    DEBUGLOG(4, "ZSTD_CCtx_loadDictionary_advanced (size: %u)", (U32)dictSize);
     ZSTD_freeCDict(cctx->cdictLocal);  /* in case one already exists */
     if (dict==NULL || dictSize==0) {   /* no dictionary mode */
         cctx->cdictLocal = NULL;
@@ -2273,7 +2273,7 @@ static size_t ZSTD_initCDict_internal(
                     ZSTD_dictMode_e dictMode,
                     ZSTD_compressionParameters cParams)
 {
-    DEBUGLOG(4, "ZSTD_initCDict_internal, mode %u", (U32)dictMode);
+    DEBUGLOG(3, "ZSTD_initCDict_internal, mode %u", (U32)dictMode);
     if ((dictLoadMethod == ZSTD_dlm_byRef) || (!dictBuffer) || (!dictSize)) {
         cdict->dictBuffer = NULL;
         cdict->dictContent = dictBuffer;
@@ -2303,7 +2303,7 @@ ZSTD_CDict* ZSTD_createCDict_advanced(const void* dictBuffer, size_t dictSize,
                                       ZSTD_dictMode_e dictMode,
                                       ZSTD_compressionParameters cParams, ZSTD_customMem customMem)
 {
-    DEBUGLOG(4, "ZSTD_createCDict_advanced, mode %u", (U32)dictMode);
+    DEBUGLOG(3, "ZSTD_createCDict_advanced, mode %u", (U32)dictMode);
     if (!customMem.customAlloc ^ !customMem.customFree) return NULL;
 
     {   ZSTD_CDict* const cdict = (ZSTD_CDict*)ZSTD_malloc(sizeof(ZSTD_CDict), customMem);
@@ -2809,7 +2809,7 @@ size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
                               ZSTD_inBuffer* input,
                               ZSTD_EndDirective endOp)
 {
-    DEBUGLOG(5, "ZSTD_compress_generic");
+    DEBUGLOG(5, "ZSTD_compress_generic, endOp=%u ", (U32)endOp);
     /* check conditions */
     if (output->pos > output->size) return ERROR(GENERIC);
     if (input->pos  > input->size)  return ERROR(GENERIC);
@@ -2856,7 +2856,6 @@ size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
 #ifdef ZSTD_MULTITHREAD
     if (cctx->appliedParams.nbThreads > 1) {
         size_t const flushMin = ZSTDMT_compressStream_generic(cctx->mtctx, output, input, endOp);
-        DEBUGLOG(5, "ZSTDMT_compressStream_generic result : %u", (U32)flushMin);
         if ( ZSTD_isError(flushMin)
           || (endOp == ZSTD_e_end && flushMin == 0) ) { /* compression completed */
             ZSTD_startNewCompression(cctx);
diff --git a/lib/compress/zstdmt_compress.c b/lib/compress/zstdmt_compress.c
index 6b6466fe56d..8347cf29d5c 100644
--- a/lib/compress/zstdmt_compress.c
+++ b/lib/compress/zstdmt_compress.c
@@ -310,7 +310,7 @@ static void ZSTDMT_releaseCCtx(ZSTDMT_CCtxPool* pool, ZSTD_CCtx* cctx)
 typedef struct {
     buffer_t src;
     const void* srcStart;
-    size_t   dictSize;
+    size_t   prefixSize;
     size_t   srcSize;
     buffer_t dstBuff;
     size_t   cSize;
@@ -333,10 +333,10 @@ void ZSTDMT_compressChunk(void* jobDescription)
 {
     ZSTDMT_jobDescription* const job = (ZSTDMT_jobDescription*)jobDescription;
     ZSTD_CCtx* const cctx = ZSTDMT_getCCtx(job->cctxPool);
-    const void* const src = (const char*)job->srcStart + job->dictSize;
+    const void* const src = (const char*)job->srcStart + job->prefixSize;
     buffer_t dstBuff = job->dstBuff;
-    DEBUGLOG(5, "ZSTDMT_compressChunk: job (first:%u) (last:%u) : dictSize %u, srcSize %u",
-                 job->firstChunk, job->lastChunk, (U32)job->dictSize, (U32)job->srcSize);
+    DEBUGLOG(5, "ZSTDMT_compressChunk: job (first:%u) (last:%u) : prefixSize %u, srcSize %u ",
+                 job->firstChunk, job->lastChunk, (U32)job->prefixSize, (U32)job->srcSize);
 
     if (cctx==NULL) {
         job->cSize = ERROR(memory_allocation);
@@ -356,20 +356,21 @@ void ZSTDMT_compressChunk(void* jobDescription)
     if (job->cdict) {
         size_t const initError = ZSTD_compressBegin_usingCDict_advanced(cctx, job->cdict, job->params.fParams, job->fullFrameSize);
         DEBUGLOG(4, "ZSTDMT_compressChunk: init using CDict");
-        assert(job->firstChunk); /* should only happen for first segment */
+        assert(job->firstChunk);  /* only allowed for first job */
         if (ZSTD_isError(initError)) { job->cSize = initError; goto _endJob; }
     } else {  /* srcStart points at reloaded section */
         ZSTD_CCtx_params jobParams = job->params;   /* do not modify job->params ! copy it, modify the copy */
         size_t const forceWindowError = ZSTD_CCtxParam_setParameter(&jobParams, ZSTD_p_forceMaxWindow, !job->firstChunk);
         U64 const pledgedSrcSize = job->firstChunk ? job->fullFrameSize : ZSTD_CONTENTSIZE_UNKNOWN;
         /* load dictionary in "content-only" mode (no header analysis) */
-        size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, job->srcStart, job->dictSize, ZSTD_dm_rawContent, jobParams, pledgedSrcSize);
+        size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, job->srcStart, job->prefixSize, ZSTD_dm_rawContent, jobParams, pledgedSrcSize);
+        DEBUGLOG(5, "ZSTD_compressBegin_advanced_internal called with windowLog = %u ", jobParams.cParams.windowLog);
         if (ZSTD_isError(initError) || ZSTD_isError(forceWindowError)) {
             job->cSize = initError;
             goto _endJob;
         }
     }
-    if (!job->firstChunk) {  /* flush and overwrite frame header when it's not first segment */
+    if (!job->firstChunk) {  /* flush and overwrite frame header when it's not first job */
         size_t const hSize = ZSTD_compressContinue(cctx, dstBuff.start, dstBuff.size, src, 0);
         if (ZSTD_isError(hSize)) { job->cSize = hSize; goto _endJob; }
         ZSTD_invalidateRepCodes(cctx);
@@ -380,9 +381,9 @@ void ZSTDMT_compressChunk(void* jobDescription)
     job->cSize = (job->lastChunk) ?
                  ZSTD_compressEnd     (cctx, dstBuff.start, dstBuff.size, src, job->srcSize) :
                  ZSTD_compressContinue(cctx, dstBuff.start, dstBuff.size, src, job->srcSize);
-    DEBUGLOG(5, "compressed %u bytes into %u bytes   (first:%u) (last:%u)",
+    DEBUGLOG(5, "compressed %u bytes into %u bytes   (first:%u) (last:%u) ",
                 (unsigned)job->srcSize, (unsigned)job->cSize, job->firstChunk, job->lastChunk);
-    DEBUGLOG(5, "dstBuff.size : %u ; => %s", (U32)dstBuff.size, ZSTD_getErrorName(job->cSize));
+    DEBUGLOG(5, "dstBuff.size : %u ; => %s ", (U32)dstBuff.size, ZSTD_getErrorName(job->cSize));
 
 _endJob:
     ZSTDMT_releaseCCtx(job->cctxPool, cctx);
@@ -618,7 +619,7 @@ static size_t ZSTDMT_compress_advanced_internal(
     size_t const overlapSize = (overlapRLog>=9) ? 0 : (size_t)1 << (params.cParams.windowLog - overlapRLog);
     unsigned nbChunks = computeNbChunks(srcSize, params.cParams.windowLog, params.nbThreads);
     size_t const proposedChunkSize = (srcSize + (nbChunks-1)) / nbChunks;
-    size_t const avgChunkSize = ((proposedChunkSize & 0x1FFFF) < 0x7FFF) ? proposedChunkSize + 0xFFFF : proposedChunkSize;   /* avoid too small last block */
+    size_t const avgChunkSize = (((proposedChunkSize-1) & 0x1FFFF) < 0x7FFF) ? proposedChunkSize + 0xFFFF : proposedChunkSize;   /* avoid too small last block */
     const char* const srcStart = (const char*)src;
     size_t remainingSrcSize = srcSize;
     unsigned const compressWithinDst = (dstCapacity >= ZSTD_compressBound(srcSize)) ? nbChunks : (unsigned)(dstCapacity / ZSTD_compressBound(avgChunkSize));  /* presumes avgChunkSize >= 256 KB, which should be the case */
@@ -628,7 +629,8 @@ static size_t ZSTDMT_compress_advanced_internal(
     assert(mtctx->cctxPool->totalCCtx == params.nbThreads);
 
     DEBUGLOG(4, "ZSTDMT_compress_advanced_internal");
-    DEBUGLOG(4, "nbChunks  : %2u   (chunkSize : %u bytes)   ", nbChunks, (U32)avgChunkSize);
+    DEBUGLOG(4, "nbChunks  : %2u (raw chunkSize : %u bytes; fixed chunkSize: %u)   ",
+                nbChunks, (U32)proposedChunkSize, (U32)avgChunkSize);
     if (nbChunks==1) {   /* fallback to single-thread mode */
         ZSTD_CCtx* const cctx = mtctx->cctxPool->cctx[0];
         if (cdict) return ZSTD_compress_usingCDict_advanced(cctx, dst, dstCapacity, src, srcSize, cdict, jobParams.fParams);
@@ -657,7 +659,7 @@ static size_t ZSTDMT_compress_advanced_internal(
 
             mtctx->jobs[u].src = g_nullBuffer;
             mtctx->jobs[u].srcStart = srcStart + frameStartPos - dictSize;
-            mtctx->jobs[u].dictSize = dictSize;
+            mtctx->jobs[u].prefixSize = dictSize;
             mtctx->jobs[u].srcSize = chunkSize;
             mtctx->jobs[u].cdict = (u==0) ? cdict : NULL;
             mtctx->jobs[u].fullFrameSize = srcSize;
@@ -888,7 +890,7 @@ static size_t ZSTDMT_createCompressionJob(ZSTDMT_CCtx* zcs, size_t srcSize, unsi
     zcs->jobs[jobID].src = zcs->inBuff.buffer;
     zcs->jobs[jobID].srcStart = zcs->inBuff.buffer.start;
     zcs->jobs[jobID].srcSize = srcSize;
-    zcs->jobs[jobID].dictSize = zcs->dictSize;
+    zcs->jobs[jobID].prefixSize = zcs->dictSize;
     assert(zcs->inBuff.filled >= srcSize + zcs->dictSize);
     zcs->jobs[jobID].params = zcs->params;
     /* do not calculate checksum within sections, but write it in header for first section */
@@ -1014,7 +1016,7 @@ size_t ZSTDMT_compressStream_generic(ZSTDMT_CCtx* mtctx,
 {
     size_t const newJobThreshold = mtctx->dictSize + mtctx->targetSectionSize;
     unsigned forwardInputProgress = 0;
-    DEBUGLOG(5, "ZSTDMT_compressStream_generic");
+    DEBUGLOG(5, "ZSTDMT_compressStream_generic ");
     assert(output->pos <= output->size);
     assert(input->pos  <= input->size);
 
diff --git a/lib/decompress/zstd_decompress.c b/lib/decompress/zstd_decompress.c
index f0cf967cb7b..a59d9441128 100644
--- a/lib/decompress/zstd_decompress.c
+++ b/lib/decompress/zstd_decompress.c
@@ -2451,14 +2451,16 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
                         return ZSTD_decompressLegacyStream(zds->legacyContext, legacyVersion, output, input);
                     }
 #endif
-                    return hSize; /* error */
+                    return hSize;   /* error */
                 }
                 if (hSize != 0) {   /* need more input */
                     size_t const toLoad = hSize - zds->lhSize;   /* if hSize!=0, hSize > zds->lhSize */
-                    if (toLoad > (size_t)(iend-ip)) {   /* not enough input to load full header */
-                        if (iend-ip > 0) {
-                            memcpy(zds->headerBuffer + zds->lhSize, ip, iend-ip);
-                            zds->lhSize += iend-ip;
+                    size_t const remainingInput = (size_t)(iend-ip);
+                    assert(iend >= ip);
+                    if (toLoad > remainingInput) {   /* not enough input to load full header */
+                        if (remainingInput > 0) {
+                            memcpy(zds->headerBuffer + zds->lhSize, ip, remainingInput);
+                            zds->lhSize += remainingInput;
                         }
                         input->pos = input->size;
                         return (MAX(ZSTD_frameHeaderSize_min, hSize) - zds->lhSize) + ZSTD_blockHeaderSize;   /* remaining header bytes + next block header */
@@ -2473,8 +2475,10 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
                 && (U64)(size_t)(oend-op) >= zds->fParams.frameContentSize) {
                 size_t const cSize = ZSTD_findFrameCompressedSize(istart, iend-istart);
                 if (cSize <= (size_t)(iend-istart)) {
+                    /* shortcut : using single-pass mode */
                     size_t const decompressedSize = ZSTD_decompress_usingDDict(zds, op, oend-op, istart, cSize, zds->ddict);
                     if (ZSTD_isError(decompressedSize)) return decompressedSize;
+                    DEBUGLOG(4, "shortcut to single-pass ZSTD_decompress_usingDDict()")
                     ip = istart + cSize;
                     op += decompressedSize;
                     zds->expected = 0;
@@ -2497,8 +2501,9 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
             }
 
             /* control buffer memory usage */
-            DEBUGLOG(4, "Control max buffer memory usage (max %u KB)",
-                        (U32)(zds->maxWindowSize >> 10));
+            DEBUGLOG(4, "Control max memory usage (%u KB <= max %u KB)",
+                        (U32)(zds->fParams.windowSize >>10),
+                        (U32)(zds->maxWindowSize >> 10) );
             zds->fParams.windowSize = MAX(zds->fParams.windowSize, 1U << ZSTD_WINDOWLOG_ABSOLUTEMIN);
             if (zds->fParams.windowSize > zds->maxWindowSize) return ERROR(frameParameter_windowTooLarge);
 
diff --git a/lib/zstd.h b/lib/zstd.h
index 15c467535a4..e4541c3f426 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -1016,7 +1016,8 @@ typedef enum {
                               * More threads improve speed, but also increase memory usage.
                               * Can only receive a value > 1 if ZSTD_MULTITHREAD is enabled.
                               * Special: value 0 means "do not change nbThreads" */
-    ZSTD_p_jobSize,          /* Size of a compression job. Each compression job is completed in parallel.
+    ZSTD_p_jobSize,          /* Size of a compression job. This value is only enforced in streaming (non-blocking) mode.
+                              * Each compression job is completed in parallel, so indirectly controls the nb of active threads.
                               * 0 means default, which is dynamically determined based on compression parameters.
                               * Job size must be a minimum of overlapSize, or 1 KB, whichever is largest
                               * The minimum size is automatically and transparently enforced */
@@ -1144,13 +1145,19 @@ typedef enum {
  *  - Compression parameters cannot be changed once compression is started.
  *  - outpot->pos must be <= dstCapacity, input->pos must be <= srcSize
  *  - outpot->pos and input->pos will be updated. They are guaranteed to remain below their respective limit.
- *  - @return provides the minimum amount of data still to flush from internal buffers
+ *  - In single-thread mode (default), function is blocking : it completed its job before returning to caller.
+ *  - In multi-thread mode, function is non-blocking : it just acquires a copy of input, and distribute job to internal worker threads,
+ *                                                     and then immediately returns, just indicating that there is some data remaining to be flushed.
+ *                                                     The function nonetheless guarantees forward progress : it will return only after it reads or write at least 1+ byte.
+ *  - Exception : in multi-threading mode, if the first call requests a ZSTD_e_end directive, it is blocking : it will complete compression before giving back control to caller.
+ *  - @return provides the minimum amount of data remaining to be flushed from internal buffers
  *            or an error code, which can be tested using ZSTD_isError().
- *            if @return != 0, flush is not fully completed, there is some data left within internal buffers.
- *  - after a ZSTD_e_end directive, if internal buffer is not fully flushed,
+ *            if @return != 0, flush is not fully completed, there is still some data left within internal buffers.
+ *            This is useful to determine if a ZSTD_e_flush or ZSTD_e_end directive is completed.
+ *  - after a ZSTD_e_end directive, if internal buffer is not fully flushed (@return != 0),
  *            only ZSTD_e_end or ZSTD_e_flush operations are allowed.
- *            It is necessary to fully flush internal buffers
- *            before starting a new compression job, or changing compression parameters.
+ *            Before starting a new compression job, or changing compression parameters,
+ *            it is required to fully flush internal buffers.
  */
 ZSTDLIB_API size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
                                           ZSTD_outBuffer* output,
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index e08a96d8fc9..6910457ff63 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -524,12 +524,12 @@ static int basicUnitTests(U32 seed, double compressibility)
     {   ZSTD_DDict* const ddict = ZSTD_createDDict(dictionary.start, dictionary.filled);
         size_t const initError = ZSTD_initDStream_usingDDict(zd, ddict);
         if (ZSTD_isError(initError)) goto _output_error;
-        inBuff.src = compressedBuffer;
-        inBuff.size = cSize;
-        inBuff.pos = 0;
         outBuff.dst = decodedBuffer;
         outBuff.size = CNBufferSize;
         outBuff.pos = 0;
+        inBuff.src = compressedBuffer;
+        inBuff.size = cSize;
+        inBuff.pos = 0;
         { size_t const r = ZSTD_decompressStream(zd, &outBuff, &inBuff);
           if (r != 0) goto _output_error; }  /* should reach end of frame == 0; otherwise, some data left, or an error */
         if (outBuff.pos != CNBufferSize) goto _output_error;   /* should regenerate the same amount */
@@ -548,12 +548,12 @@ static int basicUnitTests(U32 seed, double compressibility)
     DISPLAYLEVEL(3, "test%3i : maxWindowSize < frame requirement : ", testNb++);
     ZSTD_initDStream_usingDict(zd, CNBuffer, dictSize);
     CHECK_Z( ZSTD_setDStreamParameter(zd, DStream_p_maxWindowSize, 1000) );  /* too small limit */
-    inBuff.src = compressedBuffer;
-    inBuff.size = cSize;
-    inBuff.pos = 0;
     outBuff.dst = decodedBuffer;
     outBuff.size = CNBufferSize;
     outBuff.pos = 0;
+    inBuff.src = compressedBuffer;
+    inBuff.size = cSize;
+    inBuff.pos = 0;
     { size_t const r = ZSTD_decompressStream(zd, &outBuff, &inBuff);
       if (!ZSTD_isError(r)) goto _output_error;  /* must fail : frame requires > 100 bytes */
       DISPLAYLEVEL(3, "OK (%s)\n", ZSTD_getErrorName(r)); }
@@ -642,12 +642,12 @@ static int basicUnitTests(U32 seed, double compressibility)
         params.fParams.contentSizeFlag = 1;
         CHECK_Z( ZSTD_initCStream_advanced(zc, dictionary.start, dictionary.filled, params, 0 /* pledgedSrcSize==0 means "empty" when params.fParams.contentSizeFlag is set */) );
     } /* cstream advanced shall write content size = 0 */
-    inBuff.src = CNBuffer;
-    inBuff.size = 0;
-    inBuff.pos = 0;
     outBuff.dst = compressedBuffer;
     outBuff.size = compressedBufferSize;
     outBuff.pos = 0;
+    inBuff.src = CNBuffer;
+    inBuff.size = 0;
+    inBuff.pos = 0;
     CHECK_Z( ZSTD_compressStream(zc, &outBuff, &inBuff) );
     if (ZSTD_endStream(zc, &outBuff) != 0) goto _output_error;
     cSize = outBuff.pos;
@@ -671,12 +671,12 @@ static int basicUnitTests(U32 seed, double compressibility)
     if (ZSTD_findDecompressedSize(compressedBuffer, cSize) != 0) goto _output_error;
 
     ZSTD_resetCStream(zc, 0); /* resetCStream should treat 0 as unknown */
-    inBuff.src = CNBuffer;
-    inBuff.size = 0;
-    inBuff.pos = 0;
     outBuff.dst = compressedBuffer;
     outBuff.size = compressedBufferSize;
     outBuff.pos = 0;
+    inBuff.src = CNBuffer;
+    inBuff.size = 0;
+    inBuff.pos = 0;
     CHECK_Z( ZSTD_compressStream(zc, &outBuff, &inBuff) );
     if (ZSTD_endStream(zc, &outBuff) != 0) goto _output_error;
     cSize = outBuff.pos;
@@ -688,7 +688,7 @@ static int basicUnitTests(U32 seed, double compressibility)
     {   ZSTD_parameters const params = ZSTD_getParams(1, 0, 0);
         CHECK_Z( ZSTDMT_initCStream_advanced(mtctx, CNBuffer, dictSize, params, CNBufferSize) );
     }
-    outBuff.dst = (char*)(compressedBuffer);
+    outBuff.dst = compressedBuffer;
     outBuff.size = compressedBufferSize;
     outBuff.pos = 0;
     inBuff.src = CNBuffer;
@@ -700,6 +700,61 @@ static int basicUnitTests(U32 seed, double compressibility)
       if (r != 0) goto _output_error; }  /* error, or some data not flushed */
     DISPLAYLEVEL(3, "OK \n");
 
+    /* Complex multithreading + dictionary test */
+    {   U32 const nbThreads = 2;
+        size_t const jobSize = 4 * 1 MB;
+        size_t const srcSize = jobSize * nbThreads;  /* we want each job to have predictable size */
+        size_t const segLength = 2 KB;
+        size_t const offset = 600 KB;   /* must be larger than window defined in cdict */
+        size_t const start = jobSize + (offset-1);
+        const BYTE* const srcToCopy = (const BYTE*)CNBuffer + start;
+        BYTE* const dst = (BYTE*)CNBuffer + start - offset;
+        DISPLAYLEVEL(3, "test%3i : compress %u bytes with multiple threads + dictionary : ", testNb++, (U32)srcSize);
+        ZSTD_CCtx_setParameter(zc, ZSTD_p_compressionLevel, 3);
+        ZSTD_CCtx_setParameter(zc, ZSTD_p_nbThreads, 2);
+        ZSTD_CCtx_setParameter(zc, ZSTD_p_jobSize, jobSize);
+        assert(start > offset);
+        assert(start + segLength < COMPRESSIBLE_NOISE_LENGTH);
+        memcpy(dst, srcToCopy, segLength);   /* create a long repetition at long distance for job 2 */
+        outBuff.dst = compressedBuffer;
+        outBuff.size = compressedBufferSize;
+        outBuff.pos = 0;
+        inBuff.src = CNBuffer;
+        inBuff.size = srcSize; assert(srcSize < COMPRESSIBLE_NOISE_LENGTH);
+        inBuff.pos = 0;
+    }
+    {   ZSTD_compressionParameters const cParams = ZSTD_getCParams(1, 4 KB, dictionary.filled);   /* intentionnally lies on estimatedSrcSize, to push cdict into targeting a small window size */
+        ZSTD_CDict* const cdict = ZSTD_createCDict_advanced(dictionary.start, dictionary.filled, ZSTD_dlm_byRef, ZSTD_dm_fullDict, cParams, ZSTD_defaultCMem);
+        DISPLAYLEVEL(5, "cParams.windowLog = %u : ", cParams.windowLog);
+        ZSTD_CCtx_refCDict(zc, cdict);
+        CHECK_Z( ZSTD_compress_generic(zc, &outBuff, &inBuff, ZSTD_e_end) );
+        ZSTD_CCtx_refCDict(zc, NULL);  /* do not keep a reference to cdict, as its lifetime ends */
+        ZSTD_freeCDict(cdict);
+    }
+    if (inBuff.pos != inBuff.size) goto _output_error;   /* entire input should be consumed */
+    cSize = outBuff.pos;
+    DISPLAYLEVEL(3, "OK \n");
+
+    DISPLAYLEVEL(3, "test%3i : decompress large frame created from multiple threads + dictionary : ", testNb++);
+    {   ZSTD_DStream* const dstream = ZSTD_createDCtx();
+        ZSTD_frameHeader zfh;
+        ZSTD_getFrameHeader(&zfh, compressedBuffer, cSize);
+        DISPLAYLEVEL(5, "frame windowsize = %u : ", (U32)zfh.windowSize);
+        outBuff.dst = decodedBuffer;
+        outBuff.size = CNBufferSize;
+        outBuff.pos = 0;
+        inBuff.src = compressedBuffer;
+        inBuff.pos = 0;
+        CHECK_Z( ZSTD_initDStream_usingDict(dstream, dictionary.start, dictionary.filled) );
+        inBuff.size = 1;  /* avoid shortcut to single-pass mode */
+        CHECK_Z( ZSTD_decompressStream(dstream, &outBuff, &inBuff) );
+        inBuff.size = cSize;
+        CHECK_Z( ZSTD_decompressStream(dstream, &outBuff, &inBuff) );
+        if (inBuff.pos != inBuff.size) goto _output_error;   /* entire input should be consumed */
+        ZSTD_freeDStream(dstream);
+    }
+    DISPLAYLEVEL(3, "OK \n");
+
     DISPLAYLEVEL(3, "test%3i : check dictionary FSE tables can represent every code : ", testNb++);
     {   unsigned const kMaxWindowLog = 24;
         unsigned value;

From 8a104fda05e357d89dc8b263a48aa00ebe5d2603 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 12 Dec 2017 12:51:49 -0800
Subject: [PATCH 118/145] Revert "Created a test case which reliably reproduces
 bug #944"

This reverts commit 5098d1fbe2a0b3d0ddb66c0880e3004feac65f3e.
---
 lib/compress/zstd_compress.c     | 11 +++--
 lib/compress/zstdmt_compress.c   | 30 ++++++------
 lib/decompress/zstd_decompress.c | 19 +++-----
 lib/zstd.h                       | 19 +++-----
 tests/zstreamtest.c              | 81 +++++---------------------------
 5 files changed, 46 insertions(+), 114 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 1072918885f..3f39875f5bd 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -477,7 +477,7 @@ size_t ZSTD_CCtx_setParametersUsingCCtxParams(
 
 ZSTDLIB_API size_t ZSTD_CCtx_setPledgedSrcSize(ZSTD_CCtx* cctx, unsigned long long pledgedSrcSize)
 {
-    DEBUGLOG(4, "ZSTD_CCtx_setPledgedSrcSize to %u bytes", (U32)pledgedSrcSize);
+    DEBUGLOG(4, " setting pledgedSrcSize to %u", (U32)pledgedSrcSize);
     if (cctx->streamStage != zcss_init) return ERROR(stage_wrong);
     cctx->pledgedSrcSizePlusOne = pledgedSrcSize+1;
     return 0;
@@ -489,7 +489,7 @@ size_t ZSTD_CCtx_loadDictionary_advanced(
 {
     if (cctx->streamStage != zcss_init) return ERROR(stage_wrong);
     if (cctx->staticSize) return ERROR(memory_allocation);  /* no malloc for static CCtx */
-    DEBUGLOG(4, "ZSTD_CCtx_loadDictionary_advanced (size: %u)", (U32)dictSize);
+    DEBUGLOG(4, "load dictionary of size %u", (U32)dictSize);
     ZSTD_freeCDict(cctx->cdictLocal);  /* in case one already exists */
     if (dict==NULL || dictSize==0) {   /* no dictionary mode */
         cctx->cdictLocal = NULL;
@@ -2273,7 +2273,7 @@ static size_t ZSTD_initCDict_internal(
                     ZSTD_dictMode_e dictMode,
                     ZSTD_compressionParameters cParams)
 {
-    DEBUGLOG(3, "ZSTD_initCDict_internal, mode %u", (U32)dictMode);
+    DEBUGLOG(4, "ZSTD_initCDict_internal, mode %u", (U32)dictMode);
     if ((dictLoadMethod == ZSTD_dlm_byRef) || (!dictBuffer) || (!dictSize)) {
         cdict->dictBuffer = NULL;
         cdict->dictContent = dictBuffer;
@@ -2303,7 +2303,7 @@ ZSTD_CDict* ZSTD_createCDict_advanced(const void* dictBuffer, size_t dictSize,
                                       ZSTD_dictMode_e dictMode,
                                       ZSTD_compressionParameters cParams, ZSTD_customMem customMem)
 {
-    DEBUGLOG(3, "ZSTD_createCDict_advanced, mode %u", (U32)dictMode);
+    DEBUGLOG(4, "ZSTD_createCDict_advanced, mode %u", (U32)dictMode);
     if (!customMem.customAlloc ^ !customMem.customFree) return NULL;
 
     {   ZSTD_CDict* const cdict = (ZSTD_CDict*)ZSTD_malloc(sizeof(ZSTD_CDict), customMem);
@@ -2809,7 +2809,7 @@ size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
                               ZSTD_inBuffer* input,
                               ZSTD_EndDirective endOp)
 {
-    DEBUGLOG(5, "ZSTD_compress_generic, endOp=%u ", (U32)endOp);
+    DEBUGLOG(5, "ZSTD_compress_generic");
     /* check conditions */
     if (output->pos > output->size) return ERROR(GENERIC);
     if (input->pos  > input->size)  return ERROR(GENERIC);
@@ -2856,6 +2856,7 @@ size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
 #ifdef ZSTD_MULTITHREAD
     if (cctx->appliedParams.nbThreads > 1) {
         size_t const flushMin = ZSTDMT_compressStream_generic(cctx->mtctx, output, input, endOp);
+        DEBUGLOG(5, "ZSTDMT_compressStream_generic result : %u", (U32)flushMin);
         if ( ZSTD_isError(flushMin)
           || (endOp == ZSTD_e_end && flushMin == 0) ) { /* compression completed */
             ZSTD_startNewCompression(cctx);
diff --git a/lib/compress/zstdmt_compress.c b/lib/compress/zstdmt_compress.c
index 8347cf29d5c..6b6466fe56d 100644
--- a/lib/compress/zstdmt_compress.c
+++ b/lib/compress/zstdmt_compress.c
@@ -310,7 +310,7 @@ static void ZSTDMT_releaseCCtx(ZSTDMT_CCtxPool* pool, ZSTD_CCtx* cctx)
 typedef struct {
     buffer_t src;
     const void* srcStart;
-    size_t   prefixSize;
+    size_t   dictSize;
     size_t   srcSize;
     buffer_t dstBuff;
     size_t   cSize;
@@ -333,10 +333,10 @@ void ZSTDMT_compressChunk(void* jobDescription)
 {
     ZSTDMT_jobDescription* const job = (ZSTDMT_jobDescription*)jobDescription;
     ZSTD_CCtx* const cctx = ZSTDMT_getCCtx(job->cctxPool);
-    const void* const src = (const char*)job->srcStart + job->prefixSize;
+    const void* const src = (const char*)job->srcStart + job->dictSize;
     buffer_t dstBuff = job->dstBuff;
-    DEBUGLOG(5, "ZSTDMT_compressChunk: job (first:%u) (last:%u) : prefixSize %u, srcSize %u ",
-                 job->firstChunk, job->lastChunk, (U32)job->prefixSize, (U32)job->srcSize);
+    DEBUGLOG(5, "ZSTDMT_compressChunk: job (first:%u) (last:%u) : dictSize %u, srcSize %u",
+                 job->firstChunk, job->lastChunk, (U32)job->dictSize, (U32)job->srcSize);
 
     if (cctx==NULL) {
         job->cSize = ERROR(memory_allocation);
@@ -356,21 +356,20 @@ void ZSTDMT_compressChunk(void* jobDescription)
     if (job->cdict) {
         size_t const initError = ZSTD_compressBegin_usingCDict_advanced(cctx, job->cdict, job->params.fParams, job->fullFrameSize);
         DEBUGLOG(4, "ZSTDMT_compressChunk: init using CDict");
-        assert(job->firstChunk);  /* only allowed for first job */
+        assert(job->firstChunk); /* should only happen for first segment */
         if (ZSTD_isError(initError)) { job->cSize = initError; goto _endJob; }
     } else {  /* srcStart points at reloaded section */
         ZSTD_CCtx_params jobParams = job->params;   /* do not modify job->params ! copy it, modify the copy */
         size_t const forceWindowError = ZSTD_CCtxParam_setParameter(&jobParams, ZSTD_p_forceMaxWindow, !job->firstChunk);
         U64 const pledgedSrcSize = job->firstChunk ? job->fullFrameSize : ZSTD_CONTENTSIZE_UNKNOWN;
         /* load dictionary in "content-only" mode (no header analysis) */
-        size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, job->srcStart, job->prefixSize, ZSTD_dm_rawContent, jobParams, pledgedSrcSize);
-        DEBUGLOG(5, "ZSTD_compressBegin_advanced_internal called with windowLog = %u ", jobParams.cParams.windowLog);
+        size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, job->srcStart, job->dictSize, ZSTD_dm_rawContent, jobParams, pledgedSrcSize);
         if (ZSTD_isError(initError) || ZSTD_isError(forceWindowError)) {
             job->cSize = initError;
             goto _endJob;
         }
     }
-    if (!job->firstChunk) {  /* flush and overwrite frame header when it's not first job */
+    if (!job->firstChunk) {  /* flush and overwrite frame header when it's not first segment */
         size_t const hSize = ZSTD_compressContinue(cctx, dstBuff.start, dstBuff.size, src, 0);
         if (ZSTD_isError(hSize)) { job->cSize = hSize; goto _endJob; }
         ZSTD_invalidateRepCodes(cctx);
@@ -381,9 +380,9 @@ void ZSTDMT_compressChunk(void* jobDescription)
     job->cSize = (job->lastChunk) ?
                  ZSTD_compressEnd     (cctx, dstBuff.start, dstBuff.size, src, job->srcSize) :
                  ZSTD_compressContinue(cctx, dstBuff.start, dstBuff.size, src, job->srcSize);
-    DEBUGLOG(5, "compressed %u bytes into %u bytes   (first:%u) (last:%u) ",
+    DEBUGLOG(5, "compressed %u bytes into %u bytes   (first:%u) (last:%u)",
                 (unsigned)job->srcSize, (unsigned)job->cSize, job->firstChunk, job->lastChunk);
-    DEBUGLOG(5, "dstBuff.size : %u ; => %s ", (U32)dstBuff.size, ZSTD_getErrorName(job->cSize));
+    DEBUGLOG(5, "dstBuff.size : %u ; => %s", (U32)dstBuff.size, ZSTD_getErrorName(job->cSize));
 
 _endJob:
     ZSTDMT_releaseCCtx(job->cctxPool, cctx);
@@ -619,7 +618,7 @@ static size_t ZSTDMT_compress_advanced_internal(
     size_t const overlapSize = (overlapRLog>=9) ? 0 : (size_t)1 << (params.cParams.windowLog - overlapRLog);
     unsigned nbChunks = computeNbChunks(srcSize, params.cParams.windowLog, params.nbThreads);
     size_t const proposedChunkSize = (srcSize + (nbChunks-1)) / nbChunks;
-    size_t const avgChunkSize = (((proposedChunkSize-1) & 0x1FFFF) < 0x7FFF) ? proposedChunkSize + 0xFFFF : proposedChunkSize;   /* avoid too small last block */
+    size_t const avgChunkSize = ((proposedChunkSize & 0x1FFFF) < 0x7FFF) ? proposedChunkSize + 0xFFFF : proposedChunkSize;   /* avoid too small last block */
     const char* const srcStart = (const char*)src;
     size_t remainingSrcSize = srcSize;
     unsigned const compressWithinDst = (dstCapacity >= ZSTD_compressBound(srcSize)) ? nbChunks : (unsigned)(dstCapacity / ZSTD_compressBound(avgChunkSize));  /* presumes avgChunkSize >= 256 KB, which should be the case */
@@ -629,8 +628,7 @@ static size_t ZSTDMT_compress_advanced_internal(
     assert(mtctx->cctxPool->totalCCtx == params.nbThreads);
 
     DEBUGLOG(4, "ZSTDMT_compress_advanced_internal");
-    DEBUGLOG(4, "nbChunks  : %2u (raw chunkSize : %u bytes; fixed chunkSize: %u)   ",
-                nbChunks, (U32)proposedChunkSize, (U32)avgChunkSize);
+    DEBUGLOG(4, "nbChunks  : %2u   (chunkSize : %u bytes)   ", nbChunks, (U32)avgChunkSize);
     if (nbChunks==1) {   /* fallback to single-thread mode */
         ZSTD_CCtx* const cctx = mtctx->cctxPool->cctx[0];
         if (cdict) return ZSTD_compress_usingCDict_advanced(cctx, dst, dstCapacity, src, srcSize, cdict, jobParams.fParams);
@@ -659,7 +657,7 @@ static size_t ZSTDMT_compress_advanced_internal(
 
             mtctx->jobs[u].src = g_nullBuffer;
             mtctx->jobs[u].srcStart = srcStart + frameStartPos - dictSize;
-            mtctx->jobs[u].prefixSize = dictSize;
+            mtctx->jobs[u].dictSize = dictSize;
             mtctx->jobs[u].srcSize = chunkSize;
             mtctx->jobs[u].cdict = (u==0) ? cdict : NULL;
             mtctx->jobs[u].fullFrameSize = srcSize;
@@ -890,7 +888,7 @@ static size_t ZSTDMT_createCompressionJob(ZSTDMT_CCtx* zcs, size_t srcSize, unsi
     zcs->jobs[jobID].src = zcs->inBuff.buffer;
     zcs->jobs[jobID].srcStart = zcs->inBuff.buffer.start;
     zcs->jobs[jobID].srcSize = srcSize;
-    zcs->jobs[jobID].prefixSize = zcs->dictSize;
+    zcs->jobs[jobID].dictSize = zcs->dictSize;
     assert(zcs->inBuff.filled >= srcSize + zcs->dictSize);
     zcs->jobs[jobID].params = zcs->params;
     /* do not calculate checksum within sections, but write it in header for first section */
@@ -1016,7 +1014,7 @@ size_t ZSTDMT_compressStream_generic(ZSTDMT_CCtx* mtctx,
 {
     size_t const newJobThreshold = mtctx->dictSize + mtctx->targetSectionSize;
     unsigned forwardInputProgress = 0;
-    DEBUGLOG(5, "ZSTDMT_compressStream_generic ");
+    DEBUGLOG(5, "ZSTDMT_compressStream_generic");
     assert(output->pos <= output->size);
     assert(input->pos  <= input->size);
 
diff --git a/lib/decompress/zstd_decompress.c b/lib/decompress/zstd_decompress.c
index a59d9441128..f0cf967cb7b 100644
--- a/lib/decompress/zstd_decompress.c
+++ b/lib/decompress/zstd_decompress.c
@@ -2451,16 +2451,14 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
                         return ZSTD_decompressLegacyStream(zds->legacyContext, legacyVersion, output, input);
                     }
 #endif
-                    return hSize;   /* error */
+                    return hSize; /* error */
                 }
                 if (hSize != 0) {   /* need more input */
                     size_t const toLoad = hSize - zds->lhSize;   /* if hSize!=0, hSize > zds->lhSize */
-                    size_t const remainingInput = (size_t)(iend-ip);
-                    assert(iend >= ip);
-                    if (toLoad > remainingInput) {   /* not enough input to load full header */
-                        if (remainingInput > 0) {
-                            memcpy(zds->headerBuffer + zds->lhSize, ip, remainingInput);
-                            zds->lhSize += remainingInput;
+                    if (toLoad > (size_t)(iend-ip)) {   /* not enough input to load full header */
+                        if (iend-ip > 0) {
+                            memcpy(zds->headerBuffer + zds->lhSize, ip, iend-ip);
+                            zds->lhSize += iend-ip;
                         }
                         input->pos = input->size;
                         return (MAX(ZSTD_frameHeaderSize_min, hSize) - zds->lhSize) + ZSTD_blockHeaderSize;   /* remaining header bytes + next block header */
@@ -2475,10 +2473,8 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
                 && (U64)(size_t)(oend-op) >= zds->fParams.frameContentSize) {
                 size_t const cSize = ZSTD_findFrameCompressedSize(istart, iend-istart);
                 if (cSize <= (size_t)(iend-istart)) {
-                    /* shortcut : using single-pass mode */
                     size_t const decompressedSize = ZSTD_decompress_usingDDict(zds, op, oend-op, istart, cSize, zds->ddict);
                     if (ZSTD_isError(decompressedSize)) return decompressedSize;
-                    DEBUGLOG(4, "shortcut to single-pass ZSTD_decompress_usingDDict()")
                     ip = istart + cSize;
                     op += decompressedSize;
                     zds->expected = 0;
@@ -2501,9 +2497,8 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
             }
 
             /* control buffer memory usage */
-            DEBUGLOG(4, "Control max memory usage (%u KB <= max %u KB)",
-                        (U32)(zds->fParams.windowSize >>10),
-                        (U32)(zds->maxWindowSize >> 10) );
+            DEBUGLOG(4, "Control max buffer memory usage (max %u KB)",
+                        (U32)(zds->maxWindowSize >> 10));
             zds->fParams.windowSize = MAX(zds->fParams.windowSize, 1U << ZSTD_WINDOWLOG_ABSOLUTEMIN);
             if (zds->fParams.windowSize > zds->maxWindowSize) return ERROR(frameParameter_windowTooLarge);
 
diff --git a/lib/zstd.h b/lib/zstd.h
index e4541c3f426..15c467535a4 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -1016,8 +1016,7 @@ typedef enum {
                               * More threads improve speed, but also increase memory usage.
                               * Can only receive a value > 1 if ZSTD_MULTITHREAD is enabled.
                               * Special: value 0 means "do not change nbThreads" */
-    ZSTD_p_jobSize,          /* Size of a compression job. This value is only enforced in streaming (non-blocking) mode.
-                              * Each compression job is completed in parallel, so indirectly controls the nb of active threads.
+    ZSTD_p_jobSize,          /* Size of a compression job. Each compression job is completed in parallel.
                               * 0 means default, which is dynamically determined based on compression parameters.
                               * Job size must be a minimum of overlapSize, or 1 KB, whichever is largest
                               * The minimum size is automatically and transparently enforced */
@@ -1145,19 +1144,13 @@ typedef enum {
  *  - Compression parameters cannot be changed once compression is started.
  *  - outpot->pos must be <= dstCapacity, input->pos must be <= srcSize
  *  - outpot->pos and input->pos will be updated. They are guaranteed to remain below their respective limit.
- *  - In single-thread mode (default), function is blocking : it completed its job before returning to caller.
- *  - In multi-thread mode, function is non-blocking : it just acquires a copy of input, and distribute job to internal worker threads,
- *                                                     and then immediately returns, just indicating that there is some data remaining to be flushed.
- *                                                     The function nonetheless guarantees forward progress : it will return only after it reads or write at least 1+ byte.
- *  - Exception : in multi-threading mode, if the first call requests a ZSTD_e_end directive, it is blocking : it will complete compression before giving back control to caller.
- *  - @return provides the minimum amount of data remaining to be flushed from internal buffers
+ *  - @return provides the minimum amount of data still to flush from internal buffers
  *            or an error code, which can be tested using ZSTD_isError().
- *            if @return != 0, flush is not fully completed, there is still some data left within internal buffers.
- *            This is useful to determine if a ZSTD_e_flush or ZSTD_e_end directive is completed.
- *  - after a ZSTD_e_end directive, if internal buffer is not fully flushed (@return != 0),
+ *            if @return != 0, flush is not fully completed, there is some data left within internal buffers.
+ *  - after a ZSTD_e_end directive, if internal buffer is not fully flushed,
  *            only ZSTD_e_end or ZSTD_e_flush operations are allowed.
- *            Before starting a new compression job, or changing compression parameters,
- *            it is required to fully flush internal buffers.
+ *            It is necessary to fully flush internal buffers
+ *            before starting a new compression job, or changing compression parameters.
  */
 ZSTDLIB_API size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
                                           ZSTD_outBuffer* output,
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 6910457ff63..e08a96d8fc9 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -524,12 +524,12 @@ static int basicUnitTests(U32 seed, double compressibility)
     {   ZSTD_DDict* const ddict = ZSTD_createDDict(dictionary.start, dictionary.filled);
         size_t const initError = ZSTD_initDStream_usingDDict(zd, ddict);
         if (ZSTD_isError(initError)) goto _output_error;
-        outBuff.dst = decodedBuffer;
-        outBuff.size = CNBufferSize;
-        outBuff.pos = 0;
         inBuff.src = compressedBuffer;
         inBuff.size = cSize;
         inBuff.pos = 0;
+        outBuff.dst = decodedBuffer;
+        outBuff.size = CNBufferSize;
+        outBuff.pos = 0;
         { size_t const r = ZSTD_decompressStream(zd, &outBuff, &inBuff);
           if (r != 0) goto _output_error; }  /* should reach end of frame == 0; otherwise, some data left, or an error */
         if (outBuff.pos != CNBufferSize) goto _output_error;   /* should regenerate the same amount */
@@ -548,12 +548,12 @@ static int basicUnitTests(U32 seed, double compressibility)
     DISPLAYLEVEL(3, "test%3i : maxWindowSize < frame requirement : ", testNb++);
     ZSTD_initDStream_usingDict(zd, CNBuffer, dictSize);
     CHECK_Z( ZSTD_setDStreamParameter(zd, DStream_p_maxWindowSize, 1000) );  /* too small limit */
-    outBuff.dst = decodedBuffer;
-    outBuff.size = CNBufferSize;
-    outBuff.pos = 0;
     inBuff.src = compressedBuffer;
     inBuff.size = cSize;
     inBuff.pos = 0;
+    outBuff.dst = decodedBuffer;
+    outBuff.size = CNBufferSize;
+    outBuff.pos = 0;
     { size_t const r = ZSTD_decompressStream(zd, &outBuff, &inBuff);
       if (!ZSTD_isError(r)) goto _output_error;  /* must fail : frame requires > 100 bytes */
       DISPLAYLEVEL(3, "OK (%s)\n", ZSTD_getErrorName(r)); }
@@ -642,12 +642,12 @@ static int basicUnitTests(U32 seed, double compressibility)
         params.fParams.contentSizeFlag = 1;
         CHECK_Z( ZSTD_initCStream_advanced(zc, dictionary.start, dictionary.filled, params, 0 /* pledgedSrcSize==0 means "empty" when params.fParams.contentSizeFlag is set */) );
     } /* cstream advanced shall write content size = 0 */
-    outBuff.dst = compressedBuffer;
-    outBuff.size = compressedBufferSize;
-    outBuff.pos = 0;
     inBuff.src = CNBuffer;
     inBuff.size = 0;
     inBuff.pos = 0;
+    outBuff.dst = compressedBuffer;
+    outBuff.size = compressedBufferSize;
+    outBuff.pos = 0;
     CHECK_Z( ZSTD_compressStream(zc, &outBuff, &inBuff) );
     if (ZSTD_endStream(zc, &outBuff) != 0) goto _output_error;
     cSize = outBuff.pos;
@@ -671,12 +671,12 @@ static int basicUnitTests(U32 seed, double compressibility)
     if (ZSTD_findDecompressedSize(compressedBuffer, cSize) != 0) goto _output_error;
 
     ZSTD_resetCStream(zc, 0); /* resetCStream should treat 0 as unknown */
-    outBuff.dst = compressedBuffer;
-    outBuff.size = compressedBufferSize;
-    outBuff.pos = 0;
     inBuff.src = CNBuffer;
     inBuff.size = 0;
     inBuff.pos = 0;
+    outBuff.dst = compressedBuffer;
+    outBuff.size = compressedBufferSize;
+    outBuff.pos = 0;
     CHECK_Z( ZSTD_compressStream(zc, &outBuff, &inBuff) );
     if (ZSTD_endStream(zc, &outBuff) != 0) goto _output_error;
     cSize = outBuff.pos;
@@ -688,7 +688,7 @@ static int basicUnitTests(U32 seed, double compressibility)
     {   ZSTD_parameters const params = ZSTD_getParams(1, 0, 0);
         CHECK_Z( ZSTDMT_initCStream_advanced(mtctx, CNBuffer, dictSize, params, CNBufferSize) );
     }
-    outBuff.dst = compressedBuffer;
+    outBuff.dst = (char*)(compressedBuffer);
     outBuff.size = compressedBufferSize;
     outBuff.pos = 0;
     inBuff.src = CNBuffer;
@@ -700,61 +700,6 @@ static int basicUnitTests(U32 seed, double compressibility)
       if (r != 0) goto _output_error; }  /* error, or some data not flushed */
     DISPLAYLEVEL(3, "OK \n");
 
-    /* Complex multithreading + dictionary test */
-    {   U32 const nbThreads = 2;
-        size_t const jobSize = 4 * 1 MB;
-        size_t const srcSize = jobSize * nbThreads;  /* we want each job to have predictable size */
-        size_t const segLength = 2 KB;
-        size_t const offset = 600 KB;   /* must be larger than window defined in cdict */
-        size_t const start = jobSize + (offset-1);
-        const BYTE* const srcToCopy = (const BYTE*)CNBuffer + start;
-        BYTE* const dst = (BYTE*)CNBuffer + start - offset;
-        DISPLAYLEVEL(3, "test%3i : compress %u bytes with multiple threads + dictionary : ", testNb++, (U32)srcSize);
-        ZSTD_CCtx_setParameter(zc, ZSTD_p_compressionLevel, 3);
-        ZSTD_CCtx_setParameter(zc, ZSTD_p_nbThreads, 2);
-        ZSTD_CCtx_setParameter(zc, ZSTD_p_jobSize, jobSize);
-        assert(start > offset);
-        assert(start + segLength < COMPRESSIBLE_NOISE_LENGTH);
-        memcpy(dst, srcToCopy, segLength);   /* create a long repetition at long distance for job 2 */
-        outBuff.dst = compressedBuffer;
-        outBuff.size = compressedBufferSize;
-        outBuff.pos = 0;
-        inBuff.src = CNBuffer;
-        inBuff.size = srcSize; assert(srcSize < COMPRESSIBLE_NOISE_LENGTH);
-        inBuff.pos = 0;
-    }
-    {   ZSTD_compressionParameters const cParams = ZSTD_getCParams(1, 4 KB, dictionary.filled);   /* intentionnally lies on estimatedSrcSize, to push cdict into targeting a small window size */
-        ZSTD_CDict* const cdict = ZSTD_createCDict_advanced(dictionary.start, dictionary.filled, ZSTD_dlm_byRef, ZSTD_dm_fullDict, cParams, ZSTD_defaultCMem);
-        DISPLAYLEVEL(5, "cParams.windowLog = %u : ", cParams.windowLog);
-        ZSTD_CCtx_refCDict(zc, cdict);
-        CHECK_Z( ZSTD_compress_generic(zc, &outBuff, &inBuff, ZSTD_e_end) );
-        ZSTD_CCtx_refCDict(zc, NULL);  /* do not keep a reference to cdict, as its lifetime ends */
-        ZSTD_freeCDict(cdict);
-    }
-    if (inBuff.pos != inBuff.size) goto _output_error;   /* entire input should be consumed */
-    cSize = outBuff.pos;
-    DISPLAYLEVEL(3, "OK \n");
-
-    DISPLAYLEVEL(3, "test%3i : decompress large frame created from multiple threads + dictionary : ", testNb++);
-    {   ZSTD_DStream* const dstream = ZSTD_createDCtx();
-        ZSTD_frameHeader zfh;
-        ZSTD_getFrameHeader(&zfh, compressedBuffer, cSize);
-        DISPLAYLEVEL(5, "frame windowsize = %u : ", (U32)zfh.windowSize);
-        outBuff.dst = decodedBuffer;
-        outBuff.size = CNBufferSize;
-        outBuff.pos = 0;
-        inBuff.src = compressedBuffer;
-        inBuff.pos = 0;
-        CHECK_Z( ZSTD_initDStream_usingDict(dstream, dictionary.start, dictionary.filled) );
-        inBuff.size = 1;  /* avoid shortcut to single-pass mode */
-        CHECK_Z( ZSTD_decompressStream(dstream, &outBuff, &inBuff) );
-        inBuff.size = cSize;
-        CHECK_Z( ZSTD_decompressStream(dstream, &outBuff, &inBuff) );
-        if (inBuff.pos != inBuff.size) goto _output_error;   /* entire input should be consumed */
-        ZSTD_freeDStream(dstream);
-    }
-    DISPLAYLEVEL(3, "OK \n");
-
     DISPLAYLEVEL(3, "test%3i : check dictionary FSE tables can represent every code : ", testNb++);
     {   unsigned const kMaxWindowLog = 24;
         unsigned value;

From 03832b7aa590ed708354b086842406bc7c4f9b3a Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 12 Dec 2017 14:01:54 -0800
Subject: [PATCH 119/145] re-added test case

messing with revert ... :(
---
 lib/compress/zstd_compress.c     | 11 ++---
 lib/compress/zstdmt_compress.c   | 30 ++++++------
 lib/decompress/zstd_decompress.c | 19 +++++---
 lib/zstd.h                       | 19 +++++---
 tests/zstreamtest.c              | 81 +++++++++++++++++++++++++++-----
 5 files changed, 114 insertions(+), 46 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 3f39875f5bd..1072918885f 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -477,7 +477,7 @@ size_t ZSTD_CCtx_setParametersUsingCCtxParams(
 
 ZSTDLIB_API size_t ZSTD_CCtx_setPledgedSrcSize(ZSTD_CCtx* cctx, unsigned long long pledgedSrcSize)
 {
-    DEBUGLOG(4, " setting pledgedSrcSize to %u", (U32)pledgedSrcSize);
+    DEBUGLOG(4, "ZSTD_CCtx_setPledgedSrcSize to %u bytes", (U32)pledgedSrcSize);
     if (cctx->streamStage != zcss_init) return ERROR(stage_wrong);
     cctx->pledgedSrcSizePlusOne = pledgedSrcSize+1;
     return 0;
@@ -489,7 +489,7 @@ size_t ZSTD_CCtx_loadDictionary_advanced(
 {
     if (cctx->streamStage != zcss_init) return ERROR(stage_wrong);
     if (cctx->staticSize) return ERROR(memory_allocation);  /* no malloc for static CCtx */
-    DEBUGLOG(4, "load dictionary of size %u", (U32)dictSize);
+    DEBUGLOG(4, "ZSTD_CCtx_loadDictionary_advanced (size: %u)", (U32)dictSize);
     ZSTD_freeCDict(cctx->cdictLocal);  /* in case one already exists */
     if (dict==NULL || dictSize==0) {   /* no dictionary mode */
         cctx->cdictLocal = NULL;
@@ -2273,7 +2273,7 @@ static size_t ZSTD_initCDict_internal(
                     ZSTD_dictMode_e dictMode,
                     ZSTD_compressionParameters cParams)
 {
-    DEBUGLOG(4, "ZSTD_initCDict_internal, mode %u", (U32)dictMode);
+    DEBUGLOG(3, "ZSTD_initCDict_internal, mode %u", (U32)dictMode);
     if ((dictLoadMethod == ZSTD_dlm_byRef) || (!dictBuffer) || (!dictSize)) {
         cdict->dictBuffer = NULL;
         cdict->dictContent = dictBuffer;
@@ -2303,7 +2303,7 @@ ZSTD_CDict* ZSTD_createCDict_advanced(const void* dictBuffer, size_t dictSize,
                                       ZSTD_dictMode_e dictMode,
                                       ZSTD_compressionParameters cParams, ZSTD_customMem customMem)
 {
-    DEBUGLOG(4, "ZSTD_createCDict_advanced, mode %u", (U32)dictMode);
+    DEBUGLOG(3, "ZSTD_createCDict_advanced, mode %u", (U32)dictMode);
     if (!customMem.customAlloc ^ !customMem.customFree) return NULL;
 
     {   ZSTD_CDict* const cdict = (ZSTD_CDict*)ZSTD_malloc(sizeof(ZSTD_CDict), customMem);
@@ -2809,7 +2809,7 @@ size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
                               ZSTD_inBuffer* input,
                               ZSTD_EndDirective endOp)
 {
-    DEBUGLOG(5, "ZSTD_compress_generic");
+    DEBUGLOG(5, "ZSTD_compress_generic, endOp=%u ", (U32)endOp);
     /* check conditions */
     if (output->pos > output->size) return ERROR(GENERIC);
     if (input->pos  > input->size)  return ERROR(GENERIC);
@@ -2856,7 +2856,6 @@ size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
 #ifdef ZSTD_MULTITHREAD
     if (cctx->appliedParams.nbThreads > 1) {
         size_t const flushMin = ZSTDMT_compressStream_generic(cctx->mtctx, output, input, endOp);
-        DEBUGLOG(5, "ZSTDMT_compressStream_generic result : %u", (U32)flushMin);
         if ( ZSTD_isError(flushMin)
           || (endOp == ZSTD_e_end && flushMin == 0) ) { /* compression completed */
             ZSTD_startNewCompression(cctx);
diff --git a/lib/compress/zstdmt_compress.c b/lib/compress/zstdmt_compress.c
index 6b6466fe56d..8347cf29d5c 100644
--- a/lib/compress/zstdmt_compress.c
+++ b/lib/compress/zstdmt_compress.c
@@ -310,7 +310,7 @@ static void ZSTDMT_releaseCCtx(ZSTDMT_CCtxPool* pool, ZSTD_CCtx* cctx)
 typedef struct {
     buffer_t src;
     const void* srcStart;
-    size_t   dictSize;
+    size_t   prefixSize;
     size_t   srcSize;
     buffer_t dstBuff;
     size_t   cSize;
@@ -333,10 +333,10 @@ void ZSTDMT_compressChunk(void* jobDescription)
 {
     ZSTDMT_jobDescription* const job = (ZSTDMT_jobDescription*)jobDescription;
     ZSTD_CCtx* const cctx = ZSTDMT_getCCtx(job->cctxPool);
-    const void* const src = (const char*)job->srcStart + job->dictSize;
+    const void* const src = (const char*)job->srcStart + job->prefixSize;
     buffer_t dstBuff = job->dstBuff;
-    DEBUGLOG(5, "ZSTDMT_compressChunk: job (first:%u) (last:%u) : dictSize %u, srcSize %u",
-                 job->firstChunk, job->lastChunk, (U32)job->dictSize, (U32)job->srcSize);
+    DEBUGLOG(5, "ZSTDMT_compressChunk: job (first:%u) (last:%u) : prefixSize %u, srcSize %u ",
+                 job->firstChunk, job->lastChunk, (U32)job->prefixSize, (U32)job->srcSize);
 
     if (cctx==NULL) {
         job->cSize = ERROR(memory_allocation);
@@ -356,20 +356,21 @@ void ZSTDMT_compressChunk(void* jobDescription)
     if (job->cdict) {
         size_t const initError = ZSTD_compressBegin_usingCDict_advanced(cctx, job->cdict, job->params.fParams, job->fullFrameSize);
         DEBUGLOG(4, "ZSTDMT_compressChunk: init using CDict");
-        assert(job->firstChunk); /* should only happen for first segment */
+        assert(job->firstChunk);  /* only allowed for first job */
         if (ZSTD_isError(initError)) { job->cSize = initError; goto _endJob; }
     } else {  /* srcStart points at reloaded section */
         ZSTD_CCtx_params jobParams = job->params;   /* do not modify job->params ! copy it, modify the copy */
         size_t const forceWindowError = ZSTD_CCtxParam_setParameter(&jobParams, ZSTD_p_forceMaxWindow, !job->firstChunk);
         U64 const pledgedSrcSize = job->firstChunk ? job->fullFrameSize : ZSTD_CONTENTSIZE_UNKNOWN;
         /* load dictionary in "content-only" mode (no header analysis) */
-        size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, job->srcStart, job->dictSize, ZSTD_dm_rawContent, jobParams, pledgedSrcSize);
+        size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, job->srcStart, job->prefixSize, ZSTD_dm_rawContent, jobParams, pledgedSrcSize);
+        DEBUGLOG(5, "ZSTD_compressBegin_advanced_internal called with windowLog = %u ", jobParams.cParams.windowLog);
         if (ZSTD_isError(initError) || ZSTD_isError(forceWindowError)) {
             job->cSize = initError;
             goto _endJob;
         }
     }
-    if (!job->firstChunk) {  /* flush and overwrite frame header when it's not first segment */
+    if (!job->firstChunk) {  /* flush and overwrite frame header when it's not first job */
         size_t const hSize = ZSTD_compressContinue(cctx, dstBuff.start, dstBuff.size, src, 0);
         if (ZSTD_isError(hSize)) { job->cSize = hSize; goto _endJob; }
         ZSTD_invalidateRepCodes(cctx);
@@ -380,9 +381,9 @@ void ZSTDMT_compressChunk(void* jobDescription)
     job->cSize = (job->lastChunk) ?
                  ZSTD_compressEnd     (cctx, dstBuff.start, dstBuff.size, src, job->srcSize) :
                  ZSTD_compressContinue(cctx, dstBuff.start, dstBuff.size, src, job->srcSize);
-    DEBUGLOG(5, "compressed %u bytes into %u bytes   (first:%u) (last:%u)",
+    DEBUGLOG(5, "compressed %u bytes into %u bytes   (first:%u) (last:%u) ",
                 (unsigned)job->srcSize, (unsigned)job->cSize, job->firstChunk, job->lastChunk);
-    DEBUGLOG(5, "dstBuff.size : %u ; => %s", (U32)dstBuff.size, ZSTD_getErrorName(job->cSize));
+    DEBUGLOG(5, "dstBuff.size : %u ; => %s ", (U32)dstBuff.size, ZSTD_getErrorName(job->cSize));
 
 _endJob:
     ZSTDMT_releaseCCtx(job->cctxPool, cctx);
@@ -618,7 +619,7 @@ static size_t ZSTDMT_compress_advanced_internal(
     size_t const overlapSize = (overlapRLog>=9) ? 0 : (size_t)1 << (params.cParams.windowLog - overlapRLog);
     unsigned nbChunks = computeNbChunks(srcSize, params.cParams.windowLog, params.nbThreads);
     size_t const proposedChunkSize = (srcSize + (nbChunks-1)) / nbChunks;
-    size_t const avgChunkSize = ((proposedChunkSize & 0x1FFFF) < 0x7FFF) ? proposedChunkSize + 0xFFFF : proposedChunkSize;   /* avoid too small last block */
+    size_t const avgChunkSize = (((proposedChunkSize-1) & 0x1FFFF) < 0x7FFF) ? proposedChunkSize + 0xFFFF : proposedChunkSize;   /* avoid too small last block */
     const char* const srcStart = (const char*)src;
     size_t remainingSrcSize = srcSize;
     unsigned const compressWithinDst = (dstCapacity >= ZSTD_compressBound(srcSize)) ? nbChunks : (unsigned)(dstCapacity / ZSTD_compressBound(avgChunkSize));  /* presumes avgChunkSize >= 256 KB, which should be the case */
@@ -628,7 +629,8 @@ static size_t ZSTDMT_compress_advanced_internal(
     assert(mtctx->cctxPool->totalCCtx == params.nbThreads);
 
     DEBUGLOG(4, "ZSTDMT_compress_advanced_internal");
-    DEBUGLOG(4, "nbChunks  : %2u   (chunkSize : %u bytes)   ", nbChunks, (U32)avgChunkSize);
+    DEBUGLOG(4, "nbChunks  : %2u (raw chunkSize : %u bytes; fixed chunkSize: %u)   ",
+                nbChunks, (U32)proposedChunkSize, (U32)avgChunkSize);
     if (nbChunks==1) {   /* fallback to single-thread mode */
         ZSTD_CCtx* const cctx = mtctx->cctxPool->cctx[0];
         if (cdict) return ZSTD_compress_usingCDict_advanced(cctx, dst, dstCapacity, src, srcSize, cdict, jobParams.fParams);
@@ -657,7 +659,7 @@ static size_t ZSTDMT_compress_advanced_internal(
 
             mtctx->jobs[u].src = g_nullBuffer;
             mtctx->jobs[u].srcStart = srcStart + frameStartPos - dictSize;
-            mtctx->jobs[u].dictSize = dictSize;
+            mtctx->jobs[u].prefixSize = dictSize;
             mtctx->jobs[u].srcSize = chunkSize;
             mtctx->jobs[u].cdict = (u==0) ? cdict : NULL;
             mtctx->jobs[u].fullFrameSize = srcSize;
@@ -888,7 +890,7 @@ static size_t ZSTDMT_createCompressionJob(ZSTDMT_CCtx* zcs, size_t srcSize, unsi
     zcs->jobs[jobID].src = zcs->inBuff.buffer;
     zcs->jobs[jobID].srcStart = zcs->inBuff.buffer.start;
     zcs->jobs[jobID].srcSize = srcSize;
-    zcs->jobs[jobID].dictSize = zcs->dictSize;
+    zcs->jobs[jobID].prefixSize = zcs->dictSize;
     assert(zcs->inBuff.filled >= srcSize + zcs->dictSize);
     zcs->jobs[jobID].params = zcs->params;
     /* do not calculate checksum within sections, but write it in header for first section */
@@ -1014,7 +1016,7 @@ size_t ZSTDMT_compressStream_generic(ZSTDMT_CCtx* mtctx,
 {
     size_t const newJobThreshold = mtctx->dictSize + mtctx->targetSectionSize;
     unsigned forwardInputProgress = 0;
-    DEBUGLOG(5, "ZSTDMT_compressStream_generic");
+    DEBUGLOG(5, "ZSTDMT_compressStream_generic ");
     assert(output->pos <= output->size);
     assert(input->pos  <= input->size);
 
diff --git a/lib/decompress/zstd_decompress.c b/lib/decompress/zstd_decompress.c
index f0cf967cb7b..a59d9441128 100644
--- a/lib/decompress/zstd_decompress.c
+++ b/lib/decompress/zstd_decompress.c
@@ -2451,14 +2451,16 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
                         return ZSTD_decompressLegacyStream(zds->legacyContext, legacyVersion, output, input);
                     }
 #endif
-                    return hSize; /* error */
+                    return hSize;   /* error */
                 }
                 if (hSize != 0) {   /* need more input */
                     size_t const toLoad = hSize - zds->lhSize;   /* if hSize!=0, hSize > zds->lhSize */
-                    if (toLoad > (size_t)(iend-ip)) {   /* not enough input to load full header */
-                        if (iend-ip > 0) {
-                            memcpy(zds->headerBuffer + zds->lhSize, ip, iend-ip);
-                            zds->lhSize += iend-ip;
+                    size_t const remainingInput = (size_t)(iend-ip);
+                    assert(iend >= ip);
+                    if (toLoad > remainingInput) {   /* not enough input to load full header */
+                        if (remainingInput > 0) {
+                            memcpy(zds->headerBuffer + zds->lhSize, ip, remainingInput);
+                            zds->lhSize += remainingInput;
                         }
                         input->pos = input->size;
                         return (MAX(ZSTD_frameHeaderSize_min, hSize) - zds->lhSize) + ZSTD_blockHeaderSize;   /* remaining header bytes + next block header */
@@ -2473,8 +2475,10 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
                 && (U64)(size_t)(oend-op) >= zds->fParams.frameContentSize) {
                 size_t const cSize = ZSTD_findFrameCompressedSize(istart, iend-istart);
                 if (cSize <= (size_t)(iend-istart)) {
+                    /* shortcut : using single-pass mode */
                     size_t const decompressedSize = ZSTD_decompress_usingDDict(zds, op, oend-op, istart, cSize, zds->ddict);
                     if (ZSTD_isError(decompressedSize)) return decompressedSize;
+                    DEBUGLOG(4, "shortcut to single-pass ZSTD_decompress_usingDDict()")
                     ip = istart + cSize;
                     op += decompressedSize;
                     zds->expected = 0;
@@ -2497,8 +2501,9 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
             }
 
             /* control buffer memory usage */
-            DEBUGLOG(4, "Control max buffer memory usage (max %u KB)",
-                        (U32)(zds->maxWindowSize >> 10));
+            DEBUGLOG(4, "Control max memory usage (%u KB <= max %u KB)",
+                        (U32)(zds->fParams.windowSize >>10),
+                        (U32)(zds->maxWindowSize >> 10) );
             zds->fParams.windowSize = MAX(zds->fParams.windowSize, 1U << ZSTD_WINDOWLOG_ABSOLUTEMIN);
             if (zds->fParams.windowSize > zds->maxWindowSize) return ERROR(frameParameter_windowTooLarge);
 
diff --git a/lib/zstd.h b/lib/zstd.h
index 15c467535a4..e4541c3f426 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -1016,7 +1016,8 @@ typedef enum {
                               * More threads improve speed, but also increase memory usage.
                               * Can only receive a value > 1 if ZSTD_MULTITHREAD is enabled.
                               * Special: value 0 means "do not change nbThreads" */
-    ZSTD_p_jobSize,          /* Size of a compression job. Each compression job is completed in parallel.
+    ZSTD_p_jobSize,          /* Size of a compression job. This value is only enforced in streaming (non-blocking) mode.
+                              * Each compression job is completed in parallel, so indirectly controls the nb of active threads.
                               * 0 means default, which is dynamically determined based on compression parameters.
                               * Job size must be a minimum of overlapSize, or 1 KB, whichever is largest
                               * The minimum size is automatically and transparently enforced */
@@ -1144,13 +1145,19 @@ typedef enum {
  *  - Compression parameters cannot be changed once compression is started.
  *  - outpot->pos must be <= dstCapacity, input->pos must be <= srcSize
  *  - outpot->pos and input->pos will be updated. They are guaranteed to remain below their respective limit.
- *  - @return provides the minimum amount of data still to flush from internal buffers
+ *  - In single-thread mode (default), function is blocking : it completed its job before returning to caller.
+ *  - In multi-thread mode, function is non-blocking : it just acquires a copy of input, and distribute job to internal worker threads,
+ *                                                     and then immediately returns, just indicating that there is some data remaining to be flushed.
+ *                                                     The function nonetheless guarantees forward progress : it will return only after it reads or write at least 1+ byte.
+ *  - Exception : in multi-threading mode, if the first call requests a ZSTD_e_end directive, it is blocking : it will complete compression before giving back control to caller.
+ *  - @return provides the minimum amount of data remaining to be flushed from internal buffers
  *            or an error code, which can be tested using ZSTD_isError().
- *            if @return != 0, flush is not fully completed, there is some data left within internal buffers.
- *  - after a ZSTD_e_end directive, if internal buffer is not fully flushed,
+ *            if @return != 0, flush is not fully completed, there is still some data left within internal buffers.
+ *            This is useful to determine if a ZSTD_e_flush or ZSTD_e_end directive is completed.
+ *  - after a ZSTD_e_end directive, if internal buffer is not fully flushed (@return != 0),
  *            only ZSTD_e_end or ZSTD_e_flush operations are allowed.
- *            It is necessary to fully flush internal buffers
- *            before starting a new compression job, or changing compression parameters.
+ *            Before starting a new compression job, or changing compression parameters,
+ *            it is required to fully flush internal buffers.
  */
 ZSTDLIB_API size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
                                           ZSTD_outBuffer* output,
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index e08a96d8fc9..6910457ff63 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -524,12 +524,12 @@ static int basicUnitTests(U32 seed, double compressibility)
     {   ZSTD_DDict* const ddict = ZSTD_createDDict(dictionary.start, dictionary.filled);
         size_t const initError = ZSTD_initDStream_usingDDict(zd, ddict);
         if (ZSTD_isError(initError)) goto _output_error;
-        inBuff.src = compressedBuffer;
-        inBuff.size = cSize;
-        inBuff.pos = 0;
         outBuff.dst = decodedBuffer;
         outBuff.size = CNBufferSize;
         outBuff.pos = 0;
+        inBuff.src = compressedBuffer;
+        inBuff.size = cSize;
+        inBuff.pos = 0;
         { size_t const r = ZSTD_decompressStream(zd, &outBuff, &inBuff);
           if (r != 0) goto _output_error; }  /* should reach end of frame == 0; otherwise, some data left, or an error */
         if (outBuff.pos != CNBufferSize) goto _output_error;   /* should regenerate the same amount */
@@ -548,12 +548,12 @@ static int basicUnitTests(U32 seed, double compressibility)
     DISPLAYLEVEL(3, "test%3i : maxWindowSize < frame requirement : ", testNb++);
     ZSTD_initDStream_usingDict(zd, CNBuffer, dictSize);
     CHECK_Z( ZSTD_setDStreamParameter(zd, DStream_p_maxWindowSize, 1000) );  /* too small limit */
-    inBuff.src = compressedBuffer;
-    inBuff.size = cSize;
-    inBuff.pos = 0;
     outBuff.dst = decodedBuffer;
     outBuff.size = CNBufferSize;
     outBuff.pos = 0;
+    inBuff.src = compressedBuffer;
+    inBuff.size = cSize;
+    inBuff.pos = 0;
     { size_t const r = ZSTD_decompressStream(zd, &outBuff, &inBuff);
       if (!ZSTD_isError(r)) goto _output_error;  /* must fail : frame requires > 100 bytes */
       DISPLAYLEVEL(3, "OK (%s)\n", ZSTD_getErrorName(r)); }
@@ -642,12 +642,12 @@ static int basicUnitTests(U32 seed, double compressibility)
         params.fParams.contentSizeFlag = 1;
         CHECK_Z( ZSTD_initCStream_advanced(zc, dictionary.start, dictionary.filled, params, 0 /* pledgedSrcSize==0 means "empty" when params.fParams.contentSizeFlag is set */) );
     } /* cstream advanced shall write content size = 0 */
-    inBuff.src = CNBuffer;
-    inBuff.size = 0;
-    inBuff.pos = 0;
     outBuff.dst = compressedBuffer;
     outBuff.size = compressedBufferSize;
     outBuff.pos = 0;
+    inBuff.src = CNBuffer;
+    inBuff.size = 0;
+    inBuff.pos = 0;
     CHECK_Z( ZSTD_compressStream(zc, &outBuff, &inBuff) );
     if (ZSTD_endStream(zc, &outBuff) != 0) goto _output_error;
     cSize = outBuff.pos;
@@ -671,12 +671,12 @@ static int basicUnitTests(U32 seed, double compressibility)
     if (ZSTD_findDecompressedSize(compressedBuffer, cSize) != 0) goto _output_error;
 
     ZSTD_resetCStream(zc, 0); /* resetCStream should treat 0 as unknown */
-    inBuff.src = CNBuffer;
-    inBuff.size = 0;
-    inBuff.pos = 0;
     outBuff.dst = compressedBuffer;
     outBuff.size = compressedBufferSize;
     outBuff.pos = 0;
+    inBuff.src = CNBuffer;
+    inBuff.size = 0;
+    inBuff.pos = 0;
     CHECK_Z( ZSTD_compressStream(zc, &outBuff, &inBuff) );
     if (ZSTD_endStream(zc, &outBuff) != 0) goto _output_error;
     cSize = outBuff.pos;
@@ -688,7 +688,7 @@ static int basicUnitTests(U32 seed, double compressibility)
     {   ZSTD_parameters const params = ZSTD_getParams(1, 0, 0);
         CHECK_Z( ZSTDMT_initCStream_advanced(mtctx, CNBuffer, dictSize, params, CNBufferSize) );
     }
-    outBuff.dst = (char*)(compressedBuffer);
+    outBuff.dst = compressedBuffer;
     outBuff.size = compressedBufferSize;
     outBuff.pos = 0;
     inBuff.src = CNBuffer;
@@ -700,6 +700,61 @@ static int basicUnitTests(U32 seed, double compressibility)
       if (r != 0) goto _output_error; }  /* error, or some data not flushed */
     DISPLAYLEVEL(3, "OK \n");
 
+    /* Complex multithreading + dictionary test */
+    {   U32 const nbThreads = 2;
+        size_t const jobSize = 4 * 1 MB;
+        size_t const srcSize = jobSize * nbThreads;  /* we want each job to have predictable size */
+        size_t const segLength = 2 KB;
+        size_t const offset = 600 KB;   /* must be larger than window defined in cdict */
+        size_t const start = jobSize + (offset-1);
+        const BYTE* const srcToCopy = (const BYTE*)CNBuffer + start;
+        BYTE* const dst = (BYTE*)CNBuffer + start - offset;
+        DISPLAYLEVEL(3, "test%3i : compress %u bytes with multiple threads + dictionary : ", testNb++, (U32)srcSize);
+        ZSTD_CCtx_setParameter(zc, ZSTD_p_compressionLevel, 3);
+        ZSTD_CCtx_setParameter(zc, ZSTD_p_nbThreads, 2);
+        ZSTD_CCtx_setParameter(zc, ZSTD_p_jobSize, jobSize);
+        assert(start > offset);
+        assert(start + segLength < COMPRESSIBLE_NOISE_LENGTH);
+        memcpy(dst, srcToCopy, segLength);   /* create a long repetition at long distance for job 2 */
+        outBuff.dst = compressedBuffer;
+        outBuff.size = compressedBufferSize;
+        outBuff.pos = 0;
+        inBuff.src = CNBuffer;
+        inBuff.size = srcSize; assert(srcSize < COMPRESSIBLE_NOISE_LENGTH);
+        inBuff.pos = 0;
+    }
+    {   ZSTD_compressionParameters const cParams = ZSTD_getCParams(1, 4 KB, dictionary.filled);   /* intentionnally lies on estimatedSrcSize, to push cdict into targeting a small window size */
+        ZSTD_CDict* const cdict = ZSTD_createCDict_advanced(dictionary.start, dictionary.filled, ZSTD_dlm_byRef, ZSTD_dm_fullDict, cParams, ZSTD_defaultCMem);
+        DISPLAYLEVEL(5, "cParams.windowLog = %u : ", cParams.windowLog);
+        ZSTD_CCtx_refCDict(zc, cdict);
+        CHECK_Z( ZSTD_compress_generic(zc, &outBuff, &inBuff, ZSTD_e_end) );
+        ZSTD_CCtx_refCDict(zc, NULL);  /* do not keep a reference to cdict, as its lifetime ends */
+        ZSTD_freeCDict(cdict);
+    }
+    if (inBuff.pos != inBuff.size) goto _output_error;   /* entire input should be consumed */
+    cSize = outBuff.pos;
+    DISPLAYLEVEL(3, "OK \n");
+
+    DISPLAYLEVEL(3, "test%3i : decompress large frame created from multiple threads + dictionary : ", testNb++);
+    {   ZSTD_DStream* const dstream = ZSTD_createDCtx();
+        ZSTD_frameHeader zfh;
+        ZSTD_getFrameHeader(&zfh, compressedBuffer, cSize);
+        DISPLAYLEVEL(5, "frame windowsize = %u : ", (U32)zfh.windowSize);
+        outBuff.dst = decodedBuffer;
+        outBuff.size = CNBufferSize;
+        outBuff.pos = 0;
+        inBuff.src = compressedBuffer;
+        inBuff.pos = 0;
+        CHECK_Z( ZSTD_initDStream_usingDict(dstream, dictionary.start, dictionary.filled) );
+        inBuff.size = 1;  /* avoid shortcut to single-pass mode */
+        CHECK_Z( ZSTD_decompressStream(dstream, &outBuff, &inBuff) );
+        inBuff.size = cSize;
+        CHECK_Z( ZSTD_decompressStream(dstream, &outBuff, &inBuff) );
+        if (inBuff.pos != inBuff.size) goto _output_error;   /* entire input should be consumed */
+        ZSTD_freeDStream(dstream);
+    }
+    DISPLAYLEVEL(3, "OK \n");
+
     DISPLAYLEVEL(3, "test%3i : check dictionary FSE tables can represent every code : ", testNb++);
     {   unsigned const kMaxWindowLog = 24;
         unsigned value;

From e28305fcca441564a57758d84da8ef73afe7d85d Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 12 Dec 2017 16:20:51 -0800
Subject: [PATCH 120/145] fix #944 : ZSTDMT with large files and dictionary now
 works correctly

windowLog is now enforced from provided compression parameters,
instead of being copied blindly from `cdict`
where it could be smaller.

also :
- fix a minor bug in zstreamtest --mt : advanced parameters must be set before init
- changed advanced parameter name to ZSTDMT_jobSize
---
 lib/compress/zstd_compress.c          | 62 +++++++++++++++------------
 lib/compress/zstd_compress_internal.h |  1 +
 lib/compress/zstdmt_compress.c        | 43 ++++++++++++-------
 lib/compress/zstdmt_compress.h        |  8 ++--
 tests/zstreamtest.c                   | 16 +++----
 5 files changed, 75 insertions(+), 55 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 1072918885f..7586c3a0c74 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -410,7 +410,7 @@ size_t ZSTD_CCtxParam_setParameter(
         return ERROR(parameter_unsupported);
 #else
         if (CCtxParams->nbThreads <= 1) return ERROR(parameter_unsupported);
-        return ZSTDMT_CCtxParam_setMTCtxParameter(CCtxParams, ZSTDMT_p_sectionSize, value);
+        return ZSTDMT_CCtxParam_setMTCtxParameter(CCtxParams, ZSTDMT_p_jobSize, value);
 #endif
 
     case ZSTD_p_overlapSizeLog :
@@ -987,15 +987,16 @@ void ZSTD_invalidateRepCodes(ZSTD_CCtx* cctx) {
 
 /*! ZSTD_copyCCtx_internal() :
  *  Duplicate an existing context `srcCCtx` into another one `dstCCtx`.
- *  The "context", in this case, refers to the hash and chain tables,
- *  entropy tables, and dictionary offsets.
  *  Only works during stage ZSTDcs_init (i.e. after creation, but before first call to ZSTD_compressContinue()).
- *  pledgedSrcSize=0 means "empty".
- *  @return : 0, or an error code */
+ *  The "context", in this case, refers to the hash and chain tables,
+ *  entropy tables, and dictionary references.
+ * `windowLog` value is enforced if != 0, otherwise value is copied from srcCCtx.
+ * @return : 0, or an error code */
 static size_t ZSTD_copyCCtx_internal(ZSTD_CCtx* dstCCtx,
                             const ZSTD_CCtx* srcCCtx,
+                            unsigned windowLog,
                             ZSTD_frameParameters fParams,
-                            unsigned long long pledgedSrcSize,
+                            U64 pledgedSrcSize,
                             ZSTD_buffered_policy_e zbuff)
 {
     DEBUGLOG(5, "ZSTD_copyCCtx_internal");
@@ -1005,6 +1006,7 @@ static size_t ZSTD_copyCCtx_internal(ZSTD_CCtx* dstCCtx,
     {   ZSTD_CCtx_params params = dstCCtx->requestedParams;
         /* Copy only compression parameters related to tables. */
         params.cParams = srcCCtx->appliedParams.cParams;
+        if (windowLog) params.cParams.windowLog = windowLog;
         params.fParams = fParams;
         ZSTD_resetCCtx_internal(dstCCtx, params, pledgedSrcSize,
                                 ZSTDcrp_noMemset, zbuff);
@@ -1050,7 +1052,9 @@ size_t ZSTD_copyCCtx(ZSTD_CCtx* dstCCtx, const ZSTD_CCtx* srcCCtx, unsigned long
     if (pledgedSrcSize==0) pledgedSrcSize = ZSTD_CONTENTSIZE_UNKNOWN;
     fParams.contentSizeFlag = (pledgedSrcSize != ZSTD_CONTENTSIZE_UNKNOWN);
 
-    return ZSTD_copyCCtx_internal(dstCCtx, srcCCtx, fParams, pledgedSrcSize, zbuff);
+    return ZSTD_copyCCtx_internal(dstCCtx, srcCCtx,
+                                0 /*windowLog from srcCCtx*/, fParams, pledgedSrcSize,
+                                zbuff);
 }
 
 
@@ -2037,12 +2041,12 @@ static size_t ZSTD_compress_insertDictionary(ZSTD_CCtx* cctx,
 
 /*! ZSTD_compressBegin_internal() :
  * @return : 0, or an error code */
-static size_t ZSTD_compressBegin_internal(ZSTD_CCtx* cctx,
+size_t ZSTD_compressBegin_internal(ZSTD_CCtx* cctx,
                              const void* dict, size_t dictSize,
                              ZSTD_dictMode_e dictMode,
                              const ZSTD_CDict* cdict,
-                                   ZSTD_CCtx_params params, U64 pledgedSrcSize,
-                                   ZSTD_buffered_policy_e zbuff)
+                             ZSTD_CCtx_params params, U64 pledgedSrcSize,
+                             ZSTD_buffered_policy_e zbuff)
 {
     DEBUGLOG(4, "ZSTD_compressBegin_internal");
     /* params are supposed to be fully validated at this point */
@@ -2052,7 +2056,7 @@ static size_t ZSTD_compressBegin_internal(ZSTD_CCtx* cctx,
     if (cdict && cdict->dictContentSize>0) {
         cctx->requestedParams = params;
         return ZSTD_copyCCtx_internal(cctx, cdict->refContext,
-                                      params.fParams, pledgedSrcSize,
+                                      params.cParams.windowLog, params.fParams, pledgedSrcSize,
                                       zbuff);
     }
 
@@ -2061,17 +2065,19 @@ static size_t ZSTD_compressBegin_internal(ZSTD_CCtx* cctx,
     return ZSTD_compress_insertDictionary(cctx, dict, dictSize, dictMode);
 }
 
-size_t ZSTD_compressBegin_advanced_internal(
-                                    ZSTD_CCtx* cctx,
+size_t ZSTD_compressBegin_advanced_internal(ZSTD_CCtx* cctx,
                                     const void* dict, size_t dictSize,
                                     ZSTD_dictMode_e dictMode,
+                                    const ZSTD_CDict* cdict,
                                     ZSTD_CCtx_params params,
                                     unsigned long long pledgedSrcSize)
 {
     DEBUGLOG(4, "ZSTD_compressBegin_advanced_internal");
     /* compression parameters verification and optimization */
     CHECK_F( ZSTD_checkCParams(params.cParams) );
-    return ZSTD_compressBegin_internal(cctx, dict, dictSize, dictMode, NULL,
+    return ZSTD_compressBegin_internal(cctx,
+                                       dict, dictSize, dictMode,
+                                       cdict,
                                        params, pledgedSrcSize,
                                        ZSTDb_not_buffered);
 }
@@ -2084,9 +2090,10 @@ size_t ZSTD_compressBegin_advanced(ZSTD_CCtx* cctx,
 {
     ZSTD_CCtx_params const cctxParams =
             ZSTD_assignParamsToCCtxParams(cctx->requestedParams, params);
-    return ZSTD_compressBegin_advanced_internal(cctx, dict, dictSize, ZSTD_dm_auto,
-                                                cctxParams,
-                                                pledgedSrcSize);
+    return ZSTD_compressBegin_advanced_internal(cctx,
+                                            dict, dictSize, ZSTD_dm_auto,
+                                            NULL /*cdict*/,
+                                            cctxParams, pledgedSrcSize);
 }
 
 size_t ZSTD_compressBegin_usingDict(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, int compressionLevel)
@@ -2507,10 +2514,10 @@ static size_t ZSTD_resetCStream_internal(ZSTD_CStream* zcs,
     assert(!((dict) && (cdict)));  /* either dict or cdict, not both */
 
     CHECK_F( ZSTD_compressBegin_internal(zcs,
-                                        dict, dictSize, dictMode,
-                                        cdict,
-                                        params, pledgedSrcSize,
-                                        ZSTDb_buffered) );
+                                         dict, dictSize, dictMode,
+                                         cdict,
+                                         params, pledgedSrcSize,
+                                         ZSTDb_buffered) );
 
     zcs->inToCompress = 0;
     zcs->inBuffPos = 0;
@@ -2534,7 +2541,7 @@ size_t ZSTD_resetCStream(ZSTD_CStream* zcs, unsigned long long pledgedSrcSize)
 }
 
 /*! ZSTD_initCStream_internal() :
- *  Note : not static, but hidden (not exposed). Used by zstdmt_compress.c
+ *  Note : for lib/compress only. Used by zstdmt_compress.c.
  *  Assumption 1 : params are valid
  *  Assumption 2 : either dict, or cdict, is defined, not both */
 size_t ZSTD_initCStream_internal(ZSTD_CStream* zcs,
@@ -2546,7 +2553,7 @@ size_t ZSTD_initCStream_internal(ZSTD_CStream* zcs,
     assert(!((dict) && (cdict)));  /* either dict or cdict, not both */
 
     if (dict && dictSize >= 8) {
-        DEBUGLOG(5, "loading dictionary of size %u", (U32)dictSize);
+        DEBUGLOG(4, "loading dictionary of size %u", (U32)dictSize);
         if (zcs->staticSize) {   /* static CCtx : never uses malloc */
             /* incompatible with internal cdict creation */
             return ERROR(memory_allocation);
@@ -2559,14 +2566,14 @@ size_t ZSTD_initCStream_internal(ZSTD_CStream* zcs,
         if (zcs->cdictLocal == NULL) return ERROR(memory_allocation);
     } else {
         if (cdict) {
-            params.cParams = ZSTD_getCParamsFromCDict(cdict);  /* cParams are enforced from cdict */
+            params.cParams = ZSTD_getCParamsFromCDict(cdict);  /* cParams are enforced from cdict; this includes windowLog (should it be enforced from `params` instead ?) */
         }
         ZSTD_freeCDict(zcs->cdictLocal);
         zcs->cdictLocal = NULL;
         zcs->cdict = cdict;
     }
 
-    params.compressionLevel = ZSTD_CLEVEL_CUSTOM;
+    params.compressionLevel = ZSTD_CLEVEL_CUSTOM; /* enforce usage of cParams, instead of a dynamic derivation from cLevel (but does that happen ?) */
     zcs->requestedParams = params;
 
     return ZSTD_resetCStream_internal(zcs, NULL, 0, ZSTD_dm_auto, zcs->cdict, params, pledgedSrcSize);
@@ -2606,10 +2613,9 @@ size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs,
                                  const void* dict, size_t dictSize,
                                  ZSTD_parameters params, unsigned long long pledgedSrcSize)
 {
-    ZSTD_CCtx_params const cctxParams =
-            ZSTD_assignParamsToCCtxParams(zcs->requestedParams, params);
+    ZSTD_CCtx_params const cctxParams = ZSTD_assignParamsToCCtxParams(zcs->requestedParams, params);
     DEBUGLOG(4, "ZSTD_initCStream_advanced: pledgedSrcSize=%u, flag=%u",
-            (U32)pledgedSrcSize, params.fParams.contentSizeFlag);
+                (U32)pledgedSrcSize, params.fParams.contentSizeFlag);
     CHECK_F( ZSTD_checkCParams(params.cParams) );
     if ((pledgedSrcSize==0) && (params.fParams.contentSizeFlag==0)) pledgedSrcSize = ZSTD_CONTENTSIZE_UNKNOWN;  /* for compatibility with older programs relying on this behavior. Users should now specify ZSTD_CONTENTSIZE_UNKNOWN. This line will be removed in the future. */
     return ZSTD_initCStream_internal(zcs, dict, dictSize, NULL /*cdict*/, cctxParams, pledgedSrcSize);
diff --git a/lib/compress/zstd_compress_internal.h b/lib/compress/zstd_compress_internal.h
index 278f2427c59..f104fe981ea 100644
--- a/lib/compress/zstd_compress_internal.h
+++ b/lib/compress/zstd_compress_internal.h
@@ -447,6 +447,7 @@ ZSTD_compressionParameters ZSTD_getCParamsFromCDict(const ZSTD_CDict* cdict);
 size_t ZSTD_compressBegin_advanced_internal(ZSTD_CCtx* cctx,
                                     const void* dict, size_t dictSize,
                                     ZSTD_dictMode_e dictMode,
+                                    const ZSTD_CDict* cdict,
                                     ZSTD_CCtx_params params,
                                     unsigned long long pledgedSrcSize);
 
diff --git a/lib/compress/zstdmt_compress.c b/lib/compress/zstdmt_compress.c
index 8347cf29d5c..a5e996d3ebf 100644
--- a/lib/compress/zstdmt_compress.c
+++ b/lib/compress/zstdmt_compress.c
@@ -350,29 +350,35 @@ void ZSTDMT_compressChunk(void* jobDescription)
             goto _endJob;
         }
         job->dstBuff = dstBuff;
-        DEBUGLOG(5, "ZSTDMT_compressChunk: allocated dstBuff of size %u", (U32)dstBuff.size);
+        DEBUGLOG(5, "ZSTDMT_compressChunk: received dstBuff of size %u", (U32)dstBuff.size);
     }
 
     if (job->cdict) {
-        size_t const initError = ZSTD_compressBegin_usingCDict_advanced(cctx, job->cdict, job->params.fParams, job->fullFrameSize);
+        size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, NULL, 0, ZSTD_dm_auto, job->cdict, job->params, job->fullFrameSize);
         DEBUGLOG(4, "ZSTDMT_compressChunk: init using CDict");
         assert(job->firstChunk);  /* only allowed for first job */
         if (ZSTD_isError(initError)) { job->cSize = initError; goto _endJob; }
     } else {  /* srcStart points at reloaded section */
+        U64 const pledgedSrcSize = job->firstChunk ? job->fullFrameSize : ZSTD_CONTENTSIZE_UNKNOWN;
         ZSTD_CCtx_params jobParams = job->params;   /* do not modify job->params ! copy it, modify the copy */
         size_t const forceWindowError = ZSTD_CCtxParam_setParameter(&jobParams, ZSTD_p_forceMaxWindow, !job->firstChunk);
-        U64 const pledgedSrcSize = job->firstChunk ? job->fullFrameSize : ZSTD_CONTENTSIZE_UNKNOWN;
-        /* load dictionary in "content-only" mode (no header analysis) */
-        size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, job->srcStart, job->prefixSize, ZSTD_dm_rawContent, jobParams, pledgedSrcSize);
-        DEBUGLOG(5, "ZSTD_compressBegin_advanced_internal called with windowLog = %u ", jobParams.cParams.windowLog);
-        if (ZSTD_isError(initError) || ZSTD_isError(forceWindowError)) {
-            job->cSize = initError;
+        if (ZSTD_isError(forceWindowError)) {
+            DEBUGLOG(5, "ZSTD_CCtxParam_setParameter error : %s ", ZSTD_getErrorName(forceWindowError));
+            job->cSize = forceWindowError;
             goto _endJob;
         }
+        /* load dictionary in "content-only" mode (no header analysis) */
+        {   size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, job->srcStart, job->prefixSize, ZSTD_dm_rawContent, NULL, jobParams, pledgedSrcSize);
+            DEBUGLOG(5, "ZSTD_compressBegin_advanced_internal called with windowLog = %u ", jobParams.cParams.windowLog);
+            if (ZSTD_isError(initError)) {
+                DEBUGLOG(5, "ZSTD_compressBegin_advanced_internal error : %s ", ZSTD_getErrorName(initError));
+                job->cSize = initError;
+                goto _endJob;
+        }   }
     }
     if (!job->firstChunk) {  /* flush and overwrite frame header when it's not first job */
         size_t const hSize = ZSTD_compressContinue(cctx, dstBuff.start, dstBuff.size, src, 0);
-        if (ZSTD_isError(hSize)) { job->cSize = hSize; goto _endJob; }
+        if (ZSTD_isError(hSize)) { job->cSize = hSize; /* save error code */ goto _endJob; }
         ZSTD_invalidateRepCodes(cctx);
     }
 
@@ -559,11 +565,13 @@ size_t ZSTDMT_sizeof_CCtx(ZSTDMT_CCtx* mtctx)
 }
 
 /* Internal only */
-size_t ZSTDMT_CCtxParam_setMTCtxParameter(
-    ZSTD_CCtx_params* params, ZSTDMT_parameter parameter, unsigned value) {
+size_t ZSTDMT_CCtxParam_setMTCtxParameter(ZSTD_CCtx_params* params,
+                                ZSTDMT_parameter parameter, unsigned value) {
+    DEBUGLOG(4, "ZSTDMT_CCtxParam_setMTCtxParameter");
     switch(parameter)
     {
-    case ZSTDMT_p_sectionSize :
+    case ZSTDMT_p_jobSize :
+        DEBUGLOG(4, "ZSTDMT_CCtxParam_setMTCtxParameter : set jobSize to %u", value);
         if ( (value > 0)  /* value==0 => automatic job size */
            & (value < ZSTDMT_JOBSIZE_MIN) )
             value = ZSTDMT_JOBSIZE_MIN;
@@ -581,9 +589,10 @@ size_t ZSTDMT_CCtxParam_setMTCtxParameter(
 
 size_t ZSTDMT_setMTCtxParameter(ZSTDMT_CCtx* mtctx, ZSTDMT_parameter parameter, unsigned value)
 {
+    DEBUGLOG(4, "ZSTDMT_setMTCtxParameter");
     switch(parameter)
     {
-    case ZSTDMT_p_sectionSize :
+    case ZSTDMT_p_jobSize :
         return ZSTDMT_CCtxParam_setMTCtxParameter(&mtctx->params, parameter, value);
     case ZSTDMT_p_overlapSectionLog :
         return ZSTDMT_CCtxParam_setMTCtxParameter(&mtctx->params, parameter, value);
@@ -817,7 +826,7 @@ size_t ZSTDMT_initCStream_internal(
     zcs->targetSectionSize = params.jobSize ? params.jobSize : (size_t)1 << (params.cParams.windowLog + 2);
     if (zcs->targetSectionSize < ZSTDMT_JOBSIZE_MIN) zcs->targetSectionSize = ZSTDMT_JOBSIZE_MIN;
     if (zcs->targetSectionSize < zcs->targetDictSize) zcs->targetSectionSize = zcs->targetDictSize;  /* job size must be >= overlap size */
-    DEBUGLOG(4, "Job Size : %u KB", (U32)(zcs->targetSectionSize>>10));
+    DEBUGLOG(4, "Job Size : %u KB (note : set to %u)", (U32)(zcs->targetSectionSize>>10), params.jobSize);
     zcs->inBuffSize = zcs->targetDictSize + zcs->targetSectionSize;
     DEBUGLOG(4, "inBuff Size : %u KB", (U32)(zcs->inBuffSize>>10));
     ZSTDMT_setBufferSize(zcs->bufPool, MAX(zcs->inBuffSize, ZSTD_compressBound(zcs->targetSectionSize)) );
@@ -955,6 +964,7 @@ static size_t ZSTDMT_createCompressionJob(ZSTDMT_CCtx* zcs, size_t srcSize, unsi
 static size_t ZSTDMT_flushNextJob(ZSTDMT_CCtx* zcs, ZSTD_outBuffer* output, unsigned blockToFlush)
 {
     unsigned const wJobID = zcs->doneJobID & zcs->jobIDMask;
+    DEBUGLOG(5, "ZSTDMT_flushNextJob");
     if (zcs->doneJobID == zcs->nextJobID) return 0;   /* all flushed ! */
     ZSTD_PTHREAD_MUTEX_LOCK(&zcs->jobCompleted_mutex);
     while (zcs->jobs[wJobID].jobCompleted==0) {
@@ -967,7 +977,8 @@ static size_t ZSTDMT_flushNextJob(ZSTDMT_CCtx* zcs, ZSTD_outBuffer* output, unsi
     {   ZSTDMT_jobDescription job = zcs->jobs[wJobID];
         if (!job.jobScanned) {
             if (ZSTD_isError(job.cSize)) {
-                DEBUGLOG(5, "compression error detected ");
+                DEBUGLOG(5, "job %u : compression error detected : %s",
+                            zcs->doneJobID, ZSTD_getErrorName(job.cSize));
                 ZSTDMT_waitForAllJobsCompleted(zcs);
                 ZSTDMT_releaseAllJobResources(zcs);
                 return job.cSize;
@@ -1099,9 +1110,11 @@ size_t ZSTDMT_compressStream(ZSTDMT_CCtx* zcs, ZSTD_outBuffer* output, ZSTD_inBu
 static size_t ZSTDMT_flushStream_internal(ZSTDMT_CCtx* mtctx, ZSTD_outBuffer* output, unsigned endFrame)
 {
     size_t const srcSize = mtctx->inBuff.filled - mtctx->dictSize;
+    DEBUGLOG(5, "ZSTDMT_flushStream_internal");
 
     if ( ((srcSize > 0) || (endFrame && !mtctx->frameEnded))
        && (mtctx->nextJobID <= mtctx->doneJobID + mtctx->jobIDMask) ) {
+           DEBUGLOG(5, "ZSTDMT_flushStream_internal : create a new job");
         CHECK_F( ZSTDMT_createCompressionJob(mtctx, srcSize, endFrame) );
     }
 
diff --git a/lib/compress/zstdmt_compress.h b/lib/compress/zstdmt_compress.h
index 269c54b1ac8..4209cf3c5ac 100644
--- a/lib/compress/zstdmt_compress.h
+++ b/lib/compress/zstdmt_compress.h
@@ -84,13 +84,13 @@ ZSTDLIB_API size_t ZSTDMT_initCStream_usingCDict(ZSTDMT_CCtx* mtctx,
 /* ZSTDMT_parameter :
  * List of parameters that can be set using ZSTDMT_setMTCtxParameter() */
 typedef enum {
-    ZSTDMT_p_sectionSize,        /* size of input "section". Each section is compressed in parallel. 0 means default, which is dynamically determined within compression functions */
-    ZSTDMT_p_overlapSectionLog   /* Log of overlapped section; 0 == no overlap, 6(default) == use 1/8th of window, >=9 == use full window */
+    ZSTDMT_p_jobSize,           /* Each job is compressed in parallel. By default, this value is dynamically determined depending on compression parameters. Can be set explicitly here. */
+    ZSTDMT_p_overlapSectionLog  /* Each job may reload a part of previous job to enhance compressionr ratio; 0 == no overlap, 6(default) == use 1/8th of window, >=9 == use full window */
 } ZSTDMT_parameter;
 
 /* ZSTDMT_setMTCtxParameter() :
  * allow setting individual parameters, one at a time, among a list of enums defined in ZSTDMT_parameter.
- * The function must be called typically after ZSTD_createCCtx().
+ * The function must be called typically after ZSTD_createCCtx() but __before ZSTDMT_init*() !__
  * Parameters not explicitly reset by ZSTDMT_init*() remain the same in consecutive compression sessions.
  * @return : 0, or an error code (which can be tested using ZSTD_isError()) */
 ZSTDLIB_API size_t ZSTDMT_setMTCtxParameter(ZSTDMT_CCtx* mtctx, ZSTDMT_parameter parameter, unsigned value);
@@ -114,7 +114,7 @@ size_t ZSTDMT_CCtxParam_setMTCtxParameter(ZSTD_CCtx_params* params, ZSTDMT_param
 
 /* ZSTDMT_CCtxParam_setNbThreads()
  * Set nbThreads, and clamp it correctly,
- * but also reset jobSize and overlapLog */ 
+ * but also reset jobSize and overlapLog */
 size_t ZSTDMT_CCtxParam_setNbThreads(ZSTD_CCtx_params* params, unsigned nbThreads);
 
 /*! ZSTDMT_initCStream_internal() :
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 6910457ff63..67ef6ce802f 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -1263,15 +1263,15 @@ static int fuzzerTests_MT(U32 seed, U32 nbTests, unsigned startTest, double comp
             }
             {   U64 const pledgedSrcSize = (FUZ_rand(&lseed) & 3) ? ZSTD_CONTENTSIZE_UNKNOWN : maxTestSize;
                 ZSTD_parameters params = ZSTD_getParams(cLevel, pledgedSrcSize, dictSize);
-                DISPLAYLEVEL(5, "Init with windowLog = %u and pledgedSrcSize = %u \n",
-                    params.cParams.windowLog, (U32)pledgedSrcSize);
+                DISPLAYLEVEL(5, "Init with windowLog = %u, pledgedSrcSize = %u, dictSize = %u \n",
+                    params.cParams.windowLog, (U32)pledgedSrcSize, (U32)dictSize);
                 params.fParams.checksumFlag = FUZ_rand(&lseed) & 1;
                 params.fParams.noDictIDFlag = FUZ_rand(&lseed) & 1;
                 params.fParams.contentSizeFlag = FUZ_rand(&lseed) & 1;
                 DISPLAYLEVEL(5, "checksumFlag : %u \n", params.fParams.checksumFlag);
-                CHECK_Z( ZSTDMT_initCStream_advanced(zc, dict, dictSize, params, pledgedSrcSize) );
                 CHECK_Z( ZSTDMT_setMTCtxParameter(zc, ZSTDMT_p_overlapSectionLog, FUZ_rand(&lseed) % 12) );
-                CHECK_Z( ZSTDMT_setMTCtxParameter(zc, ZSTDMT_p_sectionSize, FUZ_rand(&lseed) % (2*maxTestSize+1)) );
+                CHECK_Z( ZSTDMT_setMTCtxParameter(zc, ZSTDMT_p_jobSize, FUZ_rand(&lseed) % (2*maxTestSize+1)) );   /* custome job size */
+                CHECK_Z( ZSTDMT_initCStream_advanced(zc, dict, dictSize, params, pledgedSrcSize) );
         }   }
 
         /* multi-segments compression test */
@@ -1288,9 +1288,9 @@ static int fuzzerTests_MT(U32 seed, U32 nbTests, unsigned startTest, double comp
                     ZSTD_inBuffer inBuff = { srcBuffer+srcStart, srcSize, 0 };
                     outBuff.size = outBuff.pos + dstBuffSize;
 
-                    DISPLAYLEVEL(5, "Sending %u bytes to compress \n", (U32)srcSize);
+                    DISPLAYLEVEL(6, "Sending %u bytes to compress \n", (U32)srcSize);
                     CHECK_Z( ZSTDMT_compressStream(zc, &outBuff, &inBuff) );
-                    DISPLAYLEVEL(5, "%u bytes read by ZSTDMT_compressStream \n", (U32)inBuff.pos);
+                    DISPLAYLEVEL(6, "%u bytes read by ZSTDMT_compressStream \n", (U32)inBuff.pos);
 
                     XXH64_update(&xxhState, srcBuffer+srcStart, inBuff.pos);
                     memcpy(copyBuffer+totalTestSize, srcBuffer+srcStart, inBuff.pos);
@@ -1337,10 +1337,10 @@ static int fuzzerTests_MT(U32 seed, U32 nbTests, unsigned startTest, double comp
                 size_t const dstBuffSize = MIN(dstBufferSize - totalGenSize, randomDstSize);
                 inBuff.size = inBuff.pos + readCSrcSize;
                 outBuff.size = outBuff.pos + dstBuffSize;
-                DISPLAYLEVEL(5, "ZSTD_decompressStream input %u bytes \n", (U32)readCSrcSize);
+                DISPLAYLEVEL(6, "ZSTD_decompressStream input %u bytes \n", (U32)readCSrcSize);
                 decompressionResult = ZSTD_decompressStream(zd, &outBuff, &inBuff);
                 CHECK (ZSTD_isError(decompressionResult), "decompression error : %s", ZSTD_getErrorName(decompressionResult));
-                DISPLAYLEVEL(5, "inBuff.pos = %u \n", (U32)readCSrcSize);
+                DISPLAYLEVEL(6, "inBuff.pos = %u \n", (U32)readCSrcSize);
             }
             CHECK (outBuff.pos != totalTestSize, "decompressed data : wrong size (%u != %u)", (U32)outBuff.pos, (U32)totalTestSize);
             CHECK (inBuff.pos != cSize, "compressed data should be fully read (%u != %u)", (U32)inBuff.pos, (U32)cSize);

From e9dc0ab8843437ff590a5acbc479ff9769b2f449 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 12 Dec 2017 18:20:04 -0800
Subject: [PATCH 121/145] zstd binary, when compiled from /tests, enables debug
 mode

adds ZSTD_DEBUG=1 and -g
---
 tests/Makefile | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tests/Makefile b/tests/Makefile
index f944fd5a867..342a648c0f8 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -76,16 +76,16 @@ allnothread: fullbench fuzzer paramgrill datagen decodecorpus
 dll: fuzzer-dll zstreamtest-dll
 
 zstd:
-	$(MAKE) -C $(PRGDIR) $@
+	$(MAKE) -C $(PRGDIR) $@ MOREFLAGS="$(DEBUGFLAGS)"
 
 zstd32:
-	$(MAKE) -C $(PRGDIR) $@
+	$(MAKE) -C $(PRGDIR) $@ MOREFLAGS="$(DEBUGFLAGS)"
 
 zstd-nolegacy:
-	$(MAKE) -C $(PRGDIR) $@
+	$(MAKE) -C $(PRGDIR) $@ MOREFLAGS="$(DEBUGFLAGS)"
 
 gzstd:
-	$(MAKE) -C $(PRGDIR) zstd HAVE_ZLIB=1
+	$(MAKE) -C $(PRGDIR) zstd HAVE_ZLIB=1 MOREFLAGS="$(DEBUGFLAGS)"
 
 fullbench32: CPPFLAGS += -m32
 fullbench fullbench32 : CPPFLAGS += $(MULTITHREAD_CPP)

From 98ccc115edffd286d44ac6501479c7462b770825 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 12 Dec 2017 19:47:38 -0800
Subject: [PATCH 122/145] added cli test veryfying bug #944

this test fails on v1.3.2
---
 tests/playTests.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/playTests.sh b/tests/playTests.sh
index 299c2d883b1..6b78b305ac8 100755
--- a/tests/playTests.sh
+++ b/tests/playTests.sh
@@ -297,6 +297,8 @@ cp $TESTFILE tmp
 $ZSTD -f tmp -D tmpDict
 $ZSTD -d tmp.zst -D tmpDict -fo result
 $DIFF $TESTFILE result
+$ECHO "- Test dictionary compression with multithreading "
+./datagen -g5M | $ZSTD -T2 -D tmpDict | $ZSTD -t -D tmpDict   # fails with v1.3.2 
 $ECHO "- Create second (different) dictionary "
 $ZSTD --train *.c ../programs/*.c ../programs/*.h -o tmpDictC
 $ZSTD -d tmp.zst -D tmpDictC -fo result && die "wrong dictionary not detected!"

From 04a1557e2808be33bf199757482c59fa4bd287da Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 13 Dec 2017 10:08:52 -0800
Subject: [PATCH 123/145] fuzzer tests : added -v command

so that unit tests get listed in CI console
and we can see which one failed.
---
 tests/Makefile | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tests/Makefile b/tests/Makefile
index f944fd5a867..79df7dd9576 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -300,10 +300,10 @@ test-fullbench32: fullbench32 datagen
 	$(QEMU_SYS) ./fullbench32 -i1 -P0
 
 test-fuzzer: fuzzer
-	$(QEMU_SYS) ./fuzzer $(FUZZERTEST) $(FUZZER_FLAGS)
+	$(QEMU_SYS) ./fuzzer -v $(FUZZERTEST) $(FUZZER_FLAGS)
 
 test-fuzzer32: fuzzer32
-	$(QEMU_SYS) ./fuzzer32 $(FUZZERTEST) $(FUZZER_FLAGS)
+	$(QEMU_SYS) ./fuzzer32 -v $(FUZZERTEST) $(FUZZER_FLAGS)
 
 test-zbuff: zbufftest
 	$(QEMU_SYS) ./zbufftest $(ZSTREAM_TESTTIME)
@@ -312,7 +312,7 @@ test-zbuff32: zbufftest32
 	$(QEMU_SYS) ./zbufftest32 $(ZSTREAM_TESTTIME)
 
 test-zstream: zstreamtest
-	$(QEMU_SYS) ./zstreamtest $(ZSTREAM_TESTTIME) $(FUZZER_FLAGS)
+	$(QEMU_SYS) ./zstreamtest -v $(ZSTREAM_TESTTIME) $(FUZZER_FLAGS)
 	$(QEMU_SYS) ./zstreamtest --mt -t1 $(ZSTREAM_TESTTIME) $(FUZZER_FLAGS)
 	$(QEMU_SYS) ./zstreamtest --newapi -t1 $(ZSTREAM_TESTTIME) $(FUZZER_FLAGS)
 	$(QEMU_SYS) ./zstreamtest --opaqueapi -t1 $(ZSTREAM_TESTTIME) $(FUZZER_FLAGS)

From b5e7f6c0f3e023d9a6622d2b227bf6d15c73c574 Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Wed, 13 Dec 2017 10:55:24 -0800
Subject: [PATCH 124/145] [fileio] Fix window size MB calculation

Test command:
```
head -c 10000 /dev/zero | ./zstd -c --zstd=wlog=12 | ./zstd -M2048 -t
```
---
 programs/fileio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/programs/fileio.c b/programs/fileio.c
index 101c99b1ae8..4758fa0ed5b 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -1149,7 +1149,7 @@ static void FIO_zstdErrorHelp(dRess_t* ress, size_t ret, char const* srcFileName
     if (ret == 0) {
         U32 const windowSize = (U32)header.windowSize;
         U32 const windowLog = BIT_highbit32(windowSize) + ((windowSize & (windowSize - 1)) != 0);
-        U32 const windowMB = (windowSize >> 20) + (windowSize & ((1 MB) - 1));
+        U32 const windowMB = (windowSize >> 20) + ((windowSize & ((1 MB) - 1)) != 0);
         assert(header.windowSize <= (U64)((U32)-1));
         assert(g_memLimit > 0);
         DISPLAYLEVEL(1, "%s : Window size larger than maximum : %llu > %u\n",

From 22727a7467ff55892c6b9775a0c190cecc8b9b5c Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Tue, 12 Dec 2017 17:37:06 -0800
Subject: [PATCH 125/145] Fix cdict compressor repcodes

---
 lib/compress/zstd_compress.c |  6 +++++
 tests/fuzzer.c               | 44 ++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 3f39875f5bd..8cd288d2a0b 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -1033,6 +1033,12 @@ static size_t ZSTD_copyCCtx_internal(ZSTD_CCtx* dstCCtx,
 
     /* copy entropy tables */
     memcpy(dstCCtx->entropy, srcCCtx->entropy, sizeof(ZSTD_entropyCTables_t));
+    /* copy repcodes */
+    {
+        int i;
+        for (i = 0; i < ZSTD_REP_NUM; ++i)
+            dstCCtx->seqStore.rep[i] = srcCCtx->seqStore.rep[i];
+    }
 
     return 0;
 }
diff --git a/tests/fuzzer.c b/tests/fuzzer.c
index 46671a15cd4..f799b5f9ee8 100644
--- a/tests/fuzzer.c
+++ b/tests/fuzzer.c
@@ -835,6 +835,50 @@ static int basicUnitTests(U32 seed, double compressibility)
         }
         DISPLAYLEVEL(4, "OK \n");
 
+        DISPLAYLEVEL(4, "test%3i : Dictionary with non-default repcodes : ", testNb++);
+        { U32 u; for (u=0; u= dictLimit) goto _output_error;
+            MEM_writeLE32(dictPtr + 0, 10);
+            MEM_writeLE32(dictPtr + 4, 10);
+            MEM_writeLE32(dictPtr + 8, 10);
+            /* Set the last 8 bytes to 'x' */
+            memset((BYTE*)dictBuffer + dictSize - 8, 'x', 8);
+        }
+        /* The optimal parser checks all the repcodes.
+         * Make sure at least one is a match >= targetLength so that it is
+         * immediately chosen. This will make sure that the compressor and
+         * decompressor agree on at least one of the repcodes.
+         */
+        {   size_t dSize;
+            BYTE data[1024];
+            ZSTD_compressionParameters const cParams = ZSTD_getCParams(19, CNBuffSize, dictSize);
+            ZSTD_CDict* const cdict = ZSTD_createCDict_advanced(dictBuffer, dictSize,
+                                            ZSTD_dlm_byRef, ZSTD_dm_auto,
+                                            cParams, ZSTD_defaultCMem);
+            memset(data, 'x', sizeof(data));
+            cSize = ZSTD_compress_usingCDict(cctx, compressedBuffer, compressedBufferSize,
+                                             data, sizeof(data), cdict);
+            ZSTD_freeCDict(cdict);
+            if (ZSTD_isError(cSize)) { DISPLAYLEVEL(5, "Compression error %s : ", ZSTD_getErrorName(cSize)); goto _output_error; }
+            dSize = ZSTD_decompress_usingDict(dctx, decodedBuffer, sizeof(data), compressedBuffer, cSize, dictBuffer, dictSize);
+            if (ZSTD_isError(dSize)) { DISPLAYLEVEL(5, "Decompression error %s : ", ZSTD_getErrorName(dSize)); goto _output_error; }
+            if (memcmp(data, decodedBuffer, sizeof(data))) { DISPLAYLEVEL(5, "Data corruption : "); goto _output_error; }
+        }
+        DISPLAYLEVEL(4, "OK \n");
+
         ZSTD_freeCCtx(cctx);
         free(dictBuffer);
         free(samplesSizes);

From 311878dec3ae7a9d9cf54883c0dac8c1ccedc81a Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 13 Dec 2017 11:48:30 -0800
Subject: [PATCH 126/145] Improved tests

- building cli from /tests preserves potential flags in MOREFLAGS (such as asan/usan)
- MT dictionary tests check for MT capability (MT is not enabled by default for zstd32)
---
 tests/Makefile     | 6 +++---
 tests/playTests.sh | 7 +++++--
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/tests/Makefile b/tests/Makefile
index faa7d019c37..853f4ee89b1 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -76,13 +76,13 @@ allnothread: fullbench fuzzer paramgrill datagen decodecorpus
 dll: fuzzer-dll zstreamtest-dll
 
 zstd:
-	$(MAKE) -C $(PRGDIR) $@ MOREFLAGS="$(DEBUGFLAGS)"
+	$(MAKE) -C $(PRGDIR) $@ MOREFLAGS+="$(DEBUGFLAGS)"
 
 zstd32:
-	$(MAKE) -C $(PRGDIR) $@ MOREFLAGS="$(DEBUGFLAGS)"
+	$(MAKE) -C $(PRGDIR) $@ MOREFLAGS+="$(DEBUGFLAGS)"
 
 zstd-nolegacy:
-	$(MAKE) -C $(PRGDIR) $@ MOREFLAGS="$(DEBUGFLAGS)"
+	$(MAKE) -C $(PRGDIR) $@ MOREFLAGS+="$(DEBUGFLAGS)"
 
 gzstd:
 	$(MAKE) -C $(PRGDIR) zstd HAVE_ZLIB=1 MOREFLAGS="$(DEBUGFLAGS)"
diff --git a/tests/playTests.sh b/tests/playTests.sh
index 6b78b305ac8..67732d3a15b 100755
--- a/tests/playTests.sh
+++ b/tests/playTests.sh
@@ -297,8 +297,11 @@ cp $TESTFILE tmp
 $ZSTD -f tmp -D tmpDict
 $ZSTD -d tmp.zst -D tmpDict -fo result
 $DIFF $TESTFILE result
-$ECHO "- Test dictionary compression with multithreading "
-./datagen -g5M | $ZSTD -T2 -D tmpDict | $ZSTD -t -D tmpDict   # fails with v1.3.2 
+if [ -n "$hasMT" ]
+then
+    $ECHO "- Test dictionary compression with multithreading "
+    ./datagen -g5M | $ZSTD -T2 -D tmpDict | $ZSTD -t -D tmpDict   # fails with v1.3.2 
+fi
 $ECHO "- Create second (different) dictionary "
 $ZSTD --train *.c ../programs/*.c ../programs/*.h -o tmpDictC
 $ZSTD -d tmp.zst -D tmpDictC -fo result && die "wrong dictionary not detected!"

From 82bc8fe0cc3746d9b02b86c1bddaed842d07f755 Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Wed, 13 Dec 2017 12:04:46 -0800
Subject: [PATCH 127/145] [fileio] Refuse to remove non-regular file

---
 programs/fileio.c  | 12 ++++++++++--
 tests/playTests.sh |  8 ++++++++
 2 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/programs/fileio.c b/programs/fileio.c
index 101c99b1ae8..a0e370bac5b 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -261,6 +261,10 @@ void FIO_setLdmHashEveryLog(unsigned ldmHashEveryLog) {
  * @result : Unlink `fileName`, even if it's read-only */
 static int FIO_remove(const char* path)
 {
+    if (!UTIL_isRegularFile(path)) {
+        DISPLAYLEVEL(2, "zstd: Refusing to remove non-regular file %s\n", path);
+        return 0;
+    }
 #if defined(_WIN32) || defined(WIN32)
     /* windows doesn't allow remove read-only files,
      * so try to make it writable first */
@@ -314,9 +318,13 @@ static FILE* FIO_openDstFile(const char* dstFileName)
         g_sparseFileSupport = ZSTD_SPARSE_DEFAULT;
     }
 
-    if (strcmp (dstFileName, nulmark)) {  /* not /dev/null */
+    if (UTIL_isRegularFile(dstFileName)) {
+        FILE* fCheck;
+        if (!strcmp(dstFileName, nulmark)) {
+            EXM_THROW(40, "%s is unexpectedly a regular file", dstFileName);
+        }
         /* Check if destination file already exists */
-        FILE* const fCheck = fopen( dstFileName, "rb" );
+        fCheck = fopen( dstFileName, "rb" );
         if (fCheck != NULL) {  /* dst file exists, authorization prompt */
             fclose(fCheck);
             if (!g_overwrite) {
diff --git a/tests/playTests.sh b/tests/playTests.sh
index 299c2d883b1..402808c0dc3 100755
--- a/tests/playTests.sh
+++ b/tests/playTests.sh
@@ -56,10 +56,12 @@ fi
 
 isWindows=false
 INTOVOID="/dev/null"
+DEVDEVICE="/dev/zero"
 case "$OS" in
   Windows*)
     isWindows=true
     INTOVOID="NUL"
+    DEVDEVICE="NUL"
     ;;
 esac
 
@@ -159,6 +161,12 @@ $ZSTD -f --rm tmp
 test ! -f tmp  # tmp should no longer be present
 $ZSTD -f -d --rm tmp.zst
 test ! -f tmp.zst   # tmp.zst should no longer be present
+$ECHO "test : should quietly not remove non-regular file"
+$ECHO hello > tmp
+$ZSTD tmp -f -o "$DEVDEVICE" 2>tmplog > "$INTOVOID"
+grep -v "Refusing to remove non-regular file" tmplog
+rm -f tmplog
+$ZSTD tmp -f -o "$INTONULL" 2>&1 | grep -v "Refusing to remove non-regular file"
 $ECHO "test : --rm on stdin"
 $ECHO a | $ZSTD --rm > $INTOVOID   # --rm should remain silent
 rm tmp

From d23eb9a098ce99ae0fe039496385687c0e574c60 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 13 Dec 2017 15:35:49 -0800
Subject: [PATCH 128/145] zstreamtest : added missing CHECK_Z()

---
 lib/compress/zstd_compress.c |  2 +-
 tests/zstreamtest.c          | 10 +++++-----
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 7586c3a0c74..34bfb737c14 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -2566,7 +2566,7 @@ size_t ZSTD_initCStream_internal(ZSTD_CStream* zcs,
         if (zcs->cdictLocal == NULL) return ERROR(memory_allocation);
     } else {
         if (cdict) {
-            params.cParams = ZSTD_getCParamsFromCDict(cdict);  /* cParams are enforced from cdict; this includes windowLog (should it be enforced from `params` instead ?) */
+            params.cParams = ZSTD_getCParamsFromCDict(cdict);  /* cParams are enforced from cdict; it includes windowLog */
         }
         ZSTD_freeCDict(zcs->cdictLocal);
         zcs->cdictLocal = NULL;
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 67ef6ce802f..70922e2e033 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -710,9 +710,9 @@ static int basicUnitTests(U32 seed, double compressibility)
         const BYTE* const srcToCopy = (const BYTE*)CNBuffer + start;
         BYTE* const dst = (BYTE*)CNBuffer + start - offset;
         DISPLAYLEVEL(3, "test%3i : compress %u bytes with multiple threads + dictionary : ", testNb++, (U32)srcSize);
-        ZSTD_CCtx_setParameter(zc, ZSTD_p_compressionLevel, 3);
-        ZSTD_CCtx_setParameter(zc, ZSTD_p_nbThreads, 2);
-        ZSTD_CCtx_setParameter(zc, ZSTD_p_jobSize, jobSize);
+        CHECK_Z( ZSTD_CCtx_setParameter(zc, ZSTD_p_compressionLevel, 3) );
+        CHECK_Z( ZSTD_CCtx_setParameter(zc, ZSTD_p_nbThreads, 2) );
+        CHECK_Z( ZSTD_CCtx_setParameter(zc, ZSTD_p_jobSize, jobSize) );
         assert(start > offset);
         assert(start + segLength < COMPRESSIBLE_NOISE_LENGTH);
         memcpy(dst, srcToCopy, segLength);   /* create a long repetition at long distance for job 2 */
@@ -726,9 +726,9 @@ static int basicUnitTests(U32 seed, double compressibility)
     {   ZSTD_compressionParameters const cParams = ZSTD_getCParams(1, 4 KB, dictionary.filled);   /* intentionnally lies on estimatedSrcSize, to push cdict into targeting a small window size */
         ZSTD_CDict* const cdict = ZSTD_createCDict_advanced(dictionary.start, dictionary.filled, ZSTD_dlm_byRef, ZSTD_dm_fullDict, cParams, ZSTD_defaultCMem);
         DISPLAYLEVEL(5, "cParams.windowLog = %u : ", cParams.windowLog);
-        ZSTD_CCtx_refCDict(zc, cdict);
+        CHECK_Z( ZSTD_CCtx_refCDict(zc, cdict) );
         CHECK_Z( ZSTD_compress_generic(zc, &outBuff, &inBuff, ZSTD_e_end) );
-        ZSTD_CCtx_refCDict(zc, NULL);  /* do not keep a reference to cdict, as its lifetime ends */
+        CHECK_Z( ZSTD_CCtx_refCDict(zc, NULL) );  /* do not keep a reference to cdict, as its lifetime ends */
         ZSTD_freeCDict(cdict);
     }
     if (inBuff.pos != inBuff.size) goto _output_error;   /* entire input should be consumed */

From 4680e85bdf6d6862564a7465b77f9243592cc66a Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Tue, 12 Dec 2017 18:32:50 -0800
Subject: [PATCH 129/145] Allow -o with multiple files

---
 programs/fileio.c  | 26 +++++++++++---------------
 programs/fileio.h  |  4 ++--
 programs/zstdcli.c | 10 ++--------
 tests/playTests.sh | 16 ++++++++++++++++
 4 files changed, 31 insertions(+), 25 deletions(-)

diff --git a/programs/fileio.c b/programs/fileio.c
index 5a8840b5b62..16c2cb8fe9c 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -939,7 +939,7 @@ int FIO_compressFilename(const char* dstFileName, const char* srcFileName,
 
 
 int FIO_compressMultipleFilenames(const char** inFileNamesTable, unsigned nbFiles,
-                                  const char* suffix,
+                                  const char* outFileName, const char* suffix,
                                   const char* dictFileName, int compressionLevel,
                                   ZSTD_compressionParameters* comprParams)
 {
@@ -955,16 +955,15 @@ int FIO_compressMultipleFilenames(const char** inFileNamesTable, unsigned nbFile
     /* init */
     if (dstFileName==NULL)
         EXM_THROW(27, "FIO_compressMultipleFilenames : allocation error for dstFileName");
-    if (suffix == NULL)
+    if (outFileName == NULL && suffix == NULL)
         EXM_THROW(28, "FIO_compressMultipleFilenames : dst unknown");  /* should never happen */
 
     /* loop on each file */
-    if (!strcmp(suffix, stdoutmark)) {
+    if (outFileName != NULL) {
         unsigned u;
-        ress.dstFile = stdout;
-        SET_BINARY_MODE(stdout);
+        ress.dstFile = FIO_openDstFile(outFileName);
         for (u=0; u1)) {
-        DISPLAY("Too many files (%u) on the command line. \n", filenameIdx);
-        CLEAN_RETURN(filenameIdx);
-    }
-
 #ifndef ZSTD_NOCOMPRESS
     /* check compression level limits */
     {   int const maxCLevel = ultra ? ZSTD_maxCLevel() : ZSTDCLI_CLEVEL_MAX;
@@ -854,7 +848,7 @@ int main(int argCount, const char* argv[])
         if ((filenameIdx==1) && outFileName)
           operationResult = FIO_compressFilename(outFileName, filenameTable[0], dictFileName, cLevel, &compressionParams);
         else
-          operationResult = FIO_compressMultipleFilenames(filenameTable, filenameIdx, outFileName ? outFileName : suffix, dictFileName, cLevel, &compressionParams);
+          operationResult = FIO_compressMultipleFilenames(filenameTable, filenameIdx, outFileName, suffix, dictFileName, cLevel, &compressionParams);
 #else
         (void)suffix;
         DISPLAY("Compression not supported\n");
@@ -872,7 +866,7 @@ int main(int argCount, const char* argv[])
         if (filenameIdx==1 && outFileName)
             operationResult = FIO_decompressFilename(outFileName, filenameTable[0], dictFileName);
         else
-            operationResult = FIO_decompressMultipleFilenames(filenameTable, filenameIdx, outFileName ? outFileName : ZSTD_EXTENSION, dictFileName);
+            operationResult = FIO_decompressMultipleFilenames(filenameTable, filenameIdx, outFileName, dictFileName);
 #else
         DISPLAY("Decompression not supported\n");
 #endif
diff --git a/tests/playTests.sh b/tests/playTests.sh
index 402808c0dc3..773da845b5b 100755
--- a/tests/playTests.sh
+++ b/tests/playTests.sh
@@ -173,6 +173,22 @@ rm tmp
 $ZSTD -f tmp && die "tmp not present : should have failed"
 test ! -f tmp.zst  # tmp.zst should not be created
 
+$ECHO "test : compress multiple files"
+$ECHO hello > tmp1
+$ECHO world > tmp2
+$ZSTD tmp1 tmp2 -o "$INTOVOID"
+$ZSTD tmp1 tmp2 -c | $ZSTD -t
+$ZSTD tmp1 tmp2 -o tmp.zst
+test ! -f tmp1.zst
+test ! -f tmp2.zst
+$ZSTD tmp1 tmp2
+$ZSTD -t tmp1.zst tmp2.zst
+$ZSTD -dc tmp1.zst tmp2.zst
+$ZSTD tmp1.zst tmp2.zst -o "$INTOVOID"
+$ZSTD -d tmp1.zst tmp2.zst -o tmp
+rm tmp*
+
+
 
 $ECHO "\n===>  Advanced compression parameters "
 $ECHO "Hello world!" | $ZSTD --zstd=windowLog=21,      - -o tmp.zst && die "wrong parameters not detected!"

From 5432ef6921b840c6cfe25f5d8ffcc87f4f91deff Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 13 Dec 2017 17:45:26 -0800
Subject: [PATCH 130/145] fixes adaptation on srcSize

This patch restores capability for each file to receive adapted compression parameters depending on its size.

The bug breaking this feature was relatively silly :
setting a parameter with a value "0" is supposed to be a no-op.
Unfortunately, it would pin down compression parameters as if they were manually set,
preventing later automatic adaptation.

Unfortunately, I'm currently short of a test case that could check this situation and trigger an error.
Compression parameters selection between tableID 0,1,2,3 is largely internal,
leaving no trace to outside world, not even in frame header.
---
 lib/compress/zstd_compress.c | 12 ++++++++++--
 programs/fileio.c            |  2 +-
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 8cd288d2a0b..d52261aae7e 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -144,6 +144,8 @@ const seqStore_t* ZSTD_getSeqStore(const ZSTD_CCtx* ctx) { return &(ctx->seqStor
 static ZSTD_compressionParameters ZSTD_getCParamsFromCCtxParams(
         ZSTD_CCtx_params CCtxParams, U64 srcSizeHint, size_t dictSize)
 {
+    DEBUGLOG(4, "ZSTD_getCParamsFromCCtxParams: srcSize = %u, dictSize = %u",
+                (U32)srcSizeHint, (U32)dictSize);
     return (CCtxParams.compressionLevel == ZSTD_CLEVEL_CUSTOM) ?
                 CCtxParams.cParams :
                 ZSTD_getCParams(CCtxParams.compressionLevel, srcSizeHint, dictSize);
@@ -151,18 +153,22 @@ static ZSTD_compressionParameters ZSTD_getCParamsFromCCtxParams(
 
 static void ZSTD_cLevelToCCtxParams_srcSize(ZSTD_CCtx_params* CCtxParams, U64 srcSize)
 {
+    DEBUGLOG(4, "ZSTD_cLevelToCCtxParams_srcSize: srcSize = %u",
+                (U32)srcSize);
     CCtxParams->cParams = ZSTD_getCParamsFromCCtxParams(*CCtxParams, srcSize, 0);
     CCtxParams->compressionLevel = ZSTD_CLEVEL_CUSTOM;
 }
 
 static void ZSTD_cLevelToCParams(ZSTD_CCtx* cctx)
 {
+    DEBUGLOG(4, "ZSTD_cLevelToCParams");
     ZSTD_cLevelToCCtxParams_srcSize(
             &cctx->requestedParams, cctx->pledgedSrcSizePlusOne-1);
 }
 
 static void ZSTD_cLevelToCCtxParams(ZSTD_CCtx_params* CCtxParams)
 {
+    DEBUGLOG(4, "ZSTD_cLevelToCCtxParams");
     ZSTD_cLevelToCCtxParams_srcSize(CCtxParams, 0);
 }
 
@@ -261,7 +267,7 @@ size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, unsigned v
     case ZSTD_p_targetLength:
     case ZSTD_p_compressionStrategy:
         if (cctx->cdict) return ERROR(stage_wrong);
-        ZSTD_cLevelToCParams(cctx);  /* Can optimize if srcSize is known */
+        if (value>0) ZSTD_cLevelToCParams(cctx);  /* Can optimize if srcSize is known */
         return ZSTD_CCtxParam_setParameter(&cctx->requestedParams, param, value);
 
     case ZSTD_p_contentSizeFlag:
@@ -422,7 +428,7 @@ size_t ZSTD_CCtxParam_setParameter(
 #endif
 
     case ZSTD_p_enableLongDistanceMatching :
-        if (value != 0) {
+        if (value) {
             ZSTD_cLevelToCCtxParams(CCtxParams);
             CCtxParams->cParams.windowLog = ZSTD_LDM_DEFAULT_WINDOW_LOG;
         }
@@ -3056,6 +3062,8 @@ ZSTD_compressionParameters ZSTD_getCParams(int compressionLevel, unsigned long l
     size_t const addedSize = srcSizeHint ? 0 : 500;
     U64 const rSize = srcSizeHint+dictSize ? srcSizeHint+dictSize+addedSize : (U64)-1;
     U32 const tableID = (rSize <= 256 KB) + (rSize <= 128 KB) + (rSize <= 16 KB);   /* intentional underflow for srcSizeHint == 0 */
+    DEBUGLOG(4, "ZSTD_getCParams: cLevel=%i, srcSize=%u, dictSize=%u => table %u",
+                compressionLevel, (U32)srcSizeHint, (U32)dictSize, tableID);
 
 #if defined(ZSTD_DEBUG) && (ZSTD_DEBUG>=1)
     static int g_monotonicTest = 1;
diff --git a/programs/fileio.c b/programs/fileio.c
index 4758fa0ed5b..8799540786d 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -446,7 +446,7 @@ static cRess_t FIO_createCResources(const char* dictFileName, int cLevel,
         DISPLAYLEVEL(5,"set nb threads = %u \n", g_nbThreads);
         CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_nbThreads, g_nbThreads) );
         /* dictionary */
-        CHECK( ZSTD_CCtx_setPledgedSrcSize(ress.cctx, srcSize) );  /* just for dictionary loading, using good compression parameters */
+        CHECK( ZSTD_CCtx_setPledgedSrcSize(ress.cctx, srcSize) );  /* just for dictionary loading, for compression parameters adaptation */
         CHECK( ZSTD_CCtx_loadDictionary(ress.cctx, dictBuffer, dictBuffSize) );
         CHECK( ZSTD_CCtx_setPledgedSrcSize(ress.cctx, ZSTD_CONTENTSIZE_UNKNOWN) );  /* reset */
 

From 2e97a6d464d85e82584ba77f971526f1f1fd106b Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Wed, 13 Dec 2017 18:50:05 -0800
Subject: [PATCH 131/145] fixed minor declaration-after-statement warning

---
 lib/compress/zstd_compress.c | 6 ++++--
 programs/fileio.c            | 4 ++++
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index d52261aae7e..02235c11a68 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -1738,6 +1738,8 @@ static size_t ZSTD_compress_frameChunk (ZSTD_CCtx* cctx,
             op += cSize;
             assert(dstCapacity >= cSize);
             dstCapacity -= cSize;
+            DEBUGLOG(5, "ZSTD_compress_frameChunk: adding a block of size %u",
+                        (U32)cSize);
     }   }
 
     if (lastFrameChunk && (op>ostart)) cctx->stage = ZSTDcs_ending;
@@ -3062,8 +3064,6 @@ ZSTD_compressionParameters ZSTD_getCParams(int compressionLevel, unsigned long l
     size_t const addedSize = srcSizeHint ? 0 : 500;
     U64 const rSize = srcSizeHint+dictSize ? srcSizeHint+dictSize+addedSize : (U64)-1;
     U32 const tableID = (rSize <= 256 KB) + (rSize <= 128 KB) + (rSize <= 16 KB);   /* intentional underflow for srcSizeHint == 0 */
-    DEBUGLOG(4, "ZSTD_getCParams: cLevel=%i, srcSize=%u, dictSize=%u => table %u",
-                compressionLevel, (U32)srcSizeHint, (U32)dictSize, tableID);
 
 #if defined(ZSTD_DEBUG) && (ZSTD_DEBUG>=1)
     static int g_monotonicTest = 1;
@@ -3073,6 +3073,8 @@ ZSTD_compressionParameters ZSTD_getCParams(int compressionLevel, unsigned long l
     }
 #endif
 
+    DEBUGLOG(4, "ZSTD_getCParams: cLevel=%i, srcSize=%u, dictSize=%u => table %u",
+                compressionLevel, (U32)srcSizeHint, (U32)dictSize, tableID);
     if (compressionLevel <= 0) compressionLevel = ZSTD_CLEVEL_DEFAULT;   /* 0 == default; no negative compressionLevel yet */
     if (compressionLevel > ZSTD_MAX_CLEVEL) compressionLevel = ZSTD_MAX_CLEVEL;
     { ZSTD_compressionParameters const cp = ZSTD_defaultCParameters[tableID][compressionLevel];
diff --git a/programs/fileio.c b/programs/fileio.c
index 8799540786d..68032af94bb 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -781,6 +781,8 @@ static int FIO_compressFilename_internal(cRess_t ress,
                         &outBuff, &inBuff, ZSTD_e_continue) );
 
             /* Write compressed stream */
+            DISPLAYLEVEL(6, "ZSTD_compress_generic,ZSTD_e_continue: generated %u bytes \n",
+                            (U32)outBuff.pos);
             if (outBuff.pos) {
                 size_t const sizeCheck = fwrite(ress.dstBuffer, 1, outBuff.pos, dstFile);
                 if (sizeCheck!=outBuff.pos)
@@ -816,6 +818,8 @@ static int FIO_compressFilename_internal(cRess_t ress,
                 EXM_THROW(26, "Compression error during frame end : %s",
                             ZSTD_getErrorName(result));
             }
+            DISPLAYLEVEL(6, "ZSTD_compress_generic,ZSTD_e_end: generated %u bytes \n",
+                            (U32)outBuff.pos);
             {   size_t const sizeCheck = fwrite(ress.dstBuffer, 1, outBuff.pos, dstFile);
                 if (sizeCheck != outBuff.pos)
                     EXM_THROW(27, "Write error : cannot write frame end into %s", dstFileName);

From 281f06e01f6ad818769e71804f625c9750ac459c Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Thu, 14 Dec 2017 11:47:02 -0800
Subject: [PATCH 132/145] saves 3-bytes on small input with streaming API

zstd streaming API was adding a null-block at end of frame for small input.

Reason is : on small input, a single block is enough.
ZSTD_CStream would size its input buffer to expect a single block of this size,
automatically triggering a flush on reaching this size.

Unfortunately, that last byte was generally received before the "end" directive (at least in `fileio`).
The later "end" directive would force the creation of a 3-bytes last block to indicate end of frame.

The solution is to not flush automatically, which is btw the expected behavior.
It happens in this case because blocksize is defined with exactly the same size as input.
Just adding one-byte is enough to stop triggering the automatic flush.

I initially looked at another solution, solving the problem directly in the compression context.
But it felt awkward.
Now, the underlying compression API `ZSTD_compressContinue()` would take the decision the close a frame
on reaching its expected end (`pledgedSrcSize`).
This feels awkward, a responsability over-reach, beyond the definition of this API.
ZSTD_compressContinue() is clearly documented as a guaranteed flush,
with ZSTD_compressEnd() generating a guaranteed end.

I faced similar issue when trying to port a similar mechanism at the higher streaming layer.
Having ZSTD_CStream end a frame automatically on reaching `pledgedSrcSize` can surprise the caller,
since it did not explicitly requested an end of frame.
The only sensible action remaining after that is to end the frame with no additional input.
This adds additional logic in the ZSTD_CStream state to check this condition.
Plus some potential confusion on the meaning of ZSTD_endStream() with no additional input (ending confirmation ? new 0-size frame ?)

In the end, just enlarging input buffer by 1 byte feels the least intrusive change.
It's also a contract remaining inside the streaming layer, so the logic is contained in this part of the code.

The patch also introduces a new test checking that size of small frame is as expected, without additional 3-bytes null block.
---
 lib/compress/zstd_compress.c | 9 +++++----
 tests/playTests.sh           | 4 +++-
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 0dec78820b2..37dd5a4b0b1 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -2527,7 +2527,8 @@ static size_t ZSTD_resetCStream_internal(ZSTD_CStream* zcs,
 
     zcs->inToCompress = 0;
     zcs->inBuffPos = 0;
-    zcs->inBuffTarget = zcs->blockSize;
+    zcs->inBuffTarget = zcs->blockSize
+                      + (zcs->blockSize == pledgedSrcSize);   /* for small input: avoid automatic flush on reaching end of block, since it would require to add a 3-bytes null block to end frame */
     zcs->outBuffContentSize = zcs->outBuffFlushedSize = 0;
     zcs->streamStage = zcss_load;
     zcs->frameEnded = 0;
@@ -2679,9 +2680,9 @@ size_t ZSTD_compressStream_generic(ZSTD_CStream* zcs,
     /* check expectations */
     DEBUGLOG(5, "ZSTD_compressStream_generic, flush=%u", (U32)flushMode);
     assert(zcs->inBuff != NULL);
-    assert(zcs->inBuffSize>0);
-    assert(zcs->outBuff!= NULL);
-    assert(zcs->outBuffSize>0);
+    assert(zcs->inBuffSize > 0);
+    assert(zcs->outBuff !=  NULL);
+    assert(zcs->outBuffSize > 0);
     assert(output->pos <= output->size);
     assert(input->pos <= input->size);
 
diff --git a/tests/playTests.sh b/tests/playTests.sh
index 9db3dadea3b..c48148fa295 100755
--- a/tests/playTests.sh
+++ b/tests/playTests.sh
@@ -126,6 +126,8 @@ $ZSTD -d > $INTOVOID && die "should have refused : compressed data from terminal
 fi
 $ECHO "test : null-length file roundtrip"
 $ECHO -n '' | $ZSTD - --stdout | $ZSTD -d --stdout
+$ECHO "test : ensure small file doesn't add 3-bytes null block"
+./datagen -g1 | $ZSTD | wc -c | grep "14"
 $ECHO "test : decompress file with wrong suffix (must fail)"
 $ZSTD -d tmpCompressed && die "wrong suffix error not detected!"
 $ZSTD -df tmp && die "should have refused : wrong extension"
@@ -324,7 +326,7 @@ $DIFF $TESTFILE result
 if [ -n "$hasMT" ]
 then
     $ECHO "- Test dictionary compression with multithreading "
-    ./datagen -g5M | $ZSTD -T2 -D tmpDict | $ZSTD -t -D tmpDict   # fails with v1.3.2 
+    ./datagen -g5M | $ZSTD -T2 -D tmpDict | $ZSTD -t -D tmpDict   # fails with v1.3.2
 fi
 $ECHO "- Create second (different) dictionary "
 $ZSTD --train *.c ../programs/*.c ../programs/*.h -o tmpDictC

From a0e0985d38d05a9a28b8b94a9f9ece8120c146a8 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Thu, 14 Dec 2017 13:32:24 -0800
Subject: [PATCH 133/145] added test on small file

on top of test on small stream
---
 tests/playTests.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tests/playTests.sh b/tests/playTests.sh
index c48148fa295..c93c58fe8d5 100755
--- a/tests/playTests.sh
+++ b/tests/playTests.sh
@@ -127,7 +127,9 @@ fi
 $ECHO "test : null-length file roundtrip"
 $ECHO -n '' | $ZSTD - --stdout | $ZSTD -d --stdout
 $ECHO "test : ensure small file doesn't add 3-bytes null block"
-./datagen -g1 | $ZSTD | wc -c | grep "14"
+./datagen -g1 > tmp1
+$ZSTD tmp1 -c | wc -c | grep "14"
+$ZSTD < tmp1  | wc -c | grep "14"
 $ECHO "test : decompress file with wrong suffix (must fail)"
 $ZSTD -d tmpCompressed && die "wrong suffix error not detected!"
 $ZSTD -df tmp && die "should have refused : wrong extension"

From f48d34edba069b33d30ad75d1684b2a8e1071520 Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Thu, 14 Dec 2017 13:00:20 -0800
Subject: [PATCH 134/145] [fileio] Merge end loop for small optimization

---
 programs/fileio.c | 51 +++++++++++++++++------------------------------
 1 file changed, 18 insertions(+), 33 deletions(-)

diff --git a/programs/fileio.c b/programs/fileio.c
index 339c968b2b4..b57ce14d2f3 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -118,12 +118,13 @@ static UTIL_time_t g_displayClock = UTIL_TIME_INITIALIZER;
     exit(error);                                                          \
 }
 
-#define CHECK(f) {                                   \
-    size_t const err = f;                            \
-    if (ZSTD_isError(err)) {                         \
+#define CHECK_V(v, f)                                \
+    v = f;                                           \
+    if (ZSTD_isError(v)) {                           \
         DEBUGLOG(1, "%s \n", #f);                    \
-        EXM_THROW(11, "%s", ZSTD_getErrorName(err)); \
-}   }
+        EXM_THROW(11, "%s", ZSTD_getErrorName(v));   \
+    }
+#define CHECK(f) { size_t err; CHECK_V(err, f); }
 
 
 /*-************************************
@@ -733,6 +734,7 @@ static int FIO_compressFilename_internal(cRess_t ress,
     U64 readsize = 0;
     U64 compressedfilesize = 0;
     U64 const fileSize = UTIL_getFileSize(srcFileName);
+    ZSTD_EndDirective directive = ZSTD_e_continue;
     DISPLAYLEVEL(5, "%s: %u bytes \n", srcFileName, (U32)fileSize);
 
     switch (g_compressionType) {
@@ -776,17 +778,20 @@ static int FIO_compressFilename_internal(cRess_t ress,
         ZSTD_CCtx_setPledgedSrcSize(ress.cctx, fileSize);
 
     /* Main compression loop */
-    while (1) {
+    do {
+        size_t result;
         /* Fill input Buffer */
         size_t const inSize = fread(ress.srcBuffer, (size_t)1, ress.srcBufferSize, srcFile);
         ZSTD_inBuffer inBuff = { ress.srcBuffer, inSize, 0 };
-        if (inSize==0) break;
         readsize += inSize;
 
-        while (inBuff.pos != inBuff.size) {
+        if (inSize == 0 || (fileSize != UTIL_FILESIZE_UNKNOWN && readsize == fileSize))
+            directive = ZSTD_e_end;
+
+        result = 1;
+        while (inBuff.pos != inBuff.size || (directive == ZSTD_e_end && result != 0)) {
             ZSTD_outBuffer outBuff = { ress.dstBuffer, ress.dstBufferSize, 0 };
-            CHECK( ZSTD_compress_generic(ress.cctx,
-                        &outBuff, &inBuff, ZSTD_e_continue) );
+            CHECK_V(result, ZSTD_compress_generic(ress.cctx, &outBuff, &inBuff, directive));
 
             /* Write compressed stream */
             DISPLAYLEVEL(6, "ZSTD_compress_generic,ZSTD_e_continue: generated %u bytes \n",
@@ -796,7 +801,8 @@ static int FIO_compressFilename_internal(cRess_t ress,
                 if (sizeCheck!=outBuff.pos)
                     EXM_THROW(25, "Write error : cannot write compressed block into %s", dstFileName);
                 compressedfilesize += outBuff.pos;
-        }   }
+            }
+        }
         if (g_nbThreads > 1) {
             if (fileSize == UTIL_FILESIZE_UNKNOWN)
                 DISPLAYUPDATE(2, "\rRead : %u MB", (U32)(readsize>>20))
@@ -813,28 +819,7 @@ static int FIO_compressFilename_internal(cRess_t ress,
                                 (U32)(readsize>>20), (U32)(fileSize>>20),
                                 (double)compressedfilesize/readsize*100);
         }
-    }
-
-    /* End of Frame */
-    {   size_t result = 1;
-        while (result != 0) {
-            ZSTD_outBuffer outBuff = { ress.dstBuffer, ress.dstBufferSize, 0 };
-            ZSTD_inBuffer inBuff = { NULL, 0, 0 };
-            result = ZSTD_compress_generic(ress.cctx,
-                        &outBuff, &inBuff, ZSTD_e_end);
-            if (ZSTD_isError(result)) {
-                EXM_THROW(26, "Compression error during frame end : %s",
-                            ZSTD_getErrorName(result));
-            }
-            DISPLAYLEVEL(6, "ZSTD_compress_generic,ZSTD_e_end: generated %u bytes \n",
-                            (U32)outBuff.pos);
-            {   size_t const sizeCheck = fwrite(ress.dstBuffer, 1, outBuff.pos, dstFile);
-                if (sizeCheck != outBuff.pos)
-                    EXM_THROW(27, "Write error : cannot write frame end into %s", dstFileName);
-            }
-            compressedfilesize += outBuff.pos;
-        }
-    }
+    } while (directive != ZSTD_e_end);
 
 finish:
     /* Status */

From 2cff66b62f68ec2e76f401629c7083f65a8198ec Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Thu, 14 Dec 2017 16:11:20 -0800
Subject: [PATCH 135/145] version bump to v1.3.3

---
 NEWS                 | 12 ++++++++++++
 doc/zstd_manual.html | 29 ++++++++++++++++++-----------
 lib/zstd.h           |  2 +-
 programs/Makefile    |  4 ++--
 programs/zstd.1      |  2 +-
 5 files changed, 34 insertions(+), 15 deletions(-)

diff --git a/NEWS b/NEWS
index 215c0a329ba..a45960b9688 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,15 @@
+v1.3.3
+perf: faster zstd_opt strategy (levels 17-19)
+fix : bug #944 : multithreading with shared ditionary and large data, reported by @gsliepen
+cli : fix : content size written in header by default
+cli : fix : improved LZ4 format support, by @felixhandte
+cli : new : hidden command `-S`, to benchmark multiple files while generating one result per file
+api : fix : support large skippable frames, by @terrelln
+api : fix : streaming interface was adding a useless 3-bytes null block to small frames
+api : change : when setting `pledgedSrcSize`, use `ZSTD_CONTENTSIZE_UNKNOWN` macro value to mean "unknown"
+build: fix : compilation under rhel6 and centos6, reported by @pixelb
+build: added `check` target
+
 v1.3.2
 new : long range mode, using --long command, by Stella Lau (@stellamplau)
 new : ability to generate and decode magicless frames (#591)
diff --git a/doc/zstd_manual.html b/doc/zstd_manual.html
index 872dbf6377d..da11ddb2a4b 100644
--- a/doc/zstd_manual.html
+++ b/doc/zstd_manual.html
@@ -1,10 +1,10 @@
 
 
 
-zstd 1.3.2 Manual
+zstd 1.3.3 Manual
 
 
-
    zstd 1.3.2 Manual
    
+
    zstd 1.3.3 Manual
    
 
    
 
    Contents
    
 
      
@@ -633,12 +633,12 @@ 
      Streaming decompression functions
      size_t ZSTD_initDSt
 
 
      Advanced Streaming compression functions
      ZSTD_CStream* ZSTD_createCStream_advanced(ZSTD_customMem customMem);
 ZSTD_CStream* ZSTD_initStaticCStream(void* workspace, size_t workspaceSize);    /**< same as ZSTD_initStaticCCtx() */
-size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize);   /**< pledgedSrcSize must be correct. If it is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN. Note that, for compatibility with older programs, a size of 0 is interepreted as "unknown". But it may change in some future version to mean "empty". */
+size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize);   /**< pledgedSrcSize must be correct. If it is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN. Note that, for compatibility with older programs, "0" also disables frame content size field. It may be enabled in the future. */
 size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t dictSize, int compressionLevel); /**< creates of an internal CDict (incompatible with static CCtx), except if dict == NULL or dictSize < 8, in which case no dict is used. Note: dict is loaded with ZSTD_dm_auto (treated as a full zstd dictionary if it begins with ZSTD_MAGIC_DICTIONARY, else as raw content) and ZSTD_dlm_byCopy.*/
 size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs, const void* dict, size_t dictSize,
-                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize : if srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
+                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize must be correct. If srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
 size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict);  /**< note : cdict will just be referenced, and must outlive compression session */
-size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, const ZSTD_CDict* cdict, ZSTD_frameParameters fParams, unsigned long long pledgedSrcSize);  /**< same as ZSTD_initCStream_usingCDict(), with control over frame parameters */
+size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, const ZSTD_CDict* cdict, ZSTD_frameParameters fParams, unsigned long long pledgedSrcSize);  /**< same as ZSTD_initCStream_usingCDict(), with control over frame parameters. pledgedSrcSize must be correct. If srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. */
 

      
 
      size_t ZSTD_resetCStream(ZSTD_CStream* zcs, unsigned long long pledgedSrcSize);
 
        start a new compression job, using same parameters from previous job.
@@ -863,7 +863,8 @@ 
      Buffer-less streaming decompression functions
      typedef
                               * More threads improve speed, but also increase memory usage.
                               * Can only receive a value > 1 if ZSTD_MULTITHREAD is enabled.
                               * Special: value 0 means "do not change nbThreads" */
-    ZSTD_p_jobSize,          /* Size of a compression job. Each compression job is completed in parallel.
+    ZSTD_p_jobSize,          /* Size of a compression job. This value is only enforced in streaming (non-blocking) mode.
+                              * Each compression job is completed in parallel, so indirectly controls the nb of active threads.
                               * 0 means default, which is dynamically determined based on compression parameters.
                               * Job size must be a minimum of overlapSize, or 1 KB, whichever is largest
                               * The minimum size is automatically and transparently enforced */
@@ -990,13 +991,19 @@ 
      Buffer-less streaming decompression functions
      typedef
   - Compression parameters cannot be changed once compression is started.
   - outpot->pos must be <= dstCapacity, input->pos must be <= srcSize
   - outpot->pos and input->pos will be updated. They are guaranteed to remain below their respective limit.
-  - @return provides the minimum amount of data still to flush from internal buffers
+  - In single-thread mode (default), function is blocking : it completed its job before returning to caller.
+  - In multi-thread mode, function is non-blocking : it just acquires a copy of input, and distribute job to internal worker threads,
+                                                     and then immediately returns, just indicating that there is some data remaining to be flushed.
+                                                     The function nonetheless guarantees forward progress : it will return only after it reads or write at least 1+ byte.
+  - Exception : in multi-threading mode, if the first call requests a ZSTD_e_end directive, it is blocking : it will complete compression before giving back control to caller.
+  - @return provides the minimum amount of data remaining to be flushed from internal buffers
             or an error code, which can be tested using ZSTD_isError().
-            if @return != 0, flush is not fully completed, there is some data left within internal buffers.
-  - after a ZSTD_e_end directive, if internal buffer is not fully flushed,
+            if @return != 0, flush is not fully completed, there is still some data left within internal buffers.
+            This is useful to determine if a ZSTD_e_flush or ZSTD_e_end directive is completed.
+  - after a ZSTD_e_end directive, if internal buffer is not fully flushed (@return != 0),
             only ZSTD_e_end or ZSTD_e_flush operations are allowed.
-            It is necessary to fully flush internal buffers
-            before starting a new compression job, or changing compression parameters.
+            Before starting a new compression job, or changing compression parameters,
+            it is required to fully flush internal buffers.
  
 

      
 
diff --git a/lib/zstd.h b/lib/zstd.h
index e4541c3f426..9ac0a73dceb 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -59,7 +59,7 @@ extern "C" {
 /*------   Version   ------*/
 #define ZSTD_VERSION_MAJOR    1
 #define ZSTD_VERSION_MINOR    3
-#define ZSTD_VERSION_RELEASE  2
+#define ZSTD_VERSION_RELEASE  3
 
 #define ZSTD_VERSION_NUMBER  (ZSTD_VERSION_MAJOR *100*100 + ZSTD_VERSION_MINOR *100 + ZSTD_VERSION_RELEASE)
 ZSTDLIB_API unsigned ZSTD_versionNumber(void);   /**< useful to check dll version */
diff --git a/programs/Makefile b/programs/Makefile
index 37788db3918..51888d3ef5c 100644
--- a/programs/Makefile
+++ b/programs/Makefile
@@ -226,8 +226,8 @@ clean:
 MD2ROFF = ronn
 MD2ROFF_FLAGS = --roff --warnings --manual="User Commands" --organization="zstd $(ZSTD_VERSION)"
 
-zstd.1: zstd.1.md
-	cat $^ | $(MD2ROFF) $(MD2ROFF_FLAGS) | sed -n '/^\.\\\".*/!p' > $@
+zstd.1: zstd.1.md ../lib/zstd.h
+	cat $< | $(MD2ROFF) $(MD2ROFF_FLAGS) | sed -n '/^\.\\\".*/!p' > $@
 
 .PHONY: man
 man: zstd.1
diff --git a/programs/zstd.1 b/programs/zstd.1
index eb53e2c3309..e0902e8582c 100644
--- a/programs/zstd.1
+++ b/programs/zstd.1
@@ -1,5 +1,5 @@
 .
-.TH "ZSTD" "1" "November 2017" "zstd 1.3.2" "User Commands"
+.TH "ZSTD" "1" "December 2017" "zstd 1.3.3" "User Commands"
 .
 .SH "NAME"
 \fBzstd\fR \- zstd, zstdmt, unzstd, zstdcat \- Compress or decompress \.zst files

From 3cbfac1cdb7635b799809575208cc0318055a1fc Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Thu, 14 Dec 2017 23:29:00 -0800
Subject: [PATCH 136/145] updated levels 15-20

taking advantage of `btopt` improved speed to tune parameters.
Levels 16-19 are stronger than previous release, making the graph more favorable.

In theory, I should also update small-size tables,
but I got lazy on that one ...
---
 lib/compress/zstd_compress.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 44f8c68a897..ebc9c1854d5 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -2955,12 +2955,12 @@ static const ZSTD_compressionParameters ZSTD_defaultCParameters[4][ZSTD_MAX_CLEV
     { 22, 20, 22,  5,  5, 16, ZSTD_lazy2   },  /* level 12 */
     { 22, 21, 22,  5,  5, 16, ZSTD_lazy2   },  /* level 13 */
     { 22, 21, 22,  6,  5, 16, ZSTD_lazy2   },  /* level 14 */
-    { 22, 21, 22,  5,  5, 16, ZSTD_btlazy2 },  /* level 15 */
-    { 23, 22, 22,  5,  5, 16, ZSTD_btlazy2 },  /* level 16 */
-    { 23, 22, 22,  4,  5, 24, ZSTD_btopt   },  /* level 17 */
-    { 23, 22, 22,  5,  4, 32, ZSTD_btopt   },  /* level 18 */
-    { 23, 23, 22,  6,  3, 48, ZSTD_btopt   },  /* level 19 */
-    { 25, 25, 23,  7,  3, 64, ZSTD_btultra },  /* level 20 */
+    { 22, 21, 22,  4,  5, 16, ZSTD_btlazy2 },  /* level 15 */
+    { 22, 21, 22,  4,  5, 48, ZSTD_btopt   },  /* level 16 */
+    { 23, 22, 22,  4,  4, 48, ZSTD_btopt   },  /* level 17 */
+    { 23, 22, 22,  5,  3, 64, ZSTD_btopt   },  /* level 18 */
+    { 23, 23, 22,  7,  3,128, ZSTD_btopt   },  /* level 19 */
+    { 25, 25, 23,  7,  3,128, ZSTD_btultra },  /* level 20 */
     { 26, 26, 24,  7,  3,256, ZSTD_btultra },  /* level 21 */
     { 27, 27, 25,  9,  3,512, ZSTD_btultra },  /* level 22 */
 },

From 78de28239fa35a0b0ae7f1ab3b4b552d1c1c5d5c Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 15 Dec 2017 13:26:39 -0800
Subject: [PATCH 137/145] minor readme formatting update

---
 README.md                  |  47 +++++++++++++++++++------------------
 doc/images/zstd_logo86.png | Bin 0 -> 5963 bytes
 2 files changed, 24 insertions(+), 23 deletions(-)
 create mode 100644 doc/images/zstd_logo86.png

diff --git a/README.md b/README.md
index 99f320c4588..50da24ba420 100644
--- a/README.md
+++ b/README.md
@@ -1,14 +1,15 @@
- __Zstandard__, or `zstd` as short version, is a fast lossless compression algorithm,
- targeting real-time compression scenarios at zlib-level and better compression ratios.
+
      Zstandard
      
 
-It is provided as an open-source BSD-licensed **C** library,
-and a command line utility producing and decoding `.zst` and `.gz` files.
-For other programming languages,
-you can consult a list of known ports on [Zstandard homepage](http://www.zstd.net/#other-languages).
+__Zstandard__, or `zstd` as short version, is a fast lossless compression algorithm,
+targeting real-time compression scenarios at zlib-level and better compression ratios.
+It's backed by a very fast entropy stage, provided by [Huff0 and FSE library](https://github.com/Cyan4973/FiniteStateEntropy).
 
-| dev branch status |
-|-------------------|
-| [![Build Status][travisDevBadge]][travisLink]   [![Build status][AppveyorDevBadge]][AppveyorLink]   [![Build status][CircleDevBadge]][CircleLink]
+The project is provided as an open-source BSD-licensed **C** library,
+and a command line utility producing and decoding `.zst`, `.gz`, `.xz` and `.lz4` files.
+Should your project require another programming language,
+a list of known ports and bindings is provided on [Zstandard homepage](http://www.zstd.net/#other-languages).
+
+Development branch status : [![Build Status][travisDevBadge]][travisLink]   [![Build status][AppveyorDevBadge]][AppveyorLink]   [![Build status][CircleDevBadge]][CircleLink]
 
 [travisDevBadge]: https://travis-ci.org/facebook/zstd.svg?branch=dev "Continuous Integration test suite"
 [travisLink]: https://travis-ci.org/facebook/zstd
@@ -17,8 +18,9 @@ you can consult a list of known ports on [Zstandard homepage](http://www.zstd.ne
 [CircleDevBadge]: https://circleci.com/gh/facebook/zstd/tree/dev.svg?style=shield "Short test suite"
 [CircleLink]: https://circleci.com/gh/facebook/zstd
 
+### Benchmarks
 
-As a reference, several fast compression algorithms were tested and compared
+For reference, several fast compression algorithms were tested and compared
 on a server running Linux Debian (`Linux version 4.8.0-1-amd64`),
 with a Core i7-6700K CPU @ 4.0GHz,
 using [lzbench], an open-source in-memory benchmark by @inikep
@@ -43,7 +45,9 @@ on the [Silesia compression corpus].
 [LZ4]: http://www.lz4.org/
 
 Zstd can also offer stronger compression ratios at the cost of compression speed.
-Speed vs Compression trade-off is configurable by small increments. Decompression speed is preserved and remains roughly the same at all settings, a property shared by most LZ compression algorithms, such as [zlib] or lzma.
+Speed vs Compression trade-off is configurable by small increments.
+Decompression speed is preserved and remains roughly the same at all settings,
+a property shared by most LZ compression algorithms, such as [zlib] or lzma.
 
 The following tests were run
 on a server running Linux Debian (`Linux version 4.8.0-1-amd64`)
@@ -56,8 +60,8 @@ Compression Speed vs Ratio | Decompression Speed
 ---------------------------|--------------------
 ![Compression Speed vs Ratio](doc/images/Cspeed4.png "Compression Speed vs Ratio") | ![Decompression Speed](doc/images/Dspeed4.png "Decompression Speed")
 
-Several algorithms can produce higher compression ratios, but at slower speeds, falling outside of the graph.
-For a larger picture including very slow modes, [click on this link](doc/images/DCspeed5.png).
+A few other algorithms can produce higher compression ratios at slower speeds, falling outside of the graph.
+For a larger picture including slow modes, [click on this link](doc/images/DCspeed5.png).
 
 
 ### The case for Small Data compression
@@ -99,19 +103,16 @@ Dictionary gains are mostly effective in the first few KB. Then, the compression
 `zstd -D dictionaryName --decompress FILE.zst`
 
 
-### Build
-
-Once you have the repository cloned, there are multiple ways provided to build Zstandard.
+### Build instructions
 
 #### Makefile
 
-If your system is compatible with a standard `make` (or `gmake`) binary generator,
-you can simply run it at the root directory.
-It will generate `zstd` within root directory.
+If your system is compatible with standard `make` (or `gmake`),
+invoking `make` in root directory will generate `zstd` cli in root directory.
 
 Other available options include:
-- `make install` : create and install zstd binary, library and man page
-- `make test` : create and run `zstd` and test tools on local platform
+- `make install` : create and install zstd cli, library and man pages
+- `make check` : create and run `zstd`, tests its behavior on local platform
 
 #### cmake
 
@@ -143,8 +144,8 @@ Zstandard is dual-licensed under [BSD](LICENSE) and [GPLv2](COPYING).
 
 ### Contributing
 
-The "dev" branch is the one where all contributions will be merged before reaching "master".
-If you plan to propose a patch, please commit into the "dev" branch or its own feature branch.
+The "dev" branch is the one where all contributions are merged before reaching "master".
+If you plan to propose a patch, please commit into the "dev" branch, or its own feature branch.
 Direct commit to "master" are not permitted.
 For more information, please read [CONTRIBUTING](CONTRIBUTING.md).
 
diff --git a/doc/images/zstd_logo86.png b/doc/images/zstd_logo86.png
new file mode 100644
index 0000000000000000000000000000000000000000..216f228061c460c4e4725785486ebcff38804fbc
GIT binary patch
literal 5963
zcmZ`+1yodByB@kjx*I`3X6O!K=*4(Tp&KuWq(1TKEx
zfB*lx>)yN9I{WPNzR&aKUVEJ#rKO=rfJ=i5000P-mB2a=Z{3GS6C3kkPA|D$0RV8z
z?d9aOl;z}@v|ORK_Kpw$Kq)FU4NFhgn5O^G7ucpn66r1r4RJ?R`dolL*{Pru6hgHt@JE#x4K)nPf`>ssvEvw@dhh^5Q+
z4DwlpB+WxP@fCKn))XP6}7+K12DYAxd`AP7T12J3N9?+@E(30b*N01f||`C9i2
zfO>qxFbDkZ_VS)4y0u4*2cX3{h|{X(EH=n7?zkD$Qe)EQ_7M$FQ0b%D|WE#YJ
zEb5bK${}i**;2X*4)ZDC#t4%6;nMRJkf=B=#-EiHyQ9U)X^QKE$~<_g?YVIxDv@%k
zm7A4|@O!tbC8Tl1k+a_Z>hXYpXtoWt<{{)MHZnv_o5pVA-iPj9d~%A7-(jmz`A5uc
z_qaBpWL@+0Mlc`6V{YvCmMGBIcVwsRcS-!iBWKZBmJI#kE^paCCNc02KN^$b8n99v
zuJ-oHP&3b{XHqK38%Tp$m1th!bKv+b@MjMtMCSf*d_EmQJ*q}Pl%2E2UJ=7?S(}^O
zgQ%4(u^YYU+Ze7&?M*n!+hi5oBC~YKD_6$P-ncw
z==fGw1f`%(tf2N3rGN?BsEAS3f@u7^E&Ha8y3j7%?28V!(xx4+ye%2
z94Oh`NCx#p2B!@5Sw2>5u;^lrqzmRj2Nrq|F)@l&r%nfkUm{_T1;zl{uFNA&6w?ku
zEu4#x+k8SglztgJ5rWZB)fJj846hDWCXCP(aw-hQ4iWU{-jTQy%=${Sd_luZ4v`p5
zvQk>aWSG6O$69=BVQg9qUl^VS59X`9V$>tL#mxyBl{3tj+CrHoB9Ma?070m_7VJi(
zLSa8Ux#!Uc!&q0?UHGz5es+Z)J#Iv;3MT3vIr6!p@xw~%`n-?Az1=blj^jw!;y|kqd(rqD};Df$^D7BDtcnqOc;JdT4u|uZk!q*??V{xpK6O0P8SK3tLfJ3$Lox0SLsL_!QmZ`?GsQX@3fgHi`Z$if)c>Q7KfZQAr-8PX(5%=`+sg%~X~@E7zO`y)Jo;
zH)Bw?uia86R9IDf@M%;ZXdqp3r-e2Pn!%nj`cPwDWNNLZt2(8rTim328nPKlAvg4b
zOWvVn*0|Ersky$QUdFfOLUcb}4N`)|A7|XTXA{5kYU^rO$^BycY%6w`W?Ny0VoPXS
zYJzpU=Iq^B{jSGW+>XO8*6z=3;Vo$aCYoyeD@<{MW_&3c8fpbvXOJVWpJlOCkI7dIX0C?DGc3ht@2rZw$DAqO=KihTWE}7L~aB(Zi5TK
zH%$zUm+K1_uNFVS?cqMw4z^jZTKX>#^xt{rokL$ld>8h`@~!bNx~&dH$hFGByCu^|
zxKi{7oA)9;&fLv*y%&$`lG}9Crg%%Gb4GG>bBILSGoS*}u)E-EG@fQ2x^#aY9m5ib
z*4?ox9btOmc+wJ;
z-$o(@Z+Pw4Mc>}QO=AH?}F?b1gwk28Z)WClFY
zIN-_^hl)c*CfIAZVVqytKXPx&*Q%b>uUjH$+q0!vj!5)yk;PY~d+OI2H1FPkJXoHI
zRX)i{WXKCpQ6eqD6;F3~HfD_E;3FlEDyG(;7Uxsw4PIST<8)KDbx0$5?b^)V_l}Y6
zcA;uPe<9BRTq%-Ul)*Z@{nEbKb#1V~YI<*7j@tZBE4RP4&Yi+D!>_~9q_1wrwb%^Vc$ynJeY%m1ywbP<
zdke4XU~_$ky{)Si>}l*9?B=RX9}EL)e|=mnjZ#<6dL#4=imbzEkZ;^LFleasfJ8%@
zAZXu08U&$s&Oqc{Kz}X0;h@vd(L%oiV@j^fAliQ7ddmeom-7e=Jv=ho>&v_;x
z4&%98GHNOFU-vt?4vvS=C-C%`HGXfZuKQ`V(mRyy2Yj)0vL)uVuyL}@rD3S?0`kJC
z$u_XE>08y!x<|_{hwq+mCRAkU)xO7`({9$X*G0*mjne
z;VSn`>hLZG%bo}+F&rRr$9d0t0XrrQ=h)B`%$@!1eBzrt{qjx8a!dP-@l~B7+ETD#
zpv~Q}NA;2DxzBath1MsMp|jOX(1e=Tht@cgthzYt;%>px;33)5c^cLzxpBXun%
zIjAdyNsvc?2gD+U%f!Sa;c8^7yaQ&lI4}VyNehQ=;h_bozhwWy
zO91~=>K~Q+yDfi7ANnGND*^m>kEL)2us-8F++$Rf!O!))QTB5TBkA?2`t6c4kTJ@R
z775XavN8*G7OEZ7lBm9XD>?btp6E9AzG-#g
zsJpGAoD(>~BeSiMPZPnUqIKeN!P;|St_|V10)*r9Cf1MpGd{p0?8~auYHHR|N6X}Wj
zB}g{=KXLlbzL|&9>eb+Uj$E!Exw%*l#6#ni1>}J&g1k;Tf{{;N1C||MKh0Q<*?qa(
zy5Q?SL;(#K`OGc51UUQD8S3wn)m!%=G|IzQlkRG|FAmAE+GjiU3bq$6_tu-T4EFMI
zD(2(rwp#n-%P)(-)q#VOyUdP-)oJSPw>~$@b7{{%1YMVcy=R*}kP(v+`b9~rqoxY6
zN27EC1J~2pAx)}Nz6RMM&~8kqYVUW$*qLnYqE;B8{4#Y0*9Kw<;aPbJ!)W4-$B6j4
zIz}ftnY9_IniJHdH0x8})7-YYM1{3OjY_H&)2tibi9TT3o3tS+1|x2~{Vfu8+d5Hn
zyOKYVix+%d>(|%PlT<0I_$1l{^EI#N=g(4>t-
z?4@}I&QvAOYp6PNgk0B)oX}$3@N0LRfh~TP&c_*nUbwQGnwH3;R*YA21l`!@LY!dq!mOy6*LTsX*3dDa)XOf
z7At|C2Ud3DfVM`uBYY`|=ywt{m72k%U1F=A+FyCKgBX~rANHjTISo%#Z%ZulW$Lso
z=4mp?1q?qi=u3J~3%KK@jvW4H6btE@MkVwnu*yroB|vb{GMYI1@LigH)m{rBwmZ#N
zR7enxFsc=qy1_XH!KzeWL^8!kcnsc(Ne6}zU{qzx$o*L5eEZ!%CdyXLt+q}M+29vl
zHjl|@^3l`Ot)U7%KnMEU%@u(l!JQ6o$~}KYNEdF^rMFz+q^BlT6Y%Px>{v=x5J9>h(Nb)Wn8
zQ}oTXa-7Vp)orisf)&OTY>lKyM%H*q1JYPs4DIfYj|dfFG{2HrHQ94+25H1lA~OhT
zD5IISz>d*Wglzc1LjT0Vc?$NvOhiOT{$scex6+F_d4
zWT>Z;fp^DJ=~PYmoW$4o<9@Zh$z$SDmG^jah1T}T`kaXo4skbdy^b_Xt@rDAYkeNP
zk<6g6UpLUhpZHBJzoJPs3Om#qzqga2Pa^`MQ55;82lj%yQ1b+FbIzG_8Ju#5VE4_a
zw6icXR`ZYjBjv$kYaLN&i+4+RJa3~H^FOl-j$zYbdw`7Ntlu%Oy`d)Sd5ZqnW4&qs
zlXc$ih=%D`&@o=J~M3(#lLsdL?yMfP=){o
zSDkhIMpykf%#Ldr6>x*Z6qV^tsiB
z0J03ZpO3oEDxw#ACoPY~yc4-iMor_VDQBjLL7f=OWmEO7KOeW|bCX1%R&TARlN}|m
zUH;v#@F6C5i^P>Kx0J!c=LiO}(p3(g6dv$zW)Wa?qe**L8~kP`Z~KjC#de0ei#jvN
zxWinVA9%G8%109EL&AK}chcp9pdnUt#W&HI(xm669bl@4Gs;`NLTs+T65n&DA~~hi
zM?HOeW=!1!w3k(k_aR}3Y(SwL9g)=@pXnT%d_E|t`k}ecgoF~+4N>n&DYA%Ee;)&d
z`LD6YbBnv98>Pz-k~ffQERvy;3R~H}CGct?rTpnq$tV@MXjS&I|8wKM&SJ>TYy)`{_?uQVRjdOPKx!uoyqGQD;SP#+K^_u1T+k!&@p
zG_0fICRL+BnY>E7S&O;tDbs?^G=x&wcyo`#<2wUj?NzC)hl@Y6pD~5r$S{N6<7*6xgmR`+uM#UVzz;b+U(NmPWW%F;>M%J97}sa&+w92erc>?NgK
z&h-*%6%x{}Cblq%icv=!GZG?a
zLH{%Em+#0EREELPqA!TunlVYuiE`$Ww~3u9z$jlcyHMPmKImfD+GQ4S0GIcdd|G7b
zdPBaNqIvsvIPZXiT6?c?GI~OkDbpR2+p@y~f1>+NQb+9-a|Duhu6JI1Us^bJ>rVA-
z(<38gdpLuuh`mlSvyd8wPyI7`Qdg3rNjxNli`SV=>0f;khUP05>4CO`sHTyO8oOU+
zs+Nni8h7tro#FBae>JpoXlIQZDT>%y$a~54ahRsi7Be~agd1fsE7DNi%%H+^wDw@w
zVsc?S`ZI7eU%VFI?A%w08VTI|il&w8dwXvF8F24?YgMTR4mj0Qv^hMqF@`#Wa5fod
zO$`VQVi%*s3Vf7ytxZ{g8<@!`^Jt7l6*Oxp1;$(3#iBd3gSnvh+mNz%5
zn`Kr?rk#EeHwkvF|~N_@%9n`87eL&)24^ms-5bEG5153Ib;v6Q9+SA7uO
zsSW|W1-6_))}hvyt+5RqCoF#D@lW!dJX2=fK?G%VK1F~Tht{jyQk%fkfVc{h`gXg#
zL#`u2oh?N~kXN0ezRCP$x(~9_Lg85~F8-Y==AHQ4bFtF+oN~+DRme?PG(qh>DLRXp
k5qmjZhCV9Cq1heQhr^_O>+l-dKeq{Gc@1!ttVPIw0Z+glQ2+n{

literal 0
HcmV?d00001


From 64d1701c6476f79961e8c20a96986688d7e0a6e7 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 15 Dec 2017 13:28:34 -0800
Subject: [PATCH 138/145] remove last paragraph

---
 README.md | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/README.md b/README.md
index 50da24ba420..3755315ad68 100644
--- a/README.md
+++ b/README.md
@@ -148,7 +148,3 @@ The "dev" branch is the one where all contributions are merged before reaching "
 If you plan to propose a patch, please commit into the "dev" branch, or its own feature branch.
 Direct commit to "master" are not permitted.
 For more information, please read [CONTRIBUTING](CONTRIBUTING.md).
-
-### Miscellaneous
-
-Zstd entropy stage is provided by [Huff0 and FSE, from Finite State Entropy library](https://github.com/Cyan4973/FiniteStateEntropy).

From 5c2f2ebfdb8fc8e074269ad4f85ac79267cc36fe Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Sat, 16 Dec 2017 12:48:13 -0800
Subject: [PATCH 139/145] zstdmt via compress_generic: reduce opportunity to
 free/create mtctx

`zstreamtest --newapi` (and `--opaqueapi`) create and destroy way too many threads
resulting in failure of tsan tests,
and potentially connected to the qemu flaky tests.

This is because, at each test, the nb of threads can be changed (random).

The `--no-big-tests` directive reduce this choice to 1/2 threads,
in order to limit memory usage, especially for qemu and 32-bits builds.
Unfortunately, swapping between 1 and 2 threads is enough to constantly create/destroy new mtctx.

This patch takes advantage of the following property :
via compress_generic, no internal mtctx is needed for nbThreads < 2.
As a consequence, when nbThreads == 2, the currently active mtctx is necessarily good.

This dramatically reduces the nb of thread creations when invoking `zstreamtest --newapi --no-big-tests`
(only when parent cctx itself is created, which is randomized to 1/256 tests).

Expected outcome :
- at a minimum : tsan tests shall now work continuously without exploding the thread counter
- at best : flaky qemu tests on `zstreamtest --newapi --no-big-tests` may stop being flaky, due to less stress from constant thread creation/destruction

Real world impact :
minimal, I don't expect users to constantly change `nbThreads` between each invocation.
If `nbThreads` remains stable, existing implementation re-uses existing mtctx.

Also : `zstreamtest --newapi` but without `--no-big-tests` doesn't benefit as much,
since this test can select a random `nbThreads` value between 1 and 4.
The current patch only reduces opportunity to free/create mtctx (for example : 2->1->2 doesn't need a new mtctx)
but doesn't completely eliminate it, since `nbThreads` can still change between 2/3/4.
A more complete solution could be to only use 2 out of 4 allocated threads, thus keeping the pool at a constant size.
This would require a larger change to `POOL_*` api though.
---
 lib/compress/zstd_compress.c   | 4 +++-
 lib/compress/zstdmt_compress.c | 9 +++++++++
 lib/compress/zstdmt_compress.h | 7 ++++++-
 3 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index ebc9c1854d5..0a2d589d714 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -2851,7 +2851,9 @@ size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
         if ((cctx->pledgedSrcSizePlusOne-1) <= ZSTDMT_JOBSIZE_MIN)
             params.nbThreads = 1; /* do not invoke multi-threading when src size is too small */
         if (params.nbThreads > 1) {
-            if (cctx->mtctx == NULL || cctx->appliedParams.nbThreads != params.nbThreads) {
+            if (cctx->mtctx == NULL || (params.nbThreads != ZSTDMT_getNbThreads(cctx->mtctx))) {
+                DEBUGLOG(4, "ZSTD_compress_generic: creating new mtctx for nbThreads=%u (previous: %u)",
+                            params.nbThreads, ZSTDMT_getNbThreads(cctx->mtctx));
                 ZSTDMT_freeCCtx(cctx->mtctx);
                 cctx->mtctx = ZSTDMT_createCCtx_advanced(params.nbThreads, cctx->customMem);
                 if (cctx->mtctx == NULL) return ERROR(memory_allocation);
diff --git a/lib/compress/zstdmt_compress.c b/lib/compress/zstdmt_compress.c
index a5e996d3ebf..659992f3461 100644
--- a/lib/compress/zstdmt_compress.c
+++ b/lib/compress/zstdmt_compress.c
@@ -459,6 +459,15 @@ size_t ZSTDMT_CCtxParam_setNbThreads(ZSTD_CCtx_params* params, unsigned nbThread
     return nbThreads;
 }
 
+/* ZSTDMT_getNbThreads():
+ * @return nb threads currently active in mtctx.
+ * mtctx must be valid */
+size_t ZSTDMT_getNbThreads(const ZSTDMT_CCtx* mtctx)
+{
+    assert(mtctx != NULL);
+    return mtctx->params.nbThreads;
+}
+
 ZSTDMT_CCtx* ZSTDMT_createCCtx_advanced(unsigned nbThreads, ZSTD_customMem cMem)
 {
     ZSTDMT_CCtx* mtctx;
diff --git a/lib/compress/zstdmt_compress.h b/lib/compress/zstdmt_compress.h
index 4209cf3c5ac..d12f0adb8d4 100644
--- a/lib/compress/zstdmt_compress.h
+++ b/lib/compress/zstdmt_compress.h
@@ -114,9 +114,14 @@ size_t ZSTDMT_CCtxParam_setMTCtxParameter(ZSTD_CCtx_params* params, ZSTDMT_param
 
 /* ZSTDMT_CCtxParam_setNbThreads()
  * Set nbThreads, and clamp it correctly,
- * but also reset jobSize and overlapLog */
+ * also reset jobSize and overlapLog */
 size_t ZSTDMT_CCtxParam_setNbThreads(ZSTD_CCtx_params* params, unsigned nbThreads);
 
+/* ZSTDMT_getNbThreads():
+ * @return nb threads currently active in mtctx.
+ * mtctx must be valid */
+size_t ZSTDMT_getNbThreads(const ZSTDMT_CCtx* mtctx);
+
 /*! ZSTDMT_initCStream_internal() :
  *  Private use only. Init streaming operation.
  *  expects params to be valid.

From f299fa39ac07b9de897a5f0e930459528fec0bb5 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 19 Dec 2017 09:43:03 +0100
Subject: [PATCH 140/145] fix a subtle issue in continue mode

The deep fuzzer tests caught a subtle bug that was probably there for a long time.
The impact of the bug is not a crash, or any other clear error signal,
rather, it reduces performance, by cutting data into smaller blocks.
Eventually, the following test would fail because it produces too many 1-byte blocks,
requiring more space than buffer can provide :
`./zstreamtest_asan --mt -s3514 -t1678312 -i1678314`

The root scenario is as follows :
- Create context, initialize it using explicit parameters or a `cdict` to pin them down, set `pledgedSrcSize=1`
- The compression parameters will not be adapted, but `windowSize` and `blockSize` will be automatically set to `1`.
  `windowSize` and `blockSize` are dynamic values, set within `ZSTD_resetCCtx_internal()`.
  The automatic adaptation makes it possible to generate smaller contexts for smaller input sizes.
- Complete compression
- New compression with same context, using same parameters, but `pledgedSrcSize=ZSTD_CONTENTSIZE_UNKNOWN`
  trigger "continue mode"
- Continue mode doesn't modify blockSize, because it used to depend on `windowLog` only,
  but in fact, it also depends on `pledgedSrcSize`.
- The "old" blocksize (1) is still there,
  next compression will use this value to cut input into blocks,
  resulting in more blocks and worse performance than necessary performance.

Given the scenario, and its possible variants, I'm surprised it did not show up before.
But I suspect it did show up, it's just that it never triggered an error, because "worse performance" is not a trigger.
The above test is a special corner case, where performance is so impacted that it reaches an error case.

The fix works, but I'm not completely pleased.
I think the current code relies too much on implied relations between variables.
This will likely break again in the future when some related part of the code change.
Unfortunately, no time to make larger changes if we want to keep the release target for zstd v1.3.3.
So a longer term fix will have to be considered after the release.

To do : create a reliable test case which triggers this scenario for CI tests.
---
 lib/common/zstd_internal.h     | 19 +++++++-------
 lib/compress/zstd_compress.c   | 45 +++++++++++++++++++++-------------
 lib/compress/zstdmt_compress.c | 20 +++++++--------
 3 files changed, 47 insertions(+), 37 deletions(-)

diff --git a/lib/common/zstd_internal.h b/lib/common/zstd_internal.h
index 2f812482550..5d2900eb768 100644
--- a/lib/common/zstd_internal.h
+++ b/lib/common/zstd_internal.h
@@ -57,20 +57,19 @@ extern "C" {
 extern int g_debuglog_enable;
 /* recommended values for ZSTD_DEBUG display levels :
  * 1 : no display, enables assert() only
- * 2 : reserved for currently active debugging path
- * 3 : events once per object lifetime (CCtx, CDict)
+ * 2 : reserved for currently active debug path
+ * 3 : events once per object lifetime (CCtx, CDict, etc.)
  * 4 : events once per frame
  * 5 : events once per block
  * 6 : events once per sequence (*very* verbose) */
-#  define RAWLOG(l, ...) {                                 \
-                if ((g_debuglog_enable) & (l<=ZSTD_DEBUG)) { \
-                    fprintf(stderr, __VA_ARGS__);            \
+#  define RAWLOG(l, ...) {                                      \
+                if ((g_debuglog_enable) & (l<=ZSTD_DEBUG)) {    \
+                    fprintf(stderr, __VA_ARGS__);               \
             }   }
-#  define DEBUGLOG(l, ...) {                                 \
-                if ((g_debuglog_enable) & (l<=ZSTD_DEBUG)) { \
-                    fprintf(stderr, __FILE__ ": ");          \
-                    fprintf(stderr, __VA_ARGS__);            \
-                    fprintf(stderr, " \n");                  \
+#  define DEBUGLOG(l, ...) {                                    \
+                if ((g_debuglog_enable) & (l<=ZSTD_DEBUG)) {    \
+                    fprintf(stderr, __FILE__ ": " __VA_ARGS__); \
+                    fprintf(stderr, " \n");                     \
             }   }
 #else
 #  define RAWLOG(l, ...)      {}    /* disabled */
diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index ebc9c1854d5..5f952b8c053 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -728,10 +728,7 @@ size_t ZSTD_estimateCStreamSize(int compressionLevel) {
 static U32 ZSTD_equivalentCParams(ZSTD_compressionParameters cParams1,
                                   ZSTD_compressionParameters cParams2)
 {
-    U32 bslog1 = MIN(cParams1.windowLog, ZSTD_BLOCKSIZELOG_MAX);
-    U32 bslog2 = MIN(cParams2.windowLog, ZSTD_BLOCKSIZELOG_MAX);
-    return (bslog1 == bslog2)   /* same block size */
-         & (cParams1.hashLog  == cParams2.hashLog)
+    return (cParams1.hashLog  == cParams2.hashLog)
          & (cParams1.chainLog == cParams2.chainLog)
          & (cParams1.strategy == cParams2.strategy)   /* opt parser space */
          & ((cParams1.searchLength==3) == (cParams2.searchLength==3));  /* hashlog3 space */
@@ -755,24 +752,29 @@ typedef enum { ZSTDb_not_buffered, ZSTDb_buffered } ZSTD_buffered_policy_e;
 /* ZSTD_sufficientBuff() :
  * check internal buffers exist for streaming if buffPol == ZSTDb_buffered .
  * Note : they are assumed to be correctly sized if ZSTD_equivalentCParams()==1 */
-static U32 ZSTD_sufficientBuff(size_t bufferSize, ZSTD_buffered_policy_e buffPol, ZSTD_compressionParameters cParams2, U64 pledgedSrcSize)
+static U32 ZSTD_sufficientBuff(size_t bufferSize1, size_t blockSize1,
+                            ZSTD_buffered_policy_e buffPol2,
+                            ZSTD_compressionParameters cParams2,
+                            U64 pledgedSrcSize)
 {
     size_t const windowSize = MAX(1, (size_t)MIN(((U64)1 << cParams2.windowLog), pledgedSrcSize));
-    size_t const blockSize = MIN(ZSTD_BLOCKSIZE_MAX, windowSize);
-    size_t const neededBufferSize = (buffPol==ZSTDb_buffered) ? windowSize + blockSize : 0;
-    return (neededBufferSize <= bufferSize);
+    size_t const blockSize2 = MIN(ZSTD_BLOCKSIZE_MAX, windowSize);
+    size_t const neededBufferSize = (buffPol2==ZSTDb_buffered) ? windowSize + blockSize2 : 0;
+    return (blockSize2 <= blockSize1) /* seqStore space depends on blockSize */
+         & (neededBufferSize <= bufferSize1);
 }
 
 /** Equivalence for resetCCtx purposes */
 static U32 ZSTD_equivalentParams(ZSTD_CCtx_params params1,
                                  ZSTD_CCtx_params params2,
-                                 size_t buffSize1,
+                                 size_t buffSize1, size_t blockSize1,
                                  ZSTD_buffered_policy_e buffPol2,
                                  U64 pledgedSrcSize)
 {
+    DEBUGLOG(4, "ZSTD_equivalentParams: pledgedSrcSize=%u", (U32)pledgedSrcSize);
     return ZSTD_equivalentCParams(params1.cParams, params2.cParams) &&
            ZSTD_equivalentLdmParams(params1.ldmParams, params2.ldmParams) &&
-           ZSTD_sufficientBuff(buffSize1, buffPol2, params2.cParams, pledgedSrcSize);
+           ZSTD_sufficientBuff(buffSize1, blockSize1, buffPol2, params2.cParams, pledgedSrcSize);
 }
 
 /*! ZSTD_continueCCtx() :
@@ -780,7 +782,11 @@ static U32 ZSTD_equivalentParams(ZSTD_CCtx_params params1,
 static size_t ZSTD_continueCCtx(ZSTD_CCtx* cctx, ZSTD_CCtx_params params, U64 pledgedSrcSize)
 {
     U32 const end = (U32)(cctx->nextSrc - cctx->base);
-    DEBUGLOG(4, "continue mode");
+    size_t const windowSize = MAX(1, (size_t)MIN(((U64)1 << params.cParams.windowLog), pledgedSrcSize));
+    size_t const blockSize = MIN(ZSTD_BLOCKSIZE_MAX, windowSize);
+    DEBUGLOG(4, "ZSTD_continueCCtx");
+
+    cctx->blockSize = blockSize;   /* previous block size could be different even for same windowLog, due to pledgedSrcSize */
     cctx->appliedParams = params;
     cctx->pledgedSrcSizePlusOne = pledgedSrcSize+1;
     cctx->consumedSrcSize = 0;
@@ -813,8 +819,11 @@ static size_t ZSTD_resetCCtx_internal(ZSTD_CCtx* zc,
     assert(!ZSTD_isError(ZSTD_checkCParams(params.cParams)));
 
     if (crp == ZSTDcrp_continue) {
-        if (ZSTD_equivalentParams(params, zc->appliedParams, zc->inBuffSize, zbuff, pledgedSrcSize)) {
-            DEBUGLOG(4, "ZSTD_equivalentParams()==1 -> continue mode");
+        if (ZSTD_equivalentParams(params, zc->appliedParams,
+                                zc->inBuffSize, zc->blockSize,
+                                zbuff, pledgedSrcSize)) {
+            DEBUGLOG(4, "ZSTD_equivalentParams()==1 -> continue mode (wLog=%u, blockSize=%u)",
+                        zc->appliedParams.cParams.windowLog, (U32)zc->blockSize);
             assert(!(params.ldmParams.enableLdm &&
                      params.ldmParams.hashEveryLog == ZSTD_LDM_HASHEVERYLOG_NOTSET));
             zc->entropy->hufCTable_repeatMode = HUF_repeat_none;
@@ -865,8 +874,8 @@ static size_t ZSTD_resetCCtx_internal(ZSTD_CCtx* zc,
                                        tableSpace + tokenSpace + bufferSpace;
             DEBUGLOG(4, "Need %uKB workspace, including %uKB for tables, and %uKB for buffers",
                         (U32)(neededSpace>>10), (U32)(tableSpace>>10), (U32)(bufferSpace>>10));
-            DEBUGLOG(4, "chainSize: %u - hSize: %u - h3Size: %u - windowSize: %u",
-                        (U32)chainSize, (U32)hSize, (U32)h3Size, (U32)windowSize);
+            DEBUGLOG(4, "chainSize: %u - hSize: %u - h3Size: %u - windowSize: %u - blockSize: %u",
+                        (U32)chainSize, (U32)hSize, (U32)h3Size, (U32)windowSize, (U32)blockSize);
 
             if (zc->workSpaceSize < neededSpace) {  /* too small : resize */
                 DEBUGLOG(4, "Need to update workSpaceSize from %uK to %uK",
@@ -1666,6 +1675,7 @@ static size_t ZSTD_compress_frameChunk (ZSTD_CCtx* cctx,
     U32 const maxDist = (U32)1 << cctx->appliedParams.cParams.windowLog;
     assert(cctx->appliedParams.cParams.windowLog <= 31);
 
+    DEBUGLOG(5, "ZSTD_compress_frameChunk (blockSize=%u)", (U32)blockSize);
     if (cctx->appliedParams.fParams.checksumFlag && srcSize)
         XXH64_update(&cctx->xxhState, src, srcSize);
 
@@ -1838,6 +1848,7 @@ static size_t ZSTD_compressContinue_internal (ZSTD_CCtx* cctx,
         cctx->lowLimit = lowLimitMax;
     }
 
+    DEBUGLOG(5, "ZSTD_compressContinue_internal (blockSize=%u)", (U32)cctx->blockSize);
     {   size_t const cSize = frame ?
                              ZSTD_compress_frameChunk (cctx, dst, dstCapacity, src, srcSize, lastFrameChunk) :
                              ZSTD_compressBlock_internal (cctx, dst, dstCapacity, src, srcSize);
@@ -2062,7 +2073,7 @@ size_t ZSTD_compressBegin_internal(ZSTD_CCtx* cctx,
                              ZSTD_CCtx_params params, U64 pledgedSrcSize,
                              ZSTD_buffered_policy_e zbuff)
 {
-    DEBUGLOG(4, "ZSTD_compressBegin_internal");
+    DEBUGLOG(4, "ZSTD_compressBegin_internal: wlog=%u", params.cParams.windowLog);
     /* params are supposed to be fully validated at this point */
     assert(!ZSTD_isError(ZSTD_checkCParams(params.cParams)));
     assert(!((dict) && (cdict)));  /* either dict or cdict, not both */
@@ -2086,7 +2097,7 @@ size_t ZSTD_compressBegin_advanced_internal(ZSTD_CCtx* cctx,
                                     ZSTD_CCtx_params params,
                                     unsigned long long pledgedSrcSize)
 {
-    DEBUGLOG(4, "ZSTD_compressBegin_advanced_internal");
+    DEBUGLOG(4, "ZSTD_compressBegin_advanced_internal: wlog=%u", params.cParams.windowLog);
     /* compression parameters verification and optimization */
     CHECK_F( ZSTD_checkCParams(params.cParams) );
     return ZSTD_compressBegin_internal(cctx,
diff --git a/lib/compress/zstdmt_compress.c b/lib/compress/zstdmt_compress.c
index a5e996d3ebf..500806188cd 100644
--- a/lib/compress/zstdmt_compress.c
+++ b/lib/compress/zstdmt_compress.c
@@ -355,7 +355,7 @@ void ZSTDMT_compressChunk(void* jobDescription)
 
     if (job->cdict) {
         size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, NULL, 0, ZSTD_dm_auto, job->cdict, job->params, job->fullFrameSize);
-        DEBUGLOG(4, "ZSTDMT_compressChunk: init using CDict");
+        DEBUGLOG(4, "ZSTDMT_compressChunk: init using CDict (windowLog=%u)", job->params.cParams.windowLog);
         assert(job->firstChunk);  /* only allowed for first job */
         if (ZSTD_isError(initError)) { job->cSize = initError; goto _endJob; }
     } else {  /* srcStart points at reloaded section */
@@ -367,9 +367,11 @@ void ZSTDMT_compressChunk(void* jobDescription)
             job->cSize = forceWindowError;
             goto _endJob;
         }
-        /* load dictionary in "content-only" mode (no header analysis) */
-        {   size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, job->srcStart, job->prefixSize, ZSTD_dm_rawContent, NULL, jobParams, pledgedSrcSize);
-            DEBUGLOG(5, "ZSTD_compressBegin_advanced_internal called with windowLog = %u ", jobParams.cParams.windowLog);
+        DEBUGLOG(5, "ZSTDMT_compressChunk: invoking ZSTD_compressBegin_advanced_internal with windowLog = %u ", jobParams.cParams.windowLog);
+        {   size_t const initError = ZSTD_compressBegin_advanced_internal(cctx,
+                                        job->srcStart, job->prefixSize, ZSTD_dm_rawContent, /* load dictionary in "content-only" mode (no header analysis) */
+                                        NULL,
+                                        jobParams, pledgedSrcSize);
             if (ZSTD_isError(initError)) {
                 DEBUGLOG(5, "ZSTD_compressBegin_advanced_internal error : %s ", ZSTD_getErrorName(initError));
                 job->cSize = initError;
@@ -637,8 +639,7 @@ static size_t ZSTDMT_compress_advanced_internal(
     assert(jobParams.nbThreads == 0);
     assert(mtctx->cctxPool->totalCCtx == params.nbThreads);
 
-    DEBUGLOG(4, "ZSTDMT_compress_advanced_internal");
-    DEBUGLOG(4, "nbChunks  : %2u (raw chunkSize : %u bytes; fixed chunkSize: %u)   ",
+    DEBUGLOG(4, "ZSTDMT_compress_advanced_internal: nbChunks=%2u (rawSize=%u bytes; fixedSize=%u)  ",
                 nbChunks, (U32)proposedChunkSize, (U32)avgChunkSize);
     if (nbChunks==1) {   /* fallback to single-thread mode */
         ZSTD_CCtx* const cctx = mtctx->cctxPool->cctx[0];
@@ -688,7 +689,7 @@ static size_t ZSTDMT_compress_advanced_internal(
                 XXH64_update(&xxh64, srcStart + frameStartPos, chunkSize);
             }
 
-            DEBUGLOG(5, "posting job %u   (%u bytes)", u, (U32)chunkSize);
+            DEBUGLOG(5, "ZSTDMT_compress_advanced_internal: posting job %u  (%u bytes)", u, (U32)chunkSize);
             DEBUG_PRINTHEX(6, mtctx->jobs[u].srcStart, 12);
             POOL_add(mtctx->factory, ZSTDMT_compressChunk, &mtctx->jobs[u]);
 
@@ -893,8 +894,7 @@ static size_t ZSTDMT_createCompressionJob(ZSTDMT_CCtx* zcs, size_t srcSize, unsi
 {
     unsigned const jobID = zcs->nextJobID & zcs->jobIDMask;
 
-    DEBUGLOG(5, "ZSTDMT_createCompressionJob");
-    DEBUGLOG(5, "preparing job %u to compress %u bytes with %u preload ",
+    DEBUGLOG(5, "ZSTDMT_createCompressionJob: preparing job %u to compress %u bytes with %u preload ",
                 zcs->nextJobID, (U32)srcSize, (U32)zcs->dictSize);
     zcs->jobs[jobID].src = zcs->inBuff.buffer;
     zcs->jobs[jobID].srcStart = zcs->inBuff.buffer.start;
@@ -945,7 +945,7 @@ static size_t ZSTDMT_createCompressionJob(ZSTDMT_CCtx* zcs, size_t srcSize, unsi
             zcs->params.fParams.checksumFlag = 0;
     }   }
 
-    DEBUGLOG(4, "posting job %u : %u bytes  (end:%u) (note : doneJob = %u=>%u)",
+    DEBUGLOG(5, "ZSTDMT_createCompressionJob: posting job %u : %u bytes  (end:%u) (note : doneJob = %u=>%u)",
                 zcs->nextJobID,
                 (U32)zcs->jobs[jobID].srcSize,
                 zcs->jobs[jobID].lastChunk,

From d88c6716633619b629129f6e7d536c1b57746263 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 19 Dec 2017 10:16:09 +0100
Subject: [PATCH 141/145] added test case for "wrong blockSize in continue
 mode"

---
 tests/fuzzer.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/tests/fuzzer.c b/tests/fuzzer.c
index f799b5f9ee8..d64b46ba3a1 100644
--- a/tests/fuzzer.c
+++ b/tests/fuzzer.c
@@ -25,6 +25,7 @@
 #include        /* free */
 #include         /* fgets, sscanf */
 #include        /* strcmp */
+#include 
 #define ZSTD_STATIC_LINKING_ONLY  /* ZSTD_compressContinue, ZSTD_compressBlock */
 #include "zstd.h"         /* ZSTD_VERSION_STRING */
 #include "zstd_errors.h"  /* ZSTD_getErrorCode */
@@ -365,6 +366,23 @@ static int basicUnitTests(U32 seed, double compressibility)
     }
     DISPLAYLEVEL(4, "OK \n");
 
+    DISPLAYLEVEL(4, "test%di : re-use CCtx with expanding block size : ", testNb++);
+    {   ZSTD_CCtx* const cctx = ZSTD_createCCtx();
+        ZSTD_parameters const params = ZSTD_getParams(1, ZSTD_CONTENTSIZE_UNKNOWN, 0);
+        assert(params.fParams.contentSizeFlag == 1);  /* block size will be adapted if pledgedSrcSize is enabled */
+        CHECK_Z( ZSTD_compressBegin_advanced(cctx, NULL, 0, params, 1 /*pledgedSrcSize*/) );
+        CHECK_Z( ZSTD_compressEnd(cctx, compressedBuffer, compressedBufferSize, CNBuffer, 1) ); /* creates a block size of 1 */
+
+        CHECK_Z( ZSTD_compressBegin_advanced(cctx, NULL, 0, params, ZSTD_CONTENTSIZE_UNKNOWN) );  /* re-use same parameters */
+        {   size_t const inSize = 2* 128 KB;
+            size_t const outSize = ZSTD_compressBound(inSize);
+            CHECK_Z( ZSTD_compressEnd(cctx, compressedBuffer, outSize, CNBuffer, inSize) );
+            /* will fail if blockSize is not resized */
+        }
+        ZSTD_freeCCtx(cctx);
+    }
+    DISPLAYLEVEL(4, "OK \n");
+
     /* Static CCtx tests */
 #define STATIC_CCTX_LEVEL 3
     DISPLAYLEVEL(4, "test%3i : create static CCtx for level %u :", testNb++, STATIC_CCTX_LEVEL);

From 574e75354bfa88084a3d162e49e7fdc9d9bc747d Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 19 Dec 2017 11:24:14 +0100
Subject: [PATCH 142/145] fuzzer: ensure existence of CHECK_Z macro beyond OS-X
 systems

---
 tests/fuzzer.c | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/tests/fuzzer.c b/tests/fuzzer.c
index d64b46ba3a1..141bf65484c 100644
--- a/tests/fuzzer.c
+++ b/tests/fuzzer.c
@@ -95,6 +95,22 @@ static unsigned FUZ_highbit32(U32 v32)
 }
 
 
+/*=============================================
+*   Test macros
+=============================================*/
+#define CHECK_Z(f) {                               \
+    size_t const err = f;                          \
+    if (ZSTD_isError(err)) {                       \
+        DISPLAY("Error => %s : %s ",               \
+                #f, ZSTD_getErrorName(err));       \
+        exit(1);                                   \
+}   }
+
+#define CHECK_V(var, fn)  size_t const var = fn; if (ZSTD_isError(var)) goto _output_error
+#define CHECK(fn)  { CHECK_V(err, fn); }
+#define CHECKPLUS(var, fn, more)  { CHECK_V(var, fn); more; }
+
+
 /*=============================================
 *   Memory Tests
 =============================================*/
@@ -144,14 +160,6 @@ static void FUZ_displayMallocStats(mallocCounter_t count)
         (U32)(count.totalMalloc >> 10));
 }
 
-#define CHECK_Z(f) {                               \
-    size_t const err = f;                          \
-    if (ZSTD_isError(err)) {                       \
-        DISPLAY("Error => %s : %s ",               \
-                #f, ZSTD_getErrorName(err));       \
-        exit(1);                                   \
-}   }
-
 static int FUZ_mallocTests(unsigned seed, double compressibility, unsigned part)
 {
     size_t const inSize = 64 MB + 16 MB + 4 MB + 1 MB + 256 KB + 64 KB; /* 85.3 MB */
@@ -257,10 +265,6 @@ static int FUZ_mallocTests(unsigned seed, double compressibility, unsigned part)
 *   Unit tests
 =============================================*/
 
-#define CHECK_V(var, fn)  size_t const var = fn; if (ZSTD_isError(var)) goto _output_error
-#define CHECK(fn)  { CHECK_V(err, fn); }
-#define CHECKPLUS(var, fn, more)  { CHECK_V(var, fn); more; }
-
 static int basicUnitTests(U32 seed, double compressibility)
 {
     size_t const CNBuffSize = 5 MB;

From 9096088f4529a25763ce4080376919c9479f48cc Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 19 Dec 2017 21:20:46 +0100
Subject: [PATCH 143/145] changed variable name for clarity, suggested by
 @terrelln

---
 lib/compress/zstd_compress.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 5f952b8c053..04b03bfcf42 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -757,11 +757,11 @@ static U32 ZSTD_sufficientBuff(size_t bufferSize1, size_t blockSize1,
                             ZSTD_compressionParameters cParams2,
                             U64 pledgedSrcSize)
 {
-    size_t const windowSize = MAX(1, (size_t)MIN(((U64)1 << cParams2.windowLog), pledgedSrcSize));
-    size_t const blockSize2 = MIN(ZSTD_BLOCKSIZE_MAX, windowSize);
-    size_t const neededBufferSize = (buffPol2==ZSTDb_buffered) ? windowSize + blockSize2 : 0;
+    size_t const windowSize2 = MAX(1, (size_t)MIN(((U64)1 << cParams2.windowLog), pledgedSrcSize));
+    size_t const blockSize2 = MIN(ZSTD_BLOCKSIZE_MAX, windowSize2);
+    size_t const neededBufferSize2 = (buffPol2==ZSTDb_buffered) ? windowSize2 + blockSize2 : 0;
     return (blockSize2 <= blockSize1) /* seqStore space depends on blockSize */
-         & (neededBufferSize <= bufferSize1);
+         & (neededBufferSize2 <= bufferSize1);
 }
 
 /** Equivalence for resetCCtx purposes */

From cafedcbbe4b36a6b7253ee3bcc4b28bbc972e433 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Tue, 19 Dec 2017 21:49:04 +0100
Subject: [PATCH 144/145] ZSTD_resetCCtx_internal: fixed order of arguments

params1 was swapped with params2.
This used to be a non-issue when testing for strict equality,
but now that some tests look for "sufficient size" `<=`, order matters.
---
 lib/compress/zstd_compress.c | 20 +++++++++++++-------
 tests/zstreamtest.c          |  2 +-
 2 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 04b03bfcf42..a9a8a265d97 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -161,7 +161,7 @@ static void ZSTD_cLevelToCCtxParams_srcSize(ZSTD_CCtx_params* CCtxParams, U64 sr
 
 static void ZSTD_cLevelToCParams(ZSTD_CCtx* cctx)
 {
-    DEBUGLOG(4, "ZSTD_cLevelToCParams");
+    DEBUGLOG(4, "ZSTD_cLevelToCParams: level=%i", cctx->requestedParams.compressionLevel);
     ZSTD_cLevelToCCtxParams_srcSize(
             &cctx->requestedParams, cctx->pledgedSrcSizePlusOne-1);
 }
@@ -169,7 +169,7 @@ static void ZSTD_cLevelToCParams(ZSTD_CCtx* cctx)
 static void ZSTD_cLevelToCCtxParams(ZSTD_CCtx_params* CCtxParams)
 {
     DEBUGLOG(4, "ZSTD_cLevelToCCtxParams");
-    ZSTD_cLevelToCCtxParams_srcSize(CCtxParams, 0);
+    ZSTD_cLevelToCCtxParams_srcSize(CCtxParams, ZSTD_CONTENTSIZE_UNKNOWN);
 }
 
 static ZSTD_CCtx_params ZSTD_makeCCtxParamsFromCParams(
@@ -247,7 +247,7 @@ static ZSTD_CCtx_params ZSTD_assignParamsToCCtxParams(
 
 size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, unsigned value)
 {
-    DEBUGLOG(4, "ZSTD_CCtx_setParameter");
+    DEBUGLOG(4, "ZSTD_CCtx_setParameter (%u, %u)", (U32)param, value);
     if (cctx->streamStage != zcss_init) return ERROR(stage_wrong);
 
     switch(param)
@@ -311,6 +311,7 @@ size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, unsigned v
 size_t ZSTD_CCtxParam_setParameter(
         ZSTD_CCtx_params* CCtxParams, ZSTD_cParameter param, unsigned value)
 {
+    DEBUGLOG(4, "ZSTD_CCtxParam_setParameter (%u, %u)", (U32)param, value);
     switch(param)
     {
     case ZSTD_p_format :
@@ -326,7 +327,7 @@ size_t ZSTD_CCtxParam_setParameter(
         return CCtxParams->compressionLevel;
 
     case ZSTD_p_windowLog :
-        DEBUGLOG(4, "setting windowLog=%u", value);
+        DEBUGLOG(4, "ZSTD_CCtxParam_setParameter: set windowLog=%u", value);
         if (value) {  /* 0 : does not change current windowLog */
             CLAMPCHECK(value, ZSTD_WINDOWLOG_MIN, ZSTD_WINDOWLOG_MAX);
             ZSTD_cLevelToCCtxParams(CCtxParams);
@@ -760,6 +761,10 @@ static U32 ZSTD_sufficientBuff(size_t bufferSize1, size_t blockSize1,
     size_t const windowSize2 = MAX(1, (size_t)MIN(((U64)1 << cParams2.windowLog), pledgedSrcSize));
     size_t const blockSize2 = MIN(ZSTD_BLOCKSIZE_MAX, windowSize2);
     size_t const neededBufferSize2 = (buffPol2==ZSTDb_buffered) ? windowSize2 + blockSize2 : 0;
+    DEBUGLOG(4, "ZSTD_sufficientBuff: windowSize2=%u from wlog=%u",
+                (U32)windowSize2, cParams2.windowLog);
+    DEBUGLOG(4, "ZSTD_sufficientBuff: blockSize2 %u <=? blockSize1 %u",
+                (U32)blockSize2, (U32)blockSize1);
     return (blockSize2 <= blockSize1) /* seqStore space depends on blockSize */
          & (neededBufferSize2 <= bufferSize1);
 }
@@ -815,14 +820,15 @@ static size_t ZSTD_resetCCtx_internal(ZSTD_CCtx* zc,
                                       ZSTD_compResetPolicy_e const crp,
                                       ZSTD_buffered_policy_e const zbuff)
 {
-    DEBUGLOG(4, "ZSTD_resetCCtx_internal: pledgedSrcSize=%u", (U32)pledgedSrcSize);
+    DEBUGLOG(4, "ZSTD_resetCCtx_internal: pledgedSrcSize=%u, wlog=%u",
+                (U32)pledgedSrcSize, params.cParams.windowLog);
     assert(!ZSTD_isError(ZSTD_checkCParams(params.cParams)));
 
     if (crp == ZSTDcrp_continue) {
-        if (ZSTD_equivalentParams(params, zc->appliedParams,
+        if (ZSTD_equivalentParams(zc->appliedParams, params,
                                 zc->inBuffSize, zc->blockSize,
                                 zbuff, pledgedSrcSize)) {
-            DEBUGLOG(4, "ZSTD_equivalentParams()==1 -> continue mode (wLog=%u, blockSize=%u)",
+            DEBUGLOG(4, "ZSTD_equivalentParams()==1 -> continue mode (wLog1=%u, blockSize1=%u)",
                         zc->appliedParams.cParams.windowLog, (U32)zc->blockSize);
             assert(!(params.ldmParams.enableLdm &&
                      params.ldmParams.hashEveryLog == ZSTD_LDM_HASHEVERYLOG_NOTSET));
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 70922e2e033..5590c9af15f 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -1527,7 +1527,7 @@ static int fuzzerTests_newAPI(U32 seed, U32 nbTests, unsigned startTest, double
                                (MAX(testLog, dictLog) / 2))) +
                                1;
             U32 const cLevel = MIN(cLevelCandidate, cLevelMax);
-            DISPLAYLEVEL(5, "t%u: cLevel : %u \n", testNb, cLevel);
+            DISPLAYLEVEL(5, "t%u: base cLevel : %u \n", testNb, cLevel);
             maxTestSize = FUZ_rLogLength(&lseed, testLog);
             DISPLAYLEVEL(5, "t%u: maxTestSize : %u \n", testNb, (U32)maxTestSize);
             oldTestLog = testLog;

From 3ddfa42fe867f18e999032ee9ace0f86a01ddb5c Mon Sep 17 00:00:00 2001
From: Shawn Landden 
Date: Tue, 19 Dec 2017 22:00:27 -0800
Subject: [PATCH 145/145] meson: fix build

used absolute paths which are deprecated in meson, also missing some sources
that got split

also move source files each to their own line so future diffs are clearer.
---
 contrib/meson/meson.build | 49 ++++++++++++++++++++++++++++++++++-----
 1 file changed, 43 insertions(+), 6 deletions(-)

diff --git a/contrib/meson/meson.build b/contrib/meson/meson.build
index 8cbdcabec72..5ff370fa716 100644
--- a/contrib/meson/meson.build
+++ b/contrib/meson/meson.build
@@ -2,14 +2,39 @@ project('zstd', 'c', license: 'BSD')
 
 libm = meson.get_compiler('c').find_library('m', required: true)
 
-lib_dir = join_paths(meson.source_root(), '..', '..', 'lib')
+lib_dir = join_paths('..', '..', 'lib')
 common_dir = join_paths(lib_dir, 'common')
 compress_dir = join_paths(lib_dir, 'compress')
 decompress_dir = join_paths(lib_dir, 'decompress')
 dictbuilder_dir = join_paths(lib_dir, 'dictBuilder')
 deprecated_dir = join_paths(lib_dir, 'deprecated')
 
-libzstd_srcs = [join_paths(common_dir, 'entropy_common.c'), join_paths(common_dir, 'fse_decompress.c'), join_paths(common_dir, 'threading.c'), join_paths(common_dir, 'pool.c'), join_paths(common_dir, 'zstd_common.c'), join_paths(common_dir, 'error_private.c'), join_paths(common_dir, 'xxhash.c'), join_paths(compress_dir, 'fse_compress.c'), join_paths(compress_dir, 'huf_compress.c'), join_paths(compress_dir, 'zstd_compress.c'), join_paths(compress_dir, 'zstdmt_compress.c'), join_paths(decompress_dir, 'huf_decompress.c'), join_paths(decompress_dir, 'zstd_decompress.c'), join_paths(dictbuilder_dir, 'cover.c'), join_paths(dictbuilder_dir, 'divsufsort.c'), join_paths(dictbuilder_dir, 'zdict.c'), join_paths(deprecated_dir, 'zbuff_common.c'), join_paths(deprecated_dir, 'zbuff_compress.c'), join_paths(deprecated_dir, 'zbuff_decompress.c')]
+libzstd_srcs = [
+    join_paths(common_dir, 'entropy_common.c'),
+    join_paths(common_dir, 'fse_decompress.c'),
+    join_paths(common_dir, 'threading.c'),
+    join_paths(common_dir, 'pool.c'),
+    join_paths(common_dir, 'zstd_common.c'),
+    join_paths(common_dir, 'error_private.c'),
+    join_paths(common_dir, 'xxhash.c'),
+    join_paths(compress_dir, 'fse_compress.c'),
+    join_paths(compress_dir, 'huf_compress.c'),
+    join_paths(compress_dir, 'zstd_compress.c'),
+    join_paths(compress_dir, 'zstd_fast.c'),
+    join_paths(compress_dir, 'zstd_double_fast.c'),
+    join_paths(compress_dir, 'zstd_lazy.c'),
+    join_paths(compress_dir, 'zstd_opt.c'),
+    join_paths(compress_dir, 'zstd_ldm.c'),
+    join_paths(compress_dir, 'zstdmt_compress.c'),
+    join_paths(decompress_dir, 'huf_decompress.c'),
+    join_paths(decompress_dir, 'zstd_decompress.c'),
+    join_paths(dictbuilder_dir, 'cover.c'),
+    join_paths(dictbuilder_dir, 'divsufsort.c'),
+    join_paths(dictbuilder_dir, 'zdict.c'),
+    join_paths(deprecated_dir, 'zbuff_common.c'),
+    join_paths(deprecated_dir, 'zbuff_compress.c'),
+    join_paths(deprecated_dir, 'zbuff_decompress.c')
+]
 
 libzstd_includes = [include_directories(common_dir, dictbuilder_dir, compress_dir, lib_dir)]
 
@@ -19,7 +44,15 @@ if get_option('legacy_support')
 
     legacy_dir = join_paths(lib_dir, 'legacy')
     libzstd_includes += [include_directories(legacy_dir)]
-    libzstd_srcs += [join_paths(legacy_dir, 'zstd_v01.c'), join_paths(legacy_dir, 'zstd_v02.c'), join_paths(legacy_dir, 'zstd_v03.c'), join_paths(legacy_dir, 'zstd_v04.c'), join_paths(legacy_dir, 'zstd_v05.c'), join_paths(legacy_dir, 'zstd_v06.c'), join_paths(legacy_dir, 'zstd_v07.c')]
+    libzstd_srcs += [
+        join_paths(legacy_dir, 'zstd_v01.c'),
+        join_paths(legacy_dir, 'zstd_v02.c'),
+        join_paths(legacy_dir, 'zstd_v03.c'),
+        join_paths(legacy_dir, 'zstd_v04.c'),
+        join_paths(legacy_dir, 'zstd_v05.c'),
+        join_paths(legacy_dir, 'zstd_v06.c'),
+        join_paths(legacy_dir, 'zstd_v07.c')
+    ]
 else
     libzstd_cflags = []
 endif
@@ -39,16 +72,20 @@ libzstd = library('zstd',
                   dependencies: libzstd_deps,
                   install: true)
 
-programs_dir = join_paths(meson.source_root(), '..', '..', 'programs')
+programs_dir = join_paths('..', '..', 'programs')
 
 zstd = executable('zstd',
-                  join_paths(programs_dir, 'bench.c'), join_paths(programs_dir, 'datagen.c'), join_paths(programs_dir, 'dibio.c'), join_paths(programs_dir, 'fileio.c'), join_paths(programs_dir, 'zstdcli.c'),
+                  join_paths(programs_dir, 'bench.c'),
+                  join_paths(programs_dir, 'datagen.c'),
+                  join_paths(programs_dir, 'dibio.c'),
+                  join_paths(programs_dir, 'fileio.c'),
+                  join_paths(programs_dir, 'zstdcli.c'),
                   include_directories: libzstd_includes,
                   c_args: ['-DZSTD_NODICT', '-DZSTD_NOBENCH'],
                   link_with: libzstd,
                   install: true)
 
-tests_dir = join_paths(meson.source_root(), '..', '..', 'tests')
+tests_dir = join_paths('..', '..', 'tests')
 datagen_c = join_paths(programs_dir, 'datagen.c')
 test_includes = libzstd_includes + [include_directories(programs_dir)]
 

    OGU}uspZfXDbeI*Mgt~)7-!GG$P2aKq5pjy=4=D)?4+3WP
z+F!VE0YUiCyu=27oGxvYLXPYE^y#O+Y2bIiHrQv6|^CFN@+0#=hXT?sCO4qG!h
z#k;ltzfBUZqv8|k8gs=u?8jAsX+p~P?%j*Cr515303t$d()rHruPRz$T(sx#kN72u
zb@q#;o#Mim$)>a=d4D+X{+qGD5H$)&nPI7=u*-Oa5ezS3LL6gQ^2wt|t@~dII?Out
zb07No?E$x92XwFro>GtVC8zceeo@Jn91yK-_o4ay6Or)*Na6s)bCCWZ&{06SJ-xv8
z&0iv)8nj{xlYJzUJAb#}L}*mcs%KkMxy17tnxXL1e>`szat7pqzyGl3gL%tv_m?v_
zq$DVob#7yDVU_nZFR!vZ=@BA-Wuho3eej0zgGtA7EnrR
zii}CA4Lpq&sQPQ>(+jHI4vO2o>!KLjsM5Kja4CH9s=E;7tv{RsDH*&$ymucx2a
zG%_ei?X4+RvgU9p9CTD;AksMoMo-S!@{QIqF7`8^WO&;v0i*3Su3vBrd0=8yN1sejq7=bPE#y7MZu!~%o*p3f
zAA1CGc7l%cY2Zj_9vm*Yb?L9!S|N|b+Mn(YY`lC;T~NJFob|b&_3b~t*9P)epz|!3
z^ZM?$BhqEcqG@<}7#$4W@StHaWjE%*Uk`@)ulZZkq4A%@!{oc9qCC#xJoB<-tf>F?
zLx=|f*5Sur5OgzZwcu?RNTr9hQSOfadU?K-Nxhn;t0Clq%q{%ems!WpCC_7Tc!tsb
z{Kmh{nF4y62s@!U%XGI1{IAVRXl5%gf;d-;0(;ME}U(kOwjh^caX>)6~?|)}aWq
z45csQiKf^0MH@5%5};051TYz7&z>;N^@4T-*Xi9?&qCU9ih=K2-JV&O%jb)hN3gwA
zl7Y{!S?4}O=8gSL?FeC(8s4lozdejI-zdkK?I;_yB`HfS-xpDx$UpH`@^h&5)!2_H
zM&>)OoMN>q_nMlw>iT=CYTQNn!qcwYeJ)odLBe@l{^uES>pP%knK{WH`h*m5el`G9mzzK5b%}
z2J`ekB`YMF{?)w!on>Q9hzJ~R(#fG5rnYXuWa2&2FQ)5GZF?5Ey4W;b2@bKnu6C7h
zKZdCC=CIduZf&lO$gb)Y&Th1JG;XWg?9|4<;eF-@E(LPai|r4wZDKL==8a7FblJYy
zQ3hxFR^u3Nkm1|nqMcP|!=mD8WqkUOMP?XJV$*Yf|E~4gW3$oS2fj*|N*75dNoU(y
zDT(dbvj;cFz`)QO{{C3tWoA-YFCy~uSz@DeV$;Di{8pb`#Xk7FXm-0rxce`$1Sl3U
zLec^)u{xq9a!vd`mMh9rGputk_K?!N|``3eCB{33iM97Bh)Jb!%Hek
z)>fgdbvJ*zQ>TYs9h!3%#erp|!V~E$Gv`QLcqW9bh6$j_>|xC2=-HP%x6&C}Z*Nrd
zmtl~p)i3$XD69SH{4L#QG}H}K^kW40_isw7wL<3@2)t3icl5eSJPaAOlCgkk83izh
zA$)6e4cI8-uJCWfS~w_A+yf;_z<2Rjr7xZ^e{L!zwJ-lf{B<#;6_zO?P2P36m-(pk
z{tQx~Q@bxiHntfhJx#ZH)N|h<83hcUGS=m+)fAOjZ@aq(Yjgk+`Cl&$Zn*4R)_P94
z?)M%?Tv76tiPgI=2cq39Ki$@PKWkLEc{iq}t<~SSCOqe~n_K126fA6;Dd
z$wZ_`Z9Uz=D(?AIN1}7OhR(B}_I${<@NX<-X@X0@E|?|;=_Z$T7pHyr4wfaT6JNi|
zW7uF^o*`s9q*a`!?^Sj03a}xRuk1x`MpK6ur8ePImMc=u2(EM_gd~P6^}`yOM5jS`
z0J*)Hpr6?1ax&4wc-a_fZFkq15n!@@skf;m#>zhjgm^|*WHKSvOR+K1J
zNWQY(Z`*7_sGfd}zuZMVDJAM!Zlk=q810Ydh6cMSX3V{(g%TzNYQ`QgW-;I!kxeGq
zf%#+!!|8?y9vpr4$Awo>#M(?xF9-_{!1$sCraq-=magTGhEEPCKMmWNDc??2rdN^o
z#&$k%J7HjR2Co|51@o$_Ls%7rT3_nL?^>nhQSl8z*%fZt92JA?i>dO2(_FQ|hX~%z
ztBfs$X-v2ymi@n@5L=EKKTx($qiL)C&4aX{N&&~(QciM^kA||B?WMDG-DTTMgw5mQ
zhV1IOwPPLGFE>9EW8f$5ZOay1ZDLc+~W2*e9%pQx!DihU`&CDbt
z^R!yCm9ckHi4&L#&f)F}zdONLan6dYa;K-+znpp#=d*+!;}~Uy|TSO3M2C7HqVuK
z&K@omSSqvjPoNYL
zk{qgPN2DZ)#D(Uem97%JITgnm+m)dizEAf;Pv(&EhIJ2uu1gT*Uo7yL_$AY&)O@7>
zBUOTpP^UD+yR3O2lUD12l6(Tw`jApENg1&UcKba-58h}mp02U@d3G@(RN@}gPi%w+
zp-KM{e^>96l_
z_^%@#tp*uEwPxMe$oYObrd}V-2TQE6(l?)sL&Qw;Y$S6HYpJB0suQ=wgKn
zbOmfKXL`=%lJ?=ezF~uT#}HMQ`t44$)$criyYNx#kk6R}VJtPtr|>$Tngye?Ccm3~
zefYP@BGxOGNzC}*-q+F19iI>aaley?QZLJH1g>V^UXh3?yH92;v!8}UYP$Vx=0LBs%_Y0criq}9(B5RE#ev*9;c>{7
z&x=;T+59QN)6vI1KA!Pc1cE2b(X
z{+J?ghg)i6tF7cGu=t;kYQcHwCrY~Gx0BxW;whL&et-~^;q>WiXaULhvAoWs+O&_u
zV`DpXLYE($Iq&p;Y<+n=)Zh2N8OFYE*@?)$W{pfFdxWtJLe}gGWr?vwMV2H=LLp}C
zd$tB4OGO!bjHP5JSz7SBqk4bJ_xGPiBd^z-d+#~V^E}UU?(|Cvi6}Waxyq(2)tn};
zF$ExAc8k*ix^%`bmb(P?z8X@hyacou;z5&jowu`Q3L^CS>MZpn8G1)5Ub8{>w0t$GLN~x%4GxmP?16reC|r
z<>DoXi-z7$haCW_&vFsbQc!0MO3dfvxu8!n_!=fuxIva}8@
z?%oPQlapyk%8!1zAm^v|%#wuim~*?rMucS4Dc-0^{mctkL)nG7ZscY-@7RPB|x
zHIty&H*v^-rTL$4ylZDbZ+Uw>yWHAqpRmd+WKv_G?83U75NxV5+~b{Ds|ipVc7p`j
z9YO4f`lQi0d{Mo0??voYRi8&(48P9Q?{J4#fQa~rRBH=XR%3{AR4b&caw;a3QKE^{
zJ%MVUg|nPrT*->46R{dcH#Ty$w6r74s?_Yng
z`nLmyL$Zz}W#_88gnRX!k4=`3$)#PWjxg<78i>NYvp8iprme0W-rQZs#KaWBVw39U
zACPsof8kLfI&sK$JJ;@o)CDyKtY~3dYHBJ7Q^9+QX1Bs#fG*94TIS~FcIj(ezda=<
z)6M5pI+ueZr^lJ#rh9iXqnbp1;okpz3N{JKYQ(+SdV{L`>gt8!H_-!V%NkGiV<=IE
zaNXng2NpmYHWfv8s?XiZAVYzsf|C9BLB>izqq4|E6^QN}A+pXx{#@|D))^70tJhM;
z9?ARnD8nNn7}kb~Fu5ny!NpiRU;xKW(8mThS50B1r_WBMaRZR}$X!Hl^WJ7@ZTKh7
zUppR)qDt39F@zH*^diz~AgQ9m$n4u)&k8)v75*HFKQ>)MLK11pZeKfdhQz750*t|=
z1`NEGgTqmP+^UE$1Jq>x5F;)a2ip|#}Ej#}g5+wYHvY?R0E+F1_W#k;JT_VGp>HbUFeKC95E7XNkCz3X*
zI;j(he|`ABcZ_Ieut!z)+TcFC%xvZ;
zh?H6Ij?1G;ZU+XC&c!C}(~;|3rEaP=
zz&Y;LAN7ff%0Fa*2wDiK%s&gmOS>vf53LaV
z1*J50w6sxG`PWCb^qU;U`%*9_j)DEgO^*QjOfjRwmjvky;w?o
zE*$1o8Ps0X6x=
z&*l;{1aklPl!JsPA`sYoOk^kKm(B5{5!pwvc)I`_rF}*C$HD)TC4w!-BVoV(x!=#1
z@O|#VF-@?BiXl8c&rlfs=Dh)9+k2UyK4@=mJtymX+>OWrEyz>KWBX#pTStwX%|AqN
zEA1leyJfJ9-UBD6>;CoyApf{;e$sqZvb96gBFWmU^v+QM88)ip?GE$tXLhX9JwD&R
zwf-ee7Q66~sdwxtsvjoGski>Fgf}N!bnK(Kr1_(WpG&W;50LE{()RkLIWkH%aD93Y
zHlln%iqaXL@?Ep_n-As-q?BVtLCOAaqu1$YSD`81{`|%)$vp6!dg^OC^3~h=>w-I7
zjsgsv4ETyk9p1-b-!FlyT$#Y+_0N98G&D2?MMY-}45*9DJrb`sj9HnRld7w$pE-NB
z)oFn%-P^;%L*4BAx66(wT$*dv_BjaMk)&Q+TA&&x5~KVf!3DTwJ7{xAAb5X9qEzf07|Rj^Mr1CM
zQ&C-+AH&6hw6Sq()I%wJPjz8|D`EkUw_oNGvnV-ZW_B-d7|gv`2ZN$(voyYgtFUH(
zxTFI#5RcXV=;j0!Rp+X*Ipd&;{@PRR14gTBmK^Ynmw
zSuvS2lY7gP^c)8<@5m2|D{=V5&|X5Tbw~HW7n^=CiZ2
z_MpxVI(_NZs!OjMp48UTy0P^gC<0O?VrH1AFsJ&=V>{`{m^4vO^PwB9&4t*8vxS6Fed
zTs-IeSjGl>PNLMn5!qJNuiw}tlF?fWb|}ZVfkK4g(~3k?t2(i+Xy3o|ofPXeXG5Ne
zk<0AGH3_4Scl{bn5(E0T11AOwC0x{1rKMnr6PYF4QtXex#t~l%thOvRwFdVG{doF?
zG5zO{$^8TXfG)A;Fy0FV?B@DqOO#T|vm2NZHm8Y(gw*!8qfLvJp?D2xJbA4oiA;9JCL!R9}ty?D8J(De#Ap_hmO?OTx;0a+5E2YF735F^gLC>
zz5Own&l^Tini8s}fTh`gyz8}C?2j?3?`n0y7!$^yseTKF&_tlYXfLv=tjemYIw6rR
z(xTTy7@Q7fD!!A|C3H=>2N*a*&_MfO56DNPV%Le(jn7$$+$UoQ7Zi^|5q96-rB!{;
zC-iZ$X5-$DWejz6oV95VWd{;pwLn5L#X4?yuKC+L3Q!kA*%*n37F&fb#{|6q-@}gt
zI849tX@gSIaZiudMY6}8)5&&cJZ~b!U-e|1U!)59R`qpx*?E_%V27kY9X9L{@!k$5
zalWtZRlPE-F2cu1@R&{8x
z3^q8R<0F-T$e2r|m44>Rd>r5Y45&A10uO4Czp}4cTe}-v&$@jcn+6l2L-s8aVU(6%
z?1<1_7`<6IA6M@7@(4ZmBYbt}*;1+5YNq#TiY}C9AiU9XJg{L`@6jsYE0Weh=XL0_8=~*khXAgI>y1lk(8NvhoIwJb?%o`SAS6=IhQ&X
zdI~FQ$7TgH;aavDqPufSIhKtUT
zFZHyP4PuyZpRT>j;D_OiS{@ieO`XH5nk1*N`U3YVSx5$V$mJmiyNPG%kIPBoV$`Ew
z;p@)LDSVF^=YKvqQ4PFBgIvWBiXyrYaprv%>~i>QU2fx9DTqcV-7x*Nqko4*w$t_?
z_7;sRYmR?QLa;z
zUY{v6O>qp3Mz3M)cJ=gWe;&B&OUGzg85tuZBNN_{-L?7=#rqR}^4vvNLXo3EHI^h|
zQ>!GmZOrFIar+4Hf$uJ?WcLs^c{*;-r(Fnq`r*U53m1665ZkzuD|N)6XmZb7$C8qH
zf92{6&9Jbr9}bM|`=C-nduv!9z8StZ7hbi0+7mQ|h@C0Ay5yzaq;4kyk63|;3x$U|
zehL>*Zr$WPJ}2NbNypk0y&v+UROUp9z-K3)N(YNK?muT!wAbGXywF5wKG<*wHG<5L
z=k5ah_KD5Mx>A%Sn{sR{BVH~n;oje2fS3>OBQte8@y-CW;DhbCNX$F!?^^*sz=ZH3
z*csky_=t1Q5^mpa1){I(s{21TMg_Ji@VLQZ^iK;#!^n&Sl5M_!!$o<;RsF{U^HRh`
z+nk&nHQwy4Zhqulc`+*Y4|?xCUg>QWNmV!RpJg{Nda`69H4Xl}Dk!d$!jK=Ea^bvc
z!BxB7ZyqTVi|mL!C=T|&us1CYnh-pJ$Q0vxG&5wZboXT_U<$qp)3Y=uE4eZ&v6Cb
z;;ZYyXGb0+vyCt2be#29HF>ufHQayY7v-FfxpLtu*Q=n8D;=H52U*Q;^oi9ZxhMc#
zYpv}(tw$8EL#36C%iVnyG6L90BfDA<-90FR2p6(Lq4nzFU?x^gUn+?zptvzKnyg)S
zg{&~a8{6AMtkrjqsm}$1ChkSa%YtzRyVFNrz_qksVLNTS?#>jC-u8d^0Vd5F$KBA)
zwsfzU@O)28n)*(*q@Krk{5!z=`=o^W_{a^Gs)?V9jcN|$^i+(uug+dq4ml9F=WJ4Y;wt}Jjf@9vYv
zcVDF%8q;89v*BE)PoMr$-XwTTiM2m@l=WRO*oGvvYiIX8Iwc-O#+qQ`U>d#_UuNpq
z0x5s!4`%1KtS9QlHL+X2c@fKfMp;MEb27c4v
z?2O!7rN6;g`x}IHT%Ozb*A1ZF9fGf~j
zor!X{&{tAkkm&SJ#`1<)iaCvsg}s@p>)*Dn@$9EfenCzkJDQkYVZKrSh#SJ|%?PFu
ziwlW$oSiIz7E{r?QTUC_&GU*bdhWE1E_g>3>7I9S5=55m+b}&Wz14O$Cf&F5+GgVH
zD?UpJ>Z;)HHj6WIV|qz`&dR-BsiXPm?2z+Z^WRtRPNbFowS6Zfuu2K#pDb9-(?$#K
zHo8^^O3nzi7>F?O{n)g;H=g5^)E%gnEpk9T?jjhR~6wtfBr04Y4KyG{_M$T
zkAJ?R9`tZo0XdA=to}ZjvorRwM^r3dw|=AAa9?Awv}yV3a*+2Yx&6fX*9Gfrb}Vc$
z4wlhBe4p4W4nF>Vk`m4JV4+y_>89`eGbO9%#8=2B
z1EWc-UBX(ydS#aJ`SqjFxEZVAG%aZ*hU#npiU2oo4zr{Z>>d!9b
zBIw>SSWxyR+>m?i)-J7Dbj2k=YQh6e4u4)8gjwue(4JJ!-9no*WDNz8(rFQql|XJw`=I
zXV$&zQ>*K^!u?w>%f4=%K2bC1xH~_zHo4dwx%upZrlY>?k5f-yP|#N4TVRDdo#E$N
zn6_Py9z{C_^n4fMoG?2*D78P!7xp7eK~MByd|1PT+Fs5J7?f0brM#M1X(M2@*>2)6
z)kViDP2ghRcJ1VLM3vTn8Dx06@Eu};rU+-U+eFfM?rHP3_pMbGL|x?(b!P8~x|~v7Dr={D+mXeO;0{3or0uYy
z`~xRf#-j4^gRvJWNi;f$+Y=(cxG6o8wM5vily%jHc|VT$8;KCkU9ccufP3DzP}C-<
zEsu)&+>KBH?JqX<{)M~Gva_?*(m8l}6M@2h8r;>8w9RN{O8~!KK-0?xMxc<6j)fuT
zGJdubEgH72OH=v_@I?&fu2J`PBV%^T%q04dlRR!-9nZzI%TKz>j}CM%izJKoxqnGO
zJ?)x}ZRt9SyoBn_rDwh`|D>qi%G6}D*DojgoR~ZYD+L-xn~{3^UHlSp-D3O
zpgQ?@tJ@Cj&q+$1j)fe3>ncV_UI0w;I0#j&)YpkB$x7iUEoX*ctCQFA8B2W{qaBZM
zJ!pJWs~BW?thr2@WhPe1{oKjbnGi*aVa|vsQIG1qTt}iQITXbhM7=VD^kt99zii+d
zcLWgfx+^YjKn61&TrO-Ajk8uRogTymfk23H3>obp@-R{v*N$jrN`fy&#AU2?%@3V7
zpDU;>%2IpD5dmvEl{jM6bYZg;BxQ-HgxVe6)Ugf7SMitDT}ajOt$K%c@jPX7o4d1{p3&(vQbXDR-3!gg
zAc5W;xPj|{He{m$h?0URDE*JxSB{nK
zPd@{GoW^Y6NW)ihnwu|M9q0;RmG@iR1Mji;&D*chnPhDi8-Cky3a3DN1{dJ`OmFo=
zES1$^SI-$5k2DHJO%|SxEv+c_bCNV}>hU9P5s2uDC9|TKU
z?5@loc@dEo`KwgOvm`+%wbCW73Ja>LQR^zyLX4cxlZw;Mv?|qWST}@VdoOP%@L}Xx>Xw!4
zZ+$H|dRh9=lN(Q?z!O#DdhO-cX8k^25aAPqfRyGEd+`*cU%24S>G!z!!TdqeN0O&d
zO+kweGT$)qg9ho1!cTD*6&%ch*;X<$qhA)lZA34@1FmBf-YOs@@xTEWjh??5SwWiH
zL3UJjU{dS_F;9SLGF8O%3us^}!V5#nAqDlz+W7yz(4W;2`n@{GNvhx;Uwsiz1K3-k
zPpjTR80}MqN}t`{cC?!~js{b`U`RtDb!uIj_S+9`p-;f2oz541SzukB7#nzKIRe|<
z@pVlOF4nwgz;{s~y&paGwNvWKO;IXo>bfuaGfSJtQFsRUtMt7(j<>%nLjQSAR~Q>G
z%YYo#9p5$6cU=Q6YxcexzIa)pFZlUFB&&UD##P!Xq=EUd5UaD8EF?rsO@>V$acA&6
zyF;DHNc8mW{H>rgY6~ms#cP4ploiP2+{j|jT|PbtE?*zM?9kaIrU;?110{#nA5#}q
z1^
zEprX5^Ru*2rBsV3MTJj}+pZrSRUM|g+2pM
zaD<;M@Pcvw(=s(7u-m;(&d?6jGC}=+@i_|W72kBwpp!YMJM+F!A
zWA_@3s{87aT!P(C!$YA@onSlEeb<7Lz(|po~5`lTmj<}Ot#|y`yFVV2lBEMQ+6P`m66b0o6
z$J2@*i<%_6PcDtm^?SwqStEO7F#5xYhuJ);_WXOZsdoYFbYsZIR-2SI;pLPRkm7c-r-!zERM?>fzy>Lih)U&WM~8)U6Pr>S2!G
z74iQC!SO~SeUkCU%&|l#Qq0&h6md2@i#*+s)#+vY%c7PZhKmUHVt1LMH|)in_VQ$W
zx~y$qD&9dN%;kh=@4UQib&DJU!Hqu&K5=49e(aO^$pqVij@Xk_z)zO5qy6DJ*?C6e
z*H?~xX}Xn8%-sCMyNSh3(yX*w**)Uz`L&q8(Ko;nU-JkH!3eotr~;q6hzdYu2eDI9
zC!u)UGK*ozHnZ$%%sWT|+~e|@`;=9|Lr*tMDD4eglvd7TVVdWpPgyyM$;kLv7OX`6kWCg&!_oVri&8
zL(K1!*Ey(c^I{nxD_O*#hPW%jmq#4v91dJ{W7KHAUYfoZX}d%JO(}ci_lsb6o|IF`
zJGEHzH1C?nduJQrf-BF%ze#%-oZvyG-)6Yvkh$fjNe8W{7%ta>e>|)G4BLWbMkv6c
zrH6gBux}ewoWZxv>G2gJIZHKHNhL2=1aK
z=K#MkL}yTs?3%?{OxIf@>sj}a-ZSwAbeg)(OnRbR+fw9ddK_heJ~9ce76L-!w`Yg3
zd+DUBC!bx$bwin(DXR)^lp+xyCO?U}_hVQlu9h|m{LMIm+<}-c+qJCX43@}#dsC3j
ze%a&v<3#9V5db`U*(jy0>UrGfkPKov$53FCde{^K!0N-0sdO
zo5Hg!DO)9!*;A3WleHK{-t_!HW!V~mI+6WVO*?B;6D1zZqT2Ywu`c%T;jCu{<#c^k
zM^7+Hh;8HLd6Vo?5;nIR<{1HdFO_<^dE;V)I4{R79yMyz7c2EXF9_={`?khnxobEY
zY3ag5TuhrO85WxWTX+>seS!rx*9Md`+52dj~f5vu*Z-tq2G8#QPy>Y=SaQjy>cimTVn@>
zUm8^$D1=}8qU2BcH3GW{EzYu);Ksj>80_Pt%1?-IeaC)F^yWhBqgIs@wJOl!yHmlR
zJq1sVBBEnlGR)t~_qneS2T7^iP}WH$WqTG`wC_7^)@FrJ;e(N9ENLlrSs#)`fQmkK
z*;c8~i^rLe!LW}XyF~2SAf!kL#iMgv9JfMT1TO_xUWr?%W`xS#k`TpD8MP$V}3W8eWQrvkT`XBcRx%{VUDGY
zoGzvXX*L*LvEbsR006+FRzh?d`H6g~4RcfXP8WUqJU5H7M!Oe!`vStT0`mHsXdSvn
zYhp#&aaQN=+6Uo#77*MV5hnJWu~FMaRP}N06#%c<+z>Jv>gu`2Nj>C|olY?xrw5;W
z$={NSb*e3~h;!gRWpPWm+SgpqGds{Agv>73(a_9(54e&}TMKWC@PH<3oRi^}AZ$YH
z-!LD+$qK?e!J+uCr~@oEu1>}UfyE#5ZF&c}8h?%a{ktO}0rNx;O6Ahr*FJUh3+`iXHvsO#Or_9eJq>d;WptANngQCpta1XG|
zKzb&@%9kvNi$P=?_G`4kOuCymlgg$>}P(@ZnP$c8Z}+oe)ogSnXsvLJf~L>*W}
zQvxofeR!UH*kur@MWq7Nx#I=o7pRnuqo!D5;4%UXG_KuLcS*R_a<_0j$p*L7s!}z|
zl{{E~mUAns1iSD-%k7fad)79}we4g$2H?zU03hgS|ukSrE!*-^3tKH{S#Hi~&jPX^0eq
zeKt-e*+jv9d4jRA;iK$rF%{OAcbixdX`2aMeuQ)?YPzXW@=}~Aov*!kpUr2~ttL9Q
z>+<-8;nvAR5M4MMB=BPTBy4MoAlMNSmIqKC@xF|AkRM-8-N1>
z@AUrPXHpz|EqVNrNZ@M$_%whJlYixMEAH%ia=nkZSa*m?hf5~*5}=Nv@I16
zZn2#bW)@5}T+o*I>0fOtehK@qsa@%@?VWUAvAd4Mo3;}h94F35Tt-k1EL|Np{V2q@
zaugv7$6azhoV-E|L6Qxw4TZd*7iX|P0DSgvhzNI?oU6P-=5@|$Lxkj2P=txqYU6Dk
zh;J`T|7p7wG!x6DjJtNl^wielqR;*znkE`+eXVA_?3~0DZ-Nag<%!6SLuf>!5Xc>I
z`&M)8+mpnL2^q6KhhiyEhuuO=SO?h{x%^gqt?5DL;s@<^)M#srd(1I!5;spjAwHI!8f
z?pJ3$A_fbj#au9aiva1C6%u)PJE(b@WFx8{#+WaiXD?Ya`$p6-dcP1_%;+ocPo-nt
zX2Fgz;fEt}fqdV+;bSYqFApzzDB_RVA%Dz{*Mgd;*r$ndm#r_I1b8v8CljEmX1mz=J~EsuC1!whaxeSc8l(W}B1vXhkOk+}G;(e}}Wr(!(9M5z=nH;b~Y
zQTFAt{&1HtRY$XW^_sMOpd>T+mn6%Uf_thuaUdswQ9Qqh2w8BtecvfLQSP8n;;3pdYk*LW`&=HutL_XYPXmUd(k11Q
zuSt8R{p%AsgF?QBvP_so3=+HAEaJMA&*t86j~IO5_+)Bs?yA7E0HaA(1P>l-nf<)a
zZ#O+8LT0tWRlM=A+*R=xRQKIDGv52riXZPXaE%%!ENu);cn1-X(HG;UPl!Y(a1@gd4EWTp(P?yxsoYTbc;<^+YR+d$7Pgx7|#IeXUfD+ZM0;fbin1qRh9bOX-PFYhy+WOSF)#0&`ay2Ql-zf%e}Hbw}a
zyp^9ki~?MKp#aiH&VTavYPbK6^-#w~aDtPSReKI$pG
zvYtfu{_NSQ99SV+3d_W$NtBP9DSNH<95G^yD~;A{)9M))r1vQCDdr3RbVf2~W8qX~
zXRz?N?WxW5meJUBRTg0NKtdqtLy-fx-(Wg_z^83h0<@){O24+Ek_Ds~uXLz8Kn8(y
z@c^s|$je6O<}Y6o7+jTV$*D|5ULQ0|#ff&-<#h!J98uDPRTawApoxMuSoi{;TgSA;
z9pcW}bRY*ni`1kkv80{IgZK286F@+vO`(H7p%xr%u9A%b1m`sIS{@u@aL&YlkJa#2
zXVv-h1tlKsd146D1mB%!&Hj3woz*ymoII19
zB+)nY)8ROJK@>}bi8y{Apa&zKSRjJ7vB1jrag2yO#T_9oE@yzGD;{OtIQ}x{Ij%(F
zW}Ya8Td^MO!qvyZoyUK~RTFu*m`IDeRO1KWHGO{YUG~i@t$sL4M4J4%7}^Z}8J-Y1
zbSR6F&V~k~Qsphl?lt*Hc5oD3pLtaY_CW`I5=U{vmBEWMBeg0xHyXP5j#dXT8G#M&
zoeVXn^=A-!iDLIw!bw|Sxp?Blh)oF_&&8Ie%9JXt%GRh|+pW#T*g-u!3PfGo%Qiya
zCGD3UhBX~C`X%fBFOFd|p+zkV=XR|d733^^7?NTEX%WNNCx@JKlF)X88?hI=WEJ#_
zrX1mNYnh!oZ+S|Np#`l9_dc)2k%5aeMax$Y`+e1rKQ(Se-sPPdTyhj0{zB1|k6soc
zyq7U!LNf>1x8w_Te_{9V$iFy27*9)BWqHdFIpJ|bPN+>*NRSdt*j@+n9>-D4=te^o
zzmVgPS|vdi?+L}fzTGEl?$}yfXK9l-N~(R<15-@%?2G$aOM(XTeMNqGN%G{J-ADZ#
z_b7%JPz$yRtCk;GtMRy9l>D2dNoCrH|9F;cxDmnRg3w@-FMZd=8SptwZ4RJ7PAu9d
znnw&2RWlE2jd7NrE_>NYaNC>Su&U(|X}8Y!l5A_adXb+OApH(aDiax5wW9aUr^f7k
zNNEd;aP?0UOuHj3n6`<8CPk+DT<@zMYQX6S>{ydRexB+xqAX9Ud5lLXi36*X32Y~+S8JW4l19sYT~@v
z$W7k+xl(`a%zvBc7wb1J@r!NzAm_(4r0hB(1ZqS_O#3zVf)@m?n{1APLwTA|i67m<%X{S2Cog4ri
zV|y&fsFxqA9(xrzW;TLymSbedu0UKFxM=xWBCz<1V7n?S_9U=^
zhiJvI-W>RvG``z9L2e+6C^Cj9XU2OgUiEkT1EJhWupz3G9CV<_nEK5SJD~oAyoQ6t
z+J>0Rbc%&F0^#Y%&OPSVYE{m?urA=fA
z6E)mQdg|JKmT!+roW)m7aRrE*{tED_7YMkS6w_&hn1fW1dbl!q6U^Ob4HMxt>p
zLkpG7)grt{^a)AegwSTy{-gEsYi?nbF}diX0Zy+P6+He-_SB=9S8F5K=wg%|FD|61
zf4xlo@27yY^%OlI_|2ZLJ~q_u4$0A$j5FUu!%otDpC}{!aauE!`%JQr`KERJc6H?Yh*Xpv%{S4EN6b)
zJx-=qw(k$)*Z7H;X#p(Rtox)c)%fv
zN9w}nQ2-8YNbY1HKIJ7=Z=`vw59ap_2Ual$>beHFUlPUEo21fODk;;lZUpdr^D&jb
zkl0I%zN++w{seP+D4TSwQf9KxA=J&L3382i86(?-yX-}_6}|$%$Pw0zpRb?Az`7W$
zq%(SJ{nZ4xxwyUCZ4glbQ@Ki2Xi!MFy-wiNI6x=~5x)wvh)SpYIn?5rZp+ZIQb&
z*Ue6>Jbs)F?G1Y+Qlzbq5y-1!|0IILQeB|;+<{u3;i)2%xFXEI|l#r&$6
zgfKQHww^R5l&SxBuJY}8l8|$*MYYb
zom;nQ;H_;a!QN_Kb>Q#+>=we=TeBxXEsi1{RoxFXcjPS_QlHW4^y2oBnM?_ous@Ps|K_pxh=s~i1q)UAf+SZ9
zs?DKfW}n>OYjxHXXkmvp-H#Ss5KqVXu^w&-1MQiE_5qx`#dhiJMYZ0qzkPdi(Q=yi2s@yJXMBI`SPm$TBw5?giE#22+oUoK
z2NrQTbqO#WLKQb2Q82Gj%yky0Uq^D-5v3_d7+7^AQx
z@-QMGsK-85R1UvQ`cps%?KzmU^1$Fi>c1-rm-@xSeTDYCuP-};diI*Ulh8PvU5UC!
zJDgg{!TGlF&cl4lG}>gqne1q$3*`};xnQJf(4(x4FP;)%0-2xGw0zQ_U$DEw5;Zil
z_g4MJ&4Oc~7WgM_{f;VFQHSgz+vBF4jzet|#5A0%@x{
z9Fi%k&b_u=s*D&UMJV8BK4|UC4MOiB6dQWYQO{xeF1ZdL8<_$BUAHttc8KL#VFkY7
zgPbqbkSjopYuAJc7FMLt$quln>2^cJIp7o!_3`|&IxwVmC8K;uiiRd3ns#WrV^*uREh|9M+3|6>d^nb6zMM}fW!#19>
zo`;5EY!5kZvsAuMn%XvEzBl-!QN+BVS^Rctv3pHe9fo{PYUt=u6=Q2v4=iaX)gia^
zSpx-p5nQHQ2$0^+`AX*q_V_?`qE_MJGjxBwZjW2><#{%iM|ffGq~=$Oa>!-088+vV
zQqoMK;#;-&r_L!hp|fpuEmJ$Kt!4)7Jh)4|iif@_8~HCBPNvb2SlR-zt*9^lgd;I7
z4C-*)BfO3q&##)z6Ai&Vsz{KCdDPguV}0&{Gr}Ky=j%;HU#YN{DdNrAg&TwBUzCX{
z=Dml<zHO#9!gF!jA_ycc2tdq$t~jF1uGT>w#<#4bk(@@aur
z>Lh7mmHkf^s`WhF3fo+7g!rya3z1kxCNV#zNXbgPM_bkb+3SaYbAJi5ZTT4;KD`x0
z_oPJ%7NkX!1g2MXdmzh}-Pjd)o1vH&6zu4;n0uV|zgZP!Qzhy@vjF~>qHhu!e2a+G
z@`rUj@0+(o9_ra&?j6`b98X@A&>_y^M)Q@h)Vw?9B{rGz~Nb
zYPtNcU8_fKd5)s2u7)0od*<5e?W381?r}FQCDiW9=bQ+&yZ%$ILN~CJ(*iAS%L(7&
z{yC#;Zod+JVu?wzA;(*&Q}9_40#LhF2T!?2Tep@Y%DpDJ@DO8rmXqmjbh|+FO_57f
zX4$wNeX^>1&>x*p2`$>F&(7H4#HwussNnmAKtM)zbCfhA?!09ZZ2OIFE65D#d8St@
z5_$hZfP^2v-T-x#uQ#4#ylJ5}S8Um+IPizY*e7~qPVIVU7iAGVaOWy>-Ln|vPu=Pm
z)-JaXcQn&#_+D{<-^guyYpqX~9u<+~2dH`4>s>CJ79WXQpYiv(cf8C=K2XXc;PRyf
zJT!)&zbOU$gsJ=Bayl2tJ~KL;^3laF|0XSREsY6x_EeznsFlE3;|RGMj;`
zn10dIA2r{j>ID+4?-vc~{va|%$wh2kSg7z;N8VOcwC%VNsgkzK82(sIfs{@{r@icS
zj^vM>L#j?trvZG-9k(D$JHYD?NBb)MUY?6vo%rE$v_IBlCxaMdjhg{k#zgt-R|eN%2qv#!>8xdDEQ9v7jDXOL__
zHEV1#GB9BzYcsSY($O^zU^giNc3rWJofNpHK6lOI
zaJwIu%vLk4H7pg>E;d3^F?40K{0)3;%RoURe*THi)r5r=MJy*&3cp8|5_s_sAZM`dJG+&X}FLlD<
zdYv_xRiLr~rO8lFWfMw-E;cXkILW1mK~VO!SGaoha+>C$wODgFD5HP-Q$`2QV^g)0RQHK7Ye(c@b46eX~1*Rbnk=Bt!D8O3ZIiac_%tL9X;4}gw~s%!-D+63pGAzJ;tsFUidfT}blg+tW1
z@<2vIV9P2{v)S$s-{;42&_!czl-IjCw6GfRvfOFN7sNV!(F}A@Lza>v7dgC!uc9?D
z@7;YoGy$ts#7c^6u%G=$+^|!Hx)23&0^%^5KgAvbn6kv8S+N|f>JT~E74s3akWGnM
zgt--$UNt}&TyG|CzopzRfP0*nZZjdf=3ndkd}EUIVdWsXt=Hd}#4eavSDLrnaTD0wV&(k@YO`D}H^L%p%VY4q?{?KQTgMIXHVzX%!
za~@_gv?9@Q(pqvxFHck(KX5E=vt*uvAe~=F#}oZ4c*2kU3LrE)YJY0O;IV7;5D})b
z`K6P?$-CufwFM40!#Iw7_o^_M0MoN;uiP5g&Z>iv8e%o0OT*XJC@04K*}qwEe0@EN
zu}Wnoq=&#Qv}Ett{~xHelklekqx`1=19l@1zkM~y#cQoj25&8)pg9nhIb`Cde!XTuITaJ#&Fp
zc{j$o>J(aB4D`uQX#(n38|Q79Ky${R4)^g|7+B`N4Bu!6mtp|CDP->9pQv3;!$0FxO`L>hRv!{aOKiY2;rYnfnzdz-D=I71f9H1d_G;vC3y7xcS&TzuZ!*o^&jdw*6b5##
zxyKC^(ZshBp@Y|0v?a)pn1RZlXj1fm4NV5gKo%kk*M|-yO5Q+W%CdxA97T0+
zkYzf>wA3TcZa$Crt2IZa5{>>@wfKQoN52bgH!H9}$vQDh<_J)O?^VO5({Zhm;jac<
zr<#~FzeyL!T)+rbNE3^Rwe;P8;qxoMY2P}ysy(dlE@a%*1
z6K*^U86M}&GRMNSE!?FXufE|xoa5sJPMe;U4cGiu!bjWzz9yVK8_@ZY^GyM@IpPgp
z=mLuRICe6gLX+S3ZPBr4G@rY-12;#BOO*m3u=ZMww`begyWUds!Yyzy8yH@QO5aM>J#
z_uv7I>_AF8((HqsB*9d`Ofsrn5@+632%Gm(B5LIwurbpHH<}OvV!vkv?gv0X3lYD0SWu@z_%4NqLZlNMJKj98O%MOXqNbRcM
z8}LvH4rZzn$-b7lGd~iYG4RB~
z$Gf|;eVQ~)<`vZPs==)`z}+Wv9GU$;Ncb(Y{Vk%mkJ2vOxy@eGbn)V^IYewk5z?u7
zDn|n(I|{!*SJA$~C+-#Zp|9KF
zMcqyTnsSAF3;0?3qVxv^dc@J$Wkn3mI8pKgNq`HRIb&9(J
zvJ6f%+)DEMY#NWQxX>`igxTBEF36^Gn|%5S9LpeR&YF~J4Q@2
zq}sT_pL5jr`2sW$1o;3*WfOidvzk{{x{WfE=)@zkg@ltO7JC%*+d0)jZ
zs_Xx;^_5{!cHh^`(A^~|EiDKrAdFIqGy+2-EgjN10@B?j2ol23EnU(j-6hh}A^E>i
z@$vUw?^n5yGiUbMvG&?)m%WW!RTz>{C)3*ucoVoLw%uRBeH)q5-j9O4pUfsT`v{6c_Mjbav}
zh@^)-wh}#RtJ!*oUe5DT-q3{njgl(~?M9VR1CW+B4;9@MuV#>WG2s+Nw?OR7EG
z**7RGJ2?uiam(XW`SK-7rAs!&AGJJXe-@1U_P=WOPX#+ef5W6j$#=W3UeCiSo+F^a
z{awHapoyb01DqFq!}bu>BXvxL(R7j&rM|iBw*f0TK>HrR6rTlBg7a(A%)&GXU#fdM
zUtDj^qPlC2y9iv-+(+}qoKc=3x@V?7LLEl$^1M=bX~I_Dq`EKpqsK7I>*Va$U$d_t
z^u3ABE|`9hZyjflBS(o_(8(5MVQDUpW4nWmgV%PEX8*)nj4~Z5s`}f`A>!iFoc+$@
z6SoQXHGkaqKK+k%U=72NHUe6^niG4pOY^X=%g4m%)h3oalybOvNoRhsPTtz}IFusu
zFrU>a{8xi?%zR)w5Xb*2py@dRGY_fSJLz+wps{qjef{6&84N~`d_9p+hB#^aVsJ_`
zXT%`3$|(?HuF_$IZ#s+VZ?0fiP68<+R?lg+8Zt*PPjm*PQfpp}7|6P_Gi;QEo#ZMq
zQSk>wAqeq*3}CHP02j!`3w)h2aA2K;Tc=n_RO|Spl%tuFn^J{Gp8xY&Sh(rI4G3)J
z#r`P?NEEwO`#A*d$>!z$G~^x*`aT%TpBKThA=E+{>)aT=3vtlgSL5-A6Uy
zvXMm6K<%!N4NY
z4t2_4)T)my4Bdz?gT8`%aH{M&U{6TrIJUnuEce+2YP+oCXA~9~qe!*B_?Hc@_X_ir
z2deI);wOrVRh;*`vj!y02?`L-k(fKtp?xkwk(BjlvX4i*Th#WE)H&1PM_6(%Y4AX*
zk9m4sGUiP2N_ktACDgse6~i5hoU}Z4f?VM*Bp#0$5r*-`;B+>v2MP01&qbk>L3i`=
zx-iJ~Vr5aZ2I4e}SE0OH`a)_*8rWyF34R0!9sjaDV0d_X-h4<*i^l#yyR@%+PxzNh
z2G)|3o5~8T>c5ynd=?->Y=p>qe_>7SorTk+O~HzGN}h+dq0yaK&!l3_-u@6BHnS2P
z;-VfM67_Jtbz6MrP+X&IMT6nje9$C9AAc)(v5ZHQqkXDg>jMfC}kSh?g
zokXZ=;?e)(>X4+GNC6~rY{9vILj~&NpjzX?`|D5%^q~0ZPlkmJs%OINh})Txk$90o
z+wC;XNpQT?T7jCq;WtFtDo<7lKXOzIKO$Xvxkv2gM!2x`68N6>
z-m7Gyi#?GfqMk~52PY!j&_ErWT~+w}d4^gmf4enb9|jMj7P3>K$1}oE(w^Q!Y4!Nm
zvJ+Op4L5BHl|^yhro5J$*&3kuQ&`AGHtZYSI%SkzM~D_tLND*0f|sgqKYlpBi^KVV
z+HW#})~woC?poyC0|+TIV9E#Yr`<5PiNT6&*xcR*E@R_AcKEf+}kS4Ar&O?y-HHVp?%=69Tv3^9z
z(z*BU$8~MyHrF1AxvI1;3wA^hqxz8PM)8MPeKpB-eAw6~_NDXDvuGtbh;dy~CYdRA
z$-Qqj|9GFL(BCG5_x@wxJAhI`*k$v(CCknL1G&-6v0hW*sMT$8Q7yP#)%mVedrc~L
z%BM(@u(=nd8T5Q;>jL>s52W)RD
z6}#HPm4Yi24J*zxWP1=CX)Gm;3`9v76Y*&1W&RfpfY~CulLMn@f}HZEJBHqT<*WXg
zzffS^u8d4oT0JHYKK>-JGFe@lFH1|n0Y~Thq$1kh*~QD
zIHRJ9di}eW_`434^NveqGe$jY(AnG(^$%CXfN2~uo-f;l$EiHduB#gi7CI!jFB%am
z=&Co&u`T(xgofVc^VL5iO)Pp|f>nNIar!^JAKe?mk;-WqfFrT~;$`1FEn{s6it>r>
zPsV;eN@@pd*#wl$vYqz&7b5H=ZAp6gY$vofj=^Z9?~mV`cY?=K@F(HunT32bqA&T5
zmd7(BHy1IfA~0WtS}+FpolexNL*ENL_jdh;KJzb<+9XV}JE~{ot>PEmi;$1;2T6s%
z9ubFlnkn;OD`Mgyb
z4~;kk&yt5;TdICp!pAfieWTqP**#laixNq$iBMw`*SIcTKh|lK(a>aFS8G8`SDi%wjeJuKfY7|m_U*LvuzG!AemN{Ngs#$vt2Bsm0T8$0K|J0u(N!~>_
z!1OLrtamoOBgD?o8DMcC9%>9MnEL9d+7XOM?pcT$66r@rPSZA(
zbrfA0B5a5SWqfRZ!v8PF1?=iN1p@;dco=2(`|blE^MmGz)ZbVNcUaR(Hetq_rh#^S
zs7hv-B)Pt={1D+pij(M@j1UBI1QOC8pwHK9+c~5TnnsURTQd|_Ibv}8Qu&3sq){pH
z$l-r7=Q;T|gE+TcfLUxj<7u
zF5YB>kc$|mT0=&HSda<*7r)he4~V3K+Akbmo&)q3hAZdsq-K;ucQ7=^coP?j|EAV%a
z92WJH%-lP{El<1leQm|<NY>fxdRA*vk-z#y)DJ)edeX{qGq+v_KH|0^>SP{F-=
zRQCgcMfPn@4F^=^<>l!`MYuIJHBewvVF0zh2c}}?_NMN&AS)Z&9w14-h)8pa&}+==
zxHDIfQf4;Fn&p*#DXHzAM3c}gD9i2mhwmh#4B@m=i6|#Xw%bwjL^Z`DFDMQ1wn)Vh
z2*c%k>ce|yeTq}uVF#onf?-OZ?GpcU^6}TTKGS-ql#sEp4Hj?GnFMx!$8*R32cuo0aVw8FAs+abqdSNiB?utq-~CX^_~!8VRd>Jv`McQ^B7hd;}2#@23
z79P+Lc!eIJ=y`OIp1CQ(L1`WjXc|xV?+8~vA{N}XQO|mz^7ypHOZUsFmkmB?VX%ze
z|5Ql1P`%Sef|nO~Rf31s?#mR;-y)^b$2XblMgS$sSl4{>v5+eaz30yZl>{Lusz#XQ
z^S%O0e@qks`Ker!g>=(0V_N(C?16F&yK9k`JGJlPh#pFVBE`G-xu1lG*gVj+4)Up8
zRE@@R6kLltRHpSd^-@UcT#MFT&gk(Hga50bX+|{o$#$@rx=c=h&qr&76ATz=A)}Q)^*_0)r77Eug*x*R!wHA2qq#Z{6$kLmY|=o^=B2bx}rcPJzN>ik<_^G
zU~#$qHm}3?tK#l|{&z?d7*MDIoft`BDCij@XNR%(Z(XX`JCH;85@iHJ#;b~|s?;8l
zm^M;ywo%=tqq&l_L*-~f_E(gl9xD%rcXx8_og0b1;TeJFEN8=esH_cS_Z`_or@;ll
z@OEg*#HlIA`#SnN(=X_Xugi&d8FN6J9+QwovZPrsy
z^otlV`U*`~-Xj)@)D0if6+-8lJzk`>B=uTKfBHa96VYu|{QRYknFUQ#S<(1n->~Hs
zWHU=c=$m=ZzO!%zJ!Z}8NLDu<9DuZU3)Sh}MF&a(?=mVXD(w7GSFs%!3}loxX`woY
zxb4*HSN$7B1U#by#+rVK`*zW;U$wn6-9FvYgSmO^Z>?`MG>VB$nQo9>)=uPx{SN^K_UNW`gd9di5oC!s)!45}rrH8489EotJUtsqMF%}IIa_dV
zuP|4acI7$kvi+c+%dU9(FP18+tg=^k{CPKp;
z$ptnV&9^k3p2Ykd!6)-i#$J0|&hlTZR}C!8dpS{U$33~eZ*}WUZeo%iCvrKk&Fd_W
z67sZVO$Js&Ibtzio0o@CQBeV?%{K!ZhG@$aGBYv;nSen#IXP9c?5-0)pLP)t!|WEx
z84eV^Sj6pa05%*^A|nIC&%js9HUQPuE=YMF9c|ARzgTnoVURRwe^{lTTj~{SAZg4(
zw?x82H?L5AUNJp@7Pit~$ydhBX;6Ne@uAjaE?vTSH8@)xVO;E=UAfsVb$>N_=68lO@VJiSjbu{%-n>-{F$HoJ
zQKqa%o)4ee>@GaXIsx{v4GZ?>`pc1^9$?E#k*(Sb`xNts
z!e(TP8uXPMoChyQBv+Nmv1`2Ik-b!DfSk)Zp|Rt0Yu`Wq6IpvpAjMAXJ=fxkXS_A=&NaH#A#ih0hwb#(s}}sOd#!Yyei$FEw@<+^E)e&A>mg
zmLbjp(J2n=OE{&qZmu3KX!#+Z%a_w?k-nQ?p=vnIWP50nr!rz*(Yn6R!d2t3p&<~a
zyWaA}$TonRXH$iWcR=y>dxO9R34tNV!Q^OfsWz-J?@tqdCV084z_GeW8}F&_h>;Z^
zIzhgkUmDm2r*E!LRTb}aa}7Qo{65bmV7C;!Hy9*OC-7=L_ws_9o%RJkmGF3&E@iHS
zSaP2i}aT+FCfJ`cNip9{4+lS9E=N;l7^3x2!ZoLgf(X_2lQ
z_0G#5@61^SS>?a_zAnCZ;+Z9x4d}4#OLLrJhfJTm4;=Qow
zKlKK~pqDz5xw%Jz;xnd}(=*jj@OBy||
z9~`|575si4!C9&;TJzI9fRFLak)!$qwebxWzndPuszEhMZDYmjX7WAQKk$T=Hh_z)
zeEn2q)y4MLRl>TE>Eb~zJnx-#auo%S9o&ns
zdyT!0{dqYutyn6f=dd)&ODV>lQ7<}y+Y;bcc!ZG?>UleSF?D(MxBTaRO1q9vd(PG>
zv~i7W>f(5M@K7_+Hh6FS#9g!NnQUl3iZBmEi0C6#PNQ`^R%tf@7_(yKqa@PBnq)o7
zc3iXfdD#uw`)@M#1=xZhsOp>c_mefrpc{G-KXW!c-N;JY<$TLmXEIZe^=Wg@uhz@k
zw=>2zJiJPI%tSkPyKwY5NYvFS4N>0X0L$}p&Yerde(wPTAl(5%fBBE!j1H1sA*pfG&@N5
zGR)XIVw4XrB>&*fQ%KS9!e$!F`v;w@h{d{BrgwvQ+TF1#bn*#B-7D^Roo(|vZ^}>z
zQ@7EF!*`a!2#0Nd%^Gi7flVo0^hUQ{ML;
z^*C`?L%`~Jj^GYvnodb$yMx#XA
z=XLbs@%rN_54-@ayE^U5N0O2jBAf!LkR848>JAT~)|LkumAMqJmaadeXxPGCMFDsv
zf>;eH^Zk>jX~UgPp-fmWN;B_nR&{R!sA~1h90BOa5^=%kl8zc-8)rM9Xg7&ubJ`QyS9?seaJU@O`$O7
zXw^b%{`nDZTEnF#;Hq*rSxI54))B=+N`9FtZj1ZX;^`H58lgYlZQA?mksfKjAn90|
zbLf(O4eQSK^ph|~iVsTPoLlarL%?M34w$XyUV(X-TP~
z)X#rdgL-$-PNk6mu%l(t_y%YD&42{atrMR{rmE
z{-*(kFP(IJW{s5l6*tOMx23`h2p{dWFKzC44|bY|M6ikKjuFG)9U+w0VX0e@ursuYZzyVCs>x+qA`vy4GY+NEEs2vJ94
zW4nWEA>*fYEzRl6SB^tsrh!r{#qMk5^tnX*a41-#@L%-tK=TIj9pC~0zxTp8x7b=N
zq?f5=L4||XC%@F8e!b5k6uNZj9(=t~z6gyfl^oHnN8G*)T^jwUHMCy2Y}Rv6%R)Hl
z+nNkR09>td54q*&4*Xt7yS91M;N&JmwoGMxjpTU&DVy8w)F;)p4qti!S)`7CVT$hs
zH;ChV4FGWzcA*sf9Y|>LV5jE@J+Bn$Ae?%PN{m8|h<0grIU>psHwa&I8(5BMORJh|
ztKEZ#iLBHC8642z5@Ua_5(+=1OE^>nXfS;%yJNmuS8}xJH
zbX<6&F0emLAF0=HB={L~D1P8S&4Q=ez(Bu3+K1;~IuJG)9tw;9j%f+F;8z;7i+9tv
zPrS>N+n`i*ou5&kT}4#+Lh#b=KXS$AU)}b7Voq?RUevN;%c^F*UeI3B?XkqRYo(ax
zfd1gleWprVgqNNX^y2g5+y%vWEaM_ZNnaum>
zL^e;2C}*Xt9#<<2d4fzfap~@{in$=aX-1xSK33f~5YW~idT%%2GoaYY>|iQ2k%(SFlpDXq4~>(K?9*WHMk%gb6Hf;UjnNE|
z%~Zxf>F;J9)81c47xx!JSjcU5wS>NkXK!XO42#Dv{NBd*I}eCO1AItS;2rA(tm3eo
zmOtG$mV{?UiSlnxOJ+fRAQ%1tPx@c{hMW7
zZ?}KK<5L~LXd==w0#Q2=Hrx1x|HSNsc<_*U9WcZj=0a1nO4XErC|;0M(xLJ;*Vb<$
zqKxYDn3^$DWlkmzo5fnl-~Py2Y|L6V1Z%EEllPe0-g)s*oRE4VArDEZDCi81laotB
zt07j4?H0z@x^m0{Q!U^>CGWC7ou~suCvG8Lyqg}Ma}v3p${wIO`6mb^69?3yt_fjc
z#kdp=*3fV_0*F9$Drq6FzQh?*mIXxHsU~-5Dl5f=_{q4F`MmfGc@8^*pu0Vo
z#i3lZ=7rrBDm8ejjeUUt`me%Ym3wBh=!!chIQ;GqNs*s6(FpMi={KFfDH>i9aEy1~
z>yRF>qCgREc}nR`7X%S}YJ}8Cg2$jRl06H~l6uV%0}rCXj)Bv%?ov>)of9q3Oo&;Q
zg|}FZiM=&;SYRngBNRv7R$|ciwXr+9Sv~B&R&Z#~Qptyh)OL&cVbs8P(*Pp9V(CWd
zR06Y8q>oLm{N3XO4fu5#WB+>@n)rjhhHXTix_gpS?a@(?5m^tSm(LaPFjB8eS&9<
zR=M<`R6iCoLKOf0FP{&x2IRqaKHKxIbOO|ms2hQ#U+&~U4PtT}hhpkMf(Mg*DCGT~
zkgoHLf%`?U9Qhk2c<;p9_UBa}3iBy}CTviZyQrc)1WC6RNSLr(3((NYLfL4|XGl1M
zl)wcZh?zYoDWpb-hLXUjC<{fzg9F%sEox`#ZKez)&B`~qIHTg4BY^$9S$#+Ur_SmP
z^1h20z=Aa8xmrJ~$HxuDWDwzDt2&uw5TQe+u>axnoF*_Ytj8u0x!5qeLPLJ6uF$YF
zyZsx>`RTwZ7)6m^FPep-*z_eRjnufCiEhKY&IpD<@6FI`=pX>ybg5=^+fsTglmK`$*SCB8(
zm(qV~%HDdwh1-jw3Ooj*K&_L)nEzDc-_sy19vLW`&zl@)&n>9jBtpl85DfZLgw3U8
z;s9+={g3Nsrh_sE6o>5+jJL`dCv<9$`
zwnQhv!KZcX0+L;3Dm}cmDSu5sPXw@lye0L28%qetZ#ifiGQxfRz2)$s4&_1l!GgI|
zMFFmGT*R|By?G7*dMI}#$|)o)I?5$Ig1T9xcDr~u2IuyjCG9^FwSDvkxCW(xk5I%P
z^9dMY)hKFtZi{wqF;`g*9?k&j$Iw9@k$UI;I^Qke`wQ>;4nQ_z#b3WFtq4@KJS|ZG
zC$s)4Awukd5+aX`3-vkd6OtAuDJv^0?X|OB2{h)1aT)oeM~_%oSiDv9Q#S~Nf46pk
zvFj6Y>k#}c6%&acAow+EllkzG+|D47XWeEziGk7JNXZBnWou5N_|B9i@H-@t*p%-WfnFpwX){Jm
z*RuC*5J;%HA)ef{Z`*$}m%iB|8-CBjDs*&m3U|}`Q9fy5YGoyO`nw5}
z0Bwq3BB|CuFK?hd3*fppDZxFgfx+N`pdb4wa3I-_UeX93UhpimU()ED(Tm?4aL)kS
z3FuLikeBGgI4hqc+lfE-J_kr%f@@FJJ<`6a#o;_uHdZ8QX6+7o`c&rh3mN?!K*+8M+*w2=xt+ho=O_&)ACA
zap(PHpXLM|W6NOACk|+2W_Xtf6e!q_riTP3LQ!61@i6&MhXDn#!l*5|3wjSeJ~c6c
z0ruz<-^~XT3Y6gvFP0Pl>PB~ak)bk4v)~XoxzkGfJq1Up#!vq)g;cSy+o3nu&$~eP
zH)u5yNRI)L6%px$!}Vclb#Fx>=F`E+$xnu5-Pe0j4jSj@=dlAg{)p7!Qe9R??>3Q~
zXMkxjc41h;QHD4KcaY7jxO>e0<%qs4a>lr;=)hj1YXOMut3a^c2G7e1-7ns)$$|G?
zpi$<};kpR3H9TXO?YA7;rsS1%ITtSjomf7Db*1~UQ_%9NuPkB2x4(t@_gByhJdUbR
z&v|M1UwH%Q`vY7)(wfgMt&fDFgSGm|A6ruz!bo+TL2@~~rfRXlul)S|TlKh4=cTN|
zA2W&ekdh8zOcC9Xn;|Ud&f=l{J>g&~F5FOQcCzXVVnt^!6P8c{#~keQq6-3lRvnV&
zC)V1EkDB1sPu)$Abm{w22$hzxN1Gp1;x3#w)`!^>&$ZKeEXS-Mi5x+2=p?Y!{%P&O
z7BEgpYV^JNste2P_)wrGeu)B!$S^%rsI_A!utE{DO^(Qys%D-vVWmY=fUW-kDA&{p
zBeFpK)`W?0=K87;ad1A)I~k_UXZ}-#JQv>&PliU6SLO?_hZpCTm~GV-Q9q0ZAl#n@
z1`QOAkZ2>)Ed4mUGeNRJg
z680CY-8;bjLBPkV3tLPjX6aRP_YUtty6!i{Pzn4#Ww||5iylPLP_%sU5u+R1&@*2zXg$ceCgw|3yUZN+
zeQmoF#cb;o*=xLVjdS*jY0~-psnGkM9$8-a?2~*NF&L+Z^flFPvil4s`*?q|&vXba
z5N1&_h%!8YV-5u_7H4xT*wxUML2GMknI8-A^OH;itBgwE@-+%$osccVY5MT4y#a~n
zW_9)VdU=p7XOC|poOgIh%f7fN(IO$S)1|v!aW)wo;4n!VRusAu4kkU-vk|~)<}mA|
z&T~dT=eh>tRL#ymF)k6Q`EDJ7-(#UF+RZ>wugdIMUWwQxW$F2*z+`^m*t4hnlmXx+j2D1{Gx}jI^Cy<>gJdgD~0rlu0wLG|c
z&RtM{1Q6-ma;c;HTnD0kbOSc&Vn=s=rt0X~CZy86E8dEG;AN(ZZf1l)08JO@VPu
z+*zyT8g?w?F|$9P)uJE&D%_z`n%XzpLd7rM*$AsL2h%G#aJWJBz&*em}=#}2H_2I
zf&$GUz*t-cPPzNd#*`=l85YQ6;sKiI0$1`B3&qQhg&!Gpp3O01d^rp2ahjBB0N8IZ
zTJ)Em__=Pbr}~fP5lS0{FCK|kO`Y9|nS!fO#b46LRHz#ev!d4#N;gV7!4q-8vRlc0%#?Q9#(X^=I%9?K{l+S49=S{%6hnXycLzR(%j
ztth}kb$oYxmb9H@q1C+6dn92CkIKaqJ9rZQfscj~`_|?+03{HW)PwAh8=BBBHLh69
zQ)!A2Qr}dwG+~inH-Ky^)eue^Lgkh+S!VypzV0Lhy}JEcUZ)OsjRaHso-8VS0Q5=I@2*G3L^9y@ZL6V52^qCL2)r8XwIfH#rAl6(Z@Ev>2f(NFb!
zDZ#e~Awtszz6~)at87YgM7T0R3D7FTdkdzHyf46mCS<1>$47SVvei%ne=Nk~Hb&~6
z%E~QNN@G6oZUL9U{~NK*F~aH`=3&}_=a@Js{|ZgKNX0LrihuLP1e;lzpiz@^?g}G&
zN3{7h3VP(H25t8=afor*m7zSJBMp^y@)1CXm$p>3CwfsX)6yY!n|ZS90Y_312F6kI
zeyKYD;=X!2Sl~J44K%oxm!@Iu_N!n%Alt<34s$oLaV#{p1kE}Xqd%-ACxai0l#7tF
zJijp}=>aSE@bJmVL#S%DFZr{xS)Ew37(3_xxNt3!f*B3N43z!eVet1}*ni#2{NH`A~s{
zV0j{a7o89NVn_PLN<}&
zMEj2Uxm&Nm+HI?LCqa&xY2_r?#Kt9!UXIg4S0tfKt2
z>i$;=owG79h5H0J4!kjaHp)NNK-Q9#9*
z&%Gul93&x%YG^2HRFJVP{9fp?XsxvJREpYMy2XFo6NLPn?ee_(eod36$<%J|PEn?{
z9?LH0=hbYfz!!-kD!uXF@0$=(wr!o!0@R6`{_K+e}bz@CF!
zi<->-U!e(CzxGNUk7`CaEzG{Ch)*R~|D7
zP>Av9j>r_8{Vm@b*_Bu?00$if%d<;L*V!&byZHV+P_Ni86aVzr#J#}>P)HmjSu(y5
z4<1{Ul;`nL?33h4Q`h=Anm0$tdL3>j+nT!|_3%@{(-kRLnA*E+DMJ<#QCtQu8~$u_
z$se!C*P0G;8vVbH@m_6i@-5yF(QMtWd~QyqmEK^9Fz-@`HEwo(K}8>`;eI^iK2syu
z)fi53dqoO*4ZuqKyd`p=bQT_SbB6`tj3w|%}v6VZ8$S$qOKIN^X
z5kTbsq*L0~TxU@TT4;F;8NlCJ_6uCqDxnFeRjvqULp<{@Y7BWCe?^mG<$wJsNO^2A
zkk2dB)F1_TUQ8r5HIJQtgbE6uyHZK264$uN5;)e2;^Irj=_EkIrT!LdT))r>rhm~1
zJR(pc$0lZj+~E(*&a!XEYJ7;@e$PnfPEqsfF-ajW`2dyGt7T>VryUr131^GV?{lKBd-peWB_l_
zLv!z`tz-HBcmwyFsMcNvG_?MB10ZTY!jEWHNxZj-`C0DxhY-IqNIj7ppzJQ!)6vmg
zoSX3CvGKlTV`Wv)*2WHQ&*+Vuw2+jO!@@+jxa>1s
zjSxrU_xCI975ZglS^pRrk%C`Sr#Bp{TJ9){yrS8vbjus7`s{5XeHD<{$6oS4-SYd)
zDxO|h4b48T6TIj&|MKOF{!OyC8z3`KMi)KGH!|>&CSugqcu3v6wtvS$@Z?g
zP3ZLEZYIBscJ=1M0EVmCzQ%8xglUpSfK7d2c5b>>(KLb1M?H|3OH
zPs){YiJM~WJbjHN7)pH;CrO+3gJtT6zP@g)Y6n9BBG6@OCjFb!7Jm{0AT_=g`HzwX
z`w0|A`9UUsir(K)3#JAHMa@VSO70w+m}xL?gf~_t3WY({gPKTfLW6?tRLl+zdIS3n
zkRC9bkHMcPD&h}b@!0&lZ`-dBr$2)Ov;(bZEj-!H-ejC1czRu-0=)H0PQV%ir{G?p
zoi<}cN0~CEcS)OEk1GU?@Yze&t#RkyzmtEQ;A1n}_QcD}bc^hCX(2Ww=~3>_U^nk3(@@e2)T#Y@C}^IJIhh3DP3
z@K>$pFY87D++R24A`Xg>|Mi;4SeZ`b^N-+8d9B6yj?gY-PF0N<)3R}jBH34V}Ja`&7XOAu9I*g@7M<4F#2j@jJH$^(MhHH(71uwmYARMyjyW3mqUeDr#wnsBQYtz59zSodHQ*(+5-kxy3KYf`WoEZoaW`
zajd#Ck&WqPpTvOmVjBs}VSg0)xyQj)RhnI%+dDh{?ng&QKj&8}dxPpHnR>9{eAh_p
z6U&Xh%saaHaX;0r1`b{=+YO@&d!4zhRh^vDAk?S3Kiq^b4xR6zJJ29idl#OD7-QM|!rEVO0pt*xzheLmRQ
z+CF>!T(MN|;NU<}OAAZg=g8Ox?pRC3SKot|U*xAg|IH(>AF*1&N=(+RoB-JWrQ_mN
zpehtBPc@^-D8DmBhI705Ye^WqvZoKYEHecW1ITq`o*rWQKX;RoX!
z#fL(%IdHSerNN&#?0tcf8I#=dX({0nf4DA9?j*ZLzxv81iT~TkeF9Q%t@!ks<)L8v
zX;gId-5%mjCZGH{l6Pk?zGwVeVjl~|?g4{vkHYrjkYdJb6x7uN+0+WtD=SG-Q&Xk2
z)yWU}$guLxg6b7%V8HMVn?a#PBLnp;qCXUA6JRA}oKo+{Jpo$#V*eF=aqY{C3C%fGpF!
zv5Ki5+91r%&PIoE^7E^zFLcKTvJPx+Z%e4D#oJlwIz3kXd<{0%PPL=*pY|4*ZhR{z
z^`m+`c04dVT+OV9iHV8z$Uv59xY|R^(nMk!S=enusCc68sPfxs=~B?{2;e_Oc=$%%
zQIsw}a<0UlzutC$qWY7XO)F6PH+tyb*QYzWD`bTe@;8?VA=-^a12XuJLDH2#i0l%#
zP55Ic#Awh0)y{biCXCYVg$;lsiI0E3J#WVcdpBj$m-}2Hg4p(<{w3)H2PqazD_IGA
zfDD|-xaro_{`qOe5w^zeOznL8Xq@uZ-8)B-&iY5X21Z{<0EZ~U2}H+M0Z)~nub)M{
zI*>|EN8?460%Cv8A&*|5{sSaERk)dA3-rcIN%{1yjR2&KzV~h(77xgje!c#ZiS;s!@Lu&I+;;LR!nXT#m#v@6wfA$uB67
zb%{4ZWiq$>0E`qoSYagHjCwMzV9J-8b=ux#9fVx0anahvy}JAf+iL*1_<~N^I;C#P
zFDH%bfXkb4pjy_j#Pd0Tr@~eLgADP4ZXoP9G_>}~M12{+*@N2t^9==%4(f%k^!iwH
zqf7X1q{aF)>Exmcz}w=Pn3|f(tDBmhUVMX-D%2m{371r_9VNAiJ^E
zSn|X#5218=cc5FhT9Hj#nBqj>FEZ)N#5?ttDov0_5vMHT42SRbI-`(EzvsallaiBz
zmzMND=H$>bF-hK2qvh3^wK~Z?Fx;MqizCU3xq-ix#KxBQSSs{Af^9yo0dzF2+H#H@pwu{KUeh>=TtckXu)hs
zXQA2sp2)xA5!-1xYHD@yGm%DqAm=t8frNq^Zd4HSY>xkzt>ni<`vTEae0JoAK)t+G
zJv)I$t@r79c1*-%uAO255T(9-`^ExNQzvOM+KkFfXBgsH24+j#R@
zz))6ok~8>D0nkM&$>t`BWC9d_KP+?pH9!b}em;WWj&YFdVaG8P-AF^;kO7-QFcRhw
z9}2?eqUWX$-{U<|qt|Qj(5Q-iBB1WW$HzwuYz9}=0Yb&#`kRR6vB(u9LlSdqYc8Nu
z6QgHtV37G$iZhClbU$gG#2};t-~$idGb`mkdC69_NwM_>7`novi1fv>pOe9TrK2Uv
zHy7YG#*~>?Lle3^=j{p*e1PPXmbc3n9uwfQm*S3gd)yI@U#)}iTSVyBze7#IdqA$v
zw?{w@FYO9ZHNe5EkpAh+d)?V&W?{k3!^>M@*hRpuQT`15{nWN2V}js*QfZwxJ0oom
zS|^s;q!fyVQIkV)GJ;hs^!TB3Ar%0MN|OL>oEIoakS@-@fi?mt9C?w1DB_L09r$BhaBn}v
zTRPfO`N&=FL3^Tz3aHWM1g3Zq$gp5c(}Jf~B@PS>@EJOq11+SeZC3NjZovZjX#gW2
zu!s%D9~mESU9@3YedizEcskZ;Z*45ZYjr;^%TgR*aUD+AV+!y=Sz{sQPrgbk^3Wa|
zbJnK!M;_F&3!vl&>)T4kpsiK8yg<2`uPeA<{Ub6R|RV&5>}#2hsUDPQB(h8IH868QV9LSDs>Gvh
z6j4}IByR}mHj1v3jU&0~hrND%&yXBMZV_vQ_Ex)$Z;Ff=*pLxyWlnL&tq$}ZkROP_
za;7p?2J`42c!({XGk~9$pjha}hn$UpMgrq4QXk_e@}v{LMwk5dvDC1M!Tpvxl-`5|
z4E70%6_?qn1HBKi#FN{1NSh0`Yt9#s?3%iYaw=g9Z>2Dlbw?XyMmbV#2!36m!Pt}(
z8=DnF5MhLD7~x=1tP+R6i>$cFNq
z#`xss2VwmY*fEsEiSYF@evQNe))z=4BO@#@6?Hj1i|9o7b-t4fubrYxHGiO>yiot#1g%U*0;M2kr
z^9(Zxjm_l&q+J(OF;C#2p-X(;rq-M(ji9bi8z5_3`K7)6E;BPKPZz{EzdPOB%d7F4
zlV@#eaxxGY)p)6*axd7RivV1uQKT}at)1q9>uNz2aRfDrK2isW5-7muRkVP4VF7pv
z?Oi6ylb|S&Y^qQ0>*KMh?G5{drcWdv+sGs0sUPii-ExHhn|OOe&Zm9S-);nq29j>n
z{pEo@A&z)5`T{qAHlKxx_oNx)#pOg9%Q|B&}zdY(5}2NVsR>55n>XcbhnQ8|7SAW`yp
z4y+_feJC?lqI|bZIT}cMNjs!vWmSMCC}gvf0K8wu&Y_^FXxpP+K>l4=G5N^xWw|b7
zKE3lBalxmYJF667w4$~(QA2``As6nMkXOb;z$8%zFsbYcm&+295EBgpNXnMT#=uy!
zwQ%4s5$0ck0D^}EOwj{_!brW74M>RaGLHW9Tl^Q0If<*1*nM+75+CFZ7#EV|&?
z1B!^W<|q*1(%#<@QUM@3Y|0MzT?PWp$4g2Zq+9=m^dx=B-n;&8G3FydSh+mXm_R_i
zA-VU&1k|X6t!5!sE=Rc=;FskWbv5Ppn#E=z8&;aj$b@_yaB|Oc0yG>}){`U162rd6
z4coY+*tls`&$xpv>yH4&CW;_-8F!A|$mWA;4#>B6?@&XTI~8)Y2x4DXW=
zT~fY?VILq!DN;~S#PWdlJuIT1Om1yi7=a4bIT@ybVmmGEG@FH3VWrVR=_Sg0jq8)n
zfWCvB1A~W`k{^Va~9c_49vvcZ4@H>wusGNP%U?IM7?+
zJyqB(B^&0AEU1HBCk}bpl@f!Nu%T|atx`i9r$zPxn0qE8Q_yKRfUB}?Y-|9v;HMZ|
z?&Q8fGiKdMo%|tvmnlhx;NDRH{9H2sQEKQ-p_U|{k(D#1$ph($0*NdK&N*mcSF+1P
zG~VLpNOU3i4|9BPLflL@&rlv5-E69{!vy95-l_7|yM89ri9!8~JOWga-rZV>l$b^FOfkBH3Jw$4E^aBJ(Lwg?>`2lhAT<;sE1z&*bF+o$9CZ^0>m(S%6p>
zoqAtiUxYdi4;tHMdH2GWAxB?IaO`?jX6Z7WD{5_QVP8Gp!+9Grm%tu;gOIeXH`Eja
z5$c~bRT&mK+i|caW|cp8(wYMk3ol9~&@ulj1-OBA;_&DN-?O_wd&}e9Ly_AI1$^`C
zRBi}T2j6BLoTC}d=ahRmII3y0Kb6`M?q&5nsI-db8^~}GJ&;sN^Yisez
z$lDu@
zw9Eui)Xiw=`DyON0yPU>$R&@S>#i1sperRvyKsD??UF(naid4>)-u_(Kz}qWc>hPk
z0)SHYkWnbSgK-fqSHFmI5<&nnRQ&(7_oneswtxS)oyby=B2ki%rEJ+pB}F33*q5^J
z!q~@J6oq6-)~qph##n|SiIklgJ7u4-?`F*Sox19}y6*e?efgyOqo$3$uj{DNsG97$LGBBn-fw(zq$<{D
zuju#NjXe!6rGd>|?wS?&ed$P&(nSMFfj}|gl|5!2Kg!G(8;vtwGacXk;{z$m2AP7u
zy@^G&f|Z8eD`PL@R*|Rd?d`MgOrp`@EfLI?s}l_!p@!Fa%=SDea>u+an
z-&jjGTUB={l|BA;1F7%zf+I&?>A25lX2pq)W!p`YS{`(@^QhY)ahF$Al)1tS5Ruvx
z{F>>Xgi4X20H)I9f*>&MPrk|OTTe1Z#f)yAXFYv=6!PPx{^Y&u?S5768LQKw(C&L_
zHLNWL@rzqr{vt}h>zd2fsKSG?jeWYi`2R_SW(B$}r41YVv&pVr@t+`fK(Tf3GWj@_
zmzNLKJmEd>w{YRAX&oyN+6XG?v}pkfZPzxwicT*LR{|*pBun>O`HXGZFcQV}Ar3n^
z(_L_;>AW+l9vo%VNM0R(<5sZ3@NbC$G>a9rIPpj61ki#)h=o^;dgh>iZOx&PW=lh>
z6jGDqP(4GT4+9r}Buwlo#>dA4{@H$;2>a(czimM?=(gTDrND!5e6d3HgP_R*sdxSk
zdD!c#F{@+okq2At1sOU6LJq-B)?{i
zwA^~GH&qHf?l!r%z26h~{EI5TxWVjAeP*SDvbbT6Q>UhtnwPz@D=M@Ldi@r9E>u-l
z<71xix{)_L2rCWB^6BG&DT|UeuYAkCDzyW{VAr@s%_7=(^^GilrKbW9J;h-6R}$Wh
z3l|>!Fq1#2P2tXFH0=Bp|Mi{J=n^q*?MiK-cc&WxdeiSFc_zpwZcr7aaO&dH{>QTwu46%&%)zyXFpN@%U0T_rl8D1pqn=XK6rD@tFzUx%PyzC1oVh(_nd
z(|;%}-Ho|Py3Sp4LQ$sccAll7YQT5{H3o_hQCBFT4wQlm2k@8g_-ilaZEI(jO7P$E6xN~Qz~mH<@?UMJT2
zB8_tAI-46bpPO}Bh*|VzsQ_u7EXp8Z*%3JwoY~X!8mCepO_Wx}aMfum>Xyl9qYh>ny~c{%1v!tVWI2LKm6im2VVB
zW0`+aep4#lLTerQqpS|S-IR$@;|cP(46-d9TlcOnKDJYNvS~~L0s?Bo!D5KhFG?sD
z#_rU7TykrU7F*|o(`m|UbA64)8qq{mGo_0G2%4;02}x|@WYPu2i`F!Eq);LSCzR2`
z7u)?BKZaJjtY$g1m3lq~!ZM#!BDzZ>X8T{Bbla`U+O^FHB`&%6e3CNx!7TfO55YC7
z`u&Q+%e5>y7MthiAm{&@4bk-R)Ec!x!=$K@Pj~m9;z|n!nwunh4MS#o8;4YA7H;1Q
zz3#$Tbr-^JSwIXdIvb$v`^CAod+A=wNdnV?!sS^ruOvn##SJ!HFB7JYhIxVtP}38z
ze8>tkxcVSp*tzogaJSNNfj!Eyyi=1rxpssAGj)?p(%1{$Fj%e3c((LAqc#=@Y;Mf|
zQ@E)GGNrg4244vJ^fsa~Ym}IHg_B1~w8%y(h0CIL>>G^}#
z+xPjKBv`lH4R03&YYJastsT;Ha$1V-c9P&oP$|EEUYvf}`xdTOXnt!*RwZ_uTg+lL
zWrJgh3~6gkao$(+UB-`m|H3B?<&%fWoSV0r2K^MKXpx
ztc<{Munn1$CqG3NITf;eC@BGdmPsppR`8?w#@)a5x0orprPLJbFsz}i!Pe`AU(qpU
z%I^aG4RL}5+)rFw*`@%%xo(_evXkK8dt2wZsD0FXh9*qe_e4$xtrWhKG*@Xu8Y_d_XiNf?%J5W~G5YJb
zs1-evzZmK2<{&T2sqYxqdkJS{Wx35AxA54>d>CXVGrNo{f{QMfAGeUc&7Y&$ADfW40azw
ze$2PJR!@|=w1CM?e9yVb4G>MlCGdRei5C5N8trbA;Wv`n$<8~V!I5KZvBD>
zSNh50_@L!-;Het?({?*+=ic?p($doGDbmh&N-P;Jj!l5x7q#Fkrw?Ctk>=JPd~du)
zkr9L>d(t~W-6HGHwKu_Vv8+4KFGMWud@uKY%)PLp>otOe_Qu~imjXvd@PUj4_u4+t_>nr%0pJkR<>AN+YE>-eD%dsC!@N|t8)FAE8$yySmT4&y?M3
zXLIS5gm~X-fTI3xSHpbbrb4zrcdCDvt9j!@qt_PmRnep-Ph^Ka3TDe??97ZR8?hw}
zTCR?MDxa0|J*SNJaTzYtDAtLJ*;5Ma2*=HGoB)xLQ}qeOR$6_DixqkF+g`N66g}YyEe{$n`gp`?tRBAUmD*YHb*WIw$1Esln{f
z3I+FqB~T^tytEJpS8cVqC~EyPr_}1Q>N4>od&}$?3V)`4>iMdn%Y$^udXkg5(Rxn9
zPWPd$V>NVP^JURFTiO_f=I9Y(75VrGyc`&3TC>gQzdCKSn&m?2LI}rAqB}Eud|$bK
z$2Y%|MQI3|X{KKPmd7pAN58RHEPP;PN&eScUS%_=^DIEQQxq8*gisk&mi5(G{aV9<
ze@KEzX5A9K_r&u+1ECmkN0V72_~ivr9c8ot1I>MPppW^F)~MxV??Ve1{pkpt!$w)l
z@=};cwG2ZELeE(LDMnQ!KHRJPo-qILTH!;fzylv0*bowlY`;nj^j*&5^7`^B%Guwv
zQWV#I;ThN@)>YX;lEWhAq4iEn>4WO`dMdpsJ}=2A=y-z3C=4&LP#G+SNk$0@*XZIc
z7%#o$1+M=56;NKyEY3MYQD{6@dj^NDe#uy}ZM~UyLvDF&V1~v|@SF?S*XIY4c&V-Z
zHLvqsk^x~J%UmSo9VR-6uZvl>#<)XpOqk`H`~?E}6V*e4La?*ej^NVr1R?#Ah2HGq
zOpB?l??z^EuKqElYsGrTMz_Gmn8(HrqoKv+LsIq1<)<4EgLzz1W000I4~uGpvxX7B
z-t4R}%T{v-RN$N2dg~p~qRci0gIWkGYh1>RxGwO%Yv(@7l5!PQ%_jBnB3!e~nCzwJ>
zW+m#>`dO);l>2944p4RqV(vja3ZE%js4Y#O`-NWC&N&eGaqXS-5+}i
zjO7no)Z#>XDj6{RX-O}C9s?auY%K+qfjGjY=K4Qupi3XWw1U@NzL}9!ylo@ODDLn3
zl9Ls8ymI%jba+JR?O;uFxsi9yM#uFP1EA;H4!pkv+c%}REn_^Az`KAh9Pc1svzSJI
z&QTvWfDk`^ViBDISgllO)N{Scwh{RnofX5KW1Uzc9cyN9!2aCU&B
zh|8{EFxZA3h(adf3vzNkjVovDoyITe;zsnOCEz2e`HEFgZ&=lrX~Fx@+PGh=&qJW{
zq3v#zosRq|GJamI6B7G@oS`hDPn0T#yuO8rgWqzH><>14SACG%0{$xOL2iKAkaQO;
z@}jhT<=)?()=yW7?29JY^t^kWFuj!BsUBCC6^onlGHt|#U$|Rxh?Z8miip!nV2cne
zZr7Z3sr^_O8DV5tFgh1n{4O#oszvAF!`J3$WLcRQF0_1FHF2b#dytC$$^3h*cG6l>
zOpEUwW}#{Y)uhq$*&R*@toA76cEPJNCRVk_pR!3k{v{pmv+^{(djbC_dEqy&(QQS8
zw(nOCq9kDu#D1H)HSIeM!zsK|r#Q?bOnMNxGUr-L9Mj#cHYCy`yi=7^jT}wSv%ei+
z6%}AHUZ5zZE)fb{lNboxzvO0FpNzP~K*gaYUE--<2?rG5l
z+j_~NogLhEduu*}gIbZBe*s%~M-fks?~9cBFKXk+p^b4}i&n7iLS9-knA#+476A9T
zg9qy@3^&^alu)avmDC&727TmUr3E9Z(T;7o`~66jj4h&ZY?9^Rtw`sAp2teu4=@ZS2g^T(&h5|idhwkLD)=D^E`!r9A5O5&taBKFgZQ{qc~HK{-fm>{fgpg@P1
zs`QIjfDJWev2#NZP`-Zi%3x~!U5{~c)>kVBnZ+d;om?(j`S@M02QG#&-T^&Q@XMIc
z^37A9hU1KXgcgVK<|)f1NtU|5nUhP}+%nL~bri1msAVX&Nij}Y`aIwG2v#kaIp!dBSGZ))JEBqrAGP-KT3IT#f*q(Xc|8B7|k^4RPi^2&JLq*%-sAcbY-0&=qffl
z_nDa_;Bx~vC+)N^$Zi$$rsFGOC=-h^FzqY>BbwED67|H9TG`+
z=B%Ugp3y6Yw!R6+BV8)!@g{K`nI(ZzEq;W^g~{!$O73!=4^Z?-LUJoeIfSA7@?{IbqTHksb&lK1=w__e+m}Y>r(4)%jI(RynvB!*os4@V0vF
z6b`fRjE5?5&R*mnXuNUWKJLonpY>dfmh&WC+)bJ7=q0z7vXr7
zL+Fcr^<(LT841IO527+$ojGhbrJs%W8t}?-B)DeVuZYKKzpECl7k}Eo@Jaf!o5ed@{hHn!CX6L_Axh38Rd0n2*#by-
zjVR|-V0+dpx6F1_gX
zk$PHi1yo37&7EUqHcw`O?2RMU4}5@fK$Z2abx+c8I0*F93`MdA=lINAXUa$-aZK3p
zyhEh?&5Wzk^})01a-T;W83(KLa>p_}tju0%2$)z#u$6i@BW^mn2$x^R^bKru6g#w^
z`6{XHMH$C%yA42
z3mNk9DY*?^sFSgCIeU_+tI&FhU)dP3WZZR!)^*gVH{58``3v6NxV&~u3WI=u;40rjs&$GFRg_C%riYN0`Dq6q&BH@sMF@0
zoL~X6;9E{2EP~7Ht*{yL#>yi$E7LLya$&=1!SWMFTYLC(Yoq_?)@o84jTO&#rkG9*
zYP^HNi!$mhip{-ZjH{3y4hUrN8|8XGzlQ}Qlal4Ss47J<<447{2_;`c=wZha>I()O
zOr^=;#m~U=F1e+Sop2)zql>9JEahEsNo_ASg<}la_09-?1$!ocNag~j4`QoN;~ZiD
zany1$_E4(M8#AEHhsYZk;dg4rZ3%MO5ykBRozg3Ts&gbv9BWmQyu8%Qh*!XfhP
z!2`MydH%L;#6qJehAJsQQJt2i(<@t=UwPs)-es+@gUdRPxXga#KIx_@f*gygeu(HQ
zm3l_GeNXm#zCFaW(B2l2)+thBm?1Z$QC}1I2sSpR0iMg#mS!_1?ns+Xm3MhP-6)>W
zkh@7OH}4$7Iaa5|O!@kKD%jE8*0IChm6d+6{R_1W&cmckUb&;2(NN`KWyVP>IJj+e
zilX=(Yk;#GIi|X;#Mz+?=|TE3Eze$t-oEVJ`5KeC={zwp!|$PBxmN+p2)(2&wx@T}
zQ0JWrwB&M1d=i;sh0)9{V6}Jz89CbQgZa3i`y_OCcm!TzE}V4d6^Avuc|bY0UM2g3
zwhRAQF)Vk@crfE7>2jZnnluzNE?}NkwNl?A=1jqiU6?}4j8Gn?GbW>Kw;AV0lP60t
zOWrQz479q0UW?~25gPXMPC1NDRU0iol*e2gpteeJ92ee~
z1h3A+Gvr;|aVZWaLfVKtJIW6d>+H5UsArU
zw}%}Gme>Y>Cv;4_v2>=0$a8^v-xWU;GJ|%zQU>`VECK3c3neTADxsr@h!7)2FTMDA
zil=pHLG2T!BBqv6xRT6ADGb#v-q0^D-o*SmYfS!<`04}>uQMJyuwDrJJUZ1J_NF7n
zDG!ZCyOl}NH45@@URCXapOZIh@Po6}w!NU}uZzBLE4(J8B-)#ZsjgG5dv}nNScFil
z8UwrDcd9?NZActQZj0ew-JG4D9qB2&sAS(d?9x+cMO!QTM9p71s7Z0JvRjd`*9?o3N!a*h
zU&jX5NnXso3xbw35_d+WtUKb}-1H*O%Xt+sa>))1)FnLq<$V3r9Sv7wryJ*3YZ5c~
z>XY^EU*inO8ZPrNnKTve^dhDY-;m}@xs#gY)RnWr1t-``6*}xb^-WOq73b48miD2K|GV&cssGN)6`?Q3-V$o<|
z0@?=UQs=nk8>G=#Jbv2040a;nLP}%zY5(=oZg0Pr6Juh&8&=#rX4O|o9=DXOW@qIL
zUC}gc4=#xTu*Ge|p=iK&x-=Cy!k4RNfdgX>->CEP@o_N~KqMtg#1C07ncdb2Zolvk
zVbBzTV6&qsUSR^kBZhraz?SId$Bog;HDW175i|W#S4u}VikdWy(6Y`t>G#z+Jk>4L
zbirUFW#y3B8WAoDdqZ56hVPHu$j6loG0xR*cI~Slh8Y#}h|cf}JEm0&Xx7Drev%$@
zyOm$;1~H{C5sdl!_{mo31c3I{0|F>@>K_M0yQc_9lkG5{`G#iWE+@mN^zPa
zm;mMs!<#_HDdP3Jt9U&a{F6UJOvOMtc}WM~#Jj78D{V)|YCV>V5hLlGz^saKif^eh
zm0zJlOg#>HL%`uS1)l)ippS(jqN2O1Fc=Kyrd!-!T6gJhCl_Fpb{2AKE!A#^l^rY2o-wc+^_T`Tp&qx=kCr_O<9hPxgn&F}#Ld6;)<1s<
zAW)tR0LMRJO!e%$x3k{HJ(}h{VgEHr6rmDzde|8;ZUcm-5iQJ0@~yVhZa-=Ge|^M0
zAjbnjgH&bTf1g-PTGR}x*x#HdyyoM7kZPA?>Xn=jAD^@5bdKq#i1}Y1eFDGGGLSMz
z+ksljWz0@bPfxM^-3YIT#D8A!-TbD$Ty^52ee*~ZT+FIH)=XYD;?93tUk|Ngon!Fk
z-$>BE763xivsiSF6i$EqgX{j-Af9nR_V=4orgHzU7yn5}72sepM>Lp=9bk@s-T2pk
z7f%CJvBEKR@SlDOs22Ei>WCG<@l9btn51D^YWK=O>!Hg>QY=%8Ff!0vHMIhA#$NVj#i
zeg1ZGVP*5;6%`MUN=XR`D&}O{nqX2!%DZ=U&;&oO9*@gl`e!!*N#-|NWv^4zS@e}G
zq!Ax954k`oSpPEtV
z#B0lxh>M!vK`GHe6Tb7q*V@jV>T0_=cg%C%_k>|&j3U0hq;`&jyNm$b+C`NMqB~=g
zD1H~nd?d{Wk=oJ&@%D1-pH3|cneVG5m9FfJx*`CJsJ9RXTV_Y?srUpn(8zDJ^X>rh
zV+b%K5BwTvOx&8Oy*`~#eY#m53rt1(xj_xv=T?^*i067@q+=9z-X6+p85`q7a{(l&
zq+!Qs6oN6GHsu(_gEFNjX+8Iaz(r*;9&i{h#6l;YG!@3Y#2mC2)&Q8lihDqxZ_9}DejkW9
z6v?rEH99%_+FD6Y{VL_;-e>6m-blN0TuFV`&X`B{!XW91?Ih=88yh|fI!{A@=##$q
zHHMkpX1fGkshxLP=y^{H{&Fyq^YSQ#f$_ZjP~&dH>b|rJ^sP1vWM!<@YrT~{LUg4d
zKfOYg8#>S>zjbr_nvgFU#f_5
z%^i3He*L=5N@kk^XdRR{4(v%{QWCq1bKhX)LQ6#`43JW*Jrp~Dpps}{x}PG|8Pu*V
zgeI(_Cbya$0HiVyi`{R}kPT@h9g!pil2pjMJ?LSvvoyTZAyhhKtGIbqZXq+QsI069
zAX6LqinuVh}+G)7+$n`E>u4YeYKrj;kT*ND-&W
z#&IBMvGkEa)Jgxt3En$Z{@=oY4UfW;rnwIn7y=e^lZ=4+ghSFk7FKE$7^qlEes)cF
z;R_*{L+eBD)4+F;_G21NlpUi@sZ|R`2(su65TNMM&cc
zI#?&6HR0Nk_W^OR-AxXC%M`csFw=ScM=O;9WxPZuVdF3j^Grg|J+yzP)dSQlF?guL
zZaQP`8mi7A>7ZNIF8_+6fvA|dOATRo(j_zhcEw9ucBN+ez~g2H+udaTo_-89b!Xm#
zw0FML(~*zqw^X3GZKVy7FVLOgHk;0O)}<(t-#5p{)|2#GTTqa&-kbdl&8>#_uIin`
zEDNY3$*hG?J@0F^Ck3T8sE@lC25SziW4m$vdDeDEzTyo@>SA4113eBI_neGP{>tOw
z{5oW0oa^^)E9)LnnAW{5p&Fy>ni+~|vRy@muY4ty@VzksxRAG;Zwv~I4+l%Jwe#sX
z*Ta_+D&`eg=Ir5&oO6{0q7StFsPWjNrEj4JQ|YjOcfHP{
znWA6#BEV8r%W#a@^2P=az&&`R*6souE
zm5|ulX4~d(J|bH)P!OL1+#vzQ3-;!tvJ|-mA0ivy)jMT?)mJCZ
zFWjmOe5DH;1*9i?wOLpzc+)5VIyR<4^Q-r
zJ7J?9Dzg(;ag`{nQE_4rzCQzR;dm$@%(wUe8no4`8rWn7cT$>rYYF=19%Ag;FzCIX
z*;E*)1wB}ABaYMiO&)F}p_bG!3A<5O=chJKw9}E-(X~H!SQt?`2N`(ig*LR^H_s;l
zTI%b7(e{M@>UNkuwQ3yz-aBQ|RRSLt0^0rqbB7tApGn&k?MrNT
zcMcVX1BQjJxs@V?ATUrA4O|xk!c*kkGo`jq3l+^-Vg={|)T{>w4-#6t19!yhe?$r`
zDUqg??h9uQu4aU~<&bzY4_%4fLiUZ5frR}Mo-ISIj8pfB>X|U$$)AEqn;8Ud@K!db
z0D*cdsN~KmKamAmacCEYP#<`gLAzp=T{&=$j;BYQ+^B5lo9MDXZ1VK0izL9#NWmMW
zh&>8*9;@l4)#*|sEV%{2{K!con8rCJi$y64;ztcEH^nM)Gy2)h3-u&CPLB01Ii#Vy
zD_NZX;C=nA%V4Wl!Hvfq-A?iml~nnQG6Pxs#t9@5!6OOcIY5ulfW?aMOPbOgx
zINNRZI>dX{v|zM3jMjLDOf@8t26d_Cw4D_ODYQzdsjwY?7?8Oeo|&OUyb=UR`XSdK
zf-$$1K1=jK5VgeH*KsSJyxTz`UD*2SYP3tucbY@q14)cY|09+GlBl?w+K2gIqe?-@
zEb;La-9v3EKYf4?R`EM~3l6VN2V6ln52y*_cP6%R7{VqM?^6t5GqyQ&o=*0>s_`0H
zaofdK#-Hk920PYRUzmfpA4uA90yMkg2d}ke+EXU4K7FaMhU}g=kS(?t0*)Q?KB15X
zsm;HeWF82VcI!u|?*ZJsH%bZN90ue)r&;$+FTeymlJ84})3^0U)8>Pt3zVY=;#AB_
zzIJpR*l!JkvQ=RO9&Il=&s`|UyKKQD8>_U@#tOI7jWkK(I%#5^1De6zoWZwSOEzh7
zCWBueHteP=O^m=Di6-2^TeGUoufMlM5Fie|ndHYjQ&4!*K~5OO6^>>_`+B&zYJ#VM7aNHs`x{+xq?B7QNHUqX0#w&)bG%_zhOs2~hAdQ8E9RoFYOhXq{b;omAa;#9&RQt(*FVRg!
z2QXQwu|KTRo)~u~8QA5;s{RL+qJisCfriCQ@vL?awbjAnBc;9E+yQIKs0YHMOfoYp
zD9xUe%DBeOwADT17Kp!3EHJnX+IZ^0jSB{?@?lhU1mttQrjum79WZcrm5pm#
zUp{l+`r&zbRUgxZ{=$_lE9m;UA^+6|BHfSOac}cIRSu#>Hu!!(!Ag|_-szSriunVB
z05LW{MDw(W$r+KP#@LJ?(tec%W0Y0bO>cAdEiUB@PS*PvW&t(77<~)~d;Mzf{PS-o
z&KT(T0Xup6l^T7!N`BCOQc!s}AgOv`zGD#-<52D%|GkXEgL$v*!r*;9ebWRtCINEF`;=*<$->POt5Hd%
z=3tIze*q9P-g03*K)fVJz#{my%`ekd-6
z9|=s_yh@p-I^RtjV*uYjGWU<1U=h0@0rMKr)qjX;ivhp`iCG*1zp`G2g)^tHgZk#q
zKrKfV;3AJjE0KISuRVTRY+8>7;-j`lF6}ZfY%!P5G1NgsNT|{^a5DuAAjmwnHm*Q0
zHh;Oi?R;ehamiN7Z);(#bth@#8N(;QUi}?-MX~TSjffV$zWLXM*l`L?#r+)Sl63bD
zft_4YsDQ~^EY?A8t%-8EVRwSJh6DTp>0AE72na~0NK48?bo&-3{j&q-4fBH^>_nLa{$4LQUKcmn4ubFC
z3X`vfZ5$W?xDxZKvwdJfRk&XFf`+cCkgYyr$wGa-P00NqgrNr>#XZv1_4^#h!FBg)7
zDT3(h1n7LfU2D4}KNL#Mu*9#l|AP55ms=CLVohPwI6!xfK-^3*ljGdX5fIRwZ{Y;Q
z)Q*l$u^oWINP|G`&@y0#?m`0u=?K6ywwB$)!qahKG>r62kD9L?74QJIgAAs~(6TOC
zBSp&4DoSluPw74V864oKv#>3V0d~Mrg1*@`YHrBHbhKQTc}(aODB2uhSQK(hPm8sRls^ln)7c8
z1{m8@#bWyND@8eBgZivHX1|E>Zf$UAf*Sqp%W7EIHb6-`0v>`ImCpC9ytR~(^B0h!w)j-}hhl5jKS@uL~8I{-zE
zJYzSrb49L)Gdl>f6p-b_8F>p@6vo@tce8Hi*V|w>cMdkYcU5&Pb|ZyK$^g`;dk28y
zWwFP(xIbUlI?F1bF7NB^LkmAcLIe?{b(#nOvIhsiEdXTu8sKQT-RuY8pm1E6B+~NJ
zP45o&a=EBKc4*6B>W%-1U%(ZnzZ1PYO+m$H5&)Cu)XD)KS2}ux@s3p4f1fJ!(uQeK
zQ?Y&0NpF`3JlF%mZ@57UJ!NgUAY*9BYn)^m*t72%E7?GxT3tgBcA$WCnj}3LP-*Hl
z&X(gs9*P{|?
z6~LbvQFIu%lA$q9b^9`efd#rWRX3;?Qg{9Nefq03*7}E~gk?mAt3V;T&_Nhqpb70T
zFWtFUrLa-c?C!Rb{&vxzPS3np1=3Ivne_N9)&5|_GLS9{O&+8Usq@w{ul#aO`o^6Z
zq}*)OScdg)2?<0x`~*S4@chuFm|rSM;Urxh4IPE`IuqF3d{vsAmDRI7PdQHe^+n_K
z&vIsNjaD19%EpZ7T=JBQZ}7Z~$60F*R=iO-f*cf(?@(zPP5JewrjS9sJTm*N<=1VD
zi8|Ja_hrI7KEH#R4|BRAdl6z_bui=oJn&bbP;LYI_45dSGNeGz{QCSw=D#%o8AUmj
z_NK-q^y|OVTj?%c$ONSrJr^;l>7gA7JzZSx#fXa@xUHj*eb?>fTMM!yeY
zJ{Um@A=5xb=Ht<7=S}EqtkgbljC`M2;J!7XRW$Isv9I8V!iNiR6b*_Ve@6qNJQ%kH
zI8a-{wh#?Kn`~p^?nfD#_UlStL3?y7*dFm3YDGDWCYffh4+3Lbwa-tp0rXxzYAHKf
zwqw_~FtcRi9CWKU5Sq1I-jVvCvK{v@)^PEC#QBX7H){Ps{^dwEp^KIK*1lI14%S-G
zLqm3eh&`NDTt2Cugg4Liy!VVCS2q$n4
zw$9DY?ki_>$>OeZyA9N-i`0HXU=IKv=!dHeMwVJg5=#`wPtA(>o
zM?E-o`Tkn-0N@5;0O5L8=MVY@UfR_760JMn6IZ)bl
z>&VVz@wU0{j8z2AhaMX^C!Vn$bIMsi&j1rkFU}>M9B5#fW{%0t+<$piJV0wZoYQ43
z9805&TdiMh^lAg6sv^$0KPM7bgAm3pwXFkcH|WYf%LeLq`B=typbrpw>
z!nA$eqM!XXA3aq;_g3bZLf%Cg4D%QI{coB$Le{d?4gG6OQA9LwL8{{gknt+4wg@P~(kBYW?))|KaVUFY^tQIb7$;KYG{{`u$7^=nse
z{PPbf0sJ8++YA1sk;fYg{@P`IL-xu)DYXov;D`N|*KS+?^AGJ2=+7>r$OiB$lK))4
zdif@5*F^We9L8Ju5>o+>u8=*^dqj5Gll0jBlZ-LPO%7aB{UmYTfjI`nl@O(ey>^QV
zOTu#CoM+J6N18d$55*#%KXfA{lZ$ymdH>;;mxVpH;p?68Lt#QfB1Q{XqNux=qha#v
zOuTJvw38+eqHTS#$94U~osujCeiBkL%9D~0|Jn7^hi6_zSbJg$GylCxe|xDo=@VCe
z{eJ5=B&GE4re!a5AZRh-`*&P|j8gMCiCVl;4U)Nlva`444|gG(V>yW(-&4-^)1ALR
z3%QykD*NJmcjzzAwS8YMl46XWWf;qE4-LNjaB%zbOh$Mv;Z>{XZ*M85$<87+EI!y9I;eHyiv{lWGE
z7bP;GEscsJ;dY!DOH5Tla}}V1WF^49zOGBo8+&h$Yt(=_^L5*{>YB`c{cN%@(Qe|+
z*K#e?^y77{J5Q8gKtKOkd3ng%_Jq;NAe9GKHcA;31
z$GGzAOQ$8uRtpyMxen3uCrNb&k1mAvtewf9DWhkrt+Tl7GgPT8I&)R8dZ1pGS+PFo
zUXU(-PyD%~iye2|85CKNV=BA;)?L6Pym^YphDnb7=uYLb;
z&alMBjIR#Y6lO3`dtu}_LCVW^NqZ=V)t-rP
zoVuQhot4|@Zg*Loq;FAO%<43k&&E3-9MO&WnmA?#MNZ;&r|V5v2J>x6AaW
zeT`nnc^clLIYU~s5Xl+M>e{x;`6Gz-2p3urq-E4VQ)J;x?c&s=BWzXZMVeWGt7~~u
zf_5BUa`PG5arNIVakQnR#5dpZpZA0GlqN@#^kCrP5qiys9CKRHcj`{a(*nkRe8
zh-GAd)q^UIAPS+cFUppv%R^m}{my1ROJxELbEA#uMxm}1aL|z(k9xcL_CR8=lJPedUxoS?v-ihUrwKG&%B#1cni%X)RaDPBAPPmEYR
z=1w$H6)4WQ7kK+j_=8F|`+Gxz%Rz-}r}G!{2{q*dj>iYcG7DFlQ|;&)bJW+8*H&5!
zm5jn|)tzdzT`TPtF`7DO#c4|g=NB!o%NzNVi}8`TCI91{#!|1j>96IxtsPo49Aeng
zVq*yrw6LTmJgW9yb^pq$k{y?lzqo9xS$$
zY~jpaN8)wdit4M_bkT01YZod5xeIz9kw1XJXM%Pg7QI-f;^o*4j6kj~_gpmK28&(%*DCE0s99rZJ^x5)!Zgw=3_)Ox+7(H~P7jC+dC9q5^
zcMXfTo+9&jao+Hzv>?@|IvQ=$G|7(ReQm0;i~+kp0@Ey57v))gw{UZvFg#xF3^!DQ
zWe4}sYLkjm;c+y+v1J2!zG4fSF2{Hczc`IZF;@-`E&AptnNs;bRTO=&;&#|c-=$eN
z%(wjxl}f2y+a32CDaOZZ$sVi9{vZmZ>mW2Gp+SM6fx=6B{
zENf_fh2V>CtWw(cM+UbacQ%O#r)}M=+gz(-Ad#Uv?sjLGV@!r#ocOvfMp2+UgE)%y
zv;?Yk*`U*PbDf??aA0O)N!w$C`jf8zsB-(cp2n<`FFZR@5<
zZ-ApOCS#BTka-nqL*MoT8H4M<5$WgcYd)XKKD$npI`+f)_0raX(AHt<%Z7SW1ephp
z5g5HVNM4}o?jW^EuYhZYS-3C=Pqj02Nxu%ZI&Q$IZBeK#%V2fYy_d7n9fn0-%re$^
zQq^t7&7Ot<{}jmWbK92cQh7_aSg6tAg?Q5feXdPSl>uHl*MZ%mMy};q)xmoZ?GC$_
z>8I`qhHu9yW*jH6i1EH=At;Z~Y$FMOjCm_o#Z~Bp_ioo9f6!@`-F@Oy(;{%-Yjnms
zo2<;s*W9JgtXS8<=vOYB)>heRx>Y^a@4
z3++ZCk>*ZZc;z-hpp!$dU)Sud85G|4tY)~9t~@CZ-WPzn$!jQF79G%Sw$9e6yFO$<
zem`vQfQ8e^^$9{-;PG_J3tPutGhh_Cb?-<;r3?Rt0ldl6{gUy^QamOYibb
zj(?FP*t50_j9Tt8B70T(HJo2N`>BIqTUR;CCL6=yQl8gYd7?ZM{>BgWRS3Ta+J|B3
z8*rLlACalxKux#@MfSLII|X%U!NCPVT=L!(fr67O?M;Ieg@w4bK*scs^Lu9ny1Hmr
zw-!S(O1Asie>Xbd>p05dX>KI4-fe4WPaYSITS
zQP0jEWXt?KzB_=uI9sr_Ra(Gv{MZ|qldm+D@V+b3xl3M_?@<>r><#ZGO$iq!ldZqi
zu^1d+;&$XF$!gVeG;;|uYhN~OMcJfFXx|>lk*&V!-E3%JI{?4zt2ro0gY-ZO@z=p8
zl-naZU{9^IDA!G=T#*LacGZPs6vg`x7DP}
zfg&;w_HvTEUkV!3JkD>3pPv)dRZS%tVFkFdf-lqXyk)t@mSr_iHrRfnh9}e0`tf`&
z$QAfG#T*wV%3NS}Ax3$)Xk41{Rah38I667ITm^lphN)(ks#U>~*K{=9z>w@6G?9Ef
zJ-4RD(Ah-UdU=D6w$(>}sf2mMQjcNq8NK
z=8ZD)tU;Vk(hwcbB%H5allN2>N+CnSccfshL4LH(b;CNsWUf@-BzP;xhV|X
z5KpbgIw7*TMzjw1IC^$yvEI4=Xk^7@J*)a?f--bA`#&DiWwE_;Frb@j4dhVLHuuIEd%b8VajOZ{cV`)7hjP6z3{
zbalwP8RpAhW|{_<{*r@N5}EW*EHy~Q9q<#=jTYQ!$u8h?7vSDKUBT+BAj|A*qo>mA
zucmZ8KGMY@Z@QSA_&#d=b&ZA&2KAMbB=xPB^-%Tbku%hTp%MnW9=E-X91gQqVr4+v
z={bzXdQX4MRZ1%)6s!(O3&2h5M2XeM2Dbkkw1M7d!7QpTELb&X~Q1xrZspDqs9Y=jl%3C5oGIF-m1icYe`r(s|
zwtbQz9pko7%A^E(qIED`3DOfuS5lVJ=aExcuB+Lu?3oV`$kwqbO;Po%)hW!d<%FLs
z(`Rz&QWA2i2e_ov`Bs=jVOJrFS=Y*YDQG>|mZo!Kl@RIKZZ_i_>AY}#hHH-M3{Ebw
zoXK&*iT1?R9YmK0;?W{b&;^M8Y?yb`8)LOg}w#2L{XL$nLsS2S>Nxlx4KN
z8S!~>X~1`QwT6)IRKKaIz`_*mb<1*q$8e9#m!pJ5^lh=kl)U>TZ*vu)lp$+rRM|BI
z-cko*+;F3OT=73tKF_SMt(?UfVvAkrs(c;Ccqj(C-JtwYuavdy-?3xlTBaXApYfih
z3YO_zM8notz^^)wHb!pP=>5ye6z|Q=L6h+Op=rq>0n;YLU@er*nisAwtfg0;PyE^*
zfFn6eaA?ImaIMOS_p(~kD>JPKo~KP`7+`!jgRue;mti8j#V8|Q;@)uU#!uV-)NE_}
zBYJ{`mA3$hj$4wb))dx67$n_axZKDvEDt>`W3xHbt-Lv|WDV7=4Z=1@CUsO(&K)KA
zjA19ilK{*>(~Ree7fM14JI+B9-eys_-T-p>z{F#%&$+1H;=w#U{A7QPblyZcYfhSqq$#&GS8-DCJ=#xd*d_5R@5snmCp=%H;=}E<
z22Mz0JnPClTqnJlCB&UUxGPs*NcL~EYhOnYG6{!S>(6&o&~UZ4OfX9N24z
z-Rtz6Qu$1U_|Q#KTj^Y#oKa?6^gC5U7j4ofGexuT_~g>cz?0xLm!tbYr9H2WXY|*=8*0xBB94K`~`6iFM
zzBDh#j~(3#jp#_-B8p$-+_D(%cOn(1qMOf>CTrD=I#Mpu>ixyU<&#?4f%_JV7iy`~
z>*1dsMijq;l73oWV)$z3vZGb^P@S=bXut4_WMbhoZR^;C8iU1#S1(_2#)dhS4Ne)B
zJ!0g_#2yi;OR6RIRI5IYY~O0b<~jPsv2oX;a2TfFAv_zbiJ!_mXA2redEMrP2%G_%y1Ci?AV
z+%9@V9WL#(9VwCKk(J6%7~E?o+Yi6v89{>^xls>-c1oVp5K39nqDfiYB2H44|{
zc?;W=Onu=5>%{(h4wGz(64!`zi=z(C
z9IfLuVcXI7&%$6V;^t@88vT+nMAZI)Fa(MnRTlND6R$n7Q
z==u3g_w}>k{&>`Yx9Bu+sAhbgwpUhSMODl2^}PqL$)-cHK|v*!r&f^Fn_hsE(QzVe
zG}yrBI;*UY!r4ft2O&b%hSA2bD!%h=JJlp1TynAm(`ivCrWh$M7LMxoFacOo1c9pv
zltzN%l)^*AXZ?I_v=;#Wduv}qKPD>JxjqV&S$pUo^dSYxCcI4DdvCc%}DI$^yUQ{b}-!t{s~>}+Mm
z=EiC~z~VDH??c54@q#uu+E$3U+%v#*RV$@Rn&D>)wrxie0wgDm@bk7A27aE`W(K%B
zBeLLc(7D#bc5IiNmZpzMq|ImQ1fl%}dkaPVCDoq(Vtv5oUA#(KjU6wMguwYxs4fB^
zW@LmIEzby)Yv>RIH1td+e3`_Anb~XkV8T8eAOT-i0f*a~y&Yoyd;lsEyLbpI7l#(R
zk+293Wbyv9Hf-x4)xL;J!a}73j?AVT%WJQc#oJ@MASnjLv{2tg==lV9*wd3L?7*Nh
zKXV)oM=H^q=iG6BN%h5<;MgNi07*%s4855@9su95pqwGT&jyLNH|hX^>TExas!2zN
z$>_WI2sy{eOpO8=Se0$g8?K*KYI_$mk0v;yXnoVg0WfUe3uQhMXO$zuXkbn=8MBq_
z@zm;?r#SNG8kXaiv!hXSe3eO=En41cB9j?v5gy1|q2U-tWEXcn$lgXDk#knGw%ar+
zuIi?!e)@y6nIyf>j8_a7=7XvWjC(hrI>kmo6(=TUH-%tSNp8P<+EVFrDppUaW$XS@
zk7Y)bhOX_cyrB?8LxmqJ@d~XwlY{Yny{B2JgTgvpv^yQ0*8YQ`m+nMr#ovio}kiE(olzDyLrKfza9|fzLw>vca^NDS5dxz}FGhsM&p#8g#H=yctEjm9|
zadvi&mgN=%B5r=z=?7KF-TE>|Vz;uo-y|v_3OKt*kK;b?-0^v+kYx}y|MItA0Ts+g
zz^@*l2s!h;vi#pOHc3gMG<>aEy9C
zBsQ@(%H-0|rf=(Zl48K`&(j&@?=-@H7xjG)|DM!$IduCge&3P@(!V;Kh=f|b3^Q1Q&S^FCpa>1m#_kgL0g=H#XY}-so
zCWHJdvDtF`!teX?P%;3-qqk0Qm~De{Kbw_=01#xWmn5hE{Eypd&8;Jp_**6!83Er7
z@sHp*N*x5*T8FZ0p<2wZ-`>0qg6!D(2KDd3&5wt1(*m{I`+Qx(TeaI?Zl`h@1la*H
z^4dFt?E9wyc6DM3q@i)`=|%#Kqj{^dRb26)qeL`+Wh_xw*-m&Cz#esr4yv&rrP0M}
z?7T5QEnWN2`ndJt8IY9$j2$~ydQ>?ky-r-j`#4XWRCnR}q{S%ctdnzHABQ!9KJ5n3
z$lY)z)bu6P;FM8%MBe6{V--hsj67%Vzcs$uAmd=FrL
z7$~wPnWzOG0;qGnlES=F5@v(C8|`~p!^W|?ARk`-a>||IrH}Z^2t!x5!+7#21dF)8
z1?{DBMqySmCj`uOgcsTj8oA3EUdi|{yjBul{>lpw6`A(`DUlL!>u^VA_qaHQh&ww7
z&y*f*Y|*yp9^fj97oUGO7t7_$W(GQCO+aVVH3-{@2uFfmQ`-axWQYN|xsY^4cCGMu
z%*;S}ywqVaUg>KnP*mcrT*%ake#E$QQ#Hz*`?sLE4sUq}BW3F2_f
z$GiaA>CM9P5>JFdnNWI9i(cm@iH6MxucJ6gW~99Ub0m}U<`lB9ioJFUG~#$)g5VK6
z%rp&mqTDlFP(A5xXX^uL-R&aoFS0#<48mA0m25yT8EA$P&4)SxLHp*dfg`vpwARD5
zW(@3h&ki$3^q2IA%Xh63W|o22f0btMmxK_t&Oab(vA`Di
zmN1du3#&uEpa!6DJ+X!i^g!tIK<^8^Z<)*+&>IjNAK`G`EUVoxR|$GH
zw?94`oz+sGyX$EO24A-pE3~W*1dKp9BGeJc>%20DeKgUiED_$Ln!VJIHVOov#kv7-
z#72`q$8W01NqcYLta=g$lZW?WyDqZZmQCFs(jGM|Goj16PBB`>TMks0^|e52!igM)_wr#qHF;pn`JcaT;X^3s#$K#Qge}7Z3piE
z`6ICF`YK?a`f=rW|4%plei)VHWd93(D`aA*Hayg@E~48eX0>}b($(dS|9AnP&rT>P
zBeAH$Zc@{AmCt_ms}0_MIQDD=iY456q1~i9m|K_A>SstL63Q;WLn`fWCLVFN_Vxc?
zqbkESp{;|o`VBGHN4g;7;}
zoTlzOvXCe{5|E%lU9?bwYSZs=HZD0EpH~ih$exnJk_@^lMqfx#Tpv$LS2_WPn1sQQ
zsoE1)fi6p_XgqlAoDcv1Xi3vN@5+o129;z7x?JtEbXCp4M5Q9w4;{~TyEKlG2RLV^
zog3Pu7PUaY(sRR0on&2Fl5Xc#LldpB6QIGR4P|O~IcWSr`jLI0^F`}z;Q4I34`-O;|rB~S9C{H)DADNB_1T@FK_Nb6&H_Yw#P+1P3m5oCh(Yo=Y9
z1Zx>Ir1&nK|A}#v90Y8wT0@(&YkwKc>q;u!wK@2PG-Uh!`cWjl
zhv!Jfj-FJxwQF+|^PV-n8$1|Y9toJxxMM2g%M>mT8YAR&<;v4QhtVZw7<7&5pTd@J
zda?x`u3YSbW{WxJzrIqoV_W}*W|aQV&k3e`oav^Zz7B8DwbX4t9Bu+mRlelep{mR{
z#n3=#a5@9@#|{$6^K_U);>dowvD7vF_&WT>CtA^L$uvvrCfZpmfm5;%B@dCxFt{xB
z!8}|zF%uv8M$B6|%M4OqomE!iF|57@^{YVm8-(9Etc=DJF%wW5hz=Xx2%s0Ov6`xO
zqp?9szM`|&gu%#aBba4waNEZeOmq9L-(|MeOq|cR{g%?}C6pgpK6AgfcmLYLAm2Vh
ze~ReWJrQ9AlP@fy_fkX@Z3@gJKSo+vwsp=_1X4l`TH3p^vurm%fmb1m!S6
zZs{-3+-@u)CHIUsgU}^hl`({1VEaw2-1g|Eu*9vc_e3OQ!kDCiB=37$1y>8ODPS$Ciz)NQNDnaeFEQgN3Cb^Hc-i&I;PiJ5hp!l!wiRi6VAdNkmceRM^z$)CvK=;S{zqf)m=I;jn5ShPX@t^7c
zD;9sn;;*^gMe@JeiActlV89IjsculPgF0JH@L2n1#yuS0rYnAOqbI+37gd=19Lkk~
z+CPQ+o*Zph=(LCg)Fzm&0x)#Q%jmi`!$yK_11vq!Hl^HlEegJ`b8wOL+x>GO;M9})g%RuEB8x>23EQ})glRmrF^k^sPwQ^F6To
zua+MbPWns-mE9o69t%3>FX6`0pkcHJ4{0xhiM`~YXNOZu)e0u(R{zuuO|Gh6@e~PG
zDNQjk1(75{aW5*e8;oH_25Q}Zhdn8ML2d@jvD%(`nalS>On!WDYau(BJI$GwU%`Tg
z$Q^h$MZtuyP!akh0VFyV5{b!KS`@QcT@R`l#xkE(>i%if>iXB7A!|rMz5Xu5M6_W)
zkybu?0UPXr?0ZTZ2Nr)We}IutK`8G8lL?K0duV9r{q`i-ui4;DE&U32WKEtyxsM_k
zEw$({iga#pUhEEfj0KhK)7{{s{T(z!o*`7B4Fddp=G5c0;y;%1{e$G`J?E+M)Wsn2
zkRK@bt&?D0Z7Xj8P2YY!zO+2EDS*f(qxw=+46Co`rR21-6VlW>>d$UZStx~MU|IIW
zAzL;;Z51@w>H$@Zb!+Dri=YZ>zY!S7n6PMmdc);nC+4K?KKl5VeY!lAp`f>YYic@j
zIKq*$8?4t}2A_00{Ry&0<@Y7KRqij<`p3`e!t=>}5=)mo_gtw1-4eX_n-ca|eZHJu
zrk{npoJVoKSb|HheR>M&@!#Th0t7P={m#f4b1b((Sxh(*FSL1(KWuzQ2lo4aKY&K{
zubI2wQ+7Q?Gulh
zg>)0Zsl-LGytAv?_{cfE%Pls>*9hv#`H!Cp-7UcgSr5fN(V^zXG}jAlxII
z_^Me0Cg!|c1yh33-!GG3Bv;HVvsrz)E!AZ{(FPhrKdK~n%GDQvd-CqEQ}1rJGcw(P
zm&61b!h02uU&Sxl|5+mA=ES!Qyjo4#z^G&g?RSquoL%_}lNk$^iDZCYuqL
z*-r#R2$hRIPp`vV*XNI+U%0F^Mk+gPS%sV^~P}2Z34X*@kwYBzJC(+5=i`l2>
z@#{0buGOD(75uzlAB`gH&*-B1o{pZCSibJi*@+J<0IX4Q{ZD-%oCO-e^^`Rc0G{CK
z=qUd8#0yZ5J(p&PQ+WMys$g}BAws%>`siz@d=B~sLXUmAW!f08tPEi%4uHDpgqa6&
zcpXqY89VtRpf-Tz(K2Y*01On4?t6k0`qc(Yy&j>VN-V8
zC?*>_$gzAjg>iBjj8*%?{Tfl-S_&(k`j<$`(+tY_dkWGd0LPdOFd;)4>5PCbXY`Fb
zQ)R196cI3DIh+u8LPeVwsiy{0uStcd{{hq
z^~hy?Q=;Q(tqZ&}pOsxMnE@0e*GW{%5inZ~9e+yIi?eG~6o{S7>P)HM2v$VR)yodI
zJ8`R}VmL?&Z>irt?L5R2`T2OGNY8R@#@cMv@L4eeP*Ksu4;Y7J!K`U`$?_bhVTUxT
z6$^32Esx)j=V*2kjL-ACBzps~BTdgpmlP#A(Wk4dUCj$U!h;Ha;E5wV2M1d^`maFV&a?c!K}+Xtqws(&_OlWSkz?`nkwkR|h8q~8(RI#yaH?mfI&L${GQp`GSfnCvQ9vl=C}@@Z(g$-d?x
zS$n1Fj88UM2K-b#$|)1Wfq%6K&Airx3Gql2bOG+Y{{dKD}_)Wh(e9UeU)x(I+@K`fS#1Cv8a&_q4o
ze{-fFCn5{BxzZ{`iAE0KijPnQ^X>C=Hj^bah+OR&Zh&UVc%n1~Ss!IBVu!`Vx*ZD2
z4O;X?FRc}Bt`@Sb97%7r-jLCYq|H3n
z(bZ}6^*TUp;d)JPvmcO;cfB}7fENLthct1_)jWE&Q)WBt)ti4QeWNHAXb30_M5l2f
z9VI-If|;t!_OhG=Bwz^NY~L*BNd!zDiU(Xabwm%)Ct{~S-(7c$N4Z*^&~O;`<9MmZ
zQ8la@7v7H+t$|(bCe)(67cv=V^a{v!A-k-~GeGb%7+_%#*Cja{4o!{M<%!HUsS!zz
zaih6h`kn1&A__FIZ!upBL-6ZSn`;q
zG>8K3myI1ePuZtQe0wy!kd_m~wY6y8(xh|?rvZ9gAHRAHbT)K;4;Zr6SN#g
z;NCKFlg}7QtlriW%Qmh9R1uxSx4ontY37DMxGH&9&k%q`%#TW)boh2WZVm@@c#&6t?GJPH0Iak<-u8^3~i952Z8_eXf`ADk>f-C_0(mWkumAvExGKq#TA?w*I)HRbPoQz7CFFVTJ;W
zGUX>J2evioWvVefP^Xx;ejQ~uO2qONJ`gQ#VNcfYFf1o8y1JpVyTBB(#O5RF^Hc2yxs(5N)r)SL
zPqVTEjQ82iKh<`GiX$_P6rGUvF5`o37XcL{w=TG{J>QGqu^h0_*vaf3@j~7YjC&=S
zcQ{#6vbldRf}Js*1`U21ZuI{Fw`v2HBLD)LJB*cS||=4!OZ@&f3Qb3aH8t7v5%#hop%;+C`T@+
zs5r?ndnx&rpc9+1Y_nSfkyGDX5?hrI&R%=B(h+&QLEC>Nyj;QBK?
zh@2J7roVcOLDhxth|NK-AR{S=xdcoaVz4hKpdsEeZ~7nzR+s~kLqRL|aZsBB6i}HC
z90ULU;VIvfK@OD3&l$v>vzRH6zc{ypLlo%Kg+=-Qp&s$12uL4=#M<;R_NoOVZ7Ur0
zs+!L29mNEQl}x|l84?Q4Ax2|&-iWe4td>r}t?#kDRqOYz{-1$@C@d&qMB1fUmO&mE
z+6O&EZMcX46iMKWEVO@a7X5i)9cua&;`|~_Iq{&hRSOIo#D2>ck_F6fyt2#eE0fQs
zgG?FFmIO@Cx_KUOUyUHQe@UZjQ=b8_h&W#$D<(_Rn?V7f<$Iv`kMr<=UA*K)-D{q+
zTg}RVerwb^a#cqQIx1juLTdqwkwyNa-LgxpX0_I&gHeduh
zE^_VbSrag$JZiwas##b$|Hj1g=jCrZvW&E%&CmlaW#J6Qt$_avO4#gx@NAJhL#Z<8
zJ3a%dvJNJoq1L+!VibmGIY)s)RS#>dZ?52@o;=lEo50=nU_g%5*)qYxbzW-?3L0nqay;yU=V
z^lF~w?-V2Bm!NhfW8`m#J>86QuXHpH@{}IgES=$GOGb1v9LmycNRSbiB9-Rr4f)@bpR{d
z0ObHsLOGY2>dXO68vyJMWTo=3OY;0QGZJtWyhr}+4dDrGTVm*fzxy~~Cu)jlZf0DAB=RpF*%l!2a
zvpV(Ab5@_FE5#V&^5x;k$c=f}CLdISUCy)}Zy5zXSy{|sF2Tg`i8a*}hqk8y6sExV
zvhW{UXSTTFezCt%Ci`#RMbV0+(?mCUPCf6|mIW!KkMLN+9LS<`1;fA`Mf4U6G=xl!
zLU~!uOy>F-8Il4(+-rYKH&rjcArjERV(;LfKv=dDw;p)m_DdQrJ#1GpTGtTWTGpaH
zbtTcNXmj?%31g-0lS7=pXr{>3>jZ7Ux#)+R8@I~z64&_4FBE~k2XTv-+_>a$ZUY(5
zLZ}N~k!x_x_MD1epRNKAM_kA=926P&&Os&~qK{yVSaM}BxZ5eQNt75q^smFh#O>kL
z%$ZYJcopl&3pC#w%1
zsl9!eB)08Ce~my-^)CG7eT+A`RG-{Wvnq96D|c;LT@PV$TAL|vY>1U_Y&VJoB%nEX
zU?GEi0AxD{K!X<3t5khD3f|!ot}fx)N}&rqn?%Rq+Hm4bSxcfd)lNT*IB8jN$~{I+
zNl4lRSp?lAuV*)Z-611CG5S_C}3O@&3V
z#Mf|kHbJP+yW8o8qhK24IzUUz$ME@%iUDLTGG26kpflw5QJ-d~+(3Nk2}i0TT!+Ks
zeTQooed`|nUlne%r5
z_ohb?0Qg=40(pN5!QO{9Ao?19?Fmtis}b<|jeu@AWFK?0OH;{V2~jXVqqqD8av-9J
z83^`_9MDpWQ@9S1%|oV}c;t?5p3QZsX9usR+1z&Cf|Bw8l--bBFrU0N2BP8dNojM*
zcQd$PHXs!;$fmKA34}Z!z6&az(@=C?CE*D!#I+6>o-5Su8jP7Fou-xeNH^IMuN(=X
z5dv#5i0K*t+JwS=mZN>eLx!Kx26?IwVqPi9;VLkwb%@K(96M8H10a>J@3I(0?6yc-
z!kL{7#;F%)*rUEfR=yU
zH0d~E7?-yp?SX(eVT>3Zr*e%}I$8??0p6I)P-_P6xa=U^-ERxDN?}Hq6g4B>QoA~z
z>KYzIjb4l4o;;T_m}x~4OGfBdWnuF%iHE4i*B1eHvtl{492)mcLo
zBpg}vofEu*)oe@w14Vq}J4XHOu(t~g#l6q#EvP;w
zE17g=ndu$4uyEWsp;Xx3ue@~VO&Z$lAnLtti;T#9SSni~S4{;bIFS+Ync6y?b-&MY
zvnt$W0RXMx5IT-^yDC1y{Clt^dGu%wWX)C=CL6(mF&)ZG^eUwtwxEz=^0;*gaC`?r
zUQcxVB7gDon)CPfc_N}258<&M&Z?e%NOD?Eg;toWa|qx~Kf!CPqUXrj(EWA-Q2g1i
zV}M7camVh6;6wb2ppG4S3#=bhbP0RKJJiQ&?Q!U$=aFk*
zdiw*vovLKiOHyIIjRsnLSQ{@9!rb#67nMNRP>HC=DH@d2M2I=^qc-UnY@y~JU;;SJ
zfN~32c<7n{=f31n=TURf9F%>qZX>OQ8}r8yW4IIcY4$jC%w!-kQa9YL$&qJYx+GgM
z2Jo^Dl?rA_nkSU{p0-2n9Kva4p|qEsR6HBFt|K}OyxR>@6g02cs$;EaC3uKI@S?8e
zr#;Ko&ckKTsN=&P$ZZ*_<(Xxu7wIvkNgNR7+fO$EJEsLSO<9Z&M4r;4;Gt|Oc4@Tu
zgp}Dgo^t~pAzTEhL{4+z8n-`BOP+St)iHYT&V}r!Gaqf8cZAFS()uG`-UunMcVH%l
z`T_kc87=6IQ?-Gz$re)7P
zt909uBvzBiX(wya{Qg$nXk6%xe?Mo+du3t3#9L$aYaX>im-YLugTE`CWK=@WO|NAIhBr|ATUa#h#`;hr>2LC8v#p
zQf#OMY$59evDMR=Y4`&m{uK0%1CxoiG~B5j&L3!z92ey2pa#qL6JOp)5mVB^J3$N>
zmo!&XG*b20WQN&(j~Ku&y_W_X+zp|^6?tt|S#KJE`Q&xoSbJ7r7xrI%{ar^TM@i+V
z@U~N3LL8bOPU7a;S1S8)XrdNM7h{)9K$Pz=kECS&H=q*)Q5h73HGrazKNt~K+b1m3
zxz(Qtes~9Bc-sP^+aGTCdXHRMr|Ec0GEoZ&6-}TXLN#{^M(8OUaI~`1wJ@!msn5WIT7{o_5p#Q>un6$DCSIOO01Idwyz*Z<4`JsO-E1-Qt4
z;AcX>D^nZ*RMrR<5upbvTsENS8zR?P>QclM^!ui&)Ib-42E!j?Hr7|;fwMkK;McaP
z1O$7HAf0^`0p3WW15on?wu0FmAr0W0YJ$wh6v%$7DxgwWcrPbKmPz9osLlBu*iDnQ
zrJ2~NgWi8=vQW{6B!Sdgl^^iqb(p2v_kh$7i|++{+?QZ1Sk}85>QQ17r2`C1;7G+Z6@2)e!f{l
zZLOITB0ui{_V+&XY{Y-}{{A$QCXk}cPqg!d14o$Xl?VDgQfGyfe#fnWq^Ph#tL(_O
zh=iu^<+vLu{$vNQMjfM9=lz!((6{s2O|)&{0yd)!&)y5|Xc#>gTLdM*P=PeFVc|HB
zVGN*9MkebZbalT%Q~%KTe{1vP`C0K;I{R$PzG53_Bp&o+NE|xBdrnFT2qrGXj;dGM
zL-E41L-c_m0$jFCuOo31znKrod9p50v-EZd2D$Z}TnF$zCKJ#Lyz-!2W4LHb7D1(G
zt2(Tntd(4Qx}{PGB2_c%%u)olO?z}6YDs3XF|CX}c3s0%lPY?AF9m~UqXy_J&@U&vkiBxPNq;-
zy=1ii>Ng6;$CWJd=heqRa$2iRAv#?o+{z|?K`X;Jc5xXnaues_FK#svT2o<6W)Kt^
z2XDDso!|sj{6H5|sFK9`oYOKuUAe{9rc2o3D$9srHVxNtW}$NkAmrDc3IM#{2Gru?
zDHw%gKr+My8O1_hJeqk3)OHXqUW$3LV!wl*Dw!l{8=U}F4u{liD+|js?-n-8>3g
zNm3-;9=hFGaAZ3ft3hJ}T>ujBdbTu~yte=*M0yyMrp}&CnYV3vxNG-~m^|)$^&PhA
zX1ZimAcwVJ%NHJTP>vV{x?I<2V9=gM9ROY|mvANr?LZYKJ+d&7UN`jGrzTVY+iK#x
z=&=IcbEHVI3BB$JD{$GXMfA~7LX^VJrO
z0SMGl1$t7np<<495mTK7&a2Pf^NtR%`-+G1`nvCM9saVt&|Ai+9zleuiQOY0gT9dZ
zhIaKZ?@e`^$lwhJ0KtCsKFe~=tL7s7dIsHf4VcHf4AuUE%2|?I+_UD(yyg5L2to&}*Z(V2aM*kH8jUcgd&;l4o&hLfCxrf-=k~(g==&GNW1`Vt!@{oiQwYL
z&|SKFZ43ZK_!@qhCa496Z_)$|#yEf#+yGRgv*tPLEAv&MukfHSZJ*wtxe7UId8k=1
zTB^cXQho){|KCYJfcau%Svpf4EsJ=pm!_~YoPEi;H>5UKvXITsc
z6njoj0TEGVc3rcA21TrwS~C%?*l{ybheOGPAg(sj4U+|c*>QOfPjqHm97XB)Ui${9
zXzn9%y`M#4-cJ*7{-WthD|@C%MUiV~#x5{WPjK)U}@lC+L!X_E-q8Dl-!MoX-
z+LsCz*5M(h48FF+x23~N1$N>6Cz?03c$#w*4hw^d&>VQVUmU>E2AQWmnh|;mT{{5U
z79=*o^SfJ$@+YaqQ-F
zG&Csf1}rPDH%22j*q~%)lpVG#m;Zrk8b?#IGbQ`S?LH-Rec%_tyKagiLHpmD2J;4R
zz9CAYGo>p954?U~04S4fr6}`b&FoO*#g3co%qGZF88|7B`--+-neb)d939~GVHAL!ouCpiFxR{sfJD(Cjt;&X0YWEg5!fR=
zCAWI~M=vxqQcB9E4Tsxf5Fy}BDapooorV;1u-SbY+pY{ry%mXw2$K(eK0cAXP4~JL
z=)zG_K#oAYs93Px$$qE*q@Th@9#|j9f%Dn3nQ1?}zDrLKYV4t!ryo-86MVWkNvPgit^GhRFo3%2Zd(d+}vR2On_qOfsSAQoA@HFUP^|ijVP^;6aK^WWUzyMh+q?pY4Fe
zQyI-Ff|4mD?0{^4Q~@352;y5{TCv}J4*afOP`z`-@<;vh;jQGv3i8iM9UG@e7_ryW
ze{m6=FA0Z%z=V=wwJPdSw4Mkd7d~HlFD$_ZuIVr#6O{@R^3QaEK|9OzR6dAR>mKY@
z95_^rW^Ezn=9MAHNmNitxucqHa&B@Ex{HV5t``xaGoI>%$&mLDG1VUK_`V5*Mtp^9
zKOvN-1x_uL9DD2YZf_3aL`c;fF;^8q*ZDxynCPX0;XDS+hnz2Yumz5BMgl@pu{rm$
zb2eG#_QI8l#qz}w5;Yy#l92-Y0dNy`{rV-#d8{*R1Ip;k7UyUBfr?)~*o=RG5i+S%
zT9}(wMHnk`1|tpC1zEy<6nCz)C8ZUHkpu(i)F4~^eqeN;L>w9KQPt;%(1nYJ{-R-1
z>bvmwxk}>c$GUjR(11Y}V5+UrhqF^X@L$)LuS&By4
z4a%7^?tYVx<{IC2N`K!6q`RGCh#L@2BJDpxeb3=OY4&bnqx*2Tv!qc`|EKUQkl;*`a*t&7sfn5lWl@r~i*`2pWoO55t#J_Ot4xti4pLLjp>r%TC-ifvF5D
zabTU#B~ZyEUHLQ%5Wb2>LJ8l#ke9~hUZvt3zsd#k$lVwB6l*kT=kklD-_bXX$uh0s
zSnGm%%2AxWcfC|$*2Ra%SBIgX1KV9D|pE<+Udo!7`Zu
z%1V|LxzA0dBnQ`3(5)TWHpZ|7$w
zCZeA~0dO$av<5T+8Npm*;)QAY2`_IE90b&qJo)9qnK$qmfJVTD`U@Zs4{kd}Lf{ZF
z2Oj1XL{ft^qt$`}rfIsK3c+9Lz54F2^$&Sko(9g^V?M?*`s=alW4`Ts$t%p$mkoLH
zb>5t#g+m$T13@hNjbsp^RDD
zTqBHOq2AjYi&?*&bSZ7Uj}S50gj3&p2LUYU!&o|hy7|K
zwplq=M<3}2@I_HEWYPU`4{A}^m>dFl()`QTs^5wBuf{n02DpMB?>~1ml1I}1OSL_W
z9*hH3P`b)A9^(x?WCcM9RZ1V358YyD!pZRl_sJwftLg&C#<_E(G%2JcfY-yu0{$2Y
z9txGxH9U)Rw)M^)O?J|X@;4^WMlR5!oDxU?9}~|c0qa#P4q;c0!@qtN>f!6*>HhgGk)u-b7MHh`Xu
z0}AYNKAhoLn4VFGu-iS*L1pU%5`qHoq!i)ssK8}uDAh|2R6^H*LoIBS{}lkkj}Tf*
z41wk~Cgdzx=!DWJ8*jvVGe^Qm6VnH+pqi$l*G<7x>v0W?Z^h-(7m
zey!uLAB
zGSvyfH&cYx&7#u*$9}#4j~`5*q7MUyJe$P5;CKCw!E14Rlmy?`!MQSuwZc}kt^m&2
zkH6oWMwbgLG2WCIs^wQT9HI=A8CtX#x&!`xy$q8WLB2^=HF(Q#izYuCIf)3=vzsUu
zGv=f=cNU|7K{&#e7QdB9$3uM*i*QPSz~cakjz4mJO&kOVG1NgdmngnYcI`mpLC!fB
z9wM%82sED|Zpp_3cA)yrZ1__iLiSXV=dCN!P2FTP0De?fJ~@5ts?4_4u&0y(c9pwC
zfIg@_nz{nm?ni_)Nw&fpBfHc04sc0Eg^f_?evr0yvI`UtBNw{rR3uYWE-|ic$ufJ0
z<|loG4KEBp*n#PrIh5)Zl-}`Q%i{vxfNexEBO6c-f(ck%$R{b}O0WR6C@!=BQLPR&
zv_UO(R^S`TufPNNA)@9G<_fNpuL&>1OVE_mA{dHZYj&xfAF5Tjq)cu)Adn4}-wgbE
zFW7jsrBW-2Q}yU=Ja=a@Iy39_pwy+L)OX@)L8ICjVm5U$L{Uz
z&8kQQCp86BZ^9Kqu8pB&b%QC{K}p{1b&hkh`A4fX>E65Y^}0a{(Bde2>`|PI4+^4g
z27Y6DhB+yRKxxY?0!dK|6bdb^zHl@P3a(%=Ir*(S&wiRiScpWIuh96oiChmgaffq<
z7b#nV?bH3%HU(kXh$bbMEs+Ll-pl!rjN7$M6pXkFK;~-gP$(dE?1eMJ@Jas8$g&5r
zqY6V*cpxiTxoi~RKE^4A=p{UHYtS%IMfft1O)QYSs_aIXP
zh)s1Od+pJb;kQ*#a4rlMEm|;(m}?-AvQWvLvjuU?ihM5d1t0a91ynw&ZVihiDN2U5
z09&K0A4Zp+MB}_8mGdBdw+CDz0cDQ}+k6fiU&O&j3|O62UPd>p1!Xgzj1IRiEwrm!
zUe15N=kdpFHH<=HG2ws5QsGi9i?0c??Y;6u0sRiJGD
zj8w_G=G>DhswkIg@`ek=wvYO$MJ$mYK<20#)1T~u-6MY}BhxJ`t2Y$7Go>O-s!8UG>
zK4Ra&m@TIImU8B(BB2F~_Q
z34EWn<4~XUl1nonArrHg(|cQ>Vh?^>)R{k7kONn4;alFr40TosAiqp_(~!!#^Sq@w
ztzsO0)u+a6+S>nLWn^(*{0MaHbd~!X|m)*;8PxF^ooe@7Vhu(lWx1g2S>|&(LkHc!
zC*Wh=Fyfw)=6ov)cz{N!SM5Ngm}s)8(l#J(d1n(5Jx@s3ziOo5z1*U<{Bs5dYEx5CWI3yC}z3y-tLPsLZ7GF=fO7
znqQTJ4RTktZa4{gTSK^17EE!J+bueJ=Rz~4yB+M+YXM@Nm}~qLBPcnM`lZ=o4jVN6
zrMa-nTu&fpa-nExdZ}0cB*z&)T1Wim?5Q42(bx>f)_NDVh;0+4B>`-PLgNA+L)mqo
zzI<(@-+j6*G;=$*Q*17Y%|MfuKB<`yQMr7f6OXgYKg8QZa}Lo(B{okjjCe{T@kZI%
zcYv}z_CGXUinEcCU7_JC>~rKKt)6?$?u#lYV{tK3Cj#GkuJ}Zew%Z6WKWul*Fvd(E
zyES{*AKgZRRB%<4-o+@x5MN`RF%(F8uDzd4rFBtBKPPu(2|A8@(pE5?!w43@g(e&Z;E1&WpHrL;`YfpX
z-WE6BI;cRAOdXc>3M4DRZ*u@Ua1_7&yRFvYbL4pXjI8Wyt$z=Ak3+)1=aiD|!k+4h
zMhYL=E81!bNoquSi6tIrBciG^d~1yHCtjg(1L$%wgCcsPlq_)rp01Oc&$!DQV8Px#
z1oRMj{9M_*^esDRIal9w^5aO-eIDw_yn!nCHNEacj$h$;ss5pEcjl3|-@v%nfZD+?c!p3%{tD=`(vk;PLVF
zv{&%Huc=48xw8bDQ@PTU%)7VQRy6kjrCO&S3iiRYFYcC$mms(7i>m@jEq6hAaOi!e
z3(yX8DD-}kUJKfbM9G
zx@J4^a4MaUN1t|x)Qka}``1(6ulVFE>0j%+`!RtVdL-laiFk{U6PCsmg)imzu{Slk
zW~Y5#PP$W5@fr+p!DKV9oOmMh?Ob8KoK|GyM_@@S3@0+DZuWOz13Z92FRdG
z_t_7te78(uW`~5T4m|{D?7rBIFHppIKaUqw+#mnA_-hU7@dM`Q&b#pgMvK6dMYlWS
z0T(S*F|!@*4u5AgN{aI;(VMp{dFvSiP%R
zGWO%}O#$=Vrf;;z@5-@v-KJ?dP_kJlht(R;RpqUitv=s2%m1sIqK|ok=NjYmt!;G8
z8)shJYKA@f{Yp&M^(q{kavUag73euSiab-={qJx;tk=Y7=&W*14V=a5S~Bl6vp!^^|-fLOnOZ(qJkh{KC-pLaUZ+kMAFai!PwoVUgF5T^JzC#0cOg$
zqQbo3Zo9MZvm|{?z%{Phtd~z9UWx}B(Qh1dS5id{i+dZoB_WlWaL2f0h4wRf%H0j$
zow`-Bqe092FpW3iStgLtyG`yCtPQU4?R4>`y>m(|e%tZ8{rFRza|aoAUR>pwEjCt4
z&(jfgaY=;P(kv^c*0VdEXtPh7iQ7iD$EVp!RDNELYWwq1fMG6QVq99`{UlzHSzt>W
zmmr-mgVub4=WK$6vTvYxj+9h??8EWS2YxA9eH=+Mub0NOl@BJ6dJtYBG+!jGT0{=E
zJioxmB)Two!E+VSaHW@RjDY_2%m^JvFT*P)Gta#)c*Y;dE~>Tj6R17lXyGT)i;+3y
z{#jkg*ssQN%$<0NKR%&&jHBAS3jRgH@I2PX;&|{DxAj@(^@)}2&pYQX7{)ASRBhw@
zf(h_yKB6C_f=pJ?syZpGm4&%t?@Q#`%d+{c>XeDJ5kyh^Tz_15>clGuJS>NKGZ
zbUSK=Pz&Kz2G!ymA%jCIjn|7QMbqtKN5-_@Hor-}l?+V^a3hLOUf?S?81(tZdPML1
zVL!g|HBPn$#QV4%{Aev~Bd*n0KCB7@uC4
zr@2!(bpKBLy?Q49StL>+;4rks&~@n;e=EN=PRxxyv!NNHPL?_Gi*|7XXkjV>ec{K>&B}`u|QdvVuxkoz$51K!H8N
zN%*e#hw@?8)Ap^5*)5iq8ni1qA$xu#mU2hm#F)osC@cd6Bn`gQzRxoS6>LBZ;=^Qfgb
z(~w@>h2$57h@aN~m>PT7PYafSM3CE#O@E&@BtC1s$ioqkY!~KAx6*xu{mKNED
z6l!%U^vbCmAM2M#@5Y9=UQ}#2XOWlZ`O>;7`-)r$r%W{W20Cm-FhEcM>g;%(ld;Js
z!rhxYl{X2S-(J#C4%uKuosrLd`$>J?h5{cy9`DKVfuyY^%(`h$Cbs|hYdmT&Q^G4E
zV>oOVPS>d+$a(K){oPY#`4$gHty|Axw|d31tKD#d>=DsgdgN16(o~uORZ^c%Nfyr9
zIFZqZBwu|q6ee#jAaN3=$r|-tkf)}%*8hOv0L`SMWY!i*E;mchUkVeo(phA!(}V^~
zr2oOrJILWPuD3$*k(}nBO}QO;2Qv8yYCs}%7pC%Zz@mVIi@rG_RUh?&5H}XF6(`ov
zrpF}x2)2Ks6wx|p8=|p5Bv;K_bSPg(JXAkV%Ws!Z+Y8M}A;L7N?u>lz;_Z)Hedka9(|Sp}za>_*zGx
z9l&(h?83C|p(m{Xbdn>0D?}V-tJ&4*2x@l;(Rm)
zI*?m^K&A@BsBBTdNHo}*Rv9Ao=fn6>o}rexX!0Eg_|~l>@d!aBNz9Qc6slyrJBAT(
zgQQ>0sKp4SpH63LwLJxl%mLvxau3)3%XAMJ_8`kHgtUBxDkCm~3(#0qMVTq=rOM0g6r!O_bTSEA
z^ut0;vNr;Hg3nzhdLCav#G(pbL_f}$y6#9}^|e)uW_DTT@eD4#f-2pPGY3d&t}JyR
zT_I@!fZXC*81);B|o2t*u1>9gg4@^t^2I
z<#IlCBxaOwLee(qF)NbB4FRC0eygNLe)(>Y!LQQ;#fTZ0Vw?bojhZBu803<5pje3H(DP9v{zS1(KNhuOK9u&Y6A>eTwy)|6*g+js2*I0k&Fgs#
zyTDLK5uag(ti~W{=-cXeOCat9%CHa-*4$q#NDMj^Ph|nYzuMw_JFV_G}?cC7UQq*LvchBDlaXLEJY&V#2(7mo~_=8HzV#1BBb}k)pi;n!z>c^
z{kE{kFmJX!hzIB6$BlY}QoyvG=OlGwHp?}$Y;ViBP(CmssQQ!E_G^o3ul8zISJV2O
z)sq(+dFwn2Bl2~2eZ0+B|h>#w{39t6gF2uXzdWH$pr-3r7?$mA@!Kjeg~
zapU<9cl~Fo$huXa9iTPG_lCmyugk|^N{Yp_FN&8Ob(2k;~0G`l3gN`xI6U9+YgKN-vQRke5*59
z9n>P~RO-u4Z@O(K0+~K!0lOd;Ip5n1y#U*h?g~!LsUg~L_&bHv!`_Hk+~d&iUB&C>
zmvac9D1d|<6Cb_NZ5PKsdkyu%4Szw50M&agRJ}(si80bU!j!_({|K;ZR00W7Pa%{x=sxr@P9W);q=e5$Gh_oT4Gxnq9ztD3)+ZpFjKW2Ob4q
z>iL=gk3ebpdt%VT>`anKyFXzGh^|Npc*?bRJIopYElk+x)zz@B`o(ycZ9DdRgr@KR
zj9`DHd`fsP*gOoE$cMLkjZevwWq8n@{gdF3^n>7V0)@y(N9M;`^B_akNj0!#zm~~`
z1mw1+K?Yqo5Q!g13z#liBrGX_FwUTVCVi*8@03r4U_&gOm%i_;g+d(NXZ?8^jAZ8t
zP0pGEK<6d`y1~g%WvR<^K7h`Lp*XO)(Lj4S@I|fRgCwBiS+xKq2nldERNEmwqm2&j
zwz6+yU3S@xqZ|r|3!S`^0Ih{96`k5R50Z1WH
z38nI|^Ey(jxsPH)pELz3aK~53--b%){qWEJ=N!lwq{?*HZ3aRy{*xv7lQN@%)PB;v{M2Nx7>aZ
zrY~P*$YfXxG_Yadp&&MX&+gJcSP{M~OP!2GrIK|d8eE#^g2f_la*UsCz^p=YIpbq<
zUt}w9S^%7<{i?V9OuUs(#(d1ZrGH3OX`6pfEyqhYpf*hSXDeIPdDp^~e1GgMD*$x3
ziVfR}-}sV;Eaugi{uk7v2+m8*>kSJ#&U)~YQ6vuwprOy`vDNZ-{q}K``W#U?1=sIG
z(kRU^=vZAagAD_shV5`i^#e7lQV6`>1_4j4)a9twJ=L&sofb+4FrbRAqZ@dp_!I^4Th+MhW|-_T1}So*=vHUY(y
zO_0>nlArX|WMeOB5#16od#u711S%_!AGnOxMuc4Abs|{i(H0;h|15tj1H5uy~BW5g|ZYtPc9yszCDw}V6pHBD5LmI
zw_#T1@y&F`9ZxkB7X{3_KQ%
zd5hI9vX1c+HK#KM75DhZW)QfFCKf7C%ADENYOoa~0h)iNA2$>fW=bLsWY;8!O6Ae}
zqIxJ)Vh-fiIW?iS{$yJK6ofjfctmkXUwqn9r>3zMZV_ZyERvJ|(`oI`DiAp8@UA&l
zE%D#fDg064GhW`UFP=VkywgXf%LZNzHTB5)b(D0uJW7BVoZJ3XoWeIoQCL>ir
z(Q*@}2lFJ$3X_}QT$S0$j^o`$uGWWf2K(XtZaUoQrLeiUU_j-~8|34HS5VRKVaFxC
z*i^&1yaIu%{w8=c-5aGO1bqYVN8hkBjGQ|n6AcqI>9-hA+B~7~es3g>*-?1DbYK;9
z^bI+3^pDs2tuUEc`?l$(KUeGfJ9wcHpC_Ej2R*Z^orCvVKS)zScBTDYtUtHM-^Kdf
zWB+@y{>bkCo~+;T@NXFWeR=(#ac61c2{RLY2>>t)J8NE<&MjK7y6Mo_UEE+XDqr{u
zd-?UlGaavuiBsEtXhP15_c_V}JR5;)gwem|=Gbl}KMfX-p7z>#vPNyekE$FDuTq
zYU)}AA!B8GWO)i@Of2$?A=p?JNH9*58)HPjUpMd0rTx(D>lx1uq<2IZBN4O+NK{^~
z-+@f0Am>q$=m}J>MV^vK%W4nK)A69p;3Lgk08n@)9C^U4p)m1Gn0v+NRv!Cl_z*#8
zcsY1%+0?r=gbEs;26|g9a#QG=Td{IYZ0H`AC;HIwp=f`$DlGh}S*G96IA|v0kVpWP
z9nZljWHNFRYCu3SCE}Dtn_?@c;lEWxbuph}))mk)cTnZsaX^`1pl*uD(ZE~k@yaxJ+XAMqK0^HkGda;3>q
z`P+eK;@@+zr?K0!#-`CI2;2V8%9XK;Q81~P1NZ%GN79#yxE?G2RD+=13n3w{Fz8p`pmKP6>_$Bnc}9wrd}A?xb_3+eL^vWU-_r?T4I5s
z6?m@^Cdu>DYT?yk2;@-gsv&x3noD18Oy*b=ZH!)=7b3OxFuU5kfd`ZFg}LxYWxSa`o^|B^JQ`<+_M1TD}K;eWQ;wPVfym%Oyz-`sz&FsSj
z9?x&tW6qO)xFq-1#uuQWrq3=)S}07PG>__9&f|czzK+JUO?XG0It95KR}|#NSKQW>qJ+E=4Zk&-igTwlrtu7nX-pd5H04a;
zw9&(9%N@xg_u+~zgBW?H?e&fT@`H#&CtxZfnEV82!(Yg}!5&
z4}0^vVA||Ej(NP)-t1iYE7$E*_c4k;K-Dj2vWo9RbolVZD6gh&)K&*uXXNW~C^FVCJ?u(1}=4
zSweh6YLUew4b)5f@5rr!!OH0Ou&^DAze!H>j$+$4?ctPP7Xy%Q^S=cygE@!LI
zu+#8I<2Gz=q;QK8>DAh@#Uy#jwCgl~m*=X?|8{wD?+jhMc(DushH}6y7B8m_64D4Q
zn$w<+gHBj!$y@E@iNyK!Uk{Y%G(AteMg*&xeaO=x?{bd<2#sIAz6^BH0}>JvBj-Uw
zY3Bk4%;i7_(u96(NCtFg$6<6=Fo>x$l69w#Yc-EJPHY-p@kh}9o(#w&tO|5;f42bd
z9tixH1Xc4m8_3GaPF5BH3-;3yMCduNf4>@PN$2$?vTb5LsFoP;UNcVL0Uo=nu_yUc
zx7U0nITFmY02Ggb!$|?fm2L6y@nz5+D+9#gVpOA_|J{H7*;;t=*b$Ay5fH)}v>O&)
z$txUj!n&+I+j;%{`}ezt&BXqb`)T`L_4MSEx9G`j1-4rm+eg}U!ZPTu(7dTn(kcha
zhvBW;x68qXug_mC$WDyGsb0V&Y7TInL903V-mfTr@2gI2d^@KGu)~Q-mB=1>B9&cN
zKPYFGAID;osr4wL06r2+KxSacV%6KzABO-P@aowBHcorXrSEY5Xj-If868Q}#9v5X
zrNh|~;pF2x00NFg0V!M9UKxOYxwy3haWBqQtIaQWU~V|ErK7{hy?w-SvUu@l&-!=!
z<4f|@ackNdPY}ZQfYDwL(5(1?Pg8pBaMI<}S<@k)NA)fM7*d$TX49IV{r!=RktMY<
z(ROIhaHgfw%)uC$U{Gua1NUVp-tCN5Pznqwk&==cnKr2m=RRZW1fyX;R!w7Dao3&y
zON&K2yjpxky9?oXx1q<(1#yP~jcv@#JV5N_H#Ro5b4!7(>+A2|5O7>xz6^x(r;xx1
z?Ulr&51Mg?XL`58zV=9Me$h_sRq3PvMykNQGka$N3=SqG2S6OFMjGbkros5+S=&h`
z@KXxh$RZjDBRfNY!ucfpkW0pg4<7=df0_w?TU#fjF1En2ROyCenX>ie*QomqDNWb3
z)Qm=oG)Ij#{h<$+(Sj?-*E98ZTVgtdi{--e=m!_*X#0R@9$;t|Mq3^<>rTT|5EwUF)=Zj
zcIWRicUTp_ct)YO*U;&jOuG5hwFQTF9;TP4pPF_i7=>sY=wJIfQS%$$Ra3`@%EU%=t7ADRJsv2eTAK(1J&0lY@H%QQ_d?=*6I5HYw}N1n$EpNb~DlXEzy
zjLIWK`=0@)$pPEjP+bE5;0$uG=J&NBQ%C*dCAA7Sk-9lHsO|ifD>ziE3_*?0D(!)T
zNku|&gux)B$It`wuQ=5{3cwH_07EH9F4+_r$maX_InU
zBSgS3wd$o=UHod3)JVIYrq%mWJTMr{)B3^OZ$0aj_-vfqa`igc_za7B1XPVOzKtx6
zuWBinmUha5Y1KtaUJK=)+J{NouVc-Mp4N@*p3`q1J$f`t2POfAmlDCuIcT6tp>00u
zcF;%reaq{pR|D&lE!DH~Eo^$4L~Yvi(&h&0lQuLM9(Drx51(_}R`QMAN{2o-8XN8A
zUJR3{_8pvp02L09+(%i4`iymn}H
Date: Fri, 13 Oct 2017 02:36:16 -0700
Subject: [PATCH 002/145] minor code formatting

added a trace during sequence encoding
---
 lib/compress/zstd_compress.c | 171 +++++++++++++++++++----------------
 lib/compress/zstd_compress.h |   2 +-
 2 files changed, 94 insertions(+), 79 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 2c46c79f1c1..e072fe30e84 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -1238,7 +1238,7 @@ static size_t ZSTD_compressLiterals (ZSTD_entropyCTables_t * entropy,
             ostart[4] = (BYTE)(cLitSize >> 10);
             break;
         }
-    default:   /* not possible : lhSize is {3,4,5} */
+    default:  /* not possible : lhSize is {3,4,5} */
         assert(0);
     }
     return lhSize+cLitSize;
@@ -1273,7 +1273,8 @@ typedef enum {
     ZSTD_defaultAllowed = 1
 } ZSTD_defaultPolicy_e;
 
-MEM_STATIC symbolEncodingType_e ZSTD_selectEncodingType(
+MEM_STATIC
+symbolEncodingType_e ZSTD_selectEncodingType(
         FSE_repeat* repeatMode, size_t const mostFrequent, size_t nbSeq,
         U32 defaultNormLog, ZSTD_defaultPolicy_e const isDefaultAllowed)
 {
@@ -1288,10 +1289,12 @@ MEM_STATIC symbolEncodingType_e ZSTD_selectEncodingType(
         *repeatMode = FSE_repeat_check;
         return set_rle;
     }
-    if (isDefaultAllowed && (*repeatMode == FSE_repeat_valid) && (nbSeq < MAX_SEQ_FOR_STATIC_FSE)) {
+    if ( isDefaultAllowed
+      && (*repeatMode == FSE_repeat_valid) && (nbSeq < MAX_SEQ_FOR_STATIC_FSE)) {
         return set_repeat;
     }
-    if (isDefaultAllowed && ((nbSeq < MIN_SEQ_FOR_DYNAMIC_FSE) || (mostFrequent < (nbSeq >> (defaultNormLog-1))))) {
+    if ( isDefaultAllowed
+      && ((nbSeq < MIN_SEQ_FOR_DYNAMIC_FSE) || (mostFrequent < (nbSeq >> (defaultNormLog-1)))) ) {
         *repeatMode = FSE_repeat_valid;
         return set_basic;
     }
@@ -1299,7 +1302,8 @@ MEM_STATIC symbolEncodingType_e ZSTD_selectEncodingType(
     return set_compressed;
 }
 
-MEM_STATIC size_t ZSTD_buildCTable(void* dst, size_t dstCapacity,
+MEM_STATIC
+size_t ZSTD_buildCTable(void* dst, size_t dstCapacity,
         FSE_CTable* CTable, U32 FSELog, symbolEncodingType_e type,
         U32* count, U32 max,
         BYTE const* codeTable, size_t nbSeq,
@@ -1317,7 +1321,7 @@ MEM_STATIC size_t ZSTD_buildCTable(void* dst, size_t dstCapacity,
     case set_repeat:
         return 0;
     case set_basic:
-        CHECK_F(FSE_buildCTable_wksp(CTable, defaultNorm, defaultMax, defaultNormLog, workspace, workspaceSize));
+        CHECK_F(FSE_buildCTable_wksp(CTable, defaultNorm, defaultMax, defaultNormLog, workspace, workspaceSize));  /* note : could be pre-calculated */
         return 0;
     case set_compressed: {
         S16 norm[MaxSeq + 1];
@@ -1339,11 +1343,13 @@ MEM_STATIC size_t ZSTD_buildCTable(void* dst, size_t dstCapacity,
     }
 }
 
-MEM_STATIC size_t ZSTD_encodeSequences(void* dst, size_t dstCapacity,
-    FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable,
-    FSE_CTable const* CTable_OffsetBits, BYTE const* ofCodeTable,
-    FSE_CTable const* CTable_LitLength, BYTE const* llCodeTable,
-    seqDef const* sequences, size_t nbSeq, int longOffsets)
+MEM_STATIC
+size_t ZSTD_encodeSequences(
+            void* dst, size_t dstCapacity,
+            FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable,
+            FSE_CTable const* CTable_OffsetBits, BYTE const* ofCodeTable,
+            FSE_CTable const* CTable_LitLength, BYTE const* llCodeTable,
+            seqDef const* sequences, size_t nbSeq, int longOffsets)
 {
     BIT_CStream_t blockStream;
     FSE_CState_t  stateMatchLength;
@@ -1380,8 +1386,12 @@ MEM_STATIC size_t ZSTD_encodeSequences(void* dst, size_t dstCapacity,
             BYTE const ofCode = ofCodeTable[n];
             BYTE const mlCode = mlCodeTable[n];
             U32  const llBits = LL_bits[llCode];
-            U32  const ofBits = ofCode;                                     /* 32b*/  /* 64b*/
+            U32  const ofBits = ofCode;
             U32  const mlBits = ML_bits[mlCode];
+            DEBUGLOG(6, "encoding: litlen:%u - matchlen:%u - offCode:%u",
+                        sequences[n].litLength,
+                        sequences[n].matchLength + MINMATCH,
+                        sequences[n].offset);                               /* 32b*/  /* 64b*/
                                                                             /* (7)*/  /* (7)*/
             FSE_encodeSymbol(&blockStream, &stateOffsetBits, ofCode);       /* 15 */  /* 15 */
             FSE_encodeSymbol(&blockStream, &stateMatchLength, mlCode);      /* 24 */  /* 24 */
@@ -1462,7 +1472,7 @@ MEM_STATIC size_t ZSTD_compressSequences_internal(seqStore_t* seqStorePtr,
 
     /* convert length/distances into codes */
     ZSTD_seqToCodes(seqStorePtr);
-    /* CTable for Literal Lengths */
+    /* build CTable for Literal Lengths */
     {   U32 max = MaxLL;
         size_t const mostFrequent = FSE_countFast_wksp(count, &max, llCodeTable, nbSeq, entropy->workspace);
         LLtype = ZSTD_selectEncodingType(&entropy->litlength_repeatMode, mostFrequent, nbSeq, LL_defaultNormLog, ZSTD_defaultAllowed);
@@ -1472,11 +1482,11 @@ MEM_STATIC size_t ZSTD_compressSequences_internal(seqStore_t* seqStorePtr,
             if (ZSTD_isError(countSize)) return countSize;
             op += countSize;
     }   }
-    /* CTable for Offsets */
+    /* build CTable for Offsets */
     {   U32 max = MaxOff;
         size_t const mostFrequent = FSE_countFast_wksp(count, &max, ofCodeTable, nbSeq, entropy->workspace);
         /* We can only use the basic table if max <= DefaultMaxOff, otherwise the offsets are too large */
-        ZSTD_defaultPolicy_e const defaultPolicy = max <= DefaultMaxOff ? ZSTD_defaultAllowed : ZSTD_defaultDisallowed;
+        ZSTD_defaultPolicy_e const defaultPolicy = (max <= DefaultMaxOff) ? ZSTD_defaultAllowed : ZSTD_defaultDisallowed;
         Offtype = ZSTD_selectEncodingType(&entropy->offcode_repeatMode, mostFrequent, nbSeq, OF_defaultNormLog, defaultPolicy);
         {   size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_OffsetBits, OffFSELog, (symbolEncodingType_e)Offtype,
                     count, max, ofCodeTable, nbSeq, OF_defaultNorm, OF_defaultNormLog, DefaultMaxOff,
@@ -1484,7 +1494,7 @@ MEM_STATIC size_t ZSTD_compressSequences_internal(seqStore_t* seqStorePtr,
             if (ZSTD_isError(countSize)) return countSize;
             op += countSize;
     }   }
-    /* CTable for MatchLengths */
+    /* build CTable for MatchLengths */
     {   U32 max = MaxML;
         size_t const mostFrequent = FSE_countFast_wksp(count, &max, mlCodeTable, nbSeq, entropy->workspace);
         MLtype = ZSTD_selectEncodingType(&entropy->matchlength_repeatMode, mostFrequent, nbSeq, ML_defaultNormLog, ZSTD_defaultAllowed);
@@ -1497,13 +1507,15 @@ MEM_STATIC size_t ZSTD_compressSequences_internal(seqStore_t* seqStorePtr,
 
     *seqHead = (BYTE)((LLtype<<6) + (Offtype<<4) + (MLtype<<2));
 
-    {   size_t const streamSize = ZSTD_encodeSequences(op, oend - op,
-                CTable_MatchLength, mlCodeTable,
-                CTable_OffsetBits, ofCodeTable,
-                CTable_LitLength, llCodeTable,
-                sequences, nbSeq, longOffsets);
-        if (ZSTD_isError(streamSize)) return streamSize;
-        op += streamSize;
+    {   size_t const bitstreamSize = ZSTD_encodeSequences(
+                                        op, oend - op,
+                                        CTable_MatchLength, mlCodeTable,
+                                        CTable_OffsetBits, ofCodeTable,
+                                        CTable_LitLength, llCodeTable,
+                                        sequences, nbSeq,
+                                        longOffsets);
+        if (ZSTD_isError(bitstreamSize)) return bitstreamSize;
+        op += bitstreamSize;
     }
 
     return op - ostart;
@@ -1517,27 +1529,27 @@ MEM_STATIC size_t ZSTD_compressSequences(seqStore_t* seqStorePtr,
 {
     size_t const cSize = ZSTD_compressSequences_internal(seqStorePtr, entropy, cParams,
                                                          dst, dstCapacity);
-    size_t const minGain = ZSTD_minGain(srcSize);
-    size_t const maxCSize = srcSize - minGain;
     /* If the srcSize <= dstCapacity, then there is enough space to write a
      * raw uncompressed block. Since we ran out of space, the block must not
      * be compressible, so fall back to a raw uncompressed block.
      */
-    int const uncompressibleError = cSize == ERROR(dstSize_tooSmall) && srcSize <= dstCapacity;
-
+    int const uncompressibleError = (cSize == ERROR(dstSize_tooSmall)) && (srcSize <= dstCapacity);
     if (ZSTD_isError(cSize) && !uncompressibleError)
         return cSize;
+
     /* Check compressibility */
-    if (cSize >= maxCSize || uncompressibleError) {
-        entropy->hufCTable_repeatMode = HUF_repeat_none;
-        entropy->offcode_repeatMode = FSE_repeat_none;
-        entropy->matchlength_repeatMode = FSE_repeat_none;
-        entropy->litlength_repeatMode = FSE_repeat_none;
-        return 0;
-    }
+    {   size_t const minGain = ZSTD_minGain(srcSize);  /* note : fixed formula, maybe should depend on compression level, or strategy */
+        size_t const maxCSize = srcSize - minGain;
+        if (cSize >= maxCSize || uncompressibleError) {
+            entropy->hufCTable_repeatMode = HUF_repeat_none;
+            entropy->offcode_repeatMode = FSE_repeat_none;
+            entropy->matchlength_repeatMode = FSE_repeat_none;
+            entropy->litlength_repeatMode = FSE_repeat_none;
+            return 0;  /* block not compressed */
+    }   }
     assert(!ZSTD_isError(cSize));
 
-    /* confirm repcodes */
+    /* block is compressed => confirm repcodes in history */
     { int i; for (i=0; irep[i] = seqStorePtr->repToConfirm[i]; }
     return cSize;
 }
@@ -1559,9 +1571,9 @@ ZSTD_blockCompressor ZSTD_selectBlockCompressor(ZSTD_strategy strat, int extDict
           ZSTD_compressBlock_btopt_extDict, ZSTD_compressBlock_btultra_extDict }
     };
     ZSTD_STATIC_ASSERT((unsigned)ZSTD_fast == 1);
+
     assert((U32)strat >= (U32)ZSTD_fast);
     assert((U32)strat <= (U32)ZSTD_btultra);
-
     return blockCompressor[extDict!=0][(U32)strat];
 }
 
@@ -1574,28 +1586,28 @@ static void ZSTD_storeLastLiterals(seqStore_t* seqStorePtr,
 
 static size_t ZSTD_compressBlock_internal(ZSTD_CCtx* zc, void* dst, size_t dstCapacity, const void* src, size_t srcSize)
 {
-    const BYTE* const base = zc->base;
-    const BYTE* const istart = (const BYTE*)src;
-    const U32 current = (U32)(istart-base);
-    size_t lastLLSize;
-    const BYTE* anchor;
-    U32 const extDict = zc->lowLimit < zc->dictLimit;
-    const ZSTD_blockCompressor blockCompressor =
-        zc->appliedParams.ldmParams.enableLdm
-            ? (extDict ? ZSTD_compressBlock_ldm_extDict : ZSTD_compressBlock_ldm)
-            : ZSTD_selectBlockCompressor(zc->appliedParams.cParams.strategy, extDict);
-
-    if (srcSize < MIN_CBLOCK_SIZE+ZSTD_blockHeaderSize+1) return 0;   /* don't even attempt compression below a certain srcSize */
+    if (srcSize < MIN_CBLOCK_SIZE+ZSTD_blockHeaderSize+1)
+        return 0;   /* don't even attempt compression below a certain srcSize */
     ZSTD_resetSeqStore(&(zc->seqStore));
-    if (current > zc->nextToUpdate + 384)
-        zc->nextToUpdate = current - MIN(192, (U32)(current - zc->nextToUpdate - 384));   /* limited update after finding a very long match */
-
-    lastLLSize = blockCompressor(zc, src, srcSize);
-
-    /* Last literals */
-    anchor = (const BYTE*)src + srcSize - lastLLSize;
-    ZSTD_storeLastLiterals(&zc->seqStore, anchor, lastLLSize);
 
+    /* limited update after a very long match */
+    {   const BYTE* const base = zc->base;
+        const BYTE* const istart = (const BYTE*)src;
+        const U32 current = (U32)(istart-base);
+        if (current > zc->nextToUpdate + 384)
+            zc->nextToUpdate = current - MIN(192, (U32)(current - zc->nextToUpdate - 384));
+    }
+    /* find and store sequences */
+    {   U32 const extDict = zc->lowLimit < zc->dictLimit;
+        const ZSTD_blockCompressor blockCompressor =
+            zc->appliedParams.ldmParams.enableLdm
+                ? (extDict ? ZSTD_compressBlock_ldm_extDict : ZSTD_compressBlock_ldm)
+                : ZSTD_selectBlockCompressor(zc->appliedParams.cParams.strategy, extDict);
+        size_t const lastLLSize = blockCompressor(zc, src, srcSize);
+        const BYTE* const anchor = (const BYTE*)src + srcSize - lastLLSize;
+        ZSTD_storeLastLiterals(&zc->seqStore, anchor, lastLLSize);
+    }
+    /* encode */
     return ZSTD_compressSequences(&zc->seqStore, zc->entropy, &zc->appliedParams.cParams, dst, dstCapacity, srcSize);
 }
 
@@ -1618,13 +1630,13 @@ static size_t ZSTD_compress_frameChunk (ZSTD_CCtx* cctx,
     BYTE* const ostart = (BYTE*)dst;
     BYTE* op = ostart;
     U32 const maxDist = (U32)1 << cctx->appliedParams.cParams.windowLog;
+    assert(cctx->appliedParams.cParams.windowLog <= 31);
 
     if (cctx->appliedParams.fParams.checksumFlag && srcSize)
         XXH64_update(&cctx->xxhState, src, srcSize);
 
     while (remaining) {
         U32 const lastBlock = lastFrameChunk & (blockSize >= remaining);
-        size_t cSize;
 
         if (dstCapacity < ZSTD_blockHeaderSize + MIN_CBLOCK_SIZE)
             return ERROR(dstSize_tooSmall);   /* not enough space to store compressed block */
@@ -1666,34 +1678,37 @@ static size_t ZSTD_compress_frameChunk (ZSTD_CCtx* cctx,
             else cctx->nextToUpdate -= correction;
             DEBUGLOG(4, "Correction of 0x%x bytes to lowLimit=0x%x\n", correction, cctx->lowLimit);
         }
-
+        /* enforce maxDist */
         if ((U32)(ip+blockSize - cctx->base) > cctx->loadedDictEnd + maxDist) {
-            /* enforce maxDist */
             U32 const newLowLimit = (U32)(ip+blockSize - cctx->base) - maxDist;
             if (cctx->lowLimit < newLowLimit) cctx->lowLimit = newLowLimit;
             if (cctx->dictLimit < cctx->lowLimit) cctx->dictLimit = cctx->lowLimit;
         }
 
-        cSize = ZSTD_compressBlock_internal(cctx, op+ZSTD_blockHeaderSize, dstCapacity-ZSTD_blockHeaderSize, ip, blockSize);
-        if (ZSTD_isError(cSize)) return cSize;
-
-        if (cSize == 0) {  /* block is not compressible */
-            U32 const cBlockHeader24 = lastBlock + (((U32)bt_raw)<<1) + (U32)(blockSize << 3);
-            if (blockSize + ZSTD_blockHeaderSize > dstCapacity) return ERROR(dstSize_tooSmall);
-            MEM_writeLE32(op, cBlockHeader24);   /* no pb, 4th byte will be overwritten */
-            memcpy(op + ZSTD_blockHeaderSize, ip, blockSize);
-            cSize = ZSTD_blockHeaderSize+blockSize;
-        } else {
-            U32 const cBlockHeader24 = lastBlock + (((U32)bt_compressed)<<1) + (U32)(cSize << 3);
-            MEM_writeLE24(op, cBlockHeader24);
-            cSize += ZSTD_blockHeaderSize;
-        }
+        {   size_t cSize = ZSTD_compressBlock_internal(cctx,
+                                op+ZSTD_blockHeaderSize, dstCapacity-ZSTD_blockHeaderSize,
+                                ip, blockSize);
+            if (ZSTD_isError(cSize)) return cSize;
+
+            if (cSize == 0) {  /* block is not compressible */
+                U32 const cBlockHeader24 = lastBlock + (((U32)bt_raw)<<1) + (U32)(blockSize << 3);
+                if (blockSize + ZSTD_blockHeaderSize > dstCapacity) return ERROR(dstSize_tooSmall);
+                MEM_writeLE32(op, cBlockHeader24);   /* 4th byte will be overwritten */
+                memcpy(op + ZSTD_blockHeaderSize, ip, blockSize);
+                cSize = ZSTD_blockHeaderSize + blockSize;
+            } else {
+                U32 const cBlockHeader24 = lastBlock + (((U32)bt_compressed)<<1) + (U32)(cSize << 3);
+                MEM_writeLE24(op, cBlockHeader24);
+                cSize += ZSTD_blockHeaderSize;
+            }
 
-        remaining -= blockSize;
-        dstCapacity -= cSize;
-        ip += blockSize;
-        op += cSize;
-    }
+            ip += blockSize;
+            assert(remaining >= blockSize);
+            remaining -= blockSize;
+            op += cSize;
+            assert(dstCapacity >= cSize);
+            dstCapacity -= cSize;
+    }   }
 
     if (lastFrameChunk && (op>ostart)) cctx->stage = ZSTDcs_ending;
     return op-ostart;
diff --git a/lib/compress/zstd_compress.h b/lib/compress/zstd_compress.h
index 94606edc935..fe98da78d8d 100644
--- a/lib/compress/zstd_compress.h
+++ b/lib/compress/zstd_compress.h
@@ -127,7 +127,7 @@ MEM_STATIC void ZSTD_storeSeq(seqStore_t* seqStorePtr, size_t litLength, const v
 #if defined(ZSTD_DEBUG) && (ZSTD_DEBUG >= 6)
     static const BYTE* g_start = NULL;
     U32 const pos = (U32)((const BYTE*)literals - g_start);
-    if (g_start==NULL) g_start = (const BYTE*)literals;
+    if (g_start==NULL) g_start = (const BYTE*)literals;  /* note : index only works for compression within a single segment */
     if ((pos > 0) && (pos < 1000000000))
         DEBUGLOG(6, "Cpos %6u :%5u literals & match %3u bytes at distance %6u",
                pos, (U32)litLength, (U32)matchCode+MINMATCH, (U32)offsetCode);

From 24ac2dbd2a56dccbfe52293ac98c3cdd8eb0c0c0 Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Fri, 13 Oct 2017 12:47:00 -0700
Subject: [PATCH 003/145] Fix invalid use of dictionary offcode table

Fixes #888.
---
 lib/compress/zstd_compress.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index e072fe30e84..3f1a7cdbae4 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -1536,6 +1536,12 @@ MEM_STATIC size_t ZSTD_compressSequences(seqStore_t* seqStorePtr,
     int const uncompressibleError = (cSize == ERROR(dstSize_tooSmall)) && (srcSize <= dstCapacity);
     if (ZSTD_isError(cSize) && !uncompressibleError)
         return cSize;
+    /* We check that dictionaries have offset codes available for the first
+     * block. After the first block, the offcode table might not have large
+     * enough codes to represent the offsets in the data.
+     */
+    if (entropy->offcode_repeatMode == FSE_repeat_valid)
+        entropy->offcode_repeatMode = FSE_repeat_check;
 
     /* Check compressibility */
     {   size_t const minGain = ZSTD_minGain(srcSize);  /* note : fixed formula, maybe should depend on compression level, or strategy */

From ced6e6189cf02b2de9bfcd7191f77cf9ff07f345 Mon Sep 17 00:00:00 2001
From: Nick Terrell 
Date: Fri, 13 Oct 2017 14:09:17 -0700
Subject: [PATCH 004/145] Add DEBUGLOG() that prints FSE encoding types

---
 lib/compress/zstd_compress.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 3f1a7cdbae4..188b8e717a7 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -1282,6 +1282,7 @@ symbolEncodingType_e ZSTD_selectEncodingType(
 #define MAX_SEQ_FOR_STATIC_FSE  1000
     ZSTD_STATIC_ASSERT(ZSTD_defaultDisallowed == 0 && ZSTD_defaultAllowed != 0);
     if ((mostFrequent == nbSeq) && (!isDefaultAllowed || nbSeq > 2)) {
+        DEBUGLOG(5, "Selected set_rle");
         /* Prefer set_basic over set_rle when there are 2 or less symbols,
          * since RLE uses 1 byte, but set_basic uses 5-6 bits per symbol.
          * If basic encoding isn't possible, always choose RLE.
@@ -1291,13 +1292,16 @@ symbolEncodingType_e ZSTD_selectEncodingType(
     }
     if ( isDefaultAllowed
       && (*repeatMode == FSE_repeat_valid) && (nbSeq < MAX_SEQ_FOR_STATIC_FSE)) {
+        DEBUGLOG(5, "Selected set_repeat");
         return set_repeat;
     }
     if ( isDefaultAllowed
       && ((nbSeq < MIN_SEQ_FOR_DYNAMIC_FSE) || (mostFrequent < (nbSeq >> (defaultNormLog-1)))) ) {
+        DEBUGLOG(5, "Selected set_basic");
         *repeatMode = FSE_repeat_valid;
         return set_basic;
     }
+    DEBUGLOG(5, "Selected set_compressed");
     *repeatMode = FSE_repeat_check;
     return set_compressed;
 }
@@ -1475,6 +1479,7 @@ MEM_STATIC size_t ZSTD_compressSequences_internal(seqStore_t* seqStorePtr,
     /* build CTable for Literal Lengths */
     {   U32 max = MaxLL;
         size_t const mostFrequent = FSE_countFast_wksp(count, &max, llCodeTable, nbSeq, entropy->workspace);
+        DEBUGLOG(5, "Building LL table");
         LLtype = ZSTD_selectEncodingType(&entropy->litlength_repeatMode, mostFrequent, nbSeq, LL_defaultNormLog, ZSTD_defaultAllowed);
         {   size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_LitLength, LLFSELog, (symbolEncodingType_e)LLtype,
                     count, max, llCodeTable, nbSeq, LL_defaultNorm, LL_defaultNormLog, MaxLL,
@@ -1487,6 +1492,7 @@ MEM_STATIC size_t ZSTD_compressSequences_internal(seqStore_t* seqStorePtr,
         size_t const mostFrequent = FSE_countFast_wksp(count, &max, ofCodeTable, nbSeq, entropy->workspace);
         /* We can only use the basic table if max <= DefaultMaxOff, otherwise the offsets are too large */
         ZSTD_defaultPolicy_e const defaultPolicy = (max <= DefaultMaxOff) ? ZSTD_defaultAllowed : ZSTD_defaultDisallowed;
+        DEBUGLOG(5, "Building OF table");
         Offtype = ZSTD_selectEncodingType(&entropy->offcode_repeatMode, mostFrequent, nbSeq, OF_defaultNormLog, defaultPolicy);
         {   size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_OffsetBits, OffFSELog, (symbolEncodingType_e)Offtype,
                     count, max, ofCodeTable, nbSeq, OF_defaultNorm, OF_defaultNormLog, DefaultMaxOff,
@@ -1497,6 +1503,7 @@ MEM_STATIC size_t ZSTD_compressSequences_internal(seqStore_t* seqStorePtr,
     /* build CTable for MatchLengths */
     {   U32 max = MaxML;
         size_t const mostFrequent = FSE_countFast_wksp(count, &max, mlCodeTable, nbSeq, entropy->workspace);
+        DEBUGLOG(5, "Building ML table");
         MLtype = ZSTD_selectEncodingType(&entropy->matchlength_repeatMode, mostFrequent, nbSeq, ML_defaultNormLog, ZSTD_defaultAllowed);
         {   size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_MatchLength, MLFSELog, (symbolEncodingType_e)MLtype,
                     count, max, mlCodeTable, nbSeq, ML_defaultNorm, ML_defaultNormLog, MaxML,

From dd18d73e7ef1b204bf514095fa8db7f9d6dfdc7b Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 13 Oct 2017 16:32:18 -0700
Subject: [PATCH 005/145] fileio: content size is enabled by default

---
 lib/compress/zstd_compress.c | 4 ++--
 programs/fileio.c            | 1 +
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index e072fe30e84..eb6f43d8cad 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -391,7 +391,7 @@ size_t ZSTD_CCtxParam_setParameter(
 
     case ZSTD_p_contentSizeFlag :
         /* Content size written in frame header _when known_ (default:1) */
-        DEBUGLOG(5, "set content size flag = %u", (value>0));
+        DEBUGLOG(4, "set content size flag = %u", (value>0));
         params->fParams.contentSizeFlag = value > 0;
         return 0;
 
@@ -401,7 +401,7 @@ size_t ZSTD_CCtxParam_setParameter(
         return 0;
 
     case ZSTD_p_dictIDFlag : /* When applicable, dictionary's dictID is provided in frame header (default:1) */
-        DEBUGLOG(5, "set dictIDFlag = %u", (value>0));
+        DEBUGLOG(4, "set dictIDFlag = %u", (value>0));
         params->fParams.noDictIDFlag = (value == 0);
         return 0;
 
diff --git a/programs/fileio.c b/programs/fileio.c
index 70158f93095..427f251b9bf 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -439,6 +439,7 @@ static cRess_t FIO_createCResources(const char* dictFileName, int cLevel,
 
 #ifdef ZSTD_NEWAPI
         {   /* frame parameters */
+            CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_contentSizeFlag, 1) );
             CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_dictIDFlag, g_dictIDFlag) );
             CHECK( ZSTD_CCtx_setParameter(ress.cctx, ZSTD_p_checksumFlag, g_checksumFlag) );
             (void)srcSize;

From fb4451664125919e0870b256797f826bad576efc Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 13 Oct 2017 17:39:13 -0700
Subject: [PATCH 006/145] ensure fParams.contentSizeFlag starts at 1

such default was failing for ZSTD_compressBegin/ZSTD_compressContinue
fixed too
---
 lib/compress/zstd_compress.c | 19 +++++++++++--------
 lib/zstd.h                   |  8 ++++----
 tests/fullbench.c            |  2 +-
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index eb6f43d8cad..b42e6996fe1 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -78,6 +78,7 @@ ZSTD_CCtx* ZSTD_createCCtx_advanced(ZSTD_customMem customMem)
     if (!cctx) return NULL;
     cctx->customMem = customMem;
     cctx->requestedParams.compressionLevel = ZSTD_CLEVEL_DEFAULT;
+    cctx->requestedParams.fParams.contentSizeFlag = 1;
     ZSTD_STATIC_ASSERT(zcss_init==0);
     ZSTD_STATIC_ASSERT(ZSTD_CONTENTSIZE_UNKNOWN==(0ULL - 1));
     return cctx;
@@ -1002,7 +1003,7 @@ void ZSTD_invalidateRepCodes(ZSTD_CCtx* cctx) {
  *  The "context", in this case, refers to the hash and chain tables, entropy
  *  tables, and dictionary offsets.
  *  Only works during stage ZSTDcs_init (i.e. after creation, but before first call to ZSTD_compressContinue()).
- *  pledgedSrcSize=0 means "empty" if fParams.contentSizeFlag=1
+ *  pledgedSrcSize=0 means "empty".
  *  @return : 0, or an error code */
 static size_t ZSTD_copyCCtx_internal(ZSTD_CCtx* dstCCtx,
                             const ZSTD_CCtx* srcCCtx,
@@ -1059,7 +1060,8 @@ size_t ZSTD_copyCCtx(ZSTD_CCtx* dstCCtx, const ZSTD_CCtx* srcCCtx, unsigned long
     ZSTD_frameParameters fParams = { 1 /*content*/, 0 /*checksum*/, 0 /*noDictID*/ };
     ZSTD_buffered_policy_e const zbuff = (ZSTD_buffered_policy_e)(srcCCtx->inBuffSize>0);
     ZSTD_STATIC_ASSERT((U32)ZSTDb_buffered==1);
-    fParams.contentSizeFlag = pledgedSrcSize>0;
+    if (pledgedSrcSize==0) pledgedSrcSize = ZSTD_CONTENTSIZE_UNKNOWN;
+    fParams.contentSizeFlag = (pledgedSrcSize != ZSTD_CONTENTSIZE_UNKNOWN);
 
     return ZSTD_copyCCtx_internal(dstCCtx, srcCCtx, fParams, pledgedSrcSize, zbuff);
 }
@@ -2077,7 +2079,7 @@ size_t ZSTD_compressBegin_usingDict(ZSTD_CCtx* cctx, const void* dict, size_t di
     ZSTD_CCtx_params const cctxParams =
             ZSTD_assignParamsToCCtxParams(cctx->requestedParams, params);
     return ZSTD_compressBegin_internal(cctx, dict, dictSize, ZSTD_dm_auto, NULL,
-                                       cctxParams, 0, ZSTDb_not_buffered);
+                                       cctxParams, ZSTD_CONTENTSIZE_UNKNOWN, ZSTDb_not_buffered);
 }
 
 size_t ZSTD_compressBegin(ZSTD_CCtx* cctx, int compressionLevel)
@@ -2797,18 +2799,18 @@ size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
             }
             DEBUGLOG(4, "call ZSTDMT_initCStream_internal as nbThreads=%u", params.nbThreads);
             CHECK_F( ZSTDMT_initCStream_internal(
-                             cctx->mtctx,
-                             prefixDict.dict, prefixDict.dictSize, ZSTD_dm_rawContent,
-                             cctx->cdict, params, cctx->pledgedSrcSizePlusOne-1) );
+                        cctx->mtctx,
+                        prefixDict.dict, prefixDict.dictSize, ZSTD_dm_rawContent,
+                        cctx->cdict, params, cctx->pledgedSrcSizePlusOne-1) );
             cctx->streamStage = zcss_load;
             cctx->appliedParams.nbThreads = params.nbThreads;
         } else
 #endif
-        {
-            CHECK_F( ZSTD_resetCStream_internal(
+        {   CHECK_F( ZSTD_resetCStream_internal(
                              cctx, prefixDict.dict, prefixDict.dictSize,
                              prefixDict.dictMode, cctx->cdict, params,
                              cctx->pledgedSrcSizePlusOne-1) );
+            assert(cctx->streamStage == zcss_load);
     }   }
 
     /* compression stage */
@@ -3034,5 +3036,6 @@ ZSTD_parameters ZSTD_getParams(int compressionLevel, unsigned long long srcSizeH
     ZSTD_compressionParameters const cParams = ZSTD_getCParams(compressionLevel, srcSizeHint, dictSize);
     memset(¶ms, 0, sizeof(params));
     params.cParams = cParams;
+    params.fParams.contentSizeFlag = 1;
     return params;
 }
diff --git a/lib/zstd.h b/lib/zstd.h
index 2194a3b23d6..7fafdbfc330 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -1000,11 +1000,11 @@ typedef enum {
                               * Special: value 0 means "do not change strategy". */
 
     /* frame parameters */
-    ZSTD_p_contentSizeFlag=200, /* Content size is written into frame header _whenever known_ (default:1)
-                              * note that content size must be known at the beginning,
-                              * it is sent using ZSTD_CCtx_setPledgedSrcSize() */
+    ZSTD_p_contentSizeFlag=200, /* Content size will be written into frame header _whenever known_ (default:1)
+                              * Content size must be known at the beginning of compression,
+                              * it is provided using ZSTD_CCtx_setPledgedSrcSize() */
     ZSTD_p_checksumFlag,     /* A 32-bits checksum of content is written at end of frame (default:0) */
-    ZSTD_p_dictIDFlag,       /* When applicable, dictID of dictionary is provided in frame header (default:1) */
+    ZSTD_p_dictIDFlag,       /* When applicable, dictionary's ID is written into frame header (default:1) */
 
     /* multi-threading parameters */
     ZSTD_p_nbThreads=400,    /* Select how many threads a compression job can spawn (default:1)
diff --git a/tests/fullbench.c b/tests/fullbench.c
index db00ce217b0..3f497b0344f 100644
--- a/tests/fullbench.c
+++ b/tests/fullbench.c
@@ -233,7 +233,7 @@ static ZSTD_CCtx* g_zcc = NULL;
 size_t local_ZSTD_compressContinue(void* dst, size_t dstCapacity, void* buff2, const void* src, size_t srcSize)
 {
     (void)buff2;
-    ZSTD_compressBegin(g_zcc, 1);
+    ZSTD_compressBegin(g_zcc, 1 /* compressionLevel */);
     return ZSTD_compressEnd(g_zcc, dst, dstCapacity, src, srcSize);
 }
 

From 3c1e3f8ec90a09558bda9b860fc1cbfdd322919c Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 13 Oct 2017 18:32:06 -0700
Subject: [PATCH 007/145] contentSizeFlag enabled by default would also fail
 for streaming and MT operations

fixed
---
 lib/compress/zstd_compress.c   | 14 ++++++++------
 lib/compress/zstdmt_compress.c | 34 +++++++++++++++++-----------------
 tests/zstreamtest.c            |  5 +++--
 3 files changed, 28 insertions(+), 25 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index b42e6996fe1..49b532b945c 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -2053,6 +2053,8 @@ size_t ZSTD_compressBegin_advanced_internal(
                                     ZSTD_CCtx_params params,
                                     unsigned long long pledgedSrcSize)
 {
+    DEBUGLOG(4, "ZSTD_compressBegin_advanced_internal");
+    DEBUGLOG(4, "contentSizeFlag : %u", params.fParams.contentSizeFlag);
     /* compression parameters verification and optimization */
     CHECK_F( ZSTD_checkCParams(params.cParams) );
     return ZSTD_compressBegin_internal(cctx, dict, dictSize, dictMode, NULL,
@@ -2281,7 +2283,7 @@ ZSTD_CDict* ZSTD_createCDict_advanced(const void* dictBuffer, size_t dictSize,
                                       ZSTD_dictMode_e dictMode,
                                       ZSTD_compressionParameters cParams, ZSTD_customMem customMem)
 {
-    DEBUGLOG(5, "ZSTD_createCDict_advanced, mode %u", (U32)dictMode);
+    DEBUGLOG(4, "ZSTD_createCDict_advanced, mode %u", (U32)dictMode);
     if (!customMem.customAlloc ^ !customMem.customFree) return NULL;
 
     {   ZSTD_CDict* const cdict = (ZSTD_CDict*)ZSTD_malloc(sizeof(ZSTD_CDict), customMem);
@@ -2356,9 +2358,9 @@ ZSTD_CDict* ZSTD_initStaticCDict(void* workspace, size_t workspaceSize,
                             + cctxSize;
     ZSTD_CDict* const cdict = (ZSTD_CDict*) workspace;
     void* ptr;
-    DEBUGLOG(5, "(size_t)workspace & 7 : %u", (U32)(size_t)workspace & 7);
+    DEBUGLOG(4, "(size_t)workspace & 7 : %u", (U32)(size_t)workspace & 7);
     if ((size_t)workspace & 7) return NULL;  /* 8-aligned */
-    DEBUGLOG(5, "(workspaceSize < neededSize) : (%u < %u) => %u",
+    DEBUGLOG(4, "(workspaceSize < neededSize) : (%u < %u) => %u",
         (U32)workspaceSize, (U32)neededSize, (U32)(workspaceSize < neededSize));
     if (workspaceSize < neededSize) return NULL;
 
@@ -2394,7 +2396,7 @@ size_t ZSTD_compressBegin_usingCDict_advanced(
     {   ZSTD_CCtx_params params = cctx->requestedParams;
         params.cParams = ZSTD_getCParamsFromCDict(cdict);
         params.fParams = fParams;
-        DEBUGLOG(5, "ZSTD_compressBegin_usingCDict_advanced");
+        DEBUGLOG(4, "ZSTD_compressBegin_usingCDict_advanced");
         return ZSTD_compressBegin_internal(cctx,
                                            NULL, 0, ZSTD_dm_auto,
                                            cdict,
@@ -2409,7 +2411,7 @@ size_t ZSTD_compressBegin_usingCDict_advanced(
 size_t ZSTD_compressBegin_usingCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict)
 {
     ZSTD_frameParameters const fParams = { 0 /*content*/, 0 /*checksum*/, 0 /*noDictID*/ };
-    DEBUGLOG(5, "ZSTD_compressBegin_usingCDict : dictIDFlag == %u", !fParams.noDictIDFlag);
+    DEBUGLOG(4, "ZSTD_compressBegin_usingCDict : dictIDFlag == %u", !fParams.noDictIDFlag);
     return ZSTD_compressBegin_usingCDict_advanced(cctx, cdict, fParams, 0);
 }
 
@@ -2585,7 +2587,7 @@ size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t di
     ZSTD_parameters const params = ZSTD_getParams(compressionLevel, 0, dictSize);
     ZSTD_CCtx_params const cctxParams =
             ZSTD_assignParamsToCCtxParams(zcs->requestedParams, params);
-    return ZSTD_initCStream_internal(zcs, dict, dictSize, NULL, cctxParams, 0);
+    return ZSTD_initCStream_internal(zcs, dict, dictSize, NULL, cctxParams, ZSTD_CONTENTSIZE_UNKNOWN);
 }
 
 size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize)
diff --git a/lib/compress/zstdmt_compress.c b/lib/compress/zstdmt_compress.c
index 7831cd3bd81..17c69c9dfe4 100644
--- a/lib/compress/zstdmt_compress.c
+++ b/lib/compress/zstdmt_compress.c
@@ -327,6 +327,7 @@ void ZSTDMT_compressChunk(void* jobDescription)
     ZSTD_CCtx* cctx = ZSTDMT_getCCtx(job->cctxPool);
     const void* const src = (const char*)job->srcStart + job->dictSize;
     buffer_t dstBuff = job->dstBuff;
+    DEBUGLOG(5, "ZSTDMT_compressChunk")
     DEBUGLOG(5, "job (first:%u) (last:%u) : dictSize %u, srcSize %u",
                  job->firstChunk, job->lastChunk, (U32)job->dictSize, (U32)job->srcSize);
 
@@ -344,22 +345,22 @@ void ZSTDMT_compressChunk(void* jobDescription)
         job->dstBuff = dstBuff;
     }
 
-    if (job->cdict) {  /* should only happen for first segment */
+    if (job->cdict) {
         size_t const initError = ZSTD_compressBegin_usingCDict_advanced(cctx, job->cdict, job->params.fParams, job->fullFrameSize);
-        DEBUGLOG(5, "using CDict");
+        DEBUGLOG(4, "ZSTDMT_compressChunk, using CDict");
+        assert(job->firstChunk); /* should only happen for first segment */
         if (ZSTD_isError(initError)) { job->cSize = initError; goto _endJob; }
     } else {  /* srcStart points at reloaded section */
-        if (!job->firstChunk) job->params.fParams.contentSizeFlag = 0;  /* ensure no srcSize control */
-        { ZSTD_CCtx_params jobParams = job->params;
-          size_t const forceWindowError =
-              ZSTD_CCtxParam_setParameter(&jobParams, ZSTD_p_forceMaxWindow, !job->firstChunk);
-          /* Force loading dictionary in "content-only" mode (no header analysis) */
-          size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, job->srcStart, job->dictSize, ZSTD_dm_rawContent, jobParams, job->fullFrameSize);
-            if (ZSTD_isError(initError) || ZSTD_isError(forceWindowError)) {
-                job->cSize = initError;
-                goto _endJob;
-            }
-    }   }
+        ZSTD_CCtx_params jobParams = job->params;
+        size_t const forceWindowError = ZSTD_CCtxParam_setParameter(&jobParams, ZSTD_p_forceMaxWindow, !job->firstChunk);
+        U64 const pledgedSrcSize = job->firstChunk ? job->fullFrameSize : ZSTD_CONTENTSIZE_UNKNOWN;
+        /* load dictionary in "content-only" mode (no header analysis) */
+        size_t const initError = ZSTD_compressBegin_advanced_internal(cctx, job->srcStart, job->dictSize, ZSTD_dm_rawContent, jobParams, pledgedSrcSize);
+        if (ZSTD_isError(initError) || ZSTD_isError(forceWindowError)) {
+            job->cSize = initError;
+            goto _endJob;
+        }
+    }
     if (!job->firstChunk) {  /* flush and overwrite frame header when it's not first segment */
         size_t const hSize = ZSTD_compressContinue(cctx, dstBuff.start, dstBuff.size, src, 0);
         if (ZSTD_isError(hSize)) { job->cSize = hSize; goto _endJob; }
@@ -367,7 +368,7 @@ void ZSTDMT_compressChunk(void* jobDescription)
     }
 
     DEBUGLOG(5, "Compressing : ");
-    DEBUG_PRINTHEX(4, job->srcStart, 12);
+    DEBUG_PRINTHEX(6, job->srcStart, 12);
     job->cSize = (job->lastChunk) ?
                  ZSTD_compressEnd     (cctx, dstBuff.start, dstBuff.size, src, job->srcSize) :
                  ZSTD_compressContinue(cctx, dstBuff.start, dstBuff.size, src, job->srcSize);
@@ -610,6 +611,7 @@ static size_t ZSTDMT_compress_advanced_internal(
     assert(jobParams.nbThreads == 0);
     assert(mtctx->cctxPool->totalCCtx == params.nbThreads);
 
+    DEBUGLOG(4, "ZSTDMT_compress_advanced_internal");
     DEBUGLOG(4, "nbChunks  : %2u   (chunkSize : %u bytes)   ", nbChunks, (U32)avgChunkSize);
     if (nbChunks==1) {   /* fallback to single-thread mode */
         ZSTD_CCtx* const cctx = mtctx->cctxPool->cctx[0];
@@ -641,7 +643,7 @@ static size_t ZSTDMT_compress_advanced_internal(
             mtctx->jobs[u].srcStart = srcStart + frameStartPos - dictSize;
             mtctx->jobs[u].dictSize = dictSize;
             mtctx->jobs[u].srcSize = chunkSize;
-            mtctx->jobs[u].cdict = mtctx->nextJobID==0 ? cdict : NULL;
+            mtctx->jobs[u].cdict = (u==0) ? cdict : NULL;
             mtctx->jobs[u].fullFrameSize = srcSize;
             mtctx->jobs[u].params = jobParams;
             /* do not calculate checksum within sections, but write it in header for first section */
@@ -777,7 +779,6 @@ size_t ZSTDMT_initCStream_internal(
     zcs->params = params;
     zcs->frameContentSize = pledgedSrcSize;
     if (dict) {
-        DEBUGLOG(4,"cdictLocal: %08X", (U32)(size_t)zcs->cdictLocal);
         ZSTD_freeCDict(zcs->cdictLocal);
         zcs->cdictLocal = ZSTD_createCDict_advanced(dict, dictSize,
                                                     ZSTD_dlm_byCopy, dictMode, /* note : a loadPrefix becomes an internal CDict */
@@ -785,7 +786,6 @@ size_t ZSTDMT_initCStream_internal(
         zcs->cdict = zcs->cdictLocal;
         if (zcs->cdictLocal == NULL) return ERROR(memory_allocation);
     } else {
-        DEBUGLOG(4,"cdictLocal: %08X", (U32)(size_t)zcs->cdictLocal);
         ZSTD_freeCDict(zcs->cdictLocal);
         zcs->cdictLocal = NULL;
         zcs->cdict = cdict;
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 8b4c836940a..36eede2939c 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -190,9 +190,10 @@ static int basicUnitTests(U32 seed, double compressibility, ZSTD_customMem custo
 
     /* Basic compression test */
     DISPLAYLEVEL(3, "test%3i : compress %u bytes : ", testNb++, COMPRESSIBLE_NOISE_LENGTH);
-    CHECK_Z( ZSTD_initCStream_usingDict(zc, CNBuffer, dictSize, 1) );
+    CHECK_Z( ZSTD_initCStream_usingDict(zc, CNBuffer, dictSize, 1 /* cLevel */) );
     outBuff.dst = (char*)(compressedBuffer)+cSize;
-    outBuff.size = compressedBufferSize;
+    assert(compressedBufferSize > cSize);
+    outBuff.size = compressedBufferSize - cSize;
     outBuff.pos = 0;
     inBuff.src = CNBuffer;
     inBuff.size = CNBufferSize;

From 213ef3b510b909b7dd95dc472e680ebea25bc450 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 13 Oct 2017 19:01:58 -0700
Subject: [PATCH 008/145] fixed ZSTD_initCStream_advanced() behavior, which
 depends on contentSizeFlag,

and a stream fuzzer test, which was incorrect
(relied on 0 being unconditionnally transformed into `ZSTD_CONTENTSIZE_UNKNOWN`)
---
 lib/compress/zstd_compress.c | 16 ++++++++++++----
 lib/zstd.h                   |  2 +-
 tests/zstreamtest.c          |  5 +++--
 3 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 49b532b945c..4ceba415c77 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -822,9 +822,8 @@ static size_t ZSTD_resetCCtx_internal(ZSTD_CCtx* zc,
                                       ZSTD_compResetPolicy_e const crp,
                                       ZSTD_buffered_policy_e const zbuff)
 {
-    DEBUGLOG(4, "ZSTD_resetCCtx_internal");
+    DEBUGLOG(4, "ZSTD_resetCCtx_internal: pledgedSrcSize=%u", (U32)pledgedSrcSize);
     assert(!ZSTD_isError(ZSTD_checkCParams(params.cParams)));
-    DEBUGLOG(4, "pledgedSrcSize: %u", (U32)pledgedSrcSize);
 
     if (crp == ZSTDcrp_continue) {
         if (ZSTD_equivalentParams(params, zc->appliedParams)) {
@@ -2500,12 +2499,15 @@ static size_t ZSTD_resetCStream_internal(ZSTD_CStream* zcs,
     return 0;   /* ready to go */
 }
 
+/* ZSTD_resetCStream():
+ * pledgedSrcSize == 0 means "unknown" */
 size_t ZSTD_resetCStream(ZSTD_CStream* zcs, unsigned long long pledgedSrcSize)
 {
     ZSTD_CCtx_params params = zcs->requestedParams;
-    params.fParams.contentSizeFlag = (pledgedSrcSize > 0);
+    DEBUGLOG(4, "ZSTD_resetCStream: pledgedSrcSize = %u", (U32)pledgedSrcSize);
+    if (pledgedSrcSize==0) pledgedSrcSize = ZSTD_CONTENTSIZE_UNKNOWN;
+    params.fParams.contentSizeFlag = 1;
     params.cParams = ZSTD_getCParamsFromCCtxParams(params, pledgedSrcSize, 0);
-    DEBUGLOG(4, "ZSTD_resetCStream");
     return ZSTD_resetCStream_internal(zcs, NULL, 0, ZSTD_dm_auto, zcs->cdict, params, pledgedSrcSize);
 }
 
@@ -2572,13 +2574,19 @@ size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict)
     return ZSTD_initCStream_usingCDict_advanced(zcs, cdict, fParams, 0);  /* note : will check that cdict != NULL */
 }
 
+/* ZSTD_initCStream_advanced() :
+ * pledgedSrcSize is optional: it can be 0, which means "unknown" if contentSizeFlag is set, or "unknown" if contentSizeFlag is not set. */
 size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs,
                                  const void* dict, size_t dictSize,
                                  ZSTD_parameters params, unsigned long long pledgedSrcSize)
 {
     ZSTD_CCtx_params const cctxParams =
             ZSTD_assignParamsToCCtxParams(zcs->requestedParams, params);
+    DEBUGLOG(4, "ZSTD_initCStream_advanced: pledgedSrcSize=%u, flag=%u",
+            (U32)pledgedSrcSize, params.fParams.contentSizeFlag);
     CHECK_F( ZSTD_checkCParams(params.cParams) );
+    if ((pledgedSrcSize==0) && (params.fParams.contentSizeFlag==0))
+        pledgedSrcSize = ZSTD_CONTENTSIZE_UNKNOWN;
     return ZSTD_initCStream_internal(zcs, dict, dictSize, NULL, cctxParams, pledgedSrcSize);
 }
 
diff --git a/lib/zstd.h b/lib/zstd.h
index 7fafdbfc330..2866866be04 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -734,7 +734,7 @@ ZSTDLIB_API ZSTD_CStream* ZSTD_initStaticCStream(void* workspace, size_t workspa
 ZSTDLIB_API size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize);   /**< pledgedSrcSize must be correct, a size of 0 means unknown.  for a frame size of 0 use initCStream_advanced */
 ZSTDLIB_API size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t dictSize, int compressionLevel); /**< creates of an internal CDict (incompatible with static CCtx), except if dict == NULL or dictSize < 8, in which case no dict is used. Note: dict is loaded with ZSTD_dm_auto (treated as a full zstd dictionary if it begins with ZSTD_MAGIC_DICTIONARY, else as raw content) and ZSTD_dlm_byCopy.*/
 ZSTDLIB_API size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs, const void* dict, size_t dictSize,
-                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize is optional and can be 0 (meaning unknown). note: if the contentSizeFlag is set, pledgedSrcSize == 0 means the source size is actually 0. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
+                                             ZSTD_parameters params, unsigned long long pledgedSrcSize);  /**< pledgedSrcSize is optional: it can be 0, which means "unknown" if contentSizeFlag is set, or "unknown" if contentSizeFlag is not set. dict is loaded with ZSTD_dm_auto and ZSTD_dlm_byCopy. */
 ZSTDLIB_API size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict);  /**< note : cdict will just be referenced, and must outlive compression session */
 ZSTDLIB_API size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, const ZSTD_CDict* cdict, ZSTD_frameParameters fParams, unsigned long long pledgedSrcSize);  /**< same as ZSTD_initCStream_usingCDict(), with control over frame parameters */
 
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 36eede2939c..3aea22709ae 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -559,7 +559,7 @@ static int basicUnitTests(U32 seed, double compressibility, ZSTD_customMem custo
     DISPLAYLEVEL(3, "test%3i : ZSTD_initCStream_advanced with pledgedSrcSize=0 and dict : ", testNb++);
     {   ZSTD_parameters params = ZSTD_getParams(5, 0, 0);
         params.fParams.contentSizeFlag = 1;
-        CHECK_Z( ZSTD_initCStream_advanced(zc, dictionary.start, dictionary.filled, params, 0) );
+        CHECK_Z( ZSTD_initCStream_advanced(zc, dictionary.start, dictionary.filled, params, 0 /* pledgedSrcSize==0 means "empty" when params.fParams.contentSizeFlag is set */) );
     } /* cstream advanced shall write content size = 0 */
     inBuff.src = CNBuffer;
     inBuff.size = 0;
@@ -833,10 +833,11 @@ static int fuzzerTests(U32 seed, U32 nbTests, unsigned startTest, double compres
             {   size_t const dictStart = FUZ_rand(&lseed) % (srcBufferSize - dictSize);
                 dict = srcBuffer + dictStart;
             }
-            {   U64 const pledgedSrcSize = (FUZ_rand(&lseed) & 3) ? 0 : maxTestSize;
+            {   U64 const pledgedSrcSize = (FUZ_rand(&lseed) & 3) ? ZSTD_CONTENTSIZE_UNKNOWN : maxTestSize;
                 ZSTD_parameters params = ZSTD_getParams(cLevel, pledgedSrcSize, dictSize);
                 params.fParams.checksumFlag = FUZ_rand(&lseed) & 1;
                 params.fParams.noDictIDFlag = FUZ_rand(&lseed) & 1;
+                params.fParams.contentSizeFlag = FUZ_rand(&lseed) & 1;
                 CHECK_Z ( ZSTD_initCStream_advanced(zc, dict, dictSize, params, pledgedSrcSize) );
         }   }
 

From beb9b4b398cb4a9b98f48a49a1c95cfbd030527c Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 13 Oct 2017 19:09:30 -0700
Subject: [PATCH 009/145] fixed ZSTDMT_initCStream() when contentSizeFlag==1 by
 default

and a wrong test in zstreamtest --mt
---
 lib/compress/zstdmt_compress.c | 2 +-
 tests/zstreamtest.c            | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/compress/zstdmt_compress.c b/lib/compress/zstdmt_compress.c
index 17c69c9dfe4..197a2c49e14 100644
--- a/lib/compress/zstdmt_compress.c
+++ b/lib/compress/zstdmt_compress.c
@@ -852,7 +852,7 @@ size_t ZSTDMT_initCStream(ZSTDMT_CCtx* zcs, int compressionLevel) {
     ZSTD_CCtx_params cctxParams = zcs->params;
     cctxParams.cParams = params.cParams;
     cctxParams.fParams = params.fParams;
-    return ZSTDMT_initCStream_internal(zcs, NULL, 0, ZSTD_dm_auto, NULL, cctxParams, 0);
+    return ZSTDMT_initCStream_internal(zcs, NULL, 0, ZSTD_dm_auto, NULL, cctxParams, ZSTD_CONTENTSIZE_UNKNOWN);
 }
 
 
diff --git a/tests/zstreamtest.c b/tests/zstreamtest.c
index 3aea22709ae..38320e8c48b 100644
--- a/tests/zstreamtest.c
+++ b/tests/zstreamtest.c
@@ -1088,13 +1088,13 @@ static int fuzzerTests_MT(U32 seed, U32 nbTests, unsigned startTest, double comp
             {   size_t const dictStart = FUZ_rand(&lseed) % (srcBufferSize - dictSize);
                 dict = srcBuffer + dictStart;
             }
-            {   U64 const pledgedSrcSize = (FUZ_rand(&lseed) & 3) ? 0 : maxTestSize;
+            {   U64 const pledgedSrcSize = (FUZ_rand(&lseed) & 3) ? ZSTD_CONTENTSIZE_UNKNOWN : maxTestSize;
                 ZSTD_parameters params = ZSTD_getParams(cLevel, pledgedSrcSize, dictSize);
                 DISPLAYLEVEL(5, "Init with windowLog = %u and pledgedSrcSize = %u \n",
                     params.cParams.windowLog, (U32)pledgedSrcSize);
                 params.fParams.checksumFlag = FUZ_rand(&lseed) & 1;
                 params.fParams.noDictIDFlag = FUZ_rand(&lseed) & 1;
-                params.fParams.contentSizeFlag = pledgedSrcSize>0;
+                params.fParams.contentSizeFlag = FUZ_rand(&lseed) & 1;
                 DISPLAYLEVEL(5, "checksumFlag : %u \n", params.fParams.checksumFlag);
                 CHECK_Z( ZSTDMT_initCStream_advanced(zc, dict, dictSize, params, pledgedSrcSize) );
                 CHECK_Z( ZSTDMT_setMTCtxParameter(zc, ZSTDMT_p_overlapSectionLog, FUZ_rand(&lseed) % 12) );

From 43914f6a206989432d68bb6abe336e1a662e913d Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Fri, 13 Oct 2017 23:47:01 -0700
Subject: [PATCH 010/145] add test for presence of decompressed size field in
 header

previous test was "display" only, it wouldn't trigger an error in case of missing header.
---
 tests/playTests.sh | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/tests/playTests.sh b/tests/playTests.sh
index bc021648cfc..013182aeb5d 100755
--- a/tests/playTests.sh
+++ b/tests/playTests.sh
@@ -600,7 +600,7 @@ $ECHO "\n**** zstd --list/-l single frame tests ****"
 ./datagen > tmp3
 $ZSTD tmp*
 $ZSTD -l *.zst
-$ZSTD -lv *.zst
+$ZSTD -lv *.zst | grep "Decompressed Size:"  # check that decompressed size is present in header
 $ZSTD --list *.zst
 $ZSTD --list -v *.zst
 
@@ -609,8 +609,6 @@ cat tmp1.zst tmp2.zst > tmp12.zst
 cat tmp12.zst tmp3.zst > tmp123.zst
 $ZSTD -l *.zst
 $ZSTD -lv *.zst
-$ZSTD --list *.zst
-$ZSTD --list -v *.zst
 
 $ECHO "\n**** zstd --list/-l error detection tests ****"
 ! $ZSTD -l tmp1 tmp1.zst
@@ -627,9 +625,9 @@ $ZSTD -lv tmp5.zst
 ! $ZSTD -lv tmp5*
 
 $ECHO "\n**** zstd --list/-l test with no content size field ****"
-./datagen -g1MB | $ZSTD > tmp6.zst
+./datagen -g513K | $ZSTD > tmp6.zst
 $ZSTD -l tmp6.zst
-$ZSTD -lv tmp6.zst
+! $ZSTD -lv tmp6.zst | grep "Decompressed Size:"  # must NOT be present in header
 
 $ECHO "\n**** zstd --list/-l test with no checksum ****"
 $ZSTD -f --no-check tmp1

From 9ef32b3cf1cbf5276f3e6dd0484de1ddd258b4a4 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Sat, 14 Oct 2017 00:02:32 -0700
Subject: [PATCH 011/145] minor : zstd -l -v display each file name

---
 programs/fileio.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/programs/fileio.c b/programs/fileio.c
index 427f251b9bf..01e901ed898 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -1927,7 +1927,7 @@ static int getFileInfo(fileInfo_t* info, const char* inFileName){
     return detectError;
 }
 
-static void displayInfo(const char* inFileName, fileInfo_t* info, int displayLevel){
+static void displayInfo(const char* inFileName, const fileInfo_t* info, int displayLevel){
     unsigned const unit = info->compressedSize < (1 MB) ? (1 KB) : (1 MB);
     const char* const unitStr = info->compressedSize < (1 MB) ? "KB" : "MB";
     double const windowSizeUnit = (double)info->windowSize / unit;
@@ -1950,8 +1950,10 @@ static void displayInfo(const char* inFileName, fileInfo_t* info, int displayLev
                     checkString, inFileName);
         }
     } else {
+        DISPLAYOUT("%s \n", inFileName);
         DISPLAYOUT("# Zstandard Frames: %d\n", info->numActualFrames);
-        DISPLAYOUT("# Skippable Frames: %d\n", info->numSkippableFrames);
+        if (info->numSkippableFrames)
+            DISPLAYOUT("# Skippable Frames: %d\n", info->numSkippableFrames);
         DISPLAYOUT("Window Size: %.2f %2s (%llu B)\n",
                    windowSizeUnit, unitStr,
                    (unsigned long long)info->windowSize);
@@ -1983,7 +1985,6 @@ static fileInfo_t FIO_addFInfo(fileInfo_t fi1, fileInfo_t fi2)
 }
 
 static int FIO_listFile(fileInfo_t* total, const char* inFileName, int displayLevel){
-    /* initialize info to avoid warnings */
     fileInfo_t info;
     memset(&info, 0, sizeof(info));
     {   int const error = getFileInfo(&info, inFileName);
@@ -2023,7 +2024,7 @@ int FIO_listMultipleFiles(unsigned numFiles, const char** filenameTable, int dis
         for (u=0; u 1 && displayLevel <= 2) {
+        if (numFiles > 1 && displayLevel <= 2) {   /* display total */
             unsigned const unit = total.compressedSize < (1 MB) ? (1 KB) : (1 MB);
             const char* const unitStr = total.compressedSize < (1 MB) ? "KB" : "MB";
             double const compressedSizeUnit = (double)total.compressedSize / unit;
@@ -2043,8 +2044,7 @@ int FIO_listMultipleFiles(unsigned numFiles, const char** filenameTable, int dis
                         total.numSkippableFrames,
                         compressedSizeUnit, unitStr, decompressedSizeUnit, unitStr,
                         ratio, checkString, total.nbFiles);
-            }
-        }
+        }   }
         return error;
     }
 }

From 5eed8e7a5538be8fc39ab94d1a61e161bbc348b9 Mon Sep 17 00:00:00 2001
From: Yann Collet 
Date: Sat, 14 Oct 2017 00:32:06 -0700
Subject: [PATCH 012/145] changed API comments to invite using macro
 ZSTD_CONTENTSIZE_UNKNOWN

to mean "pledgedSrcSize is not known at init time" instead of `0`.
Note that, a few prototypes created and documented with `0` to mean "unknown" still interpret "0" as unknown,
to avoid breaking 3rd party applications which depend on this behavior.
But this value is no longer recommended to mean "unknown".

In some future version, it might be possible to switch "0" to mean "empty",
as is already the case for several prototypes.
The advantage is that pledgedSrcSize field would have same behavior accross entire API,
making it easier to reason about.

Note that all concerned prototypes belong to the "experimental" API section.

srcSize is controlled at end of compression,
so if someone uses "0" to mean "unknown" while it effectively means "empty",
this is immediately caught by the compression function, which generates an error code : ZSTD_ERROR_srcSize_wrong
---
 doc/zstd_manual.html | 26 ++++++++++++++------------
 lib/zstd.h           | 18 ++++++++++--------
 2 files changed, 24 insertions(+), 20 deletions(-)

diff --git a/doc/zstd_manual.html b/doc/zstd_manual.html
index b4720adac26..7ef86a23dba 100644
--- a/doc/zstd_manual.html
+++ b/doc/zstd_manual.html
@@ -631,10 +631,10 @@