Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include intermediate certs in TLS handshake #5453

Merged
merged 9 commits into from
Jul 14, 2023

Conversation

eddyashton
Copy link
Member

This doesn't make a difference for most of our end-to-end tests, but does for an ACME-endorsed interface. We end up with Service Cert <- Intermediate <- Root (potentially multiple intermediates, but one is sufficient for demo). We store in the KV, and attempt to set as the server cert, the chain (/bundle) [Service Cert] \n [Intermediate], but when passing that to OpenSSL we actually end up calling PEM_read_bio_X509 which only extracts [Service Cert], and drop the Intermediate on the ground.

That means that a connecting browser, which likely only has Root in their trust store, can't build a complete chain and won't trust CCF.

The solution is retain the chain (an X509_STACK in OpenSSLese), though we have to split the bundle and remember it separately, and then pass it through to use_cert_and_key.

The end-to-end test we have using Pebble to obtain ACME endorsements had missed this, because it fetched both the Root and Intermediate from Pebble, and added them to the trust store for the test connection. I've changed that to only fetch the Root, which should match what we get in a real service.

A pure digression, but something I've not been able to find documentation on - I assume that for a real ACME endorsement we end up with a longer chain, with some overlap between the ACME service's response and the CAs in a system's trust store. ie if we end up with Service Cert <- This Let's Encrypt Instance <- Let's Encrypt Master <- ISRG Root, there's a chance that Let's Encrypt returns a chain including Let's Encrypt Master, and that's also in a system trust store. So maybe there's some overlap and redundancy, if we blindly retain the entire chain. But I don't think we can make any assessment of where the cut-off point is, so we assume the endorser/ACME protocol is handling it sensibly, and retain whatever they give us.

@eddyashton eddyashton requested a review from a team July 13, 2023 17:08
@achamayou achamayou added auto-backport Automatically backport this PR to LTS branch 4.x-todo PRs which should be backported to 4.x labels Jul 13, 2023
@ghost
Copy link

ghost commented Jul 13, 2023

present_intermediate_certs@73128 aka 20230714.12 vs main ewma over 20 builds from 72744 to 73119

Click to see table

main

build_id build_number Commit latency factor ls_virtual_cft^ pi_basic_mt_virtual_cft^ pi_basic_mt_sgx_cft^ pi_basic_mt_sgx_cft_mem pi_ls_virtual_cft^ pi_basic_virtual_cft^ ls_jwt_virtual_cft^ ls_sgx_cft^ ls_sgx_cft_mem pi_ls_jwt_virtual_cft^ pi_ls_sgx_cft^ pi_ls_sgx_cft_mem ls_js_virtual_cft^ pi_basic_sgx_cft^ pi_basic_sgx_cft_mem ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem ls_full_js_virtual_cft^ pi_ls_jwt_sgx_cft^ pi_ls_jwt_sgx_cft_mem ls_js_jwt_virtual_cft^ ls_js_sgx_cft^ ls_js_sgx_cft_mem hist_sgx_cft^ ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem RB put (/s)^ CHAMP put (/s)^ RB get (/s)^ CHAMP get (/s)^
72744 20230710.14 0.800586 45532.2 57523.5 35473.9 2.30851e+07 47676.7 55193.2 12589.1 19954.9 1.88908e+07 12995.6 20200.9 1.25993e+07 4534.16 22878 1.25993e+07 6715.39 1.67936e+07 3421.51 6521.4 6.30784e+06 3301.99 1743.27 1.05021e+07 48992.4 1452.47 1.05021e+07 1431.64 1.05021e+07 834304 1.17933e+06 8.11619e+06 3.06853e+07
72798 20230710.26 0.812772 45615.6 56675 35632.9 2.51822e+07 47320.7 55111.1 12475.3 19893.6 1.88908e+07 12598 20164.3 1.05021e+07 4518.38 22812.5 1.25993e+07 6324.2 1.67936e+07 3573.76 6427.2 6.30784e+06 3304.65 1739 1.05021e+07 47752.2 1435.26 1.05021e+07 1427.5 1.05021e+07 834517 1.17285e+06 8.13706e+06 3.13562e+07
72814 20230711.1 0.822951 45650.5 61833.3 35372.8 2.51822e+07 47315.6 55255 12539.8 20061.4 1.88908e+07 12841.6 20238.1 1.25993e+07 4455.94 22988 1.25993e+07 6721.4 1.67936e+07 3537.7 6564.4 6.30784e+06 3292.8 1745.99 1.05021e+07 46633.5 1438.48 1.05021e+07 1424.39 1.05021e+07 838546 1.1806e+06 8.15394e+06 3.07757e+07
72829 20230711.7 0.791713 45915.2 56906 34466.6 2.30851e+07 47081 54845.7 12541.4 19901.7 1.88908e+07 12672.2 20225 1.25993e+07 4509.76 22820.9 1.25993e+07 6678.88 1.67936e+07 3554.71 6560.5 6.30784e+06 3323.41 1748.32 1.05021e+07 48662.1 1439.48 1.05021e+07 1431.08 1.05021e+07 824913 1.17963e+06 8.13434e+06 3.0945e+07
72849 20230711.11 0.812991 45717.3 58794.4 35982.6 2.30851e+07 47912.9 54851.1 12423.3 19765.2 1.88908e+07 12836.8 20171.1 1.25993e+07 4462.62 22899.3 1.25993e+07 6314.4 1.67936e+07 3432.06 6474 6.30784e+06 3293.33 1739.82 1.05021e+07 46454.3 1436.66 1.05021e+07 1431.05 1.05021e+07 838987 1.17507e+06 8.15345e+06 3.10148e+07
72854 20230711.13 0.807603 46040.2 70109.4 35512.6 2.51822e+07 47781 55305.6 12411.2 19837 1.88908e+07 12938.3 20099.6 1.25993e+07 4412.3 22889.3 1.25993e+07 6707.87 1.67936e+07 3532.04 6528 6.30784e+06 3273.64 1734.98 1.05021e+07 43626.8 1433.09 1.05021e+07 1428.14 1.05021e+07 832428 1.17996e+06 8.15352e+06 3.07563e+07
72870 20230711.17 0.800785 43528.1 59075.4 35619.8 2.51822e+07 47370 55203 12448.6 19955.2 1.88908e+07 12865.2 20222.9 1.25993e+07 4534.37 22924.8 1.25993e+07 6725.15 1.67936e+07 3442.38 6524 6.30784e+06 3281.94 1741.44 1.05021e+07 48492.2 1437.05 1.05021e+07 1429.98 1.05021e+07 834837 1.18e+06 8.15209e+06 3.13826e+07
72899 20230712.1 0.817629 46005.4 61732.8 35539.2 2.51822e+07 46267.2 54817.6 12361 20098 1.88908e+07 12878 20256.9 1.25993e+07 4513.9 22921.3 1.25993e+07 6721.85 1.67936e+07 3525.42 6584.3 6.30784e+06 3277.82 1745.58 1.05021e+07 50523.7 1442.14 1.05021e+07 1430.43 1.05021e+07 833552 1.18361e+06 8.1731e+06 3.07896e+07
72916 20230712.9 0.797894 46029.2 64831.7 35694.8 2.51822e+07 47114.3 55431.5 12397.4 19992.6 1.88908e+07 12809.2 20209.5 1.25993e+07 4414.02 22903 1.25993e+07 6686.76 1.67936e+07 3533.23 6519.5 6.30784e+06 3276.07 1736.56 1.05021e+07 45948 1439.82 1.05021e+07 1428.56 1.05021e+07 830755 1.18136e+06 8.08135e+06 3.08918e+07
72928 20230712.13 0.799723 45762.6 61495.9 35535.6 2.51822e+07 46880.5 55048.8 12558.5 20046.2 1.67936e+07 12672.7 20161.4 1.25993e+07 4453.04 23021.4 1.25993e+07 6710.58 1.67936e+07 3523.86 6564.9 6.30784e+06 3283.39 1767.93 1.05021e+07 44788.5 1445.06 1.05021e+07 1436 1.05021e+07 840580 1.18275e+06 8.15465e+06 3.08862e+07
72981 20230712.26 0.813809 45598.2 65880.8 35275.6 2.51822e+07 44233.5 54556.6 12513.5 19966.3 1.88908e+07 12865.7 20119.8 1.25993e+07 4495.36 22909.6 1.25993e+07 6379.89 1.67936e+07 3520 6516.4 6.30784e+06 3275.66 1747.26 1.05021e+07 48089.6 1443.44 1.05021e+07 1432.78 1.05021e+07 840750 1.17558e+06 8.15374e+06 3.10944e+07
72986 20230712.27 0.812176 45590.3 56481.4 35645.6 2.51822e+07 47008.4 54528.8 12483.4 19693.3 1.88908e+07 12724.4 19960 1.25993e+07 4350.1 22888.3 1.25993e+07 6335.98 1.67936e+07 3379.21 6467.1 6.30784e+06 3283.36 1742.45 1.05021e+07 50037.2 1438.04 1.05021e+07 1405.44 1.05021e+07 834873 1.17802e+06 8.14599e+06 3.09174e+07
73004 20230712.33 0.812968 43847.4 62858.9 35930.1 2.51822e+07 46156.8 54946.3 12457.4 19911.9 1.88908e+07 12783.8 20179.4 1.25993e+07 4408.68 22792.5 1.25993e+07 6707.62 1.67936e+07 3491.91 6563.5 6.30784e+06 3301.96 1750.57 1.05021e+07 47692.9 1438.9 1.05021e+07 1436.09 1.05021e+07 829483 1.17554e+06 8.14939e+06 3.07674e+07
73023 20230712.38 0.776197 43776.5 63921.5 35346.2 2.51822e+07 46663.1 55196.6 12275.1 19803.6 1.88908e+07 12853.7 20142.2 1.25993e+07 4545.13 22794.7 1.25993e+07 6378.49 1.67936e+07 3535.92 6475.7 6.30784e+06 3295.26 1739.62 1.05021e+07 47073.2 1437.25 1.05021e+07 1425.26 1.05021e+07 842747 1.1838e+06 8.15358e+06 3.13394e+07
73033 20230713.1 0.85485 45771 65726.9 35459.9 2.51822e+07 47061.7 56071.3 12590.7 19966.7 1.88908e+07 12839.3 20161.4 1.25993e+07 4508.37 22890.6 1.25993e+07 6446.22 1.67936e+07 3557.15 6478.2 6.30784e+06 3303.04 1741.88 1.05021e+07 48582 1441.24 1.05021e+07 1426.37 1.05021e+07 818389 1.18097e+06 8.153e+06 3.073e+07
73052 20230713.8 0.802062 45795.1 70372.8 35363.7 2.51822e+07 47174.7 55397.1 12396.3 19736 1.88908e+07 12835.5 20080.7 1.25993e+07 4474.16 22670.8 1.25993e+07 6366.89 1.67936e+07 3550.68 6497.5 6.30784e+06 3257.04 1752.97 1.05021e+07 47685.4 1433.62 1.05021e+07 1435.81 1.05021e+07 822746 1.17828e+06 8.15413e+06 3.03948e+07
73066 20230713.13 0.795208 45653.8 68936.8 35726.5 2.30851e+07 47465.1 55269.1 12490.2 19952.2 1.88908e+07 12819.2 20043.5 1.25993e+07 4546.34 22849.1 1.25993e+07 6673.85 1.67936e+07 3555.72 6469.8 6.30784e+06 3310.87 1739.9 1.05021e+07 43220.4 1436.77 1.05021e+07 1429.29 1.05021e+07 833479 1.17929e+06 8.15378e+06 3.10439e+07
73091 20230713.22 0.793568 46198.4 63682.1 35187.3 2.51822e+07 47551.3 55228 12584.9 19794.1 1.88908e+07 12689.9 20195.1 1.25993e+07 4479.41 22861.7 1.25993e+07 6681.43 1.67936e+07 3544.88 6508 6.30784e+06 3271.82 1746.01 1.05021e+07 44792.2 1442.07 1.05021e+07 1430.15 1.05021e+07 837043 1.17886e+06 8.15332e+06 3.07771e+07
73098 20230714.1 0.790865 45877.5 64627 35632.3 2.51822e+07 47559.3 54965.2 12609.9 19988.6 1.88908e+07 12662.7 20230 1.25993e+07 4538.04 22972.5 1.25993e+07 6374.19 1.67936e+07 3556.02 6511.7 6.30784e+06 3275.15 1745.51 1.05021e+07 48445.8 1432.05 1.05021e+07 1425.52 1.05021e+07 843704 1.18342e+06 8.15209e+06 3.16739e+07
73119 20230714.10 0.819362 45837.8 69624.9 36055.4 2.51822e+07 47413.6 55492.5 12496 20021.7 1.88908e+07 12661.6 20101.8 1.25993e+07 4370.56 22955.1 1.25993e+07 6359.6 1.67936e+07 3530.65 6473.5 6.30784e+06 3285.52 1736.18 1.05021e+07 48956 1435.26 1.05021e+07 1421.37 1.05021e+07 830236 1.18254e+06 8.13234e+06 2.99709e+07

present_intermediate_certs

build_id build_number Commit latency factor pi_basic_mt_virtual_cft^ ls_virtual_cft^ pi_ls_virtual_cft^ pi_basic_virtual_cft^ ls_jwt_virtual_cft^ ls_sgx_cft^ ls_sgx_cft_mem pi_ls_jwt_virtual_cft^ pi_ls_sgx_cft^ pi_ls_sgx_cft_mem pi_basic_mt_sgx_cft^ pi_basic_mt_sgx_cft_mem ls_js_virtual_cft^ pi_basic_sgx_cft^ pi_basic_sgx_cft_mem ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem ls_full_js_virtual_cft^ pi_ls_jwt_sgx_cft^ pi_ls_jwt_sgx_cft_mem ls_js_jwt_virtual_cft^ ls_js_sgx_cft^ ls_js_sgx_cft_mem hist_sgx_cft^ ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem RB put (/s)^ CHAMP put (/s)^ RB get (/s)^ CHAMP get (/s)^
73086 20230713.20 0.822387 56280.7 45928.3 47635.9 55799.8 12417.6 19731.7 1.88908e+07 12667.7 20107.5 1.25993e+07 35729.4 2.51822e+07 4348.28 22840.4 1.25993e+07 6678.5 1.67936e+07 3568.62 6581.9 6.30784e+06 3297.34 1742.71 1.05021e+07 50753.3 1439.63 1.05021e+07 1428.64 1.05021e+07 829155 1.17906e+06 8.13425e+06 3.14574e+07
73114 20230714.8 0.770148 58785.2 45622.1 46658.3 56244 12826.9 20023.8 1.88908e+07 12837.1 20207.7 1.25993e+07 35636.6 2.30851e+07 4380.89 22965.7 1.25993e+07 6712.83 1.67936e+07 3545.18 6544.4 6.30784e+06 3307.27 1744.33 1.05021e+07 50719.3 1442.45 1.05021e+07 1426.9 1.05021e+07 822169 1.17326e+06 8.15202e+06 3.0822e+07
73128 20230714.12 0.795603 61879.1 45989.3 47739.7 56427.1 12407.5 19938.3 1.88908e+07 13010.5 20169 1.25993e+07 35776.2 2.51822e+07 4489.96 22979.8 1.25993e+07 6695.66 1.67936e+07 3537.63 6512.9 6.30784e+06 3299.26 1746 1.05021e+07 48751.4 1442.39 1.05021e+07 1432.81 1.05021e+07 831328 1.18118e+06 8.15533e+06 3.04336e+07

images

@eddyashton eddyashton enabled auto-merge (squash) July 14, 2023 08:57
@eddyashton eddyashton merged commit 467c57d into microsoft:main Jul 14, 2023
@ghost
Copy link

ghost commented Jul 14, 2023

💔 All backports failed

Status Branch Result
release/4.x Backport failed because of merge conflicts

You might need to backport the following PRs to release/4.x:
- Add getVersionOfPreviousWrite to TypeScript TypedKvMap (#5451)

Manual backport

To create the backport manually run:

backport --pr 5453

Questions ?

Please refer to the Backport tool documentation and see the Github Action logs for details

@eddyashton eddyashton added the backported This PR was successfully backported to LTS branch label Jul 14, 2023
eddyashton added a commit to eddyashton/CCF that referenced this pull request Jul 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
4.x-todo PRs which should be backported to 4.x auto-backport Automatically backport this PR to LTS branch backported This PR was successfully backported to LTS branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants