Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check validity times in default auth policies #4786

Merged
merged 27 commits into from
Jan 12, 2023

Conversation

eddyashton
Copy link
Member

@eddyashton eddyashton commented Dec 21, 2022

Resolves #4785.

We avoid using the untrusted host time in most framework code, as the threat model permits it to be manipulated by a malicious host. This leads to surprising permissiveness in some of our crypto validation, for instance accepting JWTs and certs which are expired. Using the untrusted host time here isn't a significant change in a malicious host's power - it allows them to DoS a caller by advancing the node's perception of time such that the caller's authentication is expired, but they cannot target this at a single user (affects the whole node), and they can likely build a more-targeted DoS already (by selectively dropping traffic). It is however a breaking change to the behaviour of these auth policies, which may now return auth errors for test info that they previously accepted.

TODO:

  • CHANGELOG

@eddyashton eddyashton requested a review from a team December 21, 2022 18:13
@eddyashton eddyashton marked this pull request as draft December 22, 2022 11:36
@eddyashton
Copy link
Member Author

Converting this to draft - looks like it introduces a major performance regression, I guess from the construction of a fresh verifier on every request. Have reduced the repetitions on the longer perf tests to confirm that.

I'll see if we can extract the validity periods more cheaply, directly from the serialised DER. If that doesn't work, I'll look at using the existing verifier cache (used for sig auth) for cert auth as well.

@ghost
Copy link

ghost commented Dec 22, 2022

expiry_times_in_auth_policies@60301 aka 20230112.18 vs main ewma over 20 builds from 59639 to 60249

Click to see table

main

build_id build_number Commit latency factor tpcc_virtual_cft^ ls_virtual_cft^ tpcc_sgx_cft^ tpcc_sgx_cft_mem ls_sgx_cft^ ls_sgx_cft_mem pi_ls_virtual_cft^ pi_ls_sgx_cft^ pi_ls_sgx_cft_mem pi_ls_jwt_virtual_cft^ ls_jwt_virtual_cft^ ls_js_virtual_cft^ ls_full_js_virtual_cft^ ls_js_jwt_virtual_cft^ hist_sgx_cft^ pi_ls_jwt_sgx_cft^ pi_ls_jwt_sgx_cft_mem ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem ls_js_sgx_cft^ ls_js_sgx_cft_mem ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem RB put (/s)^ CHAMP put (/s)^ RB get (/s)^ CHAMP get (/s)^
59639 20230104.19 0.793442 17099 43700.9 6317.48 8.19242e+07 15711.9 1.53396e+07 47658.5 15654 8.78598e+06 13311.4 12386.3 4631.54 3538.75 3445.45 27936.5 5888.9 6.95098e+06 5834.29 1.50774e+07 1565.38 7.73741e+06 1289.07 7.73741e+06 1297.73 6.95098e+06 825987 1.18309e+06 8.15566e+06 3.10906e+07
59665 20230105.2 0.79042 17170.6 43827.9 6325.77 8.21863e+07 15559.4 1.53396e+07 47798.4 15682.3 8.78598e+06 13342.3 14174.5 4513.65 3622.04 3476.38 22620.3 5861.5 6.68883e+06 5834.47 1.50774e+07 1570.57 7.73741e+06 1284.7 7.73741e+06 1315.74 6.95098e+06 837979 1.17992e+06 8.17323e+06 3.07162e+07
59678 20230105.6 0.79786 17180.4 43679.2 6287.89 8.24484e+07 15447.8 1.53396e+07 44918.6 15613.9 8.78598e+06 13323.5 12415.2 4590.03 3582.73 3448.03 27185.3 5818 6.68883e+06 5834.47 1.50774e+07 1571.65 7.73741e+06 1284.57 7.73741e+06 1300.41 6.95098e+06 818802 1.15908e+06 8.14269e+06 3.08531e+07
59719 20230105.16 0.816841 17292 43799.1 6305.07 8.24484e+07 15717.1 1.53396e+07 47855.8 15978.5 8.2617e+06 13712.7 12652 4681.08 3583.49 3465.9 23815.7 5810.9 4.59168e+06 5800.21 1.50774e+07 1572.4 7.73741e+06 1293.32 7.47526e+06 1309.53 6.95098e+06 832333 1.17719e+06 8.15582e+06 3.13903e+07
59747 20230105.22 0.794968 17255.8 43731.7 6282.56 8.21863e+07 15541.7 1.50774e+07 48926.1 15864.1 8.2617e+06 12965.6 12297.4 4606.8 3539.16 3455.56 27253 5731.4 4.59168e+06 5853.15 1.4291e+07 1568 7.73741e+06 1270.88 7.47526e+06 1290.39 6.95098e+06 830778 1.17671e+06 8.15459e+06 3.08021e+07
59781 20230106.3 0.766149 17131.4 45865.6 6304.56 8.21863e+07 15485.8 1.53396e+07 47560.6 15996.4 8.2617e+06 13760.5 12520 4556.31 3545.8 3482.61 29148.2 5767.3 4.59168e+06 5836.68 1.50774e+07 1571.69 7.73741e+06 1285.69 7.21312e+06 1301.23 6.95098e+06 830718 1.18073e+06 8.13347e+06 3.12057e+07
59796 20230106.7 0.761166 17319.1 43805.5 6393.56 8.19242e+07 15526.1 1.50774e+07 49223.5 15994 8.2617e+06 13222.1 14194.6 4558.98 3539.1 3405.32 22169.4 5776.3 4.59168e+06 5856.54 1.50774e+07 1573.51 7.73741e+06 1295.62 7.47526e+06 1310.43 6.95098e+06 838036 1.17707e+06 8.15342e+06 3.07106e+07
59857 20230106.21 0.824187 17277.1 43619.5 5669.72 8.32349e+07 15450.5 1.53396e+07 49466.7 15764.6 8.2617e+06 13908.8 12589.5 4670.24 3547.54 3478.18 22136.6 5734.8 4.59168e+06 5829.53 1.45532e+07 1556.45 7.73741e+06 1288.43 7.47526e+06 1286.07 6.95098e+06 834573 1.18175e+06 8.14777e+06 3.08434e+07
59868 20230106.24 0.790629 17180.8 45688.2 6320.95 8.19242e+07 15386.8 1.50774e+07 47530.3 15929.1 8.2617e+06 13969.6 12598.1 4494.25 3549.42 3319.78 24101.1 5683.6 4.32954e+06 5817.54 1.50774e+07 1574.04 7.73741e+06 1284.61 7.21312e+06 1300.25 6.95098e+06 838278 1.18118e+06 8.13463e+06 3.16206e+07
59884 20230106.28 0.795828 17323.4 43547.6 6286.33 8.24484e+07 15226.6 1.50774e+07 49261.4 15824.1 8.2617e+06 13946.8 12335 4496.59 3644.43 3497.16 27214.7 5691.5 4.59168e+06 5567.66 1.48153e+07 1549.25 7.73741e+06 1274.97 7.47526e+06 1294.91 6.95098e+06 833582 1.17879e+06 8.15352e+06 3.14975e+07
59913 20230109.3 0.801341 17030.4 43745.9 6306.03 8.19242e+07 15416.5 1.50774e+07 49415.6 15958.3 8.2617e+06 13103.9 12604.9 4577.92 3575.21 3419.69 22710.6 5774.8 4.59168e+06 5792.56 1.48153e+07 1576.66 7.73741e+06 1284.27 7.21312e+06 1294.81 6.95098e+06 831861 1.17817e+06 8.13638e+06 3.10176e+07
59965 20230109.15 0.780423 17207.7 45587.6 6324.3 8.21863e+07 15694.6 1.50774e+07 48721.1 15977.1 8.2617e+06 13982.3 14053 4691.8 3651.59 3447.26 27856.7 5780.6 4.59168e+06 5818.76 1.50774e+07 1569.53 7.73741e+06 1288.17 7.21312e+06 1309.81 6.95098e+06 835622 1.17871e+06 8.15426e+06 3.07291e+07
60058 20230110.2 0.7761 17177 45887.3 6398.3 8.24484e+07 15764.4 1.53396e+07 47185 15979.8 8.2617e+06 14027.2 12793.6 4427.6 3553.85 3323.97 28237.9 5811.5 4.59168e+06 5893.84 1.4291e+07 1572.98 7.73741e+06 1285.54 7.21312e+06 1307.16 6.95098e+06 840375 1.17776e+06 8.17301e+06 3.09067e+07
60117 20230110.20 0.793709 17246.8 45684.9 6323.27 8.21863e+07 15710.9 1.53396e+07 46985.6 15956.6 8.2617e+06 13606.6 12624.1 4479.73 3536.62 3340.99 24286.8 5785.1 4.59168e+06 5869.63 1.48153e+07 1567.43 7.73741e+06 1286.53 7.73741e+06 1296.55 6.95098e+06 835130 1.17371e+06 8.15566e+06 3.13385e+07
60151 20230111.5 0.791139 17171.3 43901.4 6343.82 8.19242e+07 15433.3 1.53396e+07 46470.5 15951.2 7.99955e+06 13117.5 12389.4 4473.32 3536.04 3461.34 21576.2 5778.4 4.59168e+06 5848.36 1.48153e+07 1578.22 7.73741e+06 1284.88 7.21312e+06 1309.21 6.95098e+06 828282 1.18042e+06 8.1099e+06 3.07678e+07
60167 20230111.8 0.758201 17139.6 45992.8 6315.59 8.19242e+07 15534.9 1.50774e+07 47619.5 15970.3 8.2617e+06 12093.6 12479.7 4485.43 3573.63 3409.95 24799.6 5782 4.59168e+06 5884.84 1.48153e+07 1570.16 7.73741e+06 1297.01 7.21312e+06 1307.84 6.95098e+06 818104 1.18208e+06 8.1379e+06 3.08169e+07
60177 20230111.10 0.795013 17324.5 41978.5 6290.6 8.24484e+07 15389.6 1.50774e+07 47072.4 15945.5 8.2617e+06 13939.1 12862.3 4483.28 3518.54 3376.87 26897.6 5797.9 4.59168e+06 5855.83 1.48153e+07 1568.94 7.73741e+06 1283.82 7.21312e+06 1297.04 6.95098e+06 834886 1.17906e+06 8.15394e+06 3.08541e+07
60205 20230111.20 0.818883 17129 45777.1 6326.66 8.19242e+07 15469 1.50774e+07 48471.7 15952.3 8.2617e+06 13013.7 12523.7 4453.97 3456.95 3417.23 26865.2 5810.4 4.59168e+06 5852.79 1.50774e+07 1572.8 7.73741e+06 1283.82 7.47526e+06 1292.92 6.95098e+06 817547 1.17827e+06 8.15949e+06 3.09095e+07
60242 20230112.2 0.801192 17468.6 45719.1 6337.48 8.19242e+07 15663 1.53396e+07 46008.3 15920.3 8.2617e+06 13174.6 12460.2 4435.54 3569.08 3280 29580.6 5776.9 4.59168e+06 5908.37 1.4291e+07 1573.91 7.73741e+06 1287.26 7.21312e+06 1307.87 6.95098e+06 829821 1.17384e+06 8.15592e+06 3.14491e+07
60249 20230112.5 0.798559 17286.4 43775.5 6272.12 8.24484e+07 15455.2 1.50774e+07 47350.2 15920.9 8.2617e+06 13596 12438.2 4492.29 3666.45 3433.92 28217.6 5746 4.59168e+06 5892.81 1.4291e+07 1568.37 7.73741e+06 1275.12 7.73741e+06 1282.5 6.95098e+06 828594 1.17891e+06 8.15433e+06 3.17598e+07

expiry_times_in_auth_policies

build_id build_number Commit latency factor tpcc_virtual_cft^ ls_virtual_cft^ tpcc_sgx_cft^ tpcc_sgx_cft_mem ls_sgx_cft^ ls_sgx_cft_mem pi_ls_virtual_cft^ pi_ls_sgx_cft^ pi_ls_sgx_cft_mem pi_ls_jwt_virtual_cft^ ls_jwt_virtual_cft^ ls_js_virtual_cft^ ls_full_js_virtual_cft^ ls_js_jwt_virtual_cft^ pi_ls_jwt_sgx_cft^ pi_ls_jwt_sgx_cft_mem ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem hist_sgx_cft^ ls_js_sgx_cft^ ls_js_sgx_cft_mem ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem RB put (/s)^ CHAMP put (/s)^ RB get (/s)^ CHAMP get (/s)^
59685 20230105.7 0.811035 17189 43738.7 6319.65 8.19242e+07 15184.9 1.50774e+07 43487.7 15151.4 8.78598e+06 12937.6 11554.3 4548.79 3539.65 3319.2 5679.9 6.68883e+06 5517.07 1.48153e+07 27950.7 1570.61 7.73741e+06 1283 7.47526e+06 1285.88 6.95098e+06 835355 1.18091e+06 8.17509e+06 3.11227e+07
59754 20230105.23 0.808523 17219.3 41674 5664.75 8.29727e+07 15133.2 1.50774e+07 45934.1 15430.6 8.2617e+06 13458.8 12772 4567.14 3529.16 3339.64 5593.7 4.59168e+06 5496.42 1.45532e+07 23002.6 1548.32 7.73741e+06 1281.09 7.21312e+06 1272.3 6.95098e+06 825250 1.18286e+06 8.17343e+06 3.08225e+07
59760 20230105.25 0.805234 17140 41887.7 6338.83 8.21863e+07 15117.9 1.53396e+07 45316.9 15455.8 8.2617e+06 13726.3 11549.8 4358.47 3657.59 3452.68 5636.6 4.59168e+06 5526.45 1.48153e+07 23839.2 1565.22 7.73741e+06 1286.66 7.47526e+06 1279.98 6.95098e+06 837581 1.17705e+06 8.1565e+06 3.12281e+07
59933 20230109.6 0.793637 17131 41826.1 6248.41 8.24484e+07 15025.6 1.58639e+07 44392.8 15504.8 8.2617e+06 13662.1 12769.3 4506.67 3556.61 3319.2 5625.7 4.59168e+06 5578.33 1.45532e+07 27665.9 1572 7.73741e+06 1279.13 7.21312e+06 1290.58 6.95098e+06 838551 1.17961e+06 8.17265e+06 3.10821e+07
60301 20230112.18 0.798787 17350.3 42035.7 6333.71 8.21863e+07 15253.9 1.50774e+07 47675.6 15546 8.2617e+06 12806.1 12899.8 4428.92 3569.48 3313.6 5626.5 4.59168e+06 5496.43 1.45532e+07 24835.9 1564.1 7.73741e+06 1280.22 7.73741e+06 1288.6 6.95098e+06 838932 1.17932e+06 8.15446e+06 3.07484e+07

images

@eddyashton
Copy link
Member Author

Major regression confirmed:
image

Will fix this before merging.

@eddyashton eddyashton self-assigned this Jan 4, 2023
@eddyashton eddyashton marked this pull request as ready for review January 4, 2023 14:38
@eddyashton
Copy link
Member Author

Converting this to draft - looks like it introduces a major performance regression, I guess from the construction of a fresh verifier on every request. Have reduced the repetitions on the longer perf tests to confirm that.

I'll see if we can extract the validity periods more cheaply, directly from the serialised DER. If that doesn't work, I'll look at using the existing verifier cache (used for sig auth) for cert auth as well.

Added a cache, and looks like the performance issue is resolved. Will give this another few runs.

@eddyashton
Copy link
Member Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

CHANGELOG.md Show resolved Hide resolved
@eddyashton eddyashton marked this pull request as draft January 5, 2023 11:57
@eddyashton
Copy link
Member Author

Perf impact is still larger than expected, especially the 5% hit on JWT auth. Returning to Draft status while this is investigated. Perhaps the construction of payload_typed is more expensive than expected, and unnecessary?

@eddyashton
Copy link
Member Author

A final thought to explore tomorrow: Is the slowdown simply because the JWT has increased in size?

@eddyashton
Copy link
Member Author

A final thought to explore tomorrow: Is the slowdown simply because the JWT has increased in size?

Yup, thanks to @achamayou for confirming this in #4821 - just increasing the size of the JWT to include time claims, with no change on the C++ side, causes a ~6% slowdown in the ls_jwt_sgx_cft benchmark. I'm surprised this small size increase (~410 bytes to ~450 bytes in 1 header) has such an impact, but I think we at least understand this now well enough to merge this PR.

@eddyashton eddyashton marked this pull request as ready for review January 12, 2023 10:38
@eddyashton eddyashton merged commit 5d7d81a into microsoft:main Jan 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Check validity periods and expiry times in default auth policies
2 participants