Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Container images release improvements for 3.0.0 #4381

Merged
merged 95 commits into from
Oct 26, 2022
Merged

Conversation

jumaffre
Copy link
Contributor

@jumaffre jumaffre commented Oct 21, 2022

Also resolves #4362

Improve Docker container image release process in line of upcoming 3.0.0 release.

Will be merged after #4367.

  • We now publish containers for both SGX, SNP and Virtual
  • app_dev and app_run dockerfiles have been cleaned up and now cater for all 3 platform types
  • Ansible playbooks have been updated to only install SGX dependencies on SGX containers
    - The containers GitHub Action that now run on main too (without publishing the containers) so that we know early when/if we've broken this pipeline
  • The CCF debian package now includes the compile target as part of its name (version field), e.g. ccf_3.0.0-sgx~dev6-69-gf8b673678

TODO:

  • Update ansible
  • Template Dockerfiles
  • Templated Github Actions containers.yml
  • Reuse --build-arg=platform=sgx for ansible if possible

@ghost
Copy link

ghost commented Oct 21, 2022

additional_containers@52316 aka 20221026.29 vs main ewma over 20 builds from 51800 to 52214

Click to see table

main

build_id build_number tpcc_virtual_cft^ tpcc_virtual_cft_mem ls_virtual_cft^ ls_virtual_cft_mem ls_jwt_virtual_cft^ ls_jwt_virtual_cft_mem tpcc_sgx_cft^ tpcc_sgx_cft_mem ls_js_virtual_cft^ ls_js_virtual_cft_mem ls_full_js_virtual_cft^ ls_full_js_virtual_cft_mem ls_js_jwt_virtual_cft^ ls_js_jwt_virtual_cft_mem ls_sgx_cft^ ls_sgx_cft_mem ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem ls_js_sgx_cft^ ls_js_sgx_cft_mem hist_sgx_cft^ ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem RB put (/s)^ CHAMP put (/s)^ RB get (/s)^ CHAMP get (/s)^
51800 20221020.52 11474.6 0 43005.4 0 10189.3 0 5792.02 8.42835e+07 4329.56 0 3606.16 0 3095.43 0 17941.5 1.66503e+07 6099.91 1.56017e+07 2362.63 9.31027e+06 24480.9 1984.09 9.57242e+06 1917.09 9.57242e+06 904628 1.37902e+06 9.22934e+06 3.56794e+07
51805 20221021.2 11181.3 0 42683.3 0 10188.4 0 5750.81 8.3497e+07 4351.77 0 3461.51 0 3212.55 0 17709.5 1.71746e+07 6168.59 1.56017e+07 2345.54 9.57242e+06 26940 1996.68 9.57242e+06 1921.35 9.04813e+06 902870 1.3806e+06 9.27948e+06 3.58669e+07
51813 20221021.5 11336.7 0 40642.5 0 11519.1 0 5793.12 8.3497e+07 4260.82 0 3425.09 0 3223.42 0 17799.1 1.6126e+07 6120.3 1.56017e+07 2348.47 9.57242e+06 28834.2 2044.46 9.31027e+06 1910.74 9.04813e+06 907679 1.37643e+06 9.22103e+06 3.59298e+07
51826 20221021.9 10567.1 0 42390.5 0 10341.2 0 5621.67 8.24484e+07 4289.94 0 3385.16 0 3174.02 0 17870.6 1.6126e+07 6151.43 1.56017e+07 2357.11 9.57242e+06 26911.8 1980.47 9.31027e+06 1926.41 9.04813e+06 885814 1.34551e+06 9.13063e+06 3.53713e+07
51832 20221021.11 10984.6 0 43765.1 0 10141 0 5611.78 8.3497e+07 4366.3 0 3463.29 0 3228.09 0 17968.1 1.63882e+07 6067.88 1.56017e+07 2340.66 9.57242e+06 24824.9 1975.36 9.57242e+06 1877.25 9.04813e+06 891090 1.36788e+06 9.21273e+06 3.56174e+07
51844 20221021.15 10939.3 0 42325.1 0 10632.1 0 5666.7 8.42835e+07 4249.92 0 3457.66 0 3218.37 0 17806.4 1.63882e+07 6079.43 1.56017e+07 2310.1 9.57242e+06 23210.3 2004.41 9.31027e+06 1914.02 9.31027e+06 908278 1.37283e+06 9.22099e+06 3.57411e+07
51922 20221023.1 11341.2 0 42755.5 0 10599.5 0 5780.97 8.21863e+07 4421.86 0 3478.67 0 3215.35 0 17755.2 1.6126e+07 6062.06 1.53396e+07 2339.54 9.57242e+06 24171.6 1976.51 9.57242e+06 1923.12 9.04813e+06 908401 1.36688e+06 9.14686e+06 3.57417e+07
51929 20221024.2 11020 0 42507.8 0 10714.4 0 5649.55 8.32349e+07 4234.49 0 3585.69 0 3241.89 0 18197.1 1.66503e+07 6173.25 1.58639e+07 2353.95 9.31027e+06 21619.3 1956.4 1.14074e+07 1881.83 9.04813e+06 924057 1.38349e+06 9.30901e+06 3.57411e+07
51934 20221024.4 11228.6 0 43074.1 0 10611.6 0 5696.81 8.37592e+07 4309.36 0 3534.43 0 3202.17 0 17882.1 1.6126e+07 6157.47 1.58639e+07 2348.42 9.31027e+06 28527.6 1978.09 1.27181e+07 1930.14 9.31027e+06 905468 1.35252e+06 9.41601e+06 3.56794e+07
51943 20221024.8 11138.8 0 41682.8 0 10100.2 0 5747.57 8.37592e+07 4213.87 0 3372.76 0 3197.46 0 17784.2 1.63882e+07 6134.12 1.56017e+07 2359.15 9.83456e+06 24870.3 1984.33 9.83456e+06 1930.04 9.31027e+06 898869 1.37809e+06 9.39876e+06 3.58669e+07
51959 20221024.15 11371.8 0 42816.8 0 10018.8 0 5782.18 8.45456e+07 4301.09 0 3541.08 0 3206.07 0 17690.6 1.6126e+07 6032.81 1.56017e+07 2359.2 9.57242e+06 24628.2 1980.58 9.57242e+06 1914.75 9.04813e+06 920195 1.36952e+06 9.33446e+06 3.58669e+07
52014 20221024.40 11398.6 0 43113.2 0 10654.3 0 5712.19 8.37592e+07 4225.14 0 3426.22 0 3251.09 0 17593.8 1.63882e+07 6139.58 1.58639e+07 2447.74 9.83456e+06 27874.6 1979.58 9.31027e+06 1921.92 9.04813e+06 906348 1.37366e+06 9.33025e+06 3.57417e+07
52041 20221024.53 11321.6 0 43611.8 0 10138.7 0 5760.17 8.55942e+07 4255.73 0 3463.17 0 3202.65 0 17977.4 1.58639e+07 6168.98 1.53396e+07 2360.75 9.57242e+06 25100.1 1990.82 9.57242e+06 1937.17 9.31027e+06 902277 1.37044e+06 9.22099e+06 3.57417e+07
52054 20221024.58 11044.1 0 41604.5 0 10104.2 0 5681.71 8.29727e+07 4372.67 0 3532.07 0 3171.67 0 17484.3 1.6126e+07 6021.95 1.56017e+07 2351.04 9.57242e+06 27719.6 2011.09 1.14074e+07 1915.13 9.04813e+06 885654 1.36824e+06 9.40308e+06 3.58669e+07
52062 20221025.1 10867.9 0 43201.8 0 10139.3 0 5619.86 8.29727e+07 4325.6 0 3455.55 0 3413.03 0 17763.6 1.6126e+07 6027.95 1.56017e+07 2351.98 9.31027e+06 28088.8 1976.69 9.31027e+06 1924.45 9.04813e+06 908070 1.35376e+06 9.20019e+06 3.47702e+07
52148 20221025.38 11063.7 0 40818 0 10463.8 0 5718.6 8.29727e+07 4339.26 0 3383.05 0 3229.84 0 17753.4 1.6126e+07 6044.86 1.58639e+07 2358.53 9.57242e+06 28238.2 1979.66 9.57242e+06 1921.17 9.04813e+06 888304 1.37246e+06 9.46824e+06 3.58036e+07
52163 20221025.43 11038.1 0 41761.6 0 10179 0 5600.9 8.32349e+07 4350.36 0 3469.75 0 3335.97 0 17696.9 1.66503e+07 6085.06 1.56017e+07 2358.74 9.83456e+06 28399.5 1983.17 9.57242e+06 1889.35 9.31027e+06 903435 1.38201e+06 9.39881e+06 3.62478e+07
52174 20221025.47 11133.2 0 43517.8 0 10340.2 0 5689.44 8.32349e+07 4330.74 0 3411.3 0 3218.81 0 17535.4 1.63882e+07 6022.79 1.58639e+07 2412.92 9.57242e+06 27570.9 1918.01 9.83456e+06 1919.16 9.31027e+06 914116 1.40687e+06 9.30901e+06 3.61831e+07
52195 20221026.2 11554.9 0 41858.6 0 10139.1 0 5681.17 8.3497e+07 4270.16 0 3340.51 0 3053.82 0 17589.3 1.66503e+07 6028.17 1.53396e+07 2370.68 9.57242e+06 27354.3 1984.11 9.57242e+06 1963.27 9.04813e+06 903111 1.34426e+06 9.21269e+06 3.58042e+07
52214 20221026.8 11377 0 43249 0 10217.2 0 5602.67 8.27106e+07 4275.83 0 3312.68 0 3029.65 0 17410.9 1.6126e+07 6036.44 1.53396e+07 2364.04 9.57242e+06 23651.4 1979.57 9.31027e+06 1879.58 9.04813e+06 884697 1.3768e+06 9.30055e+06 3.57411e+07

additional_containers

build_id build_number tpcc_sgx_cft^ tpcc_sgx_cft_mem tpcc_virtual_cft^ tpcc_virtual_cft_mem ls_sgx_cft^ ls_sgx_cft_mem ls_virtual_cft^ ls_virtual_cft_mem ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem ls_jwt_virtual_cft^ ls_jwt_virtual_cft_mem ls_js_sgx_cft^ ls_js_sgx_cft_mem ls_js_virtual_cft^ ls_js_virtual_cft_mem ls_full_js_virtual_cft^ ls_full_js_virtual_cft_mem ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem ls_js_jwt_virtual_cft^ ls_js_jwt_virtual_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem hist_sgx_cft^ RB put (/s)^ CHAMP put (/s)^ RB get (/s)^ CHAMP get (/s)^
52213 20221026.7 5461.36 8.32349e+07 10791.7 0 17545.8 1.71746e+07 40596.5 0 6109.37 1.56017e+07 9983.86 0 2339.14 9.83456e+06 4269.96 0 3613.59 0 1976.52 9.31027e+06 3186.91 0 1916.73 9.04813e+06 24357.6 897767 1.38424e+06 9.21273e+06 3.58042e+07
52316 20221026.29 5599.56 8.29727e+07 11163.2 0 16257 1.63882e+07 42697.2 0 5509.96 1.56017e+07 10401.9 0 2069.12 8.78598e+06 4333.3 0 3497.8 0 1690 8.78598e+06 3227.5 0 1623.91 8.2617e+06 24574.2 822761 1.18438e+06 8.15475e+06 3.09553e+07

images

@jumaffre jumaffre mentioned this pull request Oct 21, 2022
7 tasks
@jumaffre jumaffre enabled auto-merge (squash) October 26, 2022 13:55
@jumaffre jumaffre merged commit d7aadbe into main Oct 26, 2022
@jumaffre jumaffre deleted the additional_containers branch October 26, 2022 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Container job needs upgrade
2 participants