-
-
Notifications
You must be signed in to change notification settings - Fork 185
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CircleCI restrained their build time again. Now a job is limited to 1 hour. Make kung fu needed #1058
Comments
So confused by this https://circleci.com/blog/building-open-source-projects-on-circleci/ Does that mean that osresearch/heads and pull requests only could receive free open source tier and unlimited build time? I have found nothing public and CircleCI doesn't answer. |
Project is suspended on osresearch/heads, will open ticket |
This information states to use large ressource in free tier. Implemented under tlaurion@2f803bb and building happening under https://app.circleci.com/pipelines/github/tlaurion/heads/851/workflows/b30064d1-1b6f-4cc8-a879-cd7fe6d02b84/jobs/2343/parallel-runs/0/steps/0-103 The documentation doesn't define what a build is under new limitation "builds on the Free plan are limited to a maximum duration of 1 hour." If the "build" time is for the total pipeline, then we are now out of the free tier for sure. If builds means steps, then we can still optimize this a bit, and change steps and workspaces and caching to match the new limitation. Will come back at documenting this once we have result of https://app.circleci.com/pipelines/github/tlaurion/heads/851/workflows/b30064d1-1b6f-4cc8-a879-cd7fe6d02b84/jobs/2343/parallel-runs/0/steps/0-103 |
The good news is that the new limit of 1h is for a "job". This just means we need to decouple things more (musl-cross-make, coreboot.buildstack and boards builds). In current CircleCI Heads usage, this means that one hour limit needs to be able to reuse workspace, build something, and be able to create a workspace archive to be passed to another job which will reuse what was built. The strategy here is to split what takes the most time into smaller tasks.
From past experience which permitted to cache that, we were able to build on top of cache under 15 minutes, without having a large resource associated with CircleCI configs. In last attempt on my side, once musl-cross-make is built separately (see tlaurion@acd6061), building with CircleCI large resource identifier a board, after musl-cross-make is built seperately, fits just under the one hour limit (but fails when creating workspace cache), see here: https://app.circleci.com/pipelines/github/tlaurion/heads/862/workflows/58f32954-1aff-4118-9ff0-17c3621cfeab, specifically https://app.circleci.com/pipelines/github/tlaurion/heads/862/workflows/58f32954-1aff-4118-9ff0-17c3621cfeab/jobs/2360 which fails after having uploaded artifacts, that is, because musl-cross of the build was included into that job; which is the part that takes the most time for a board's build time if not already existing. So if we were building coreboot's musl-cross, we could still use the free tier. But Heads doesn't currently permit to only build musl-cross. The only supported coreboot's module supported make statements are:
We would need an independent coreboot.buildstack make target. @osresearch @Thrilleratplay @Tonux599 : if make I tried to create coreboot.buildstack under modules/coreboot but failed. Any insights/PR so that we could have the buildstack (musl-cross toolchain) built for a BOARD=xyz (so a specific version?) |
Once again, new public info is available.
Let's see.... What can be done.... |
Interestingly enough, it seems that #984 is mostly complete
|
Testing build of #1015 with the needed changes under https://app.circleci.com/pipelines/github/tlaurion/heads/873/workflows/fdc61f6b-8c1b-4bda-a2e9-82fb97f43e65 |
@Tonux599 as you can see under this testing branch, kgpe-d16 builds (workstation's kernel) is taking more then 40 minutes to build, which make the build fails.... https://app.circleci.com/pipelines/github/tlaurion/heads?branch=test_parallel I will have to remove kgpe-d16 board of CircleCI as current mitigation to have other builds succeed since CircleCI is stopping merge of other PR regression testing for other boards. |
CircleCI: We currently drop coreboot 4.11 builds. - There is a file missing in the builds. Not sure why/how this is happening src/soc/intel/fsp_broadwell_de/romstage/romstage.c:41:10: fatal error: build.h: No such file or directory Example:https://app.circleci.com/pipelines/github/tlaurion/heads/877/workflows/7d0248d2-459c-42ad-b741-8fd56a75d527/jobs/2487 - kgpe-d16_workstation building for all GPUs is unfortunately taking too much time to build (40 minutes). - Not sure why, but it seems that the kernel build paralellization is not working for 4.11 while it works for 4.13 Makefile: Uncomment MAKE_JOBS which passes the number of jobs to numbers cores by default and --max-load of 16 CircleCI: Remove CPUS statement to use Makefile default modules/newt: force build with one make job, otherwise there is a race condition in module which fails randomly expecting build modules. (TODO: FIX) Interestingly, building all coreboot 4.13 boards is happening on a clean commit just above 1h limit. More details: - CircleCI changed job build time to a maximum of 1h each. - CircleCI now permits parallelization of 30 jobs - 6000 build minutes a month. - Still waiting for osresearch/heads CircleCI project to be unlocked (currently not recognized as open source project?!)
CircleCI: We currently drop coreboot 4.11 builds. - There is a file missing in the builds. Not sure why/how this is happening src/soc/intel/fsp_broadwell_de/romstage/romstage.c:41:10: fatal error: build.h: No such file or directory Example:https://app.circleci.com/pipelines/github/tlaurion/heads/877/workflows/7d0248d2-459c-42ad-b741-8fd56a75d527/jobs/2487 - kgpe-d16_workstation building for all GPUs is unfortunately taking too much time to build (40 minutes). - Not sure why, but it seems that the kernel build paralellization is not working for 4.11 while it works for 4.13 Makefile: Uncomment MAKE_JOBS which passes the number of jobs to numbers cores by default and --max-load of 16 CircleCI: Remove CPUS statement to use Makefile default modules/newt: force build with one make job, otherwise there is a race condition in module which fails randomly expecting build modules. (TODO: FIX) Interestingly, building all coreboot 4.13 boards is happening on a clean commit just above 1h limit. More details: - CircleCI changed job build time to a maximum of 1h each. - CircleCI now permits parallelization of 30 jobs - 6000 build minutes a month. - Still waiting for osresearch/heads CircleCI project to be unlocked (currently not recognized as open source project?!)
Not sure what to do from here.
Last failed attempt on my instance https://app.circleci.com/pipelines/github/tlaurion/heads/847/workflows/518245a6-5802-4854-888a-6e44bbc7d110
@MrChromebox raised the issue.
What now? Should we raise money to get paid plan on CircleCI?
@MrChromebox @jans23?
The text was updated successfully, but these errors were encountered: