Reduce builds by building noarch and no-GPU packages in common pipeline

[tiran]

Fromager uses variants to build wheels for multiple GPU architectures. The large majority of Python packages are platform antagonistic (py3-none-any wheels) or just CPU, OS, and Python version specific (cp311-cp311-linux_x86_64 wheels or cp311-abi3-linux_x86_64 wheels). Very few packages are GPU specific.

Internally we are building about 230 wheels for CUDA and ROCm.

• 76% are purelib wheels (py3-none-any) and work on any system
• 21% are platlib wheels without GPU specific bindings
• 3% are platlib wheels with GPU specific code (torch, llama-cpp-python, pyarrow, flash-attn, vllm + vllm-flash-attn, and xformers)

The numbers are similar for Intel Gaudi, with some more Gaudi-specific, pre-built wheels.

We could speed up our pipeline by building 97% of the packages in a container with just CPU tools first, then do the remaining packages in GPU-specific containers. Maybe start by adding a flag to mark a package as GPU specific?

[dhellmann]

A big part of the philosophy behind fromager is that wheels come as a "set" and that the user shouldn't have to manage all of the intermediate dependencies. So, splitting up the build into those that can run with different system dependencies would break that design goal.

We've recently added features to bootstrap and build-sequence to skip building something if it already exists locally or on the wheel server. That has dramatically decreased the build times we're seeing, since we don't even process wheels that already exist. @shubhbapna has some other work queued up to use the metadata in wheels to let us skip re-processing the source code to determine dependencies during bootstrapping, and that should give us another boost.

Following that approach will also give us benefits as we branch out into building other sets of wheels for other products that share the same variant settings we have now but only intersect partially in the wheels that are built, because if a package was built for set A and can be reused in set B then it will be.