De-Template Track Finding, main branch (2024.10.01.)

[krasznaa]

So... this is how I've been imagining for a while now, how we should provide the reconstruction algorithms/tools. Not exposing any template based configurability in the APIs of the algorithms. 🤔

I wanted to demonstrate this on the track finding, because:

• it might just be our most elaborate piece of code;
• contrary to how it looks, we actually only ever used a single template specialization of it so far.

I moved everything from finding_algorithm.ipp into a private header under core/src/finding/. That code is still implemented as a templated function, with the same level of configurability that traccc::finding_algorithm had. But now it will be up to us to create well documented user APIs to that code, in the form of algorithms that perform one specific type of track finding each. 🤔

On the naming front I'm very open to good ideas. The name I chose now (traccc::host::default_detector_const_field_ckf_algorithm) is just meant to demonstrate that we may need to make the algorithm names quite descriptive in the end. Depending on how many different setups we may come up with.

I believe this code organization could be used for all algorithms that use detray::detector. What do you lot think?

P.S. I also tweaked the algorithmic code a little. Replacing some simple for-loops with some STL algorithms. I don't think it would change the code's performance, but I agree with @stephenswat that it's generally good practice to use STL algorithms wherever we can.

[beomki-yeo]

Didn't read the full PR - but I don't think I like this
Can't you just make a header file that defines the types that we want

// algorithms.hpp
using host_default_detector_const_field_ckf_algorithm = host::finding_algorithm<default_navigator_type, default_stepper_type>
using cuda_default_detector_const_field_ckf_algorithm =
cuda::finding_algorithm<default_navigator_type, default_stepper_type>
.. etc

[krasznaa]

Didn't read the full PR - but I don't think I like this Can't you just make a header file that defines the types that we want

// algorithms.hpp
using host_default_detector_const_field_ckf_algorithm = host::finding_algorithm<default_navigator_type, default_stepper_type>
using cuda_default_detector_const_field_ckf_algorithm =
cuda::finding_algorithm<default_navigator_type, default_stepper_type>
.. etc

I'm pondering about this still. 🤔 I worry about the understandability of the API. Let me tweak the proposal a bit more...

[krasznaa]

You guys might like this version even less, but here it goes... How about using some good old function overloading? 🤔

Again, I'd really like to avoid having templating "at the algorithm level". 🤔

[krasznaa]

Once we sort out #713, I'll use that to characterize these changes in a bit more detail. 🤔

[krasznaa]

So... Now that #713 can help with characterizing the changes of this PR.

• In the current code, to get track finding into traccc_seq_example, the build of seq_example.cpp does everything. And it uses 1.55 GB of memory to do so. (With GCC 11.)
• With this PR's code, bringing track finding to life in traccc_seq_example is split among the following source files:
  • ckf_algorithm.cpp: 350 MB
  • ckf_algorithm_teldet_cfield.cpp: 560 MB
  • ckf_algorithm_defdet_cfield.cpp: 810 MB
  • seq_example.cpp: 1.43 GB

So, we win a little more than 100 MB on compiling seq_example.cpp. While adding 3 more source files, with large, but semi-acceptable memory needs each.

As I discussed it with @stephenswat before lunch, I expect that after de-templating spacepoint creation and track fitting, seq_example.cpp should go down below 1 GB of memory. But I have no hard proof that it will actually happen.

As an interesting tidbit: I didn't do an exhaustive search yet for the biggest memory users, but read_detector.cpp, with 1.8 GB, seems to be a surprisingly large one. Clearly, constructing a geometry from JSON files is not an easy task for the compiler. 🤔

For @paulgessinger: Linking libActsCore.so takes a little more than 1 GB of memory. Underlining my previous statements that it's good to keep an eye on the linking steps as well. 🤔

[krasznaa]

@stephenswat, @beomki-yeo, have a look again. How about organizing the code like this?

[sonarqubecloud]

Quality Gate passed

Issues
13 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.3% Duplication on New Code

See analysis details on SonarCloud

[stephenswat]

This looks fine as far as I'm concerned. 👍

[paulgessinger]

For @paulgessinger: Linking libActsCore.so takes a little more than 1 GB of memory. Underlining my previous statements that it's good to keep an eye on the linking steps as well. 🤔

@krasznaa I've been looking into using CMake launchers for this (COMPILER, LINKER), that avoids having to abuse CTEST_USE_LAUNCHERS for this purpose, and is non-intrusive.

[krasznaa]

Okay, taking an executive decision here... 🤔