Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate callables for each PTO #123

Closed
wants to merge 5 commits into from
Closed

Conversation

felixhekhorn
Copy link
Contributor

Closes #119

  • in my first attempt I only checked tests are actually running
  • it still has some repetitions which maybe can be avoided
  • we can not cache the "dispatcher" - so the functions we generate dynamically since nb wants a definitive address (which we haven't since we generate on the fly); however, this is maybe acceptable since they are short and the hard work (the actual elements) are fully cached

we should assign this some priority to make the package usable again from the outside

@felixhekhorn felixhekhorn added the refactor Refactor code label May 27, 2022
@codecov
Copy link

codecov bot commented May 27, 2022

Codecov Report

Merging #123 (6106879) into develop (c316a2b) will decrease coverage by 46.58%.
The diff coverage is 33.33%.

Impacted file tree graph

@@             Coverage Diff              @@
##           develop     #123       +/-   ##
============================================
- Coverage   100.00%   53.41%   -46.59%     
============================================
  Files           67       68        +1     
  Lines         3214     3263       +49     
============================================
- Hits          3214     1743     -1471     
- Misses           0     1520     +1520     
Flag Coverage Δ
isobench 53.41% <33.33%> (-0.31%) ⬇️
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/eko/harmonics/__init__.py 32.25% <18.18%> (-67.75%) ⬇️
src/eko/evolution_operator/__init__.py 81.45% <20.00%> (-18.55%) ⬇️
...eko/matching_conditions/operator_matrix_element.py 39.49% <20.00%> (-60.51%) ⬇️
src/eko/matching_conditions/__init__.py 48.38% <40.00%> (-51.62%) ⬇️
src/eko/quad_ker_base.py 43.75% <43.75%> (ø)
src/eko/harmonics/polygamma.py 11.39% <0.00%> (-88.61%) ⬇️
src/eko/anomalous_dimensions/as3.py 19.04% <0.00%> (-80.96%) ⬇️
src/eko/matching_conditions/as3/aqqNS.py 20.00% <0.00%> (-80.00%) ⬇️
src/eko/matching_conditions/as3/aHq.py 23.52% <0.00%> (-76.48%) ⬇️
src/eko/kernels/singlet.py 24.47% <0.00%> (-75.53%) ⬇️
... and 56 more

@felixhekhorn
Copy link
Contributor Author

Actually, I think
https://github.com/N3PDF/eko/blob/143fe85e8ed3678da043e04694cbfe6df405aa8e/src/eko/matching_conditions/operator_matrix_element.py#L104-L107
the if condition could even be removed since the new compilation guard fulfils the same purpose ...

@felixhekhorn
Copy link
Contributor Author

  • we can not cache the "dispatcher" - so the functions we generate dynamically since nb wants a definitive address (which we haven't since we generate on the fly); however, this is maybe acceptable since they are short and the hard work (the actual elements) are fully cached

unfortunately in practice this is visible: for the first step in matching we pay 20s - compared to the ~7-8s for the second step; but for the moment I don't see a way around ...

@alecandido
Copy link
Member

alecandido commented May 27, 2022

  • we can not cache the "dispatcher" - so the functions we generate dynamically since nb wants a definitive address (which we haven't since we generate on the fly); however, this is maybe acceptable since they are short and the hard work (the actual elements) are fully cached

unfortunately in practice this is visible: for the first step in matching we pay 20s - compared to the ~7-8s for the second step; but for the moment I don't see a way around ...

The way out is to use the generator to actually write the function in a file, such that there is a given address. But I'm not sure that we want to do it, playing with files might have a long tail...

@felixhekhorn
Copy link
Contributor Author

@alecandido this is my second attempt; I'm not too happy, because there is too much copy and paste still in the game ... to be specific: these 11 lines https://github.com/N3PDF/eko/blob/61068792c2c53cc9d719dfd054fde4d769af14f8/src/eko/matching_conditions/operator_matrix_element.py#L119-L135 are copied almost exact 3 times ...

I guess, I should start introducing function pointers ... what do you think?

@alecandido
Copy link
Member

alecandido commented May 31, 2022

Wrap the three assignments into a single function:
https://github.com/N3PDF/eko/blob/61068792c2c53cc9d719dfd054fde4d769af14f8/src/eko/matching_conditions/operator_matrix_element.py#L119-L124
the early return can be just propagated (you early return in the function inside, and then check again in the one outside).

Also these two lines can be packed together in a single function:
https://github.com/N3PDF/eko/blob/61068792c2c53cc9d719dfd054fde4d769af14f8/src/eko/matching_conditions/operator_matrix_element.py#L133-L135

You know, they are small, so you might wonder if it's worth to bundle them into a function. I tell you that is worth: if you need to change, a single place will be affected.

In this way you go down to 8 lines. And I know they are repeated, but code is not only measured in lines: they contain almost nothing, the differences are relevant.
It's true that in principle you can get most of it in a single line, passing around the function pointers, but there is an extra value in simplicity: function pointers are complicated objects, then you might get closer to signatures, and we are screwed again.
In any case, you'd spend your time to make a more complicated call that would span 3 lines in any case, so just to remove 5 lines.

Factorize what is simple to gather, keep explicit everything else. We are always in time to further improve.

return ker[indices[self.mode0], indices[self.mode1]]
# AD mode
k = 0 if self.mode0 == 100 else 1
l = 0 if self.mode1 == 100 else 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a look at code factor, and most of the remaining issues are with poor names.

On one side, the immediate feeling is that it is too picky, and for sure it is not able to recognize long living variables from local indices and so on. But on the other, it is only (or mostly) complaining about one letter variables, that we might really consider too little, since it is for sure not meaningful at all (almost anonymous).

Here, e.g., I would make the minimal effort to improve, and just call the two of them fl1 and fl2 (or fli and flo, for I/O).

@felixhekhorn felixhekhorn marked this pull request as draft July 11, 2022 14:47
@felixhekhorn felixhekhorn mentioned this pull request Jul 29, 2022
14 tasks
@giacomomagni
Copy link
Collaborator

Should we close this?

@alecandido
Copy link
Member

alecandido commented Dec 20, 2022

Should we close this?

At some point, definitely. But @felixhekhorn mentioned that he wanted to resume the project, when possible.

We can close and restart from scratch later on, or wait for the replacement to close.
I'm fine with both.

@felixhekhorn
Copy link
Contributor Author

At some point, definitely. But @felixhekhorn mentioned that he wanted to resume the project, when possible.

yes, with more and more math coming in (QED, N3LO, polarized, FF), i.e. more and more complicated compilation, I consider this will become relevant again.

We can close and restart from scratch later on, or wait for the replacement to close. I'm fine with both.

since a lot of things changed meanwhile, I guess we can convert the idea into an issue (mentioning the alternative proposal of using C/Rust) and close this PR ...

@alecandido
Copy link
Member

This is outdated, and by now on hold.

Since we might want to step out Numba, maybe it will also be solved in a different way.

@alecandido alecandido changed the base branch from develop to master January 27, 2023 14:10
@felixhekhorn
Copy link
Contributor Author

Shifted as issue in #204

@felixhekhorn felixhekhorn deleted the feature/split-numba-pto branch January 27, 2023 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
refactor Refactor code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Matching failed to compile
3 participants