Allow evolution and convolution with different ekos for a single grid

[giacomomagni]

Adress #180.

[felixhekhorn]

Do you want to make this a draft or shall I comment right away?

[giacomomagni]

Please feel free to comment, but note that this are dirty changes and ugly solutions, so be patient 😬

[Radonirinaunimi]

As soon as this is ready and good to go (and perhaps modulo some cosmetic changes), we can test it with NNPDF/pineappl#289 - which should already give exactly what we want.

[Radonirinaunimi]

For the benchmarks, we don't necessarily need a new pineappl release. For testing purposes and until NNPDF/pineappl#289 is merged we can do the following in pyproject.toml:

pineappl = { git = "https://github.com/NNPDF/pineappl.git", branch = "extend_eko_convolution", subdirectory = "pineappl_py" }

[Radonirinaunimi]

The benchmarks still fail when I tried locally because of some breaking changes that have been introduced recently (?). In particular regarding the renaming of lumi->channel: NNPDF/pineappl@aef0982

cc @cschwan

[scarlehoff]

The inversion method is also different.

I'm running the ekos now with this branch so that I can give you the objets that reproduce the bug on my side.

[scarlehoff]

I can reproduce the error with a new environment:

pip install .[nnpdf]
pineko fonll tcards 41000000
pineko fonll ekos 41000000 HERA_NC_318GEV_EM-SIGMARED --overwrite
pineko fonll fks 41000000 HERA_NC_318GEV_EM-SIGMARED --overwrite

ekos, fktables and other artifacts can be found here: https://cernbox.cern.ch/s/RAADfwb3qk6HjyK (there's a few empty folders because there are inheritances and it doesn't work well when I copy the links from afs to eos -which is the filesystem of lxplus from which I can create public links-)

env

Package Version

---

absl-py 2.1.0
alabaster 0.7.16
astunparse 1.6.3
babel 2.16.0
blessings 1.7
certifi 2024.7.4
charset-normalizer 3.3.2
click 8.1.7
cloudpickle 3.0.0
commonmark 0.9.1
contourpy 1.2.1
cycler 0.12.1
dask 2024.8.0
distributed 2024.8.0
docutils 0.21.2
eko 0.14.6
flatbuffers 24.3.25
fonttools 4.53.1
fsspec 2024.6.1
future 1.0.0
gast 0.6.0
google-pasta 0.2.0
grpcio 1.65.4
h5py 3.11.0
hyperopt 0.2.7
idna 3.7
imagesize 1.4.1
importlib_metadata 8.2.0
importlib_resources 6.4.2
Jinja2 3.1.4
keras 3.5.0
kiwisolver 1.4.5
latexcodec 3.0.0
lhapdf-management 0.5
libclang 18.1.1
llvmlite 0.42.0
locket 1.0.0
lz4 4.3.3
Markdown 3.6
MarkupSafe 2.1.5
matplotlib 3.7.5
ml-dtypes 0.4.0
msgpack 1.0.8
namex 0.0.8
networkx 3.2.1
nnpdf 4.0.9.post1280.dev0+958470844
numba 0.59.1
numpy 1.26.4
opt-einsum 3.3.0
optree 0.12.1
packaging 24.1
pandas 2.2.2
partd 1.4.2
pillow 10.4.0
pineappl 0.8.2
pineko 0.4.5.post178+725db3d
pip 21.2.3
prompt_toolkit 3.0.47
protobuf 4.25.4
psutil 6.0.0
py4j 0.10.9.7
pybtex 0.24.0
pybtex-docutils 1.0.3
Pygments 2.18.0
pymongo 3.13.0
pyparsing 3.1.2
python-dateutil 2.9.0.post0
pytz 2024.1
PyYAML 6.0.2
reportengine 0.30.dev0
requests 2.32.3
rich 12.6.0
ruamel.yaml 0.17.40
ruamel.yaml.clib 0.2.8
scipy 1.13.1
seaborn 0.13.2
setuptools 53.0.0
six 1.16.0
snowballstemmer 2.2.0
sortedcontainers 2.4.0
Sphinx 7.4.7
sphinxcontrib-applehelp 2.0.0
sphinxcontrib-bibtex 2.6.2
sphinxcontrib-devhelp 2.0.0
sphinxcontrib-htmlhelp 2.1.0
sphinxcontrib-jsmath 1.0.1
sphinxcontrib-qthelp 2.0.0
sphinxcontrib-serializinghtml 2.0.0
tblib 3.0.0
tensorboard 2.17.1
tensorboard-data-server 0.7.2
tensorflow 2.17.0
tensorflow-io-gcs-filesystem 0.37.1
termcolor 2.4.0
tomli 2.0.1
toolz 0.12.1
tornado 6.4.1
tqdm 4.66.5
typing_extensions 4.12.2
tzdata 2024.1
urllib3 2.2.2
validobj 1.2
wcwidth 0.2.13
Werkzeug 3.0.3
wheel 0.44.0
wrapt 1.16.0
zict 3.0.0
zipp 3.20.0

[giacomomagni]

@scarlehoff, so now with the template card matching exactly your settings, eko from this branch,
new or old grid and eko and fktable from this branch I do not find the bug.

I'll now inspect your ekos...

difference old grid, new eko

LHAPDF 6.4.0 loading /project/theorie/gmagni/miniconda3/envs/nnpdf/share/LHAPDF/NNPDF40_nnlo_as_01180/NNPDF40_nnlo_as_01180_0000.dat
NNPDF40_nnlo_as_01180 PDF set, member #0, version 1; LHAPDF ID = 331100
     allow_different_ekos      main      abs_diff      rel_diff
0                1.431205  1.431205 -2.172040e-12 -1.517631e-10
1                1.393423  1.393423 -2.899903e-13 -2.081136e-11
2                1.366727  1.366727 -2.884359e-13 -2.110413e-11
3                1.288318  1.288318 -2.535749e-13 -1.968264e-11
4                1.363930  1.363930 -3.503864e-13 -2.568946e-11
5                1.324558  1.324558 -2.498002e-13 -1.885913e-11
6                1.180105  1.180105  1.484812e-12  1.258204e-10
7                1.327288  1.327288 -2.577938e-13 -1.942260e-11
8                1.210373  1.210373  1.405764e-12  1.161431e-10
9                1.058666  1.058666 -1.690426e-12 -1.596751e-10
10               0.904670  0.904670  6.575851e-13  7.268785e-11
11               0.765871  0.765871 -1.183387e-12 -1.545151e-10
12               1.272911  1.272911  1.403988e-12  1.102974e-10
13               1.231978  1.231978  1.353362e-12  1.098527e-10
14               1.093105  1.093105 -1.655565e-12 -1.514552e-10
15               0.934928  0.934928  6.024070e-13  6.443354e-11
16               0.788414  0.788414 -1.217026e-12 -1.543639e-10
17               0.677735  0.677735 -1.052158e-12 -1.552463e-10
18               0.576294  0.576294 -6.877832e-13 -1.193459e-10
19               0.496117  0.496117 -9.744427e-13 -1.964141e-10
20               0.427571  0.427571 -1.120826e-12 -2.621376e-10
21               0.368362  0.368362 -1.092904e-12 -2.966924e-10
22               0.325425  0.325425 -5.396239e-13 -1.658214e-10
23               1.220661  1.220661  1.015632e-12  8.320346e-11
24               1.112946  1.112946 -1.630696e-12 -1.465206e-10
25               0.956404  0.956404  5.553336e-13  5.806477e-11
26               0.805117  0.805117 -1.229683e-12 -1.527335e-10
27               0.689732  0.689732 -1.058820e-12 -1.535118e-10
28               0.583865  0.583865 -7.083223e-13 -1.213162e-10
29               0.500366  0.500366 -9.854340e-13 -1.969428e-10
30               0.429142  0.429142 -1.066980e-12 -2.486311e-10
31               0.367727  0.367727 -1.088407e-12 -2.959822e-10
32               0.323516  0.323516 -5.219158e-13 -1.613261e-10
33               0.264223  0.264223  1.676437e-14  6.344770e-12
34               0.144370  0.144370 -4.829470e-15 -3.345213e-12
35               1.178929  1.178929 -8.046896e-13 -6.825598e-11
36               1.121619  1.121619 -1.588063e-12 -1.415866e-10
37               0.971998  0.971998  3.562706e-13  3.665342e-11
38               0.818175  0.818175 -1.401435e-12 -1.712879e-10
39               0.699260  0.699260 -1.051159e-12 -1.503245e-10
40               0.589902  0.589902 -7.408518e-13 -1.255889e-10
41               0.503755  0.503755 -9.880985e-13 -1.961466e-10
42               0.430396  0.430396 -1.056433e-12 -2.454561e-10
43               0.367228  0.367228 -1.083467e-12 -2.950391e-10
44               0.322009  0.322009 -5.074829e-13 -1.575989e-10
45               0.261942  0.261942  1.654232e-14  6.315251e-12
46               0.141963  0.141963 -4.884981e-15 -3.441019e-12
47               1.101881  1.101881 -1.137535e-12 -1.032357e-10
48               0.990724  0.990724  3.154144e-13  3.183675e-11
49               0.837385  0.837385 -1.425304e-12 -1.702090e-10
50               0.713844  0.713844 -1.064926e-12 -1.491820e-10
51               0.599273  0.599273 -7.664980e-13 -1.279045e-10
52               0.509059  0.509059 -9.959811e-13 -1.956513e-10
53               0.432404  0.432404 -1.042333e-12 -2.410553e-10
54               0.366535  0.366535 -1.071809e-12 -2.924165e-10
55               0.319768  0.319768 -4.864442e-13 -1.521239e-10
56               0.258520  0.258520  1.565414e-14  6.055294e-12
57               0.138364  0.138364 -4.551914e-15 -3.289807e-12
58               1.044191  1.044191  7.549517e-13  7.230012e-11
59               0.996820  0.996820  2.843281e-13  2.852350e-11
60               0.850581  0.850581 -1.427858e-12 -1.678685e-10
61               0.724812  0.724812 -1.074030e-12 -1.481805e-10
62               0.606550  0.606550 -7.885914e-13 -1.300126e-10
63               0.513265  0.513265 -9.903189e-13 -1.929451e-10
64               0.434086  0.434086 -1.027289e-12 -2.366559e-10
65               0.366146  0.366146 -1.065426e-12 -2.909841e-10
66               0.318195  0.318195 -4.711787e-13 -1.480784e-10
67               0.256044  0.256044  1.548761e-14  6.048797e-12
68               0.135745  0.135745 -4.468648e-15 -3.291945e-12
69               0.020857  0.020857 -1.249001e-16 -5.988467e-13
70               0.975756  0.975756 -2.864375e-14 -2.935546e-12
71               0.862854  0.862854 -1.438849e-12 -1.667546e-10
72               0.737335  0.737335 -1.087797e-12 -1.475308e-10
73               0.615373  0.615373 -8.137935e-13 -1.322439e-10
74               0.518554  0.518554 -1.000644e-12 -1.929683e-10
75               0.436385  0.436385 -1.014966e-12 -2.325847e-10
76               0.365980  0.365980 -1.058098e-12 -2.891137e-10
77               0.316622  0.316622 -4.547474e-13 -1.436247e-10
78               0.253392  0.253392  1.515454e-14  5.980672e-12
79               0.132873  0.132873 -4.357625e-15 -3.279547e-12
80               0.921142  0.921142 -1.112666e-12 -1.207920e-10
81               0.868284  0.868284 -1.446288e-12 -1.665685e-10
82               0.746887  0.746887 -1.074030e-12 -1.438008e-10
83               0.622828  0.622828 -8.383294e-13 -1.346005e-10
84               0.523260  0.523260 -1.011191e-12 -1.932484e-10
85               0.438647  0.438647 -1.005918e-12 -2.293230e-10
86               0.366204  0.366204 -1.063372e-12 -2.903770e-10
87               0.315682  0.315682 -4.428125e-13 -1.402715e-10
88               0.251551  0.251551  1.504352e-14  5.980317e-12
89               0.130777  0.130777 -4.052314e-15 -3.098645e-12
90               0.019460  0.019460  4.163336e-17  2.139435e-13
91               0.866775  0.866775 -1.435296e-12 -1.655904e-10
92               0.756410  0.756410 -1.079692e-12 -1.427389e-10
93               0.631614  0.631614 -8.619772e-13 -1.364722e-10
94               0.529165  0.529165 -1.020184e-12 -1.927914e-10
95               0.441767  0.441767 -1.012579e-12 -2.292111e-10
96               0.366969  0.366969 -1.052602e-12 -2.868367e-10
97               0.315101  0.315101 -4.336531e-13 -1.376234e-10
98               0.249908  0.249908  1.521006e-14  6.086269e-12
99               0.128726  0.128726 -3.969047e-15 -3.083329e-12
100              0.845044  0.845044 -1.468492e-12 -1.737769e-10
101              0.762949  0.762949 -1.085687e-12 -1.423014e-10
102              0.639632  0.639632 -8.799628e-13 -1.375734e-10
103              0.534953  0.534953 -1.030509e-12 -1.926355e-10
104              0.445090  0.445090 -1.038114e-12 -2.332369e-10
105              0.368163  0.368163 -1.004363e-12 -2.728036e-10
106              0.315075  0.315075 -4.232725e-13 -1.343403e-10
107              0.248901  0.248901  1.443290e-14  5.798660e-12
108              0.127231  0.127231 -3.802514e-15 -2.988663e-12
109              0.767864  0.767864 -1.076805e-12 -1.402339e-10
110              0.650886  0.650886 -9.114931e-13 -1.400389e-10
111              0.543829  0.543829 -1.041056e-12 -1.914308e-10
112              0.450573  0.450573 -1.038003e-12 -2.303742e-10
113              0.370627  0.370627 -1.000644e-12 -2.699865e-10
114              0.315807  0.315807 -4.120038e-13 -1.304606e-10
115              0.248214  0.248214  1.451617e-14  5.848248e-12
116              0.125669  0.125669 -3.719247e-15 -2.959562e-12
117              0.017926  0.017926  1.942890e-16  1.083868e-12
118              0.750426  0.750426 -9.695578e-13 -1.292010e-10
119              0.668063  0.668063 -9.264811e-13 -1.386817e-10
120              0.559651  0.559651 -1.123324e-12 -2.007187e-10
121              0.461124  0.461124 -1.018630e-12 -2.209013e-10
122              0.376197  0.376197 -1.008749e-12 -2.681434e-10
123              0.318522  0.318522 -4.027334e-13 -1.264380e-10
124              0.248545  0.248545  1.457168e-14  5.862791e-12
125              0.124198  0.124198 -3.691492e-15 -2.972263e-12
126              0.017365  0.017365  2.636780e-16  1.518480e-12
127              0.689771  0.689771 -9.804380e-13 -1.421396e-10
128              0.593159  0.593159 -1.222245e-12 -2.060568e-10
129              0.486219  0.486219 -1.062095e-12 -2.184397e-10
130              0.391073  0.391073 -1.038503e-12 -2.655518e-10
131              0.327452  0.327452 -3.812506e-13 -1.164293e-10
132              0.252404  0.252404  1.354472e-14  5.366282e-12
133              0.123518  0.123518 -3.691492e-15 -2.988627e-12
134              0.016756  0.016756  3.157197e-16  1.884165e-12
135              0.631652  0.631652 -1.308065e-12 -2.070862e-10
136              0.546746  0.546746 -1.218914e-12 -2.229395e-10
137              0.430899  0.430899 -1.129152e-12 -2.620457e-10
138              0.353535  0.353535 -3.855805e-13 -1.090642e-10
139              0.266634  0.266634  1.415534e-14  5.308896e-12
140              0.126101  0.126101 -3.608225e-15 -2.861371e-12
141              0.016365  0.016365  3.920475e-16  2.395654e-12
142              0.603286  0.603286 -1.217693e-12 -2.018434e-10
143              0.501115  0.501115 -1.275757e-12 -2.545838e-10
144              0.402437  0.402437 -4.086731e-13 -1.015495e-10
145              0.295010  0.295010  1.498801e-14  5.080507e-12
146              0.133527  0.133527 -3.580469e-15 -2.681452e-12
147              0.016462  0.016462  4.302114e-16  2.613377e-12
148              0.577670  0.577670 -1.474487e-12 -2.552473e-10
149              0.470997  0.470997 -4.642953e-13 -9.857704e-11
150              0.337669  0.337669  1.831868e-14  5.425043e-12
151              0.145597  0.145597 -3.413936e-15 -2.344789e-12
152              0.016974  0.016974  4.614364e-16  2.718542e-12
153              0.415504  0.415504  2.331468e-14  5.611180e-12
154              0.171563  0.171563 -3.552714e-15 -2.070798e-12
155              0.018347  0.018347  5.932754e-16  3.233585e-12
156              0.200463  0.200463 -3.330669e-15 -1.661490e-12
157              0.020233  0.020233  6.036838e-16  2.983643e-12
158              0.023523  0.023523  7.042977e-16  2.994122e-12
Thanks for using LHAPDF 6.4.0. Please make sure to cite the paper:
  Eur.Phys.J. C75 (2015) 3, 132  (http://arxiv.org/abs/1412.7420)

[scarlehoff]

Are we using the same version of eko, and pineappl? (and pineko, I'm using this commit: 725db3d) ?

Before we continue, if you do

pineappl convolute HERA_NC_318GEV_EM_SIGMARED.pineappl.lz4 NNPDF40_nnlo_as_01180

from fktables/41000000 in the last folder that I linked, do you get the same results I do?

Results

 b      Q2            x                y             integ
      [GeV^2]        []               []               []
---+-----+-----+------+------+---------+---------+------------
  0    60    60 0.0008 0.0008   0.74111   0.74111 4.7527486e-1
  1    90    90 0.0013 0.0013    0.6841    0.6841 4.6824195e-1
  2    90    90 0.0015 0.0015   0.59289   0.59289 4.7514910e-1
  3    90    90  0.002  0.002   0.44466   0.44466 4.8092986e-1
  4   120   120 0.0016 0.0016   0.74111   0.74111 4.5819655e-1
  5   120   120  0.002  0.002   0.59289   0.59289 4.7249652e-1
  6   120   120 0.0032 0.0032   0.37055   0.37055 4.8040902e-1
  7   150   150  0.002  0.002   0.74111   0.74111 4.5715597e-1
  8   150   150 0.0032 0.0032   0.46319   0.46319 4.7957520e-1
  9   150   150  0.005  0.005   0.29644   0.29644 4.8026955e-1
 10   150   150  0.008  0.008   0.18528   0.18528 4.7421085e-1
 11   150   150  0.013  0.013   0.11402   0.11402 4.6529739e-1
 12   200   200 0.0026 0.0026   0.76011   0.76011 4.5561647e-1
 13   200   200 0.0032 0.0032   0.61759   0.61759 4.7171272e-1
 14   200   200  0.005  0.005   0.39526   0.39526 4.8309162e-1
 15   200   200  0.008  0.008   0.24704   0.24704 4.8025731e-1
 16   200   200  0.013  0.013   0.15202   0.15202 4.7171968e-1
 17   200   200   0.02   0.02  0.098814  0.098814 4.6076735e-1
 18   200   200  0.032  0.032  0.061759  0.061759 4.4361258e-1
 19   200   200   0.05   0.05  0.039526  0.039526 4.2104216e-1
 20   200   200   0.08   0.08  0.024704  0.024704 3.9167733e-1
 21   200   200   0.13   0.13  0.015202  0.015202 3.5461824e-1
 22   200   200   0.18   0.18  0.010979  0.010979 3.1924756e-1
 23   250   250 0.0033 0.0033   0.74859   0.74859 4.5951508e-1
 24   250   250  0.005  0.005   0.49407   0.49407 4.8239125e-1
 25   250   250  0.008  0.008   0.30879   0.30879 4.8419464e-1
 26   250   250  0.013  0.013   0.19003   0.19003 4.7648900e-1
 27   250   250   0.02   0.02   0.12352   0.12352 4.6509128e-1
 28   250   250  0.032  0.032  0.077199  0.077199 4.4690594e-1
 29   250   250   0.05   0.05  0.049407  0.049407 4.2311428e-1
 30   250   250   0.08   0.08  0.030879  0.030879 3.9234228e-1
 31   250   250   0.13   0.13  0.019003  0.019003 3.5370930e-1
 32   250   250   0.18   0.18  0.013724  0.013724 3.1724851e-1
 33   250   250   0.25   0.25 0.0098814 0.0098814 2.6197443e-1
 34   250   250    0.4    0.4 0.0061759 0.0061759 1.4416114e-1
 35   300   300 0.0039 0.0039   0.76011   0.76011 4.6072801e-1
 36   300   300  0.005  0.005   0.59289   0.59289 4.7852282e-1
 37   300   300  0.008  0.008   0.37055   0.37055 4.8658670e-1
 38   300   300  0.013  0.013   0.22803   0.22803 4.8020682e-1
 39   300   300   0.02   0.02   0.14822   0.14822 4.6857936e-1
 40   300   300  0.032  0.032  0.092638  0.092638 4.4958523e-1
 41   300   300   0.05   0.05  0.059289  0.059289 4.2480528e-1
 42   300   300   0.08   0.08  0.037055  0.037055 3.9289651e-1
 43   300   300   0.13   0.13  0.022803  0.022803 3.5300185e-1
 44   300   300   0.18   0.18  0.016469  0.016469 3.1567412e-1
 45   300   300   0.25   0.25  0.011858  0.011858 2.5968205e-1
 46   300   300    0.4    0.4 0.0074111 0.0074111 1.4174774e-1
 47   400   400 0.0053 0.0053   0.74577   0.74577 4.6801159e-1
 48   400   400  0.008  0.008   0.49407   0.49407 4.8768018e-1
 49   400   400  0.013  0.013   0.30404   0.30404 4.8559196e-1
 50   400   400   0.02   0.02   0.19763   0.19763 4.7405536e-1
 51   400   400  0.032  0.032   0.12352   0.12352 4.5388719e-1
 52   400   400   0.05   0.05  0.079051  0.079051 4.2755971e-1
 53   400   400   0.08   0.08  0.049407  0.049407 3.9386462e-1
 54   400   400   0.13   0.13  0.030404  0.030404 3.5200404e-1
 55   400   400   0.18   0.18  0.021959  0.021959 3.1333711e-1
 56   400   400   0.25   0.25   0.01581   0.01581 2.5624472e-1
 57   400   400    0.4    0.4 0.0098814 0.0098814 1.3814025e-1
 58   500   500 0.0066 0.0066   0.74859   0.74859 4.7257487e-1
 59   500   500  0.008  0.008   0.61759   0.61759 4.8449758e-1
 60   500   500  0.013  0.013   0.38005   0.38005 4.8910351e-1
 61   500   500   0.02   0.02   0.24704   0.24704 4.7834054e-1
 62   500   500  0.032  0.032    0.1544    0.1544 4.5740849e-1
 63   500   500   0.05   0.05  0.098814  0.098814 4.2988193e-1
 64   500   500   0.08   0.08  0.061759  0.061759 3.9478461e-1
 65   500   500   0.13   0.13  0.038005  0.038005 3.5139504e-1
 66   500   500   0.18   0.18  0.027448  0.027448 3.1169630e-1
 67   500   500   0.25   0.25  0.019763  0.019763 2.5375980e-1
 68   500   500    0.4    0.4  0.012352  0.012352 1.3551616e-1
 69   500   500   0.65   0.65 0.0076011 0.0076011 2.1319376e-2
 70   650   650 0.0085 0.0085   0.75564   0.75564 4.7787306e-1
 71   650   650  0.013  0.013   0.49407   0.49407 4.9180459e-1
 72   650   650   0.02   0.02   0.32115   0.32115 4.8351072e-1
 73   650   650  0.032  0.032   0.20072   0.20072 4.6199077e-1
 74   650   650   0.05   0.05   0.12846   0.12846 4.3303832e-1
 75   650   650   0.08   0.08  0.080287  0.080287 3.9622627e-1
 76   650   650   0.13   0.13  0.049407  0.049407 3.5098491e-1
 77   650   650   0.18   0.18  0.035683  0.035683 3.1004862e-1
 78   650   650   0.25   0.25  0.025692  0.025692 2.5109771e-1
 79   650   650    0.4    0.4  0.016057  0.016057 1.3263995e-1
 80   800   800 0.0105 0.0105   0.75287   0.75287 4.8321766e-1
 81   800   800  0.013  0.013   0.60809   0.60809 4.9187717e-1
 82   800   800   0.02   0.02   0.39526   0.39526 4.8774829e-1
 83   800   800  0.032  0.032   0.24704   0.24704 4.6620723e-1
 84   800   800   0.05   0.05    0.1581    0.1581 4.3609737e-1
 85   800   800   0.08   0.08  0.098814  0.098814 3.9782433e-1
 86   800   800   0.13   0.13  0.060809  0.060809 3.5102077e-1
 87   800   800   0.18   0.18  0.043917  0.043917 3.0905240e-1
 88   800   800   0.25   0.25  0.031621  0.031621 2.4924907e-1
 89   800   800    0.4    0.4  0.019763  0.019763 1.3054175e-1
 90   800   800   0.65   0.65  0.012162  0.012162 1.9786938e-2
 91  1000  1000  0.013  0.013   0.76011   0.76011 4.8810734e-1
 92  1000  1000   0.02   0.02   0.49407   0.49407 4.9232608e-1
 93  1000  1000  0.032  0.032   0.30879   0.30879 4.7163577e-1
 94  1000  1000   0.05   0.05   0.19763   0.19763 4.4025369e-1
 95  1000  1000   0.08   0.08   0.12352   0.12352 4.0023625e-1
 96  1000  1000   0.13   0.13  0.076011  0.076011 3.5158493e-1
 97  1000  1000   0.18   0.18  0.054897  0.054897 3.0841006e-1
 98  1000  1000   0.25   0.25  0.039526  0.039526 2.4759802e-1
 99  1000  1000    0.4    0.4  0.024704  0.024704 1.2848894e-1
100  1200  1200  0.014  0.014   0.84698   0.84698 4.8653947e-1
101  1200  1200   0.02   0.02   0.59289   0.59289 4.9575061e-1
102  1200  1200  0.032  0.032   0.37055   0.37055 4.7703299e-1
103  1200  1200   0.05   0.05   0.23715   0.23715 4.4461908e-1
104  1200  1200   0.08   0.08   0.14822   0.14822 4.0297511e-1
105  1200  1200   0.13   0.13  0.091213  0.091213 3.5261170e-1
106  1200  1200   0.18   0.18  0.065876  0.065876 3.0833222e-1
107  1200  1200   0.25   0.25  0.047431  0.047431 2.4658343e-1
108  1200  1200    0.4    0.4  0.029644  0.029644 1.2699291e-1
109  1500  1500   0.02   0.02   0.74111   0.74111 4.9856822e-1
110  1500  1500  0.032  0.032   0.46319   0.46319 4.8527234e-1
111  1500  1500   0.05   0.05   0.29644   0.29644 4.5172879e-1
112  1500  1500   0.08   0.08   0.18528   0.18528 4.0771364e-1
113  1500  1500   0.13   0.13   0.11402   0.11402 3.5485802e-1
114  1500  1500   0.18   0.18  0.082345  0.082345 3.0899514e-1
115  1500  1500   0.25   0.25  0.059289  0.059289 2.4588520e-1
116  1500  1500    0.4    0.4  0.037055  0.037055 1.2542911e-1
117  1500  1500   0.65   0.65  0.022803  0.022803 1.8125212e-2
118  2000  2000 0.0219 0.0219   0.90241   0.90241 5.0184841e-1
119  2000  2000  0.032  0.032   0.61759   0.61759 4.9909397e-1
120  2000  2000   0.05   0.05   0.39526   0.39526 4.6517713e-1
121  2000  2000   0.08   0.08   0.24704   0.24704 4.1720873e-1
122  2000  2000   0.13   0.13   0.15202   0.15202 3.6010628e-1
123  2000  2000   0.18   0.18   0.10979   0.10979 3.1160287e-1
124  2000  2000   0.25   0.25  0.079051  0.079051 2.4619450e-1
125  2000  2000    0.4    0.4  0.049407  0.049407 1.2395625e-1
126  2000  2000   0.65   0.65  0.030404  0.030404 1.7522694e-2
127  3000  3000  0.032  0.032   0.92638   0.92638 5.1879694e-1
128  3000  3000   0.05   0.05   0.59289   0.59289 4.9548677e-1
129  3000  3000   0.08   0.08   0.37055   0.37055 4.4065182e-1
130  3000  3000   0.13   0.13   0.22803   0.22803 3.7444715e-1
131  3000  3000   0.18   0.18   0.16469   0.16469 3.2034299e-1
132  3000  3000   0.25   0.25   0.11858   0.11858 2.5000862e-1
133  3000  3000    0.4    0.4  0.074111  0.074111 1.2327234e-1
134  3000  3000   0.65   0.65  0.045607  0.045607 1.6869558e-2
135  5000  5000 0.0547 0.0547   0.90324   0.90324 5.4276710e-1
136  5000  5000   0.08   0.08   0.61759   0.61759 4.9864859e-1
137  5000  5000   0.13   0.13   0.38005   0.38005 4.1332462e-1
138  5000  5000   0.18   0.18   0.27448   0.27448 3.4605955e-1
139  5000  5000   0.25   0.25   0.19763   0.19763 2.6413680e-1
140  5000  5000    0.4    0.4   0.12352   0.12352 1.2584652e-1
141  5000  5000   0.65   0.65  0.076011  0.076011 1.6438562e-2
142  8000  8000 0.0875 0.0875   0.90344   0.90344 5.6046209e-1
143  8000  8000   0.13   0.13   0.60809   0.60809 4.8243214e-1
144  8000  8000   0.18   0.18   0.43917   0.43917 3.9449448e-1
145  8000  8000   0.25   0.25   0.31621   0.31621 2.9237090e-1
146  8000  8000    0.4    0.4   0.19763   0.19763 1.3325934e-1
147  8000  8000   0.65   0.65   0.12162   0.12162 1.6511126e-2
148 12000 12000   0.13   0.13   0.91213   0.91213 5.5805927e-1
149 12000 12000   0.18   0.18   0.65876   0.65876 4.6260513e-1
150 12000 12000   0.25   0.25   0.47431   0.47431 3.3488338e-1
151 12000 12000    0.4    0.4   0.29644   0.29644 1.4531489e-1
152 12000 12000   0.65   0.65   0.18243   0.18243 1.7008139e-2
153 20000 20000   0.25   0.25   0.79051   0.79051 4.1254118e-1
154 20000 20000    0.4    0.4   0.49407   0.49407 1.7126302e-1
155 20000 20000   0.65   0.65   0.30404   0.30404 1.8368832e-2
156 30000 30000    0.4    0.4   0.74111   0.74111 2.0015181e-1
157 30000 30000   0.65   0.65   0.45607   0.45607 2.0247655e-2
158 50000 50000   0.65   0.65   0.76011   0.76011 2.3531551e-2

Just to discard that it is a problem of the metadata of the final fktable.

[giacomomagni]

So the bug was indeed when using the old grids and generating opcards.
Pineko was checking if "polarized" was True, but the thing was "False".

[scarlehoff]

Thanks @giacomomagni, I can confirm this fixes the issue for me!

[felixhekhorn]

thanks @giacomomagni ! good job - and indeed your conjecture was correct.

Can this then be merged?

[scarlehoff]

I'd like to go through it one more time

[scarlehoff]

Thanks for this. Since this has by now been tested in a variety of scenarios I'm relatively sure it will be safe but I'm redoing theory 41000000 to see whether something changed in the final FKs, better safe than sorry.

My comments are mostly in the direction of future maintainability when someone else is tasked with pineko, given that afaia half of the people that have participated in this PR will not be with us in about six month time

[scarlehoff]

Thank you very much @giacomomagni and apologies for the wait, I can reproduce 41_000_000 from master to whatever np.allclose precision gives.

[scarlehoff]

Before merging, I see that both @felixhekhorn and @andreab1997 had requested changes. From his last message I guess @felixhekhorn is ok with merging, not sure about @andreab1997 ?

[andreab1997]

Before merging, I see that both @felixhekhorn and @andreab1997 had requested changes. From his last message I guess @felixhekhorn is ok with merging, not sure about @andreab1997 ?

Give me just the time to have a look and I 'll let you know