Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Composite Output #105

Merged
merged 152 commits into from
Oct 10, 2022
Merged
Changes from 1 commit
Commits
Show all changes
152 commits
Select commit Hold shift + click to select a range
5fdecc7
Move output into a subpackage
alecandido Mar 13, 2022
d9f5fdb
Split output in modules
alecandido Mar 13, 2022
10b50f7
Break output into modules (and break everything)
alecandido Jul 18, 2022
dd94430
Add new output serialization
alecandido Jul 18, 2022
b3943a1
Recover one legacy test
alecandido Mar 16, 2022
211be29
Recover all legacy tests
alecandido Mar 16, 2022
04a74b6
Recover to manipulation tests
alecandido Mar 17, 2022
08af72a
Recover all manipulation tests
alecandido Mar 17, 2022
b4d0067
Recover all output tests
alecandido Mar 17, 2022
bc58d46
Simplify items iteration
alecandido Mar 17, 2022
7667286
Recover apply tests
alecandido Mar 17, 2022
d289c4f
Recover everything but runner
alecandido Jul 18, 2022
dde03e4
Recover all runner tests
alecandido Jul 18, 2022
0ce9ead
Recover all tests
alecandido Jul 18, 2022
42e1b90
Move ekobox loading to legacy
alecandido Mar 17, 2022
4783232
Replace bleeding-edge syntax for dict union
alecandido Mar 17, 2022
fe2f555
Update ekobox benchmarks
alecandido Jul 18, 2022
cfc2288
Replace remaining interpolation_xgrids
alecandido Mar 17, 2022
b856682
Recover ekobox but product
alecandido Jul 18, 2022
e36ccdb
Simplify ekobox product
alecandido Jul 18, 2022
9bf1160
Upgrade product: do not recompute existing operators
alecandido Mar 18, 2022
c72157c
Introduce xgrid type
alecandido Jul 18, 2022
a237971
Recover tests involving xgrid
alecandido Jul 18, 2022
6b4b37d
Remove bleeding edge syntax
alecandido Mar 18, 2022
cd557a1
Add runcards classes
alecandido Mar 25, 2022
148fcf0
Reshuffle files
alecandido Jul 18, 2022
442f3d7
Fix tests to run
felixhekhorn Mar 21, 2022
039a055
Fix tests to run
felixhekhorn Mar 21, 2022
579dffc
Remove unreachable code, improve tests
felixhekhorn Mar 21, 2022
eea43a2
Add ekobox.mock test
felixhekhorn Mar 21, 2022
406b2b4
Start mocking lhapdf, revert flavors
felixhekhorn Mar 21, 2022
9903dca
Add genpdf.export tests
felixhekhorn Mar 22, 2022
3b6717f
Improve lhapdf fixtures
felixhekhorn Mar 22, 2022
9cb6407
Improve generate_pdf
felixhekhorn Mar 22, 2022
65ab083
Add first genpdf test and improve former
felixhekhorn Mar 22, 2022
726808f
Recover shutil.move in py3.8 and lower
felixhekhorn Mar 23, 2022
8f3c854
Improve genpdf tests
felixhekhorn Mar 23, 2022
275e0ca
Complete genpdf.generate_pdf test
felixhekhorn Mar 23, 2022
fbe9e9d
Resolve fake_pdf name clash
felixhekhorn Mar 23, 2022
d47788c
Complete genpdf.cli tests
felixhekhorn Mar 23, 2022
a96aca4
Add ekobox install cli test
felixhekhorn Mar 23, 2022
2e19e57
Rename ekobox info file
felixhekhorn Mar 23, 2022
9ba039e
Complete coverage
alecandido Jul 18, 2022
15a2161
Rename gen_theory
felixhekhorn Mar 24, 2022
dcf29c9
Rename gen_op
felixhekhorn Mar 24, 2022
0d7c491
Apply some review suggestions
felixhekhorn Mar 25, 2022
0b1be6d
Remove bleeding edge type declaration syntax
alecandido Apr 1, 2022
f2f12b8
Add typing, progress on benchmark failure
alecandido Apr 5, 2022
5d4aab2
Simplify and update runner layout
alecandido Jul 18, 2022
ad6366f
Check grids sizes before comparison
alecandido Jul 18, 2022
976faef
Combine ekos with chosen precision
alecandido Apr 7, 2022
8157ea8
Remove annotation about improving deprecated interface
alecandido Apr 7, 2022
59f37b2
Add loading semantics to operator objects
alecandido Apr 7, 2022
b5ff85a
Bind operator object to a given path
alecandido Apr 7, 2022
0718d15
Attempt valid tarfile creation... fail
alecandido Apr 7, 2022
d3efb9e
Simplify output tests, removing unittest framework
alecandido Apr 8, 2022
b7ff707
Bootstrap new directory layout
alecandido Apr 8, 2022
3a15ea3
Split operator file bootstraping in separate function
alecandido Apr 8, 2022
33b70d0
Add methods to retrieve runcards
alecandido Apr 8, 2022
a186b7f
Rename from_dict constructor
alecandido Apr 8, 2022
8093d99
Init eko CLI
alecandido Apr 8, 2022
159b541
Add self managed item retrieval, through context
alecandido Apr 8, 2022
ad6dd31
Test items self mnagement, not yet fully implemented
alecandido Apr 8, 2022
8b71862
Add IO methods for Operator
alecandido Apr 9, 2022
08cfeb9
Extend EKO documentation
alecandido Apr 9, 2022
dedd220
Rearrange EKO iterators
alecandido Apr 9, 2022
80e9500
Add autoload and autodump
alecandido Apr 10, 2022
2673d1d
Remove user and group info from tar elements
alecandido Apr 10, 2022
54edfbc
Fix compatibility layer to use subsections
alecandido Jul 19, 2022
2fb64a6
Update compatibility tests as well
alecandido Jul 19, 2022
425485e
Disable workflows run on WIP branch
alecandido Jul 19, 2022
5b74d36
Update operator card example
alecandido Jul 19, 2022
aea1aa3
Update error identifiers
alecandido Jul 19, 2022
588d1e6
Fix operator runcard layout in OME tests
alecandido Jul 19, 2022
9c40e95
Drop alphas, sanify q2 to dump
alecandido Jul 20, 2022
38a37be
Convert all dictionaries to operators
alecandido Jul 20, 2022
d40b6c7
Fix double write to archive
alecandido Jul 20, 2022
f227a8e
Fix flavor reshaping, set operator
alecandido Jul 20, 2022
822d552
fix ome test
giacomomagni Jul 20, 2022
f5da5c0
Check file exists in tar
felixhekhorn Jul 20, 2022
9551261
Unleash workflows
alecandido Jul 20, 2022
d3e825a
Add docstrings for DictLike
alecandido Jul 25, 2022
dbe7869
Init operator docstrings
alecandido Jul 25, 2022
034f7e5
Add compatibility test
felixhekhorn Jul 26, 2022
76f38ac
Add CLI test
felixhekhorn Jul 26, 2022
aec2d6a
Fix and test evol_pdf
felixhekhorn Jul 26, 2022
0e95377
Comment runcards
felixhekhorn Jul 26, 2022
976bc98
Fix runner test
felixhekhorn Jul 26, 2022
d5885a8
Revert "Comment runcards"
alecandido Jul 27, 2022
c426b5f
Change all {input,target}basis names to {}pids
alecandido Jul 27, 2022
a621d84
Remove PathLike in favor of standard library os.PathLike
alecandido Jul 27, 2022
d84e24d
Provide native deep copy for EKO
alecandido Jul 27, 2022
d5c5816
Remove the option to delete item setting it -> use delitem
alecandido Jul 27, 2022
1b8bb42
Replace file and directory names with module level constants
alecandido Jul 27, 2022
a8a5818
Refactor save, to take stream as input
alecandido Jul 27, 2022
1be6fa4
Replace duplicated string by function call
alecandido Jul 27, 2022
1227f76
Fix operator save regression
alecandido Jul 28, 2022
0088000
Remove dead code from legacy
alecandido Jul 28, 2022
21f7b43
Remove dead code from manipulate
alecandido Jul 28, 2022
e4078f7
Remove dead code from struct
alecandido Jul 28, 2022
e6d5a36
Document dataclass attributes in place
alecandido Jul 28, 2022
a57be9d
Fix tests
felixhekhorn Jul 28, 2022
16c6fdd
Document configs and rotations, remove useless default constructor
alecandido Jul 28, 2022
eba344c
Restore defaults, since None unsupported in manipulate
alecandido Jul 28, 2022
ff02f25
Serialize properly grids
alecandido Jul 28, 2022
d529c4b
Extend compatibility layer
alecandido Jul 28, 2022
d74207e
Fix compatibility tests
alecandido Jul 28, 2022
cdaed45
Skip compatibility for new cards
felixhekhorn Jul 28, 2022
5f457a4
Load legacy YAML output
alecandido Jul 28, 2022
0cd40be
Fix ekomark/apply
felixhekhorn Jul 28, 2022
fae2622
Drop InterpolatorDispatcher constructor from dict
alecandido Jul 28, 2022
3a799f8
Fix most interpolation tests
alecandido Jul 28, 2022
aafa27f
Fix get_interpolation test
alecandido Jul 28, 2022
ac117bf
On the way to fix test_utils
felixhekhorn Jul 28, 2022
6c2a999
Merge branch 'feature/composite-output' of github.com:N3PDF/eko into …
felixhekhorn Jul 28, 2022
3767943
Propagate new InterpolatorDispatcher construction
alecandido Jul 28, 2022
4e9b67b
Propagate also into OME tests
alecandido Jul 28, 2022
3e668ab
Fix test_utils
felixhekhorn Jul 28, 2022
18c33c3
Merge branch 'feature/composite-output' of github.com:N3PDF/eko into …
felixhekhorn Jul 28, 2022
fd632fc
Fix manipulate compatibility with InterpolatorDispatcher
alecandido Jul 28, 2022
984b605
Fix eko.interpolator
felixhekhorn Jul 28, 2022
18c8c31
Fix test_op_cards
felixhekhorn Jul 28, 2022
a9b4532
Fix compatibility tests
alecandido Jul 28, 2022
6cb49a2
Fix info_file and its test
felixhekhorn Jul 28, 2022
996fbf4
Fix legacy tests
alecandido Jul 28, 2022
225c1be
Fix last output test
alecandido Jul 28, 2022
e4d059a
Restore one further test in runner
alecandido Jul 28, 2022
df64b99
Fix evolve pdf tests
alecandido Jul 28, 2022
c00fc3b
Fix last runner tests
alecandido Jul 28, 2022
73be7ef
Propagate InterpolatorDispatcher update to isolated benchmarks
alecandido Jul 28, 2022
0d92a57
Keep using legacy runcards in ekobox, including benchmarks
alecandido Jul 28, 2022
3ba82e4
Only support old layout of opcard in ekobox
felixhekhorn Jul 29, 2022
03db302
Provide some more docstrings for output structs
alecandido Jul 29, 2022
e37e40d
Complete test_interpolation
felixhekhorn Jul 29, 2022
67d5339
Complete runner
felixhekhorn Jul 29, 2022
aeddb04
Provide even more docstrings
alecandido Jul 29, 2022
af88b47
Add more struct tests
felixhekhorn Jul 29, 2022
bc89ac6
Add rotations tests
felixhekhorn Jul 29, 2022
f8eb456
Complete most tests
felixhekhorn Jul 29, 2022
44b581c
Fix numba and LHA
felixhekhorn Jul 29, 2022
4e6c8a3
Recover test_ev_operator
felixhekhorn Jul 29, 2022
7f38cad
Add tests for runcards objects
alecandido Jul 29, 2022
67bd77d
Fix legacy typing compatibility
alecandido Jul 29, 2022
2090ddd
Revert HERA20 fix
felixhekhorn Jul 29, 2022
1a7788d
Merge branch 'feature/composite-output' of github.com:N3PDF/eko into …
felixhekhorn Jul 29, 2022
f5a730a
Poetry update
alecandido Jul 30, 2022
e565186
Merge branch 'develop' into feature/composite-output
felixhekhorn Sep 5, 2022
8b8e3e3
Revert logger.info to lazy evaluation
felixhekhorn Sep 5, 2022
3f7dc23
Try fix poetry
felixhekhorn Sep 5, 2022
be82c99
Cast couplings order for numba
felixhekhorn Sep 5, 2022
fdd02fd
Remove unlink in load_tar
felixhekhorn Sep 19, 2022
47e30e5
Update docs
felixhekhorn Sep 19, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Simplify and update runner layout
alecandido committed Jul 18, 2022
commit 5d4aab21f23724526d92c109edccabf551fdbc33
19 changes: 15 additions & 4 deletions src/eko/output/manipulate.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,23 @@
# -*- coding: utf-8 -*-
import logging
import warnings
from typing import Optional

import numpy as np

from .. import basis_rotation as br
from .. import interpolation
from .struct import EKO

logger = logging.getLogger(__name__)


def xgrid_reshape(eko, targetgrid=None, inputgrid=None, inplace=True):
def xgrid_reshape(
eko: EKO,
targetgrid: Optional[np.ndarray] = None,
inputgrid: Optional[np.ndarray] = None,
inplace: bool = True,
):
"""
Changes the operators to have in the output targetgrid and/or in the input inputgrid.

@@ -23,7 +30,6 @@ def xgrid_reshape(eko, targetgrid=None, inputgrid=None, inplace=True):
inputgrid : None or list
xgrid for the input
"""
__import__("pdb").set_trace()
# calling with no arguments is an error
if targetgrid is None and inputgrid is None:
raise ValueError("Nor inputgrid nor targetgrid was given")
@@ -86,7 +92,12 @@ def xgrid_reshape(eko, targetgrid=None, inputgrid=None, inplace=True):
elem.error = errs


def flavor_reshape(eko, targetbasis=None, inputbasis=None, inplace=True):
def flavor_reshape(
eko: EKO,
targetbasis: Optional[np.ndarray] = None,
inputbasis: Optional[np.ndarray] = None,
inplace: bool = True,
):
"""
Changes the operators to have in the output targetbasis and/or in the input inputbasis.

@@ -147,7 +158,7 @@ def flavor_reshape(eko, targetbasis=None, inputbasis=None, inplace=True):
eko.rotations.targetpids = [0] * len(eko.rotations.targetpids)


def to_evol(eko, source=True, target=False, inplace=True):
def to_evol(eko: EKO, source: bool = True, target: bool = False, inplace: bool = True):
"""
Rotate the operator into evolution basis.

50 changes: 43 additions & 7 deletions src/eko/output/struct.py
Original file line number Diff line number Diff line change
@@ -4,6 +4,7 @@

import numpy as np

from .. import basis_rotation as br
from .. import interpolation
from .. import version as vmod

@@ -64,6 +65,12 @@ class Rotations(DictLike):
targetpids: Optional[np.ndarray] = None
inputpids: Optional[np.ndarray] = None

@classmethod
def default(cls, xgrid: interpolation.XGrid, pids: np.ndarray):
return cls(
targetgrid=xgrid.raw, inputgrid=xgrid.raw, targetpids=pids, inputpids=pids
)


@dataclass
class EKO:
@@ -78,6 +85,7 @@ class EKO:
configs: Configs
rotations: Rotations
debug: Debug
pids: np.ndarray = np.array(br.flavor_basis_pids)
version: str = vmod.__version__
data_version: str = vmod.__data_version__

@@ -107,14 +115,42 @@ def Q2grid(self):
return np.array(list(self._operators))

@classmethod
def from_dict(cls, dictionary):
def from_dict(cls, runcard: dict):
"""Make structure from runcard-like dictionary.

This constructor is made to be used with loaded runcards, in order to
minimize the amount of code needed to init a new object (you just to
load the runcard and call this function).

Note
----
An object is initialized with no rotations, since the role of rotations
is to keep the current state of the output object after manipulations
happened.
Since a new object is here in the process of being created, no rotation
has to be logged.

Parameters
----------
runcard : dict
a dictionary containing the runcard's content

Returns
-------
EKO
the generated structure

"""
xgrid = interpolation.XGrid(runcard["xgrid"])
pids = runcard.get("pids", cls.pids)
return cls(
xgrid=interpolation.XGrid(dictionary["xgrid"]),
Q02=dictionary["Q0"] ** 2,
_operators={q2: None for q2 in dictionary["Q2grid"]},
configs=Configs.from_dict(dictionary["configs"]),
rotations=Rotations.from_dict(dictionary.get("rotations", {})),
debug=Debug.from_dict(dictionary.get("debug", {})),
xgrid=xgrid,
pids=pids,
Q02=runcard["Q0"] ** 2,
_operators={q2: None for q2 in runcard["Q2grid"]},
configs=Configs.from_dict(runcard["configs"]),
rotations=Rotations.default(xgrid, pids),
debug=Debug.from_dict(runcard.get("debug", {})),
)

@classmethod
81 changes: 34 additions & 47 deletions src/eko/runner.py
Original file line number Diff line number Diff line change
@@ -4,14 +4,15 @@
"""
import copy
import logging
from typing import Optional

import numpy as np

from . import basis_rotation as br
from . import compatibility, interpolation, msbar_masses
from .couplings import Couplings
from .evolution_operator.grid import OperatorGrid
from .output import EKO, manipulate
from .output.struct import Rotations
from .thresholds import ThresholdsAtlas

logger = logging.getLogger(__name__)
@@ -39,7 +40,7 @@ class Runner:
o888ooooood8 o888o o888o `Y8bood8P'
"""

def __init__(self, theory_card, operators_card):
def __init__(self, theory_card: dict, operators_card: dict):
self.out = EKO()

new_theory, new_operators = compatibility.update(theory_card, operators_card)
@@ -59,72 +60,58 @@ def __init__(self, theory_card, operators_card):
# strong coupling
sc = Couplings.from_dict(new_theory, masses=masses)
# setup operator grid
self.op_grid = OperatorGrid.from_dict(
new_theory,
new_operators,
tc,
sc,
bfd,
)
self.op_grid = OperatorGrid.from_dict(new_theory, new_operators, tc, sc, bfd)

# save bases manipulations for a post processing step
rot = operators_card.get("rotations", {})
self.post_process = dict(
inputgrid=rot.get("inputgrid", bfd.xgrid.raw),
targetgrid=rot.get("targetgrid", bfd.xgrid.raw),
inputbasis=rot.get("inputbasis"),
targetbasis=rot.get("targetbasis"),
)

if "rotations" not in operators_card:
operators_card["rotations"] = {}
operators_card["rotations"]["inputgrid"] = bfd.xgrid.raw
operators_card["rotations"]["targetgrid"] = bfd.xgrid.raw
self.post_process = {
key: rot.get(key, None)
for key in ("inputgrid", "targetgrid", "inputpids", "targetpids")
}

self.out = EKO.from_dict(dict(Q0=np.sqrt(tc.q2_ref), **operators_card))

def get_output(self) -> EKO:
"""
Collects all data for output (to run the evolution)
"""Run evolution and generate output operator.

Two steps are applied sequentially:

1. evolution is performed, computing the evolution operator in an
internal flavor and x-space basis
2. bases manipulations specified in the runcard are applied

Returns
-------
ret : eko.output.EKO
output instance
EKO
output instance

"""
# add all operators
self.out.rotations.inputpids = np.array(br.flavor_basis_pids)
self.out.rotations.targetpids = np.array(br.flavor_basis_pids)
self.out.rotations.inputgrid = self.out.xgrid.grid
self.out.rotations.targetgrid = self.out.xgrid.grid
for final_scale, op in self.op_grid.compute().items():
__import__("pdb").set_trace()
self.out[float(final_scale)] = op

def similar_to_none(name: str) -> Optional[np.ndarray]:
grid = self.post_process[name]

if grid is None or np.allclose(grid, getattr(Rotations, name), atol=1e-12):
return None

return np.array(grid)

# reshape xgrid
inputgrid = (
self.post_process["inputgrid"]
if self.post_process["inputgrid"] is not self.out.xgrid.grid
else None
)
targetgrid = (
self.post_process["targetgrid"]
if self.post_process["targetgrid"] is not self.out.xgrid.grid
else None
)
inputgrid = similar_to_none("inputgrid")
targetgrid = similar_to_none("targetgrid")
if inputgrid is not None or targetgrid is not None:
manipulate.xgrid_reshape(
self.out, targetgrid=targetgrid, inputgrid=inputgrid
)

# reshape flavors
inputbasis = self.post_process["inputbasis"]
if inputbasis is not None:
inputbasis = np.array(inputbasis)
targetbasis = self.post_process["targetbasis"]
if targetbasis is not None:
targetbasis = np.array(targetbasis)
if inputbasis is not None or targetbasis is not None:
inputpids = similar_to_none("inputpids")
targetpids = similar_to_none("targetpids")
if inputpids is not None or targetpids is not None:
manipulate.flavor_reshape(
self.out, targetbasis=targetbasis, inputbasis=inputbasis
self.out, targetbasis=targetpids, inputbasis=inputpids
)

return copy.deepcopy(self.out)
1 change: 1 addition & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
@@ -65,6 +65,7 @@ def fake_output(self):
Q2grid = self.mk_g([q2_out], len(pids), len(xgrid))
d = dict(
xgrid=xgrid,
pids=pids,
rotations=dict(
targetgrid=xgrid,
inputgrid=xgrid,