Move and Rename SDECData to VisualizationData

[KasukabeDefenceForce]

📝 Description

Type: 🪲 bugfix | 🚀 feature | ☣️ breaking change | 🚦 testing | 📝 documentation | 🎢 infrastructure

This PR moves and shifts sdec data to VisualizationData for clarity purposes and also refactors some of its methods to improve readability.

📌 Resources

Examples, notebooks, and links to useful references.

🚦 Testing

How did you test these changes?

• Testing pipeline
• Other method (describe)
• My changes can't be tested (explain why)

☑️ Checklist

• I requested two reviewers for this pull request
• I updated the documentation according to my changes
• I built the documentation by applying the build_docs label

Note: If you are not allowed to perform any of these actions, ping (@) a contributor.

[tardis-bot]

*beep* *bop*
Hi human,
I ran ruff on the latest commit (b993fdd).
Here are the outputs produced.
Results can also be downloaded as artifacts here.
Summarised output:

10	E902  	[ ] No such file or directory (os error 2)
8	E999  	[ ] SyntaxError: Expected ',', found ':'
7	RET505	[ ] Unnecessary `else` after `return` statement
7	W293  	[*] Blank line contains whitespace
6	G004  	[ ] Logging statement uses f-string
6	F401  	[*] `matplotlib.cm` imported but unused
4	RET506	[ ] Unnecessary `elif` after `raise` statement
3	I001  	[*] Import block is un-sorted or un-formatted
3	W291  	[*] Trailing whitespace
2	PTH117	[ ] `os.path.isabs()` should be replaced by `Path.is_absolute()`
2	W292  	[*] No newline at end of file
2	W605  	[*] Invalid escape sequence: `\A`
1	ANN204	[ ] Missing return type annotation for special method `__getitem__`
1	D202  	[*] No blank lines allowed after function docstring (found 1)
1	D406  	[*] Section name should end with a newline ("Returns")
1	D407  	[*] Missing dashed underline after section ("Returns")
1	F541  	[*] f-string without any placeholders

Complete output(might be large):

.github/workflows/release.yml:7:4: E999 SyntaxError: Expected an expression
.github/workflows/release.yml:94:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:97:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:104:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:107:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:110:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:115:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:119:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:144:23: W292 [*] No newline at end of file
.mailmap:1:38: E999 SyntaxError: Expected an expression
.mailmap:290:39: W292 [*] No newline at end of file
CHANGELOG.md:4:15: E999 SyntaxError: Expected ',', found ':'
CITATION.cff:3:1: E999 SyntaxError: Invalid annotated assignment target
README.rst:1:1: E999 SyntaxError: Expected a statement
docs/contributing/development/benchmarks.rst:1:1: E999 SyntaxError: Expected a statement
docs/io/output/how_to_rpacket_tracking.ipynb:cell 12:1:62: W291 [*] Trailing whitespace
docs/physics/plasma/equilibrium/tardis_solver_cmfgen.ipynb:1:1: E902 No such file or directory (os error 2)
docs/physics/setup/model.ipynb:cell 2:2:1: I001 [*] Import block is un-sorted or un-formatted
docs/physics/setup/model.ipynb:cell 25:19:12: F541 [*] f-string without any placeholders
docs/quickstart.ipynb:cell 12:1:39: W291 [*] Trailing whitespace
docs/quickstart.ipynb:cell 12:5:35: W291 [*] Trailing whitespace
docs/quickstart.ipynb:cell 16:10:26: W605 [*] Invalid escape sequence: `\A`
docs/quickstart.ipynb:cell 16:11:40: W605 [*] Invalid escape sequence: `\A`
docs/resources/credits.rst:1:1: E999 SyntaxError: Expected a statement
docs/workflows/v_inner_solver_workflow.ipynb:1:1: E902 No such file or directory (os error 2)
tardis/io/configuration/config_reader.py:53:29: G004 Logging statement uses f-string
tardis/io/configuration/config_reader.py:117:9: RET505 Unnecessary `else` after `return` statement
tardis/io/configuration/config_reader.py:141:13: RET505 Unnecessary `else` after `return` statement
tardis/io/configuration/config_reader.py:218:29: G004 Logging statement uses f-string
tardis/io/configuration/schemas/montecarlo_definitions.yml:1:13: E999 SyntaxError: Expected an expression
tardis/io/model/parse_geometry_configuration.py:50:12: PTH117 `os.path.isabs()` should be replaced by `Path.is_absolute()`
tardis/io/model/parse_radiation_field_configuration.py:118:13: G004 Logging statement uses f-string
tardis/model/base.py:340:12: PTH117 `os.path.isabs()` should be replaced by `Path.is_absolute()`
tardis/model/base.py:375:21: G004 Logging statement uses f-string
tardis/opacities/macro_atom/macroatom_state.py:1:1: I001 [*] Import block is un-sorted or un-formatted
tardis/opacities/opacity_state.py:210:9: ANN204 Missing return type annotation for special method `__getitem__`
tardis/opacities/opacity_state.py:216:9: D407 [*] Missing dashed underline after section ("Returns")
tardis/opacities/opacity_state.py:216:9: D406 [*] Section name should end with a newline ("Returns")
tardis/opacities/opacity_state.py:251:5: D202 [*] No blank lines allowed after function docstring (found 1)
tardis/opacities/tests/__init__.py:1:1: E902 No such file or directory (os error 2)
tardis/plasma/equilibrium/rates/__init__.py:2:5: F401 `tardis.plasma.equilibrium.rates.collision_strengths.UpsilonCMFGENSolver` imported but unused; consider removing, adding to `__all__`, or using a redundant alias
tardis/plasma/equilibrium/rates/__init__.py:3:5: F401 `tardis.plasma.equilibrium.rates.collision_strengths.UpsilonRegemorterSolver` imported but unused; consider removing, adding to `__all__`, or using a redundant alias
tardis/plasma/equilibrium/rates/__init__.py:6:5: F401 `tardis.plasma.equilibrium.rates.collisional_rates.ThermalCollisionalRateSolver` imported but unused; consider removing, adding to `__all__`, or using a redundant alias
tardis/plasma/equilibrium/rates/__init__.py:9:5: F401 `tardis.plasma.equilibrium.rates.radiative_rates.RadiativeRatesSolver` imported but unused; consider removing, adding to `__all__`, or using a redundant alias
tardis/plasma/equilibrium/rates/collisional_ionization_rates.py:1:1: E902 No such file or directory (os error 2)
tardis/plasma/equilibrium/rates/collisional_ionization_strengths.py:1:1: E902 No such file or directory (os error 2)
tardis/plasma/equilibrium/rates/photoionization_rates.py:1:1: E902 No such file or directory (os error 2)
tardis/plasma/equilibrium/rates/photoionization_strengths.py:1:1: E902 No such file or directory (os error 2)
tardis/radiation_field/validate_radiation_field.py:1:1: E902 No such file or directory (os error 2)
tardis/spectrum/formal_integral.py:1:1: I001 [*] Import block is un-sorted or un-formatted
tardis/spectrum/formal_integral.py:358:13: RET506 Unnecessary `else` after `raise` statement
tardis/spectrum/formal_integral.py:698:5: RET505 Unnecessary `else` after `return` statement
tardis/spectrum/formal_integral.py:734:5: RET505 Unnecessary `else` after `return` statement
tardis/visualization/plot_util.py:47:5: RET505 Unnecessary `else` after `return` statement
tardis/visualization/tools/liv_plot.py:4:25: F401 [*] `matplotlib.cm` imported but unused
tardis/visualization/tools/liv_plot.py:7:18: F401 [*] `pandas` imported but unused
tardis/visualization/tools/liv_plot.py:313:13: RET506 Unnecessary `elif` after `raise` statement
tardis/visualization/tools/sdec_plot.py:105:13: RET506 Unnecessary `else` after `raise` statement
tardis/visualization/tools/sdec_plot.py:273:13: RET506 Unnecessary `else` after `raise` statement
tardis/visualization/tools/visualization_data.py:204:9: RET505 Unnecessary `else` after `return` statement
tardis/visualization/tools/visualization_data.py:334:9: RET505 Unnecessary `else` after `return` statement
tardis/workflows/simple_tardis_workflow.py:230:17: G004 Logging statement uses f-string
tardis/workflows/simple_tardis_workflow.py:449:17: G004 Logging statement uses f-string
tardis/workflows/util.py:1:1: E902 No such file or directory (os error 2)
tardis/workflows/v_inner_solver.py:1:1: E902 No such file or directory (os error 2)
Found 65 errors.
[*] 23 fixable with the `--fix` option.

[codecov]

Codecov Report

Attention: Patch coverage is 65.88235% with 29 lines in your changes missing coverage. Please review.

Project coverage is 70.04%. Comparing base (7e37d5f) to head (9b996fb).

Files with missing lines	Patch %	Lines
tardis/visualization/tools/visualization_data.py	67.24%	19 Missing ⚠️
tardis/visualization/plot_util.py	46.66%	8 Missing ⚠️
tardis/visualization/tools/liv_plot.py	90.00%	1 Missing ⚠️
tardis/visualization/tools/sdec_plot.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2928      +/-   ##
==========================================
- Coverage   70.39%   70.04%   -0.35%     
==========================================
  Files         221      222       +1     
  Lines       16147    16165      +18     
==========================================
- Hits        11366    11323      -43     
- Misses       4781     4842      +61     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[tardis-bot]

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (7e37d5f) and the latest commit (b71e734).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

Significantly changed benchmarks:

All benchmarks:

Benchmarks that have stayed the same:

| Change   | Before [7e37d5fe] <master>   | After [b71e734c]    | Ratio   | Benchmark (Parameter)                                                                                                               |
|----------|------------------------------|---------------------|---------|-------------------------------------------------------------------------------------------------------------------------------------|
|          | 1.91±5μs                     | 1.37±0.4μs          | ~0.72   | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_line                    |
|          | 511±200ns                    | 531±100ns           | 1.04    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation                                      |
|          | 48.7±20μs                    | 50.7±30μs           | 1.04    | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_line_emission                                         |
|          | 46.1±20μs                    | 48.0±20μs           | 1.04    | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_line_scatter                                          |
|          | 2.73±0ms                     | 2.77±0.01ms         | 1.01    | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('scatter')                                     |
|          | 735±0.3ns                    | 742±1ns             | 1.01    | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_thomson_scatter                                       |
|          | 62.8±0.01ms                  | 63.1±0.2ms          | 1.01    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_rpacket_trackers_to_dataframe                  |
|          | 2.08±0m                      | 2.08±0m             | 1.00    | spectrum_formal_integral.BenchmarkTransportMontecarloFormalIntegral.time_FormalIntegrator_functions                                 |
|          | 37.8±0.04μs                  | 37.7±0.01μs         | 1.00    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_generate_rpacket_tracker_list                  |
|          | 7.88±2μs                     | 7.91±2μs            | 1.00    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_volley                                    |
|          | 3.73±0.02ms                  | 3.71±0.01ms         | 0.99    | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('macroatom')                                   |
|          | 206±0.2ns                    | 203±0.1ns           | 0.99    | spectrum_formal_integral.BenchmarkTransportMontecarloFormalIntegral.time_intensity_black_body                                       |
|          | 571±100ns                    | 561±200ns           | 0.98    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_photoabsorption_opacity_calculation                              |
|          | 39.2±0.03s                   | 38.4±0s             | 0.98    | run_tardis.BenchmarkRunTardis.time_run_tardis                                                                                       |
|          | 1.06±0m                      | 1.04±0m             | 0.98    | run_tardis.BenchmarkRunTardis.time_run_tardis_rpacket_tracking                                                                      |
|          | 1.22±0μs                     | 1.18±0μs            | 0.97    | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_boundary                |
|          | 2.53±0.4ms                   | 2.45±0.5ms          | 0.97    | transport_montecarlo_single_packet_loop.BenchmarkTransportMontecarloSinglePacketLoop.time_single_packet_loop                        |
|          | 23.2±6μs                     | 22.3±5μs            | 0.96    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_generate_rpacket_last_interaction_tracker_list |
|          | 1.74±0.02ms                  | 1.66±0ms            | 0.95    | transport_montecarlo_main_loop.BenchmarkTransportMontecarloMontecarloMainLoop.time_montecarlo_main_loop                             |
|          | 3.10±0.6μs                   | 2.95±0.5μs          | 0.95    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_bad_vpacket                                       |
|          | 6.33±1μs                     | 5.96±1μs            | 0.94    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket                                           |
|          | 571±100ns                    | 532±200ns           | 0.93    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_pair_creation_opacity_calculation                                |
|          | 2.05±2μs                     | 1.90±2μs            | 0.93    | transport_montecarlo_estimators_radfield_estimator_calcs.BenchmarkMontecarloMontecarloNumbaPacket.time_update_line_estimators       |
|          | 3.21±0.5μs                   | 2.98±0.5μs          | 0.93    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_within_shell                              |

If you want to see the graph of the results, you can check it here

[wkerzendorf]

Let's make a notebook showing these in actions. Each plot should be redone with plotly and Matplotlib to check how well that works.

[andrewfullard]

The class is duplicating a lot of information from the simulation. Could the spectrum, in particular, be separated and simply used via the spectrum class rather than copying the data into this new class? Similarly, the geometry information (v_inner, r_inner etc) is being duplicated. It may be more useful to duplicate the entire geometry class from the simulation if that is what is needed for visualization. If the geometry parts are NOT all needed, then referencing the simulation rather than duplicating the data would be best.

As far as the notebook goes, it would be most useful to demonstrate the packet tracking information that is contained in the class, since the spectrum information is already plotted elsewhere e.g. in the quickstart notebook.

Please comment etc with your ideas for any better ways to package the data necessary for the SDEC and LIV plots and how that might look. Multiple classes may be needed, but in general it would be good to limit duplication as I noted above.