Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Numba montecarlo #929

Merged
merged 119 commits into from
Nov 17, 2020
Merged
Show file tree
Hide file tree
Changes from 109 commits
Commits
Show all changes
119 commits
Select commit Hold shift + click to select a range
883c605
commenting out the montecarlo part
wkerzendorf May 1, 2019
059cbff
Add new numba stuff
wkerzendorf May 1, 2019
532b414
add more numba conversions
wkerzendorf May 1, 2019
7df78ae
getting closer - TARDIS now fails
wkerzendorf May 1, 2019
90bfb22
final numba test - seems to work.
wkerzendorf May 1, 2019
d22a1c8
all sorts of nonsense
wkerzendorf May 1, 2019
394167f
commit stuff
wkerzendorf May 2, 2019
f6b0be3
several edits later
wkerzendorf May 2, 2019
0449387
still working on fixing numba_test
wkerzendorf May 2, 2019
c53425f
add numba stuff
wkerzendorf May 2, 2019
b26410b
branch point to diverge from c
wkerzendorf May 2, 2019
38aae41
working numba implementation
wkerzendorf May 3, 2019
7ce0bf0
numba rewrite
wkerzendorf May 3, 2019
c10a239
finished simple implementation
wkerzendorf May 3, 2019
9702834
cleaned up version
wkerzendorf May 3, 2019
bfd571d
rewrite problem
wkerzendorf May 3, 2019
a16f23d
further simplified r_packet
wkerzendorf May 3, 2019
17007c8
fixing simple implementation
wkerzendorf May 3, 2019
11e76db
more fixes
wkerzendorf May 3, 2019
44e2fe3
several fixes
wkerzendorf May 3, 2019
735ed5e
final simple version
wkerzendorf May 3, 2019
af2c440
adding electron scattering
wkerzendorf May 4, 2019
a27a9da
add tau_electron
wkerzendorf May 4, 2019
77c014d
add e scattering
wkerzendorf May 4, 2019
c440853
add electron scattering
wkerzendorf May 4, 2019
ee7f007
add first vpacket stuff
wkerzendorf May 4, 2019
54f3151
further restructure of the numba montecarlo
wkerzendorf May 5, 2019
24bc8b2
adding a packet collection
wkerzendorf May 5, 2019
d59c8f7
update and clean up the numba interface
wkerzendorf May 5, 2019
a0f30b0
add estimators
wkerzendorf May 5, 2019
2970800
fix packet init
wkerzendorf May 6, 2019
1ef454f
some optimizations
wkerzendorf May 6, 2019
c4179a2
add vpacket stuff
wkerzendorf May 6, 2019
7878bbd
several changes to vpacket
wkerzendorf May 17, 2019
3d711a3
Clean up rpacket
wkerzendorf Jun 4, 2019
77a8a3f
Add vpacket development
wkerzendorf Jun 6, 2019
d2df2d7
Final V-Packet addition
wkerzendorf Jun 13, 2019
a520234
adding initial macroatom setup
wkerzendorf Jun 14, 2019
5e4d347
further work on macro_atom
wkerzendorf Jun 14, 2019
7b8c86a
debugged macro_atom - very close now
wkerzendorf Jun 27, 2019
2a9ec25
add tracking to rpack4ets
wkerzendorf Nov 26, 2019
c728696
make the numba branch compatible with the rest of tardis
wkerzendorf Nov 27, 2019
51aed35
rewire montecarlo with new numba part
wkerzendorf Dec 1, 2019
7fe7d4f
add temporary_v_packet_bins
wkerzendorf Dec 1, 2019
e8fcf4d
document montecarlo part
wkerzendorf Dec 1, 2019
fc79fe8
recalcuate distance_electron
wkerzendorf Dec 1, 2019
6a1d065
add estimators
wkerzendorf Dec 2, 2019
b02f53c
finish estimators
wkerzendorf Dec 2, 2019
4d9dec8
fix formal integral
wkerzendorf Dec 2, 2019
0e8b087
fix estimators
wkerzendorf Dec 4, 2019
7375efe
work on full relativity
wkerzendorf Jan 15, 2020
df884c4
several changes
wkerzendorf May 13, 2020
feee984
Debugging relativity on numba_montecarlo (#1136)
arjunsavel Jul 2, 2020
73ba9db
Explicitly pass number of vpackets to montecarlo_main_loop (#1244)
arjunsavel Aug 10, 2020
5d6b790
Final numba PR (#1285)
arjunsavel Sep 2, 2020
7deb107
modify virtual packet spawn range
wkerzendorf Sep 4, 2020
8e1aebe
use spawn range
wkerzendorf Sep 4, 2020
e7a92e8
Co-authored-by: cvogl@mpa-garching.mpg.de
wkerzendorf Sep 4, 2020
67788d4
add some virtual packet spawn
wkerzendorf Sep 4, 2020
5eb5433
Reformat with black
chvogl Sep 4, 2020
47cf44a
Fix merge conflicts
chvogl Sep 4, 2020
da3c287
Change the Numba code to reflect the C Code
chvogl Sep 9, 2020
3eb8e8f
Vpacket updates
andrewfullard Oct 1, 2020
ca1f528
Vpacket print removed
andrewfullard Oct 1, 2020
705be39
Re-add C formal integral (#1309)
chvogl Oct 5, 2020
0feb68e
Fixed `calculate_distance_line` showing nu_diff < 0 (#1317)
Rodot- Oct 10, 2020
99f4b18
change to use math instead of np
wkerzendorf Oct 12, 2020
979fbda
update sigma thomson use from numba_config
wkerzendorf Oct 12, 2020
8ddef2d
Merge pull request #1318 from wkerzendorf/numba_montecarlo_profile1
Rodot- Oct 12, 2020
4fc9bda
Numba montecarlo tests merged (#1319)
chvogl Oct 15, 2020
4c6177f
change jitclass import to numba.experimental
wkerzendorf Oct 15, 2020
078220f
fix for line_interaction_type
wkerzendorf Oct 16, 2020
a80166a
setup of verysimple numba fixtures and tests
wkerzendorf Oct 16, 2020
6f53ae4
change tests to currently skip
wkerzendorf Oct 17, 2020
e01305a
Fixed vpacket trace within shell test
andrewfullard Oct 19, 2020
f5a56eb
More vpacket testing, if statement in vpacket fix
andrewfullard Oct 19, 2020
0f37d87
Fix packet fixture scope, add interaction.py tests
andrewfullard Oct 19, 2020
4c12843
Transition probabilities to numba
andrewfullard Oct 21, 2020
38a2129
Change numba error mode to numpy
andrewfullard Oct 21, 2020
bbc4772
No more ** exponents
andrewfullard Oct 21, 2020
93005f1
pyne version fix
andrewfullard Oct 21, 2020
9c9a3d3
Syntax fix
andrewfullard Oct 21, 2020
fbf01ba
Fix error_model njit arg
andrewfullard Oct 21, 2020
3ebff83
Rewrote close line and last line tests to avoid branching
Rodot- Oct 21, 2020
8b4c55f
Added static packet for individual testing
andrewfullard Oct 22, 2020
7a326ef
Fix vpacket close line test, rpacket line ID
andrewfullard Oct 23, 2020
8d50445
xfail macro_atom test for now
andrewfullard Oct 23, 2020
a14fda8
Added test that breaks tracing vpackets
andrewfullard Oct 23, 2020
f209cde
Even better bad vpacket test
andrewfullard Oct 23, 2020
f25c042
Fix nu_diff < 0 error (for now...)
andrewfullard Oct 23, 2020
22ffc8b
NumbaPlasma initialization test
andrewfullard Oct 26, 2020
aae1adb
numba macro_atom test, numba seed set fixture
andrewfullard Oct 26, 2020
1c44001
More packet tests
andrewfullard Oct 26, 2020
7e0aa5c
Currently failing C comparison test written
andrewfullard Oct 26, 2020
a340ee9
Improved C comparison test (still fails)
andrewfullard Oct 30, 2020
0569f9e
More C comparison improvements
andrewfullard Oct 30, 2020
a97349a
Back to the future (with a simpler C test)
andrewfullard Nov 2, 2020
e05f76d
More packets, cleanup
andrewfullard Nov 2, 2020
a6a0383
Fixed test?
andrewfullard Nov 2, 2020
1123fa1
Fixed C compare, added legacy mode flag
andrewfullard Nov 3, 2020
f1953b4
Cleaned out old fixtures
andrewfullard Nov 4, 2020
6513d18
Vpacket logging partially restored
andrewfullard Nov 11, 2020
b69986d
Logging into HDF, probably
andrewfullard Nov 11, 2020
dba60f3
Improved virtual packet logging
andrewfullard Nov 12, 2020
e68ce56
Only add vpackets that have data to the log
andrewfullard Nov 13, 2020
b72ed62
Simplified output of vpacket logging
andrewfullard Nov 13, 2020
4458145
Fix for 1D arrays
andrewfullard Nov 13, 2020
884c55a
Fix Vpacket collection spec
andrewfullard Nov 13, 2020
900cd81
Static seed for random test
andrewfullard Nov 13, 2020
dee956b
Apply Wolfgang's suggestions from code review
andrewfullard Nov 13, 2020
ab039d2
Update tardis/io/schemas/montecarlo.yml
andrewfullard Nov 13, 2020
8ba9ed8
Update tardis/montecarlo/__init__.py
andrewfullard Nov 13, 2020
44a14ed
Merge branch 'master' into numba_montecarlo
andrewfullard Nov 13, 2020
ecc5779
Fix syntax errors
andrewfullard Nov 13, 2020
59a0cf7
black reformatting for numba montecarlo
wkerzendorf Nov 13, 2020
cb9791c
Fix vpacket collection set properties test
andrewfullard Nov 13, 2020
76dead8
xfail tests with reasons
andrewfullard Nov 14, 2020
1285575
Test fixes
andrewfullard Nov 16, 2020
d14025e
Update debug_numba.rst
andrewfullard Nov 17, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions docs/development/continuous_integration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -203,11 +203,7 @@ After defining the variable, one could optionally encrypt it using this lock sym
Variables are referenced as $(variable_name), as seen in the InstallSSHKey@0 task in the hostName and sshPublicKey inputs.

Azure provides a list of agent hosts that can run the pipeline on a virtual machine. In our pipelines, we
<<<<<<< HEAD
use the vm_Images: Ubuntu 16.04 and macOs-10.14.
=======
use the vm_Images: Ubuntu 16.04 and macOS-10.13.
>>>>>>> Add periods, correct capitalizations, correct typos, and shuffle commas in continuous integration page.

In a job, you can list a single vm as::

Expand Down
14 changes: 14 additions & 0 deletions docs/development/debug_numba.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
**************************
Debugging numba_montecarlo
**************************
To facilitate more in-depth debugging when interfacing with the `montecarlo_numba`
module, we provide a set of debugging configurations. PyCharm debugging
configurations, in addition to related scripts and .yml files, are contained in
`tardis.scripts.debug`. Currently, these include the ability to run TARDIS
in asingle-packet mode, with the packet seed identified at debug time.
`tardis_example_single.yml` is the configuration filethat is used to set up the
single-packet TARDIS run; `run_numba_single.py` is thePython script that runs
this .yml file; `run_numba_single.xml` is the PyCharmdebug configuration file
that can be used in conjunction with the above files.


2 changes: 1 addition & 1 deletion docs/development/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ to the Astropy team for designing it.
.. toctree::
:maxdepth: 2

running_tests
issues
debug_numba



Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 8 additions & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,14 @@ Mission Statement
Physics overview <physics/index>
physics/plasma/index

.. toctree::
:maxdepth: 2
:caption: Testing TARDIS
:hidden:

testing/automated_tests
testing/numba


.. toctree::
:maxdepth: 2
Expand Down
16 changes: 8 additions & 8 deletions docs/running/custom_source.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,10 @@
" \n",
" def __init__(self, seed, truncation_wavelength):\n",
" super().__init__(seed)\n",
" self.rng = np.random.default_rng(seed=seed)\n",
" self.truncation_wavelength = truncation_wavelength\n",
" \n",
" def create_packets(self, T, no_of_packets,\n",
" def create_packets(self, T, no_of_packets, rng,\n",
" drawing_sample_size=None):\n",
" \"\"\"\n",
" Packet source that generates a truncated Blackbody source.\n",
Expand All @@ -70,11 +71,10 @@
" Only wavelengths higher than the truncation wavelength\n",
" will be sampled.\n",
" \"\"\"\n",
" \n",
" \n",
"\n",
" # Use mus and energies from normal blackbody source.\n",
" mus = self.create_zero_limb_darkening_packet_mus(no_of_packets)\n",
" energies = self.create_uniform_packet_energies(no_of_packets)\n",
" mus = self.create_zero_limb_darkening_packet_mus(no_of_packets, self.rng)\n",
" energies = self.create_uniform_packet_energies(no_of_packets, self.rng)\n",
"\n",
" # If not specified, draw 2 times as many packets and reject any beyond no_of_packets.\n",
" if drawing_sample_size is None:\n",
Expand All @@ -86,15 +86,15 @@
" \n",
" # Draw nus from blackbody distribution and reject based on truncation_frequency.\n",
" # If more nus.shape[0] > no_of_packets use only the first no_of_packets.\n",
" nus = self.create_blackbody_packet_nus(T, drawing_sample_size)\n",
" nus = self.create_blackbody_packet_nus(T, drawing_sample_size, self.rng)\n",
" nus = nus[nus<truncation_frequency][:no_of_packets]\n",
" \n",
" \n",
" # Only required if the truncation wavelength is too big compared to the maximum \n",
" # of the blackbody distribution. Keep sampling until nus.shape[0] > no_of_packets.\n",
" while nus.shape[0] < no_of_packets:\n",
" additional_nus = self.create_blackbody_packet_nus(\n",
" T, drawing_sample_size\n",
" T, drawing_sample_size, self.rng\n",
" )\n",
" mask = additional_nus < truncation_frequency\n",
" additional_nus = additional_nus[mask][:no_of_packets]\n",
Expand Down Expand Up @@ -170,4 +170,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}
27 changes: 18 additions & 9 deletions tardis/io/schemas/montecarlo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,21 +33,17 @@ properties:
multipleOf: 1.0
default: 0
description: Setting the number of virtual packets for the last iteration.
virtual_spectrum_range:
virtual_spectrum_spawn_range:
type: object
default: {}
properties:
start:
type: quantity
default: 50 angstrom
stop:
default: 0 angstrom
end:
type: quantity
default: 250000 angstrom
num:
type: number
multipleOf: 1.0
default: 1000000
description: Limits of virtual packet spectrum (giving maximum and minimum packet
default: inf angstrom
description: Limits of virtual packet spawn spectrum (giving maximum and minimum packet
frequency)
enable_reflective_inner_boundary:
type: boolean
Expand All @@ -68,6 +64,19 @@ properties:
default: false
description: Enables a more complete treatment of relativitic effects. This includes
angle aberration as well as use of the fully general Doppler formula.
debug_packets:
type: boolean
default: false
description: Decide whether to go into debugging mode.
andrewfullard marked this conversation as resolved.
Show resolved Hide resolved
logger_buffer:
type: number
default: 1
description: Provides option to not log every line.
single_packet_seed:
type:
- number
default: -1
description: If debug_packets is true, this is the seed for the only packet.

required:
- no_of_packets
Expand Down
5 changes: 5 additions & 0 deletions tardis/io/schemas/plasma.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ properties:
default: false
description: disable electron scattering process in montecarlo part - non-physical
only for tests
disable_line_scattering:
type: boolean
default: false
description: disable line scattering process in montecarlo part - non-physical
only for tests
ionization:
type: string
enum:
Expand Down
4 changes: 4 additions & 0 deletions tardis/io/schemas/spectrum.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,7 @@ properties:
default: False
description: If True bias v-packet emission based on the electron
scattering optical depth
virtual_packet_logging:
type: boolean
default: False
description: If True, enable virtual packet logging output
2 changes: 1 addition & 1 deletion tardis/io/util.py
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ def get_properties(self):
@property
def full_hdf_properties(self):
# If tardis was compiled --with-vpacket-logging, add vpacket properties
if hasattr(self, "virt_logging") and self.virt_logging == 1:
if hasattr(self, "virt_logging") and self.virt_logging:
self.hdf_properties.extend(self.vpacket_hdf_properties)

return self.optional_hdf_properties + self.hdf_properties
Expand Down
2 changes: 1 addition & 1 deletion tardis/montecarlo/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
from tardis.montecarlo.base import *
from tardis.montecarlo.base import *
andrewfullard marked this conversation as resolved.
Show resolved Hide resolved
Loading