Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arbor cable cell exporter and backend #393

Merged
merged 53 commits into from
Jan 17, 2023
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
3d9e7fa
Arbor cable cell label_dict and decor generation from create_hoc (tes…
lukasgd Mar 10, 2022
fc52a06
Arbor cable cell, label dict and decor generation fixed and done sepa…
lukasgd Mar 10, 2022
3e20642
Unit tests for create_acc (Arbor cable cell output)
lukasgd Mar 14, 2022
dbf766f
Fixed Arbor mechanism output, separated create_hoc from create_acc ex…
lukasgd Mar 31, 2022
fba5376
Separate module for create_acc
lukasgd Apr 7, 2022
8bd6539
Improved documentation for create_acc
lukasgd Apr 7, 2022
d2f2f42
Fixed membrane capacitance conversion to Arbor, formatting
lukasgd Apr 8, 2022
22ad6da
Fixed pycodestyle errors
lukasgd Apr 11, 2022
2228979
Fixed tox tests/docs, replace_axon handling fixed (unsupported in Arb…
lukasgd May 16, 2022
760f53c
Merge branch 'master' into arbor_integration
lukasgd May 16, 2022
315457c
Merge branch 'master' into arbor_integration
wvangeit Jun 1, 2022
dccc375
Merge branch 'master' into arbor_integration
wvangeit Jun 23, 2022
f72690d
Merge branch 'master' into arbor_integration
wvangeit Jun 24, 2022
e88168f
Support for replace_axon in Arbor, cable cell construction from JSON/…
lukasgd Jul 28, 2022
c0af192
Formatting fix, support for myelinated section in Arbor cable cell (c…
lukasgd Jul 28, 2022
27827fb
Merge branch 'master' into arbor_integration
wvangeit Jul 29, 2022
de52dea
Making Arbor an extra dependency
lukasgd Jul 29, 2022
08c9157
Merge branch 'arbor_integration' of github.com:lukasgd/BluePyOpt into…
lukasgd Jul 29, 2022
152033e
Moved Arbor all to default properties, explicit mechanism qualificati…
lukasgd Aug 2, 2022
f2ca1ed
Support for range parameters in Arbor via iexprs, Arbor mechanism cat…
lukasgd Aug 5, 2022
2c67258
Added existing section lists to instantiated cell to prevent crash on…
lukasgd Aug 5, 2022
ab004f6
Arbor package data fixes
lukasgd Aug 8, 2022
0a8a932
Merge branch 'master' into arbor_integration
lukasgd Aug 8, 2022
e685990
Validation of simulation output with Arbor vs. Neuron for L5PC mechs …
lukasgd Aug 11, 2022
26f3af1
Minor changes on ACC exporter
lukasgd Aug 24, 2022
a7453f8
Adding ACC morphology output with axon replacement
lukasgd Aug 25, 2022
79dda2e
Merge branch 'master' into arbor_integration
wvangeit Aug 29, 2022
fac077d
Support for point processes in Arbor cable cell exporter
lukasgd Sep 24, 2022
6aa15de
Arbor locations/labels, stimuli, protocols and optimization in simple…
lukasgd Oct 2, 2022
cde9b96
Using all segments of Neuron sections in axon replacement
lukasgd Oct 3, 2022
6d76ed1
L5PC optimization with Arbor
lukasgd Oct 4, 2022
3978d31
Axon replacement with Arbor morphologies, split_at/join_at and Neuron…
lukasgd Oct 12, 2022
5d4ad03
Removing old axon replacement impl and synapse implementation with sp…
lukasgd Oct 12, 2022
d98041d
Support for general Arbor labels, expsyn example fixed, Arbor iexpr g…
lukasgd Oct 17, 2022
c425101
External mechanism catalogues for Arbor simulator
lukasgd Oct 18, 2022
9443b12
avoid using string concatenation in joining paths
anilbey Oct 19, 2022
d958c14
Removing assertions, adding another Arbor iexpr generation test
lukasgd Oct 20, 2022
f2d93e3
Merge branch 'arbor_integration' of github.com:lukasgd/BluePyOpt into…
lukasgd Oct 20, 2022
4bc6424
More iexpr tests, mech metadata type in ACC exporter, Arbor label mov…
lukasgd Oct 20, 2022
e83dbc0
Renamed l5pc Arbor-Neuron validation
lukasgd Oct 20, 2022
ebd32d0
Adding docstrings to create_acc, replaced os by pathlib
lukasgd Oct 21, 2022
9bbc68e
Python 3 exception handling in protocols, better create_hoc error mes…
lukasgd Oct 23, 2022
bcada48
Use kwargs for cable cell constructor and swap label-dict and decor
lukasgd Oct 24, 2022
b973162
tox use the same EXTRA_ARBOR defined in setup.py
anilbey Oct 24, 2022
9dd09e6
create a submodule for arbor's dsl inside parameterscalers
anilbey Oct 25, 2022
4148974
Renaming Arbor iexpr module, adding docstring
lukasgd Oct 25, 2022
73772ea
Integrating Anil's review including large extension of the create_acc…
lukasgd Nov 7, 2022
e85d695
Fixing expsyn generate_acc docs
lukasgd Nov 7, 2022
93c0685
Fixing create_acc tests
lukasgd Nov 9, 2022
02a00fb
Anil's review for ACC exporter, fixing create_acc tests
lukasgd Dec 21, 2022
04ce614
Integrating more feedback from Anil, fixed L5PC ACC test
lukasgd Jan 4, 2023
5c9a5bf
Merge branch 'master' into arbor_integration
lukasgd Jan 10, 2023
70fddf8
Merge branch 'master' into arbor_integration
lukasgd Jan 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
include versioneer.py
include bluepyopt/_version.py
include bluepyopt/ephys/templates/cell_template.jinja2
include bluepyopt/ephys/templates/acc/_json_template.jinja2
include bluepyopt/ephys/templates/acc/decor_acc_template.jinja2
include bluepyopt/ephys/templates/acc/label_dict_acc_template.jinja2

include.txt
include AUTHORS.txt
Expand Down
451 changes: 451 additions & 0 deletions bluepyopt/ephys/create_acc.py

Large diffs are not rendered by default.

97 changes: 63 additions & 34 deletions bluepyopt/ephys/create_hoc.py
Original file line number Diff line number Diff line change
Expand Up @@ -112,46 +112,37 @@ def _generate_parameters(parameters):
return global_params, ordered_section_params, range_params, location_order


def create_hoc(
def _read_template(template_dir, template_filename):
"""Read Jinja2 hoc template to render"""
if template_dir is None:
template_dir = os.path.abspath(
os.path.join(
os.path.dirname(__file__),
'templates'))

template_path = os.path.join(template_dir, template_filename)
with open(template_path) as template_file:
template = template_file.read()
template = jinja2.Template(template)
return template


def _get_template_params(
mechs,
parameters,
morphology=None,
ignored_globals=(),
replace_axon=None,
template_name='CCell',
template_filename='cell_template.jinja2',
disable_banner=None,
template_dir=None,
custom_jinja_params=None,):
'''return a string containing the hoc template
disable_banner=None):
'''return parameters to render Jinja2 templates with simulator descriptions

Args:
mechs (): All the mechs for the hoc template
parameters (): All the parameters in the hoc template
morpholgy (str): Name of morphology
ignored_globals (iterable str): HOC coded is added for each
NrnGlobalParameter
that exists, to test that it matches the values set in the parameters.
This iterable contains parameter names that aren't checked
replace_axon (str): String replacement for the 'replace_axon' command.
Must include 'proc replace_axon(){ ... }
template (str): file name of the jinja2 template
template_dir (str): dir name of the jinja2 template
custom_jinja_params (dict): dict of additional jinja2 params in case
of a custom template
'''

if template_dir is None:
template_dir = os.path.abspath(
os.path.join(
os.path.dirname(__file__),
'templates'))

template_path = os.path.join(template_dir, template_filename)
with open(template_path) as template_file:
template = template_file.read()
template = jinja2.Template(template)

global_params, section_params, range_params, location_order = \
_generate_parameters(parameters)
channels = _generate_channels_by_location(mechs, location_order)
Expand All @@ -169,19 +160,57 @@ def create_hoc(
else:
banner = None

return dict(global_params=global_params,
ignored_global_params=ignored_global_params,
section_params=section_params,
range_params=range_params,
location_order=location_order,
channels=channels,
banner=banner)


def create_hoc(mechs,
parameters,
morphology=None,
ignored_globals=(),
replace_axon=None,
lukasgd marked this conversation as resolved.
Show resolved Hide resolved
template_name='CCell',
template_filename='cell_template.jinja2',
disable_banner=None,
template_dir=None,
custom_jinja_params=None):
'''return a string containing the hoc template

Args:
mechs (): All the mechs for the hoc template
parameters (): All the parameters in the hoc template
morpholgy (str): Name of morphology
ignored_globals (iterable str): HOC coded is added for each
NrnGlobalParameter
that exists, to test that it matches the values set in the parameters.
This iterable contains parameter names that aren't checked
replace_axon (str): String replacement for the 'replace_axon' command.
Must include 'proc replace_axon(){ ... }
template_filename (str): file name of the jinja2 template
template_dir (str): dir name of the jinja2 template
custom_jinja_params (dict): dict of additional jinja2 params in case
of a custom template
'''

template = _read_template(template_dir, template_filename)

template_params = _get_template_params(mechs,
parameters,
ignored_globals,
disable_banner)
re_init_rng = _generate_reinitrng(mechs)

if custom_jinja_params is None:
custom_jinja_params = {}

return template.render(template_name=template_name,
banner=banner,
channels=channels,
morphology=morphology,
section_params=section_params,
range_params=range_params,
global_params=global_params,
re_init_rng=re_init_rng,
replace_axon=replace_axon,
ignored_global_params=ignored_global_params,
re_init_rng=re_init_rng,
wvangeit marked this conversation as resolved.
Show resolved Hide resolved
**template_params,
**custom_jinja_params)
126 changes: 90 additions & 36 deletions bluepyopt/ephys/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
import collections
import string

from . import create_hoc
from . import create_hoc, create_acc
wvangeit marked this conversation as resolved.
Show resolved Hide resolved
from . import morphologies

import logging
Expand Down Expand Up @@ -223,7 +223,7 @@ def create_empty_cell(

return template_function()

def instantiate(self, sim=None):
def instantiate_morphology(self, sim=None):
"""Instantiate model in simulator"""

# TODO replace this with the real template name
Expand All @@ -240,6 +240,11 @@ def instantiate(self, sim=None):

self.morphology.instantiate(sim=sim, icell=self.icell)

def instantiate(self, sim=None):
"""Instantiate model in simulator"""

self.instantiate_morphology(sim)

if self.mechanisms is not None:
for mechanism in self.mechanisms:
mechanism.instantiate(sim=sim, icell=self.icell)
Expand Down Expand Up @@ -280,11 +285,12 @@ def check_nonfrozen_params(self, param_names): # pylint: disable=W0613
'set before simulation' %
param_name)

def create_hoc(self, param_values,
ignored_globals=(), template='cell_template.jinja2',
disable_banner=False,
template_dir=None):
"""Create hoc code for this model"""
def _create_sim_desc(self, param_values,
ignored_globals=(), template=None,
disable_banner=False,
template_dir=None,
sim_desc_creator=None):
"""Create simulator description for this model"""

to_unfreeze = []
for param in self.params.values():
Expand All @@ -294,41 +300,89 @@ def create_hoc(self, param_values,

template_name = self.name
morphology = os.path.basename(self.morphology.morphology_path)
if self.morphology.do_replace_axon:
replace_axon = self.morphology.replace_axon_hoc

if sim_desc_creator is create_hoc.create_hoc:
if self.morphology.do_replace_axon:
replace_axon = self.morphology.replace_axon_hoc
else:
replace_axon = None

if (
self.morphology.morph_modifiers is not None
and self.morphology.morph_modifiers_hoc is None
):
logger.warning('You have provided custom morphology'
' modifiers, but no corresponding hoc files.')
elif (
self.morphology.morph_modifiers is not None
and self.morphology.morph_modifiers_hoc is not None
):
if replace_axon is None:
replace_axon = ''
for morph_modifier_hoc in self.morphology.morph_modifiers_hoc:
replace_axon += '\n'
replace_axon += morph_modifier_hoc
elif sim_desc_creator is create_acc.create_acc:
if self.morphology.do_replace_axon:
if self.icell is None:
raise ValueError('Need to instantiate_morphology'
' on CellModel before creating'
' JSON/ACC-description with'
' axon replacement.')
replace_axon = [dict(nseg=section.nseg,
length=section.L,
radius=0.5 * section.diam,
tag=morphologies._arb_tags['axon'])
for section in self.icell.axon]
# Requires safe iteration over myelin section
# replace_axon += [dict(nseg=section.nseg,
# length=section.L,
# radius=0.5 * section.diam,
# tag=morphologies._arb_tags['myelin'])
# for section in self.icell.myelin]
else:
replace_axon = None
else:
replace_axon = None

if (
self.morphology.morph_modifiers is not None
and self.morphology.morph_modifiers_hoc is None
):
logger.warning('You have provided custom morphology modifiers, \
but no corresponding hoc files.')
elif (
self.morphology.morph_modifiers is not None
and self.morphology.morph_modifiers_hoc is not None
):
if replace_axon is None:
replace_axon = ''
for morph_modifier_hoc in self.morphology.morph_modifiers_hoc:
replace_axon += '\n'
replace_axon += morph_modifier_hoc

ret = create_hoc.create_hoc(mechs=self.mechanisms,
parameters=self.params.values(),
morphology=morphology,
ignored_globals=ignored_globals,
replace_axon=replace_axon,
template_name=template_name,
template_filename=template,
template_dir=template_dir,
disable_banner=disable_banner)
raise ValueError('Unsupported sim_desc_creator %s '
'(choose either create_hoc.create_hoc or '
'create_acc.create_acc)', str(sim_desc_creator))

ret = sim_desc_creator(mechs=self.mechanisms,
wvangeit marked this conversation as resolved.
Show resolved Hide resolved
parameters=self.params.values(),
morphology=morphology,
ignored_globals=ignored_globals,
replace_axon=replace_axon,
template_name=template_name,
template_filename=template,
template_dir=template_dir,
disable_banner=disable_banner)

self.unfreeze(to_unfreeze)

return ret

def create_hoc(self, param_values,
ignored_globals=(), template='cell_template.jinja2',
disable_banner=False,
template_dir=None):
"""Create hoc code for this model"""
return self._create_sim_desc(param_values,
ignored_globals, template,
disable_banner,
template_dir,
sim_desc_creator=create_hoc.create_hoc)

def create_acc(self, param_values,
ignored_globals=(), template='acc/*_template.jinja2',
disable_banner=False,
template_dir=None):
"""Create hoc code for this model"""
return self._create_sim_desc(param_values,
ignored_globals, template,
disable_banner,
template_dir,
sim_desc_creator=create_acc.create_acc)

def __str__(self):
"""Return string representation"""

Expand Down
Loading