Bugfix hkl #11

jrmlhermitte · 2018-03-19T21:20:15Z

This fixes a bug in hklpy. Fix the setting of the limits by using the API of hklpy.
Previously, we were doing something like:

        p = self._geometry.axis_get(name)
        # the third param, "1" means units are degrees
        # "0" means radians.
        p.min_max_set(value[0], value[1], 1)

This is incorrect, as the Geometry object does not return a pointer to the axis, but rather a copy. After the modification, one should re-insert it with:

self._geometry.axis_set(name, p)

This somehow worked with previous versions of hklpy. We suspect that this has something to do with the gobject library wrapping the C modules into python, but we're not sure.

I verified with the simuated tardis here:

In [1]: tardis.calc['delta']
Out[1]: Parameter(name='delta (internally: gamma)', limits=(-180.0, 180.0), value=0.0, fit=True, inverted=False, units='Degree')

In [2]: tardis.calc['delta']=-10,100

In [3]: tardis.calc['delta']
Out[3]: Parameter(name='delta (internally: gamma)', limits=(-10.0, 100.0), value=0.0, fit=True, inverted=False, units='Degree')

In [4]: tardis.calc._axis_name_to_original
Out[4]: 
OrderedDict([('theta', 'mu'),
             ('mu', 'omega'),
             ('chi', 'chi'),
             ('phi', 'phi'),
             ('delta', 'gamma'),
             ('gamma', 'delta')])

In [5]: axis = tardis.calc._geometry.axis_get('gamma')

In [6]: axis.min_max_get(1)
Out[6]: (-10.0, 100.0)

licode · 2018-03-19T21:33:28Z

tardis.calc['delta']=-10,100, set value or limits?
tardis.calc['delta'].limits=-10,100 ?

jrmlhermitte · 2018-03-19T21:37:11Z

it sets the limits

licode · 2018-03-19T21:38:51Z

hkl/calc.py

+        p = self._geometry.axis_get(name)
+        # the third param, "1" means units are degrees
+        # "0" means radians.
+        p.min_max_set(value[0], value[1], 1)


do we use unit as 0 or 1?

tacaswell · 2018-03-19T21:41:00Z

Could you also fix this by passing the geometry object + an axis name into the Parameter class and having it reach back and touch it's parent?

jrmlhermitte · 2018-03-20T11:54:09Z

Thanks @tacaswell the previous implementation was incorrect.
I think we'll test this at CSX. I don't like this fix.
Since CSX is the only user, perhaps what we should do is use this for now and think about a longer term fix in the future (as you suggested, thanks!)

licode · 2018-03-20T15:11:20Z

hkl/engine.py

+        if self._inverted:
+            low, high = -high, -low
+        self._param.min_max_set(low, high, self._units)
+        if self._geometry is not None:


There is a raise error on geometry in init, so self._geometry will not be None?

licode · 2018-03-20T15:11:38Z

hkl/engine.py

+        if self._inverted:
+            value *= -1.0
+
+        if self._geometry is not None:


licode · 2018-03-20T15:12:22Z

hkl/engine.py

+    def __init__(self, param, *args, geometry=None, **kwargs):
+        '''
+            Like calc parameter but needs reference to a geometry object.
+            Updates to the parameter should be propagated back to the geometry.


can you add some docs on param and geometry?

licode · 2018-03-20T15:13:46Z

hkl/engine.py

+        self.param_name = param.name_get()
+        # you may wonder why we stash geometry object rather than axis
+        # this is because the axis params can only be updated from the geometry
+        if geometry is None:


or make geometry not as optional.

good idea. i made it required

licode · 2018-03-20T15:20:07Z

hkl/engine.py

+    @property
+    def fit(self):
+        '''True if the parameter can be fit or not'''
+        axis = self._geometry.axis_get(self.param_name)


maybe it is easier to move this line to init? axis is used in every function.

unfortunately, we can't. axis_get returns a copy every time. We need to fetch an updated copy at the time of modification. It would also probably be good to use a locking mechanism to ensure no one else is modifying the object as this is happening in the long run. (like epics callbacks which are not GIL locked). but we should see what CSX needs first

all python with in a given process uses the same GIL. If it were not there would be segfaults all over the place.

I thought the threads that ran the epics callbacks were not GIL locked because they ran in some C thread. so they are?

Yes, part of interpreting python byte code (from any thread) is acquiring the GIL.

licode · 2018-03-20T15:20:40Z

some comments. thanks for the work!

jrmlhermitte · 2018-03-20T15:52:03Z

thanks, sounds good, i added a test as well (not tested with travis just tested manually)

licode · 2018-03-20T18:17:37Z

LGTM. We also tested it at CSX, and everything works. Let's also wait for other's input?

jrmlhermitte · 2018-03-21T11:25:11Z

yes, i would say this is good to merge. We can wait for @tacaswell or others again. But let's aim to get this before next coming cycle.
Thanks for the thorough review @licode !

tacaswell · 2018-03-21T23:01:23Z

Can you squash this down to one or two commits with good commit messages and put the tests is a function?

Ensure that changes to value, limits and fit update the parent axis added tests, force geometry to be given to CalcParameter

jrmlhermitte · 2018-03-21T23:59:03Z

ok. I just noticed there were tests one level up. In master branch, some failed but with this bugfix they all passed. I added an extra test explicitly testing the issue here.

tacaswell · 2018-04-20T01:11:17Z

.travis.yml

@@ -44,6 +44,8 @@ install:
  # ophyd version not tagged publicly just yet:
  # (dependencies pulled in above)
  - pip install git+http://github.com/nsls-ii/ophyd/
+  # this is an ophyd dep
+  - pip install typing


This is part of the standard library as of 3.5 which is the minimum that ophyd / bluesky currently support.

danielballan

I left a couple small suggestions. Looks good otherwise.

danielballan · 2018-05-01T17:42:55Z

hkl/calc.py

@@ -405,10 +405,11 @@ def units(self):
        return self._unit_name

    def __getitem__(self, axis):
-        return Parameter(self._get_axis_by_name(axis),
+        return CalcParameter(self._get_axis_by_name(axis),


The whitespace is funny here.

danielballan · 2018-05-01T17:44:01Z

hkl/engine.py

+        axis = self._geometry.axis_get(self.param_name)
+        low, high = axis.min_max_get(self._units)
+        if self._inverted:
+            return [-high, -low]


This path returns a list and the other path returns tuple. Aside from the general benefits of consistency, return mutable types (like list) have been the source of confounding bugs in the past. Better make this one a tuple: -high, -low.

danielballan · 2018-05-01T17:46:54Z

.travis.yml


  - pip install coveralls pytest pytest-cov
  - python setup.py develop

  # this is important:
-  - conda install -c lightsource2 readline
+  #- conda install -c lightsource2-tag readline


The fact that this is now commented out seems out of sync with the remark above it ("this is important"). Should it be removed entirely?

oh, i changed it to lightsource2-tag out when testing, then found i didn't need it so commented it out. @tacaswell can you confirm we don't need this anymore? worked for me when testing locally, and works on Travis

This is no longer important because we have removed the readline linking in epics-base so we should no longer have issues with crossed versions of readline. There may still need to be one more local change applied to our pyepics, but we should be able to use the stock readline again.

great thanks! in either case that was for Travis, and it passed without. great job in solving that annoying problem!

jrmlhermitte · 2018-05-01T17:53:55Z

Thanks @danielballan . I made the changes. I'll wait for @tacaswell 's comment on readline ( here )

tacaswell · 2018-05-01T20:35:05Z

hkl/engine.py

@@ -267,3 +267,70 @@ def _repr_info(self):
    def __repr__(self):
        return '{}({})'.format(self.__class__.__name__,
                               ', '.join(self._repr_info()))
+
+# the parameter for the calc object
+# this is a fix for backwards compatibility


This is not a back-compatibilty fix

tacaswell · 2018-05-01T21:10:09Z

.travis.yml

@@ -9,8 +9,9 @@ cache:

 matrix:
  include:
-  - python: 3.4
-  - python: 3.5
+  # we have removed the py35 dep because it does not work


just remove these

tacaswell · 2018-05-01T21:10:29Z

.travis.yml


  # ophyd version not tagged publicly just yet:
  # (dependencies pulled in above)
-  - pip install git+http://github.com/nsls-ii/ophyd/
+  #- pip install ophyd


same here, just remove this line

tacaswell · 2018-05-01T21:11:04Z

.travis.yml

  - conda config --add channels soft-matter

  # MAKE THE CONDA RECIPE
  - conda create -n $CONDA_ENV python=$TRAVIS_PYTHON_VERSION
  - source activate $CONDA_ENV

 install:
-  - conda install numpy pyepics prettytable filestore ipython pyolog pip
+  - conda install numpy pyepics prettytable ipython pyolog pip


why do we need pyolog! Not worth addressing...

licode reviewed Mar 19, 2018

View reviewed changes

jrmlhermitte force-pushed the bugfix-hkl branch from 8b2943e to d48a3ab Compare March 20, 2018 01:13

licode reviewed Mar 20, 2018

View reviewed changes

hkl/engine.py Outdated

if self._inverted:

value *= -1.0

if self._geometry is not None:

Copy link

Contributor

licode Mar 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here?

licode reviewed Mar 20, 2018

View reviewed changes

Julien R M Lhermitte added 2 commits March 21, 2018 19:46

Bugfix for hkl

428150f

Ensure that changes to value, limits and fit update the parent axis added tests, force geometry to be given to CalcParameter

Moved test to test file

b69caad

jrmlhermitte force-pushed the bugfix-hkl branch from f3a0c58 to b69caad Compare March 21, 2018 23:57

Add ophyd dep to tests

cbe9fa1

tacaswell reviewed Apr 20, 2018

View reviewed changes

CI: tweak versions of python run + dependency installation

5a6c145

tacaswell added this to the 18 cycle 2 milestone Apr 20, 2018

Julien R M Lhermitte added 4 commits April 30, 2018 18:07

removed typing dep which comes by default py3.5+

a2cfbe6

fix travis yml file

07354a2

add strict requirement to pygobject in travis

18ff248

py36 only

ee45762

danielballan reviewed May 1, 2018

View reviewed changes

remove readline

1ae1d11

Julien R M Lhermitte added 3 commits May 1, 2018 13:51

Return tuple, not list

9523e3b

Some PEP8

169dd09

change to tuple again

48d9de1

tacaswell reviewed May 1, 2018

View reviewed changes

remove some comments

a54f2f2

tacaswell merged commit 4023ec0 into master May 1, 2018

tacaswell deleted the bugfix-hkl branch May 1, 2018 22:56

Bugfix hkl #11

Bugfix hkl #11

Conversation

jrmlhermitte commented Mar 19, 2018 • edited Loading

licode commented Mar 19, 2018 • edited Loading

jrmlhermitte commented Mar 19, 2018

Choose a reason for hiding this comment

tacaswell commented Mar 19, 2018

jrmlhermitte commented Mar 20, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

licode commented Mar 20, 2018

jrmlhermitte commented Mar 20, 2018

licode commented Mar 20, 2018

jrmlhermitte commented Mar 21, 2018 • edited Loading

tacaswell commented Mar 21, 2018

jrmlhermitte commented Mar 21, 2018

Choose a reason for hiding this comment

danielballan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielballan May 1, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrmlhermitte May 1, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrmlhermitte commented May 1, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrmlhermitte commented Mar 19, 2018 •

edited

Loading

licode commented Mar 19, 2018 •

edited

Loading

jrmlhermitte commented Mar 21, 2018 •

edited

Loading

danielballan May 1, 2018 •

edited

Loading

jrmlhermitte May 1, 2018 •

edited

Loading