Refactor Attribute/Module/Pyrpl (not ready at all) (#83)

* fixes in memory and lockbox gui polished

* state functional, load_setup_attributes call delegated to pyrpl"

* slight change of stategy: module calls load_setup_attributes if parent is not a module, i.e. only for top-level modules

* na starts to work with new API

* defaultbranches removed from memory.py, was never used and hard to maintain

* module_property instantiates itself upon first call of getter

* stages are properly saved

* stages are visible in gui

* sequence doesnt work any more. Why?

* NA,SA, Scope unitests are OK with the new "run" API

* Small changes to make Lockbox python 3-compatible

dict keys/values are not indexable in python 3 (makes sense because
there is no ordering)

* Small changes to make Lockbox python 3 compatible

dict keys/values are not directly indexable

* adds Benchmark async sleep figures

* changes in timer benchmark

* rerun benchmark

* asyncio in benchmark

* benchmark async improved

* Async acquisition logic implemented for network analyzer.

Seems to work inside a coroutine in python 3 and manually calling
result() in python 2...

* scope.run.singe is also coroutine compatible

for instance, thi is is now possible with python 3 (so cool !!!)

from asyncio import ensure_future

async def my_coroutine(n):
for i in range(n):
n = ensure_future(p.networkanalyzer.run.single())
s = ensure_future(p.rp.scope.run.single())
x, y = await n
x, y = await s

t = ensure_future(my_coroutine(12))

* lockbox almost ready

* Scope is working (no unit tests with futures)

* async_utils is compatible with python 2.7 again (little differences between asyncio.Future and concurrent.Future)

* ModuleDict

* ModuleDict can be modified dynamically

* AcquisitionManager removed on scope

* na and scope are working nicely with RunFutures... Need to implement Module._setup_on_load...

* further lockbox cleanup

* fixed interferometer error signal (mean has nothing to do in that signal)

* PdhInterferometer

* bugfix in list memory branch

* FilterRegister is more efficient in terms of read operations

* FilterRegister bugfix

* Specan using future operational

* Replaces _callback_attributes by an option "callback=False" in Attribute constructor...

We have to decide if we want to invert the logic (callback=True by
default...)

* removed _callback_attributes from all modules except for Lockbox

* test_na ok (including same timing as before the refactor)

* oups, forgot to add the files

* old unittests for scope/na/specan OK...

* Fixes 2 small bugs (save_curve in na + cancel single_run future when pause/stop)

* stage gui working, small improvements

* scientific notation for float inputs

* import error with IIR in simulation mode fixed

* bugfix to enable loading pyrpl in simulation mode. Some startup problems are still present from the merge operation

* forgot a file

* all conflicts resolved

* stage.enable replaces setup

* callback  --> call_setup

* nicer gui

* setup_attributes order is fixed, removed workaround from scope

* lockbox bugfix

* bugfix for FabryPerot startup

* scroll bar postponed

* asg square wave

* disabled ruamel support for now

* calibration errors fix started

* Fixes PyrplFuture.result(timeout) in python 2

* Also works in python 3

* debugging new islocked

* finally fixes scope timeout bug in python 2

* towards working lock

* input.py cleanup

* Adds a section "Using asynchronous functions with python 3" in tutorial

* sequence seems to work

* bugfix

* FPF lock re-established

* is_locked and autolock work again

* python run_pyrpl.py config=test works now

* typo

* setpoint units and input_from_output improved

* is_locked for InputFromOutput

* cleaned logger spam into logger.debug

* improved signal structure

* nicer GUI experience

* further cleanup of lockbox api

* lockbox: smaller bugfixes and improvements

* interferometer bugfix

* second interferometer file

* all obvious bugs removed from lockbox

* more lockbox bugfixes, lockbox in a working state

* attribute_widget

* lockbox gui cleaner

* lambda syntax not working for functions for whatever reason

* loop class

* more loop examples

* more loop examples

* started coarse

* loops can be written even more concise now

* better pdh calibration

* firts draft for trigger module

* trigger module ready for testing

* trigger module ready for test

* error_threshold -> is_locked_threshold

* autodoc current version

* typo

* coarse lockbox prototype

* coarse search cleaned

* bugfixes in lockbox

* loops improved

* InputIq submodules reduced

* trigger module compiled

* python code modification for trigger module

* addressing sphinx issues

* cleanup

* bugfixes

* sphinx started

* coarse search lockbox advanced

* specan test repaired

* threshold unittest passing for registers

* comment added

* ch

* bugfixes in trig + coarse_lockbox

* nosetests repaired

296 Tests in 164s

* trig unittest

* better test

* more unit test

* setup develop works

* cleaned up trash folder

* nosetest fixes

* cleanup and nosetests fixes - tests passing

* more file cleanup

* more cleanup

* pyrpl config file storage

* simplified lockbox directory structure

* all cleaned up for now

* added dsp trig module trigger to scope trigger options - fpga code must be recompiled

* cleaned dependencies and bugfixed

* bugfix in scope

* incomplete modification of scope inputs

* scope input changes

* now working

* attributes cleaned

* attribute buxfixes and curve_viewer module

* first ugly version of curve viewer working

* CurveProperty can be used anywhere to plot curves

* small fixes

* register unit tests are passing

* about 10 errors in instruments during nosetests

* pyrpl gui hide/show implemented

* vivado compilation errors fixed

* implemented input=off in fpga

* Gui can be open in python 3

* scope seems to work fine again (_trigger_source-->_trigger_source_register was not done everywhere)

* nice unittest with ProxyProperty (not testing widget connectivity yet)

* started signals

* na test whitespace

* Na amplitude to 0 instead of output_off when paused or stopped

* added legend to scope channels

* test_scope passes now (had to change the test, but I wonder how the test could pass in the past)

* minor fixes

* next autodoc attempts

* UnexpectedPyrplError and bugfix for high-level dsp signals

* loop cleanup around lockbox

* ProxyProperty passes all unittests and is indistinguishable from a real property in all aspects

* proxy bugs all fixed, implemented cleanup in lockbox, logical inputs fully working

* python 3 changes

* started iir

* asg bug fixed

* further fixes

* speed issues in unittests are more clear now

* better error message

* all unittests are passing

* na test threshold less tight

* Scope is cleaned and seems ok

* Scope is clean and seems OK

* global config to configure test thresholds

* removed some commnents from acquisition_module

* travis fix attempts

* towards python 3 unittests passing

* iir advanced

* Adds a comment to justify why running_state is handled separately...

* more advanced iir gui

* not working at the moment

* iir working again

* Started very preliminary display_units in specan

* iir gui getting closer

* actual file

* iir theory proofread

* travis test with a remote redpitaya

* attempts to fix python 2.7 error in travis

* travis fix for python 3.6

* python 3.5 fix quamash installation

* python 3.5 travis file fix

* travis fix

* basestring -> str everywhere for python 3

* redid dependencies

* travis fixes

* more setup.py simplification

* triggers tragvis

* travis bug

* pyqt4 added to travis script

* might also run on python 2.7 now

* pylab import bug

* hoping to pass unittest on travis

* better commication time info for travis

* na running states bug on travis fixed

* test_read_write_time still fails on travis

* comm time

* rw time

* trying to fix all tests

* non-scientific notation in global_config

* fixes nosetests error with output

* finetuning travis

* minor bugs

* travis relaunch

* drastically reduced .travis.yml

* travis fix

* corrections

* test_pid_na_2 is slightly affected by long delay (okay, since an integrator is involved)

* exclude py3.6

* test python 3.5 before 2.7

* delete curves after scope test

* fixes na_test issue for good

* exclude python 3.3 from travis

* quamash for python >= 3.4

* global_config_fix

* another config bug

* travis order

* curve delete bug removed

.gitignore

@@ -70,6 +70,7 @@ nosetests.xml
 coverage.xml
 *,cover
 .hypothesis/
+xunit.xml
 
 # Translations
 *.mo
@@ -79,7 +80,7 @@ coverage.xml
 *.log
 
 # Sphinx documentation
-docs/_build/
+!doc/sphinx/build/html
 
 # PyBuilder
 target/
@@ -89,7 +90,7 @@ target/
 
 
 #notebooks
-docs/my_notebooks/
+doc/my_notebooks/
 
 *.pydevproject
 *.project
@@ -100,8 +101,14 @@ docs/my_notebooks/
 #config backups
 pyrpl/config/*.bak
 
+#user config
+pyrpl/config/user_config/*.yml
+pyrpl/config/user_config/*.bak
+pyrpl/config/user_config
+
+
 #test files made by Xueshi
 test_import.py
-run.py
+run.py 
 rpiq_gui.py
 rpiq.ui

.travis.yml

@@ -1,23 +1,29 @@
+# for setup: must go into travis setup and insert ip address and other data of
+# the redpitaya device, and disable concurrent builds to avoid conflicts
+# between tests of different python versions
+
+# lots of stuff here comes from https://gist.github.com/dan-blanchard/7045057
+
 env:
-- AUTOCORRETPEP8=0 REDPITAYA_SKIPGUITEST=1
+- AUTOCORRETPEP8=0 DISPLAY=:99.0
 # REDPITAYA variables are defined in travis account (encrypted)
 
 language: python
+# nosetests is only executed for 2.7 and 3.5, but we still test the
+# installation for other python versions
 python:
   - "2.7"
-  - "3.3"
-  - "3.4"
   - "3.5"
+  - "3.4"
+  - "3.3"
+#  - "3.6" # still not working
 
-# lots of stuff here comes from https://gist.github.com/dan-blanchard/7045057
-
 notifications:
   email: false
 
 before_install:
   - sudo apt-get update
-  # We do this conditionally because it saves us some downloading if the
-  # version is the same.
+  # We do this conditionally because it saves us some downloading if the version is the same.
   - if [[ "$TRAVIS_PYTHON_VERSION" == "2.7" ]]; then
       wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh -O miniconda.sh;
     else
@@ -31,40 +37,43 @@ before_install:
   - conda update -q conda
   # Useful for debugging any issues with conda
   - conda info -a
-  # The next couple lines fix a crash with multiprocessing on Travis and are not specific to using Miniconda
+  # The next lines fix a crash with multiprocessing on Travis and are not specific to using Miniconda
   - sudo rm -rf /dev/shm
   - sudo ln -s /run/shm /dev/shm
-#before_install:
-# - sudo apt-get update
-# - sudo apt-get -y install python-pip
-# - sudo pip install --upgrade pip
-# - sudo pip install pep8
-# - sudo pip install autopep8
-# - sudo pip install -r requirements.txt .
-##- apt-get install libblas-dev liblapack-dev libatlas-base-dev gfortran
+  # starts gui support, see https://docs.travis-ci.com/user/gui-and-headless-browsers/
+  - sh -e /etc/init.d/xvfb start
+  # give it some time to start
+  - sleep 3
 
 install:
-  - conda create -q -n test-environment python=$TRAVIS_PYTHON_VERSION numpy scipy paramiko matplotlib pandas nose pip
+  # we get issues with building numpy etc. if we do not include those in the next line
+  - conda create -q -n test-environment python=$TRAVIS_PYTHON_VERSION numpy scipy paramiko pandas nose pip pyqt=4
   - source activate test-environment
-  - if [[ "$TRAVIS_PYTHON_VERSION" != "3.3" ]]; then
-      conda install -c ufechner pyqtgraph;
-    fi
-  - pip install scp
-  # there are some bugs in the installation of these packages in higher python versions
-  - if [[ "$TRAVIS_PYTHON_VERSION" != "3.5" ]] && [[ "$TRAVIS_PYTHON_VERSION" != "3.4" ]]; then
-      conda install --yes -c dan_blanchard python-coveralls nose-cov; 
+  # python 2.7 doesnt come with coveralls, but we dont want to put it into the
+  # requirements in setup.py
+  - if [[ "$TRAVIS_PYTHON_VERSION" == "2.7" ]]; then
+      conda install --yes -c dan_blanchard python-coveralls nose-cov;
     fi
+  # alternative location:
+  # conda install --yes -c conda-forge coveralls;
+  # overwrite default global config file with a custom one for travis (allows slower communication time)
+  - \cp ./travis_global_config.yml ./pyrpl/config/global_config.yml
   - python setup.py install
 
-#before_script: ./autocorrectpep8_script.sh
-
 # Run test
 script:
-  #see: https://github.com/rtfd/readthedocs.org/issues/1951
-  - export DISPLAY=:99.0
-  - sh -e /etc/init.d/xvfb start
-  - nosetests --with-coverage --cover-package=pyrpl --logging-level=DEBUG
+  - if [[ "$TRAVIS_PYTHON_VERSION" == "2.7" ]] || [[ "$TRAVIS_PYTHON_VERSION" == "3.5" ]]; then
+      nosetests;
+    fi
 
 # Calculate coverage
 after_success:
   - coveralls --config_file .coveragerc
+
+# upload autodoc to github pages
+# deploy:
+#  provider: pages
+#  skip_cleanup: true
+#  github_token: $GITHUB_TOKEN # Set in travis-ci.org dashboard
+#  on:
+#    branch: master