[Bug] config option error using a NI scanner with two AO outputs (2D scanning only)

[syim70]

Version

current main

What is affected by the bug?

I tried to use a NI scanner with two AO outputs, but config file's scanner gui option "optimizer_plot_dimensions: [2]" does not work. It seems like Qudi requires "z axis" even if there is no additional AO.

When does the bug occur?

1. Latest release (both qudi-core and qudi-iqo-modules): "optimizer_plot_dimensions: [2]" is not working even with scanner_dummy.
2. Optimizer_fix_1D branch (#155): "optimizer_plot_dimensions: [2]" is working with scanner_dummy, but not with a real hardware (NI DAQ with two AOs).

How do we replicate the issue?

If users choose only 2D scanning, ni scanner (or optimizer) should not require more axes.

Expected behavior

Please, let me have an options in config file with 2D scanning only.

Relevant log output

No response

Additional Comments

No response

Contact Details

[timoML]

Can you please paste the full error that you experience with #155 and nicard?
The branch name should be "1d_scanner_optimize".

[syim70]

As you noted, I double checked the branch name "1d_scanner_optimize."
Here are two cases.
First, ni_scanner, with a full list of AOs in config file (as default).
Second, ni_scanner, without unavailable AOs (and z axis) in config file.
Thanks.

1. For ni_scanner, with a full list of AOs

Invalid analog source channels encountered. Following sources will be ignored:
  ao2, ao3
Valid analog input channels are:
  ai0, ai1, ai2, ai3, ai4, ai5, ai6, ai7, ai8, ai9, ai10, ai11, ai12, ai13, ai14, ai15
Exception during activation:
Traceback (most recent call last):
  File "...\lib\site-packages\qudi\core\module.py", line 301, in __activation_callback
    self.on_activate()
  File "...\qudi-iqo-modules\src\qudi\hardware\ni_x_series\ni_x_series_finite_sampling_io.py", line 246, in on_activate
    raise ValueError(
ValueError: The channels "ao2, ao3", specified in the config, were not recognized.

Traceback (most recent call last):
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 201, in activate_module
    self._modules[module_name].activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 502, in activate
    module.activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 502, in activate
    module.activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 502, in activate
    module.activate()
  [Previous line repeated 1 more time]
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 543, in activate
    raise RuntimeError(f'Failed to activate {self.module_base} module "{self.name}"!')
RuntimeError: Failed to activate hardware module "ni_io"!

2. For ni_scanner, without unavailable AOs (and z axis)

Exception during activation:
Traceback (most recent call last):
  File "...\lib\site-packages\qudi\core\module.py", line 301, in __activation_callback
    self.on_activate()
  File "...\qudi-iqo-modules\src\qudi\logic\scanning_probe_logic.py", line 85, in on_activate
    constr = self.scanner_constraints
  File "...\qudi-iqo-modules\src\qudi\logic\scanning_probe_logic.py", line 152, in scanner_constraints
    return self._scanner().get_constraints()
  File "...\lib\site-packages\qudi\util\overload.py", line 144, in __getattribute__
    attr = getattr(obj, name)
AttributeError: 'NiScanningProbeInterfuse' object has no attribute 'get_constraints'. Did you mean: '_constraints'?

Traceback (most recent call last):
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 201, in activate_module
    self._modules[module_name].activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 502, in activate
    module.activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 502, in activate
    module.activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 543, in activate
    raise RuntimeError(f'Failed to activate {self.module_base} module "{self.name}"!')
RuntimeError: Failed to activate logic module "scanning_probe_logic"!

[TobiasSpohn]

Regarding point 2)

Are you sure that you are running an unmodified scanning_probe_logic.py file as it is in branch 1d_scanner_optimize?
Because in the ScanningProbeLogic.scanner_constraints there is no method get_constraints from the scanner called, only the exisiting constraints property, as you can see from the code of the link above.
Looking at the repository at an earlier stage the get_constraints method is called. Try using the latest version of the file.

[syim70]

Dear TobiasSpohn,
Thank you for your note, and sorry for the confusing error report.
I found I did not correctly pull the branch (I am not familiar with git).
Here are the errors I encountered.

1. For ni_scanner, with a full list of AOs

Invalid analog source channels encountered. Following sources will be ignored:
  ao2, ao3
Valid analog input channels are:
  ai0, ai1, ai2, ai3, ai4, ai5, ai6, ai7, ai8, ai9, ai10, ai11, ai12, ai13, ai14, ai15
Exception during activation:
Traceback (most recent call last):
  File "...\lib\site-packages\qudi\core\module.py", line 301, in __activation_callback
    self.on_activate()
  File "...\qudi-iqo-modules\src\qudi\hardware\ni_x_series\ni_x_series_finite_sampling_io.py", line 246, in on_activate
    raise ValueError(
ValueError: The channels "ao3, ao2", specified in the config, were not recognized.

Traceback (most recent call last):
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 201, in activate_module
    self._modules[module_name].activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 502, in activate
    module.activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 502, in activate
    module.activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 502, in activate
    module.activate()
  [Previous line repeated 1 more time]
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 543, in activate
    raise RuntimeError(f'Failed to activate {self.module_base} module "{self.name}"!')
RuntimeError: Failed to activate hardware module "ni_io"!

2. For ni_scanner, without unavailable AOs (and z axis)

Scan settings in Status Variable empty or invalid, using defaults.
Traceback (most recent call last):
  File "...\qudi-iqo-modules\src\qudi\logic\scanning_optimize_logic.py", line 93, in on_activate
    self._check_scan_settings()
  File "...\qudi-iqo-modules\src\qudi\logic\scanning_optimize_logic.py", line 377, in _check_scan_settings
    ax.position.check(self.scan_range[name])
KeyError: 'x'
Exception during activation:
Traceback (most recent call last):
  File "...\lib\site-packages\qudi\core\module.py", line 301, in __activation_callback
    self.on_activate()
  File "...\qudi-iqo-modules\src\qudi\gui\scanning\scannergui.py", line 186, in on_activate
    self._init_static_dockwidgets()
  File "...\qudi-iqo-modules\src\qudi\gui\scanning\scannergui.py", line 423, in _init_static_dockwidgets
    self.optimizer_dockwidget = OptimizerDockWidget(axes=self._scanning_logic().scanner_axes,
  File "...\qudi-iqo-modules\src\qudi\gui\scanning\optimizer_dockwidget.py", line 60, in __init__
    for i_col, n_dim in enumerate(plot_dims):
TypeError: 'ConfigOption' object is not iterable

Traceback (most recent call last):
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 201, in activate_module
    self._modules[module_name].activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 543, in activate
    raise RuntimeError(f'Failed to activate {self.module_base} module "{self.name}"!')
RuntimeError: Failed to activate gui module "scanner_gui"!

[timoML]

Can you poste the relevant parts (the modules mentioned here) of your config file for both cases?

[TobiasSpohn]

Hi @syim70,

For me the first error seems like you are trying to configure input channels as outputs. Try looking into the config to see if you configure the right channels for inputs and outputs, as I don't know, whether you could use output channels (ao) as input channels (ai).

The second error also seems like you are running an outdated version of the gui or logic files. The error TypeError: 'ConfigOption' object is not iterable occurs when a ConfigOption fails to correctly initialize as normally it should be converted to another type upon initialization, in this case it should be a Tuple. We removed the plot_dims ConfigOption in favor of a StatusVariable that can be set e.g. in the GUI. So the plot_dims variable should no longer be a ConfigOption in the first place. Try updating the files for the entire toolchain.
The KeyError: 'x' is just a warning that occurs on first loading of the toolchain when defaults are initialized and once you successfully load the toolchain it should no longer be present.

[syim70]

Hi, timoML,

I post my config file for the both cases. Thanks.

1. For ni_scanner, with a full list of AOs

gui:
    scanner_gui:
        module.Class: 'scanning.scannergui.ScannerGui'
        options:
            image_axes_padding: 0.02
            default_position_unit_prefix: null  # optional, use unit prefix characters, e.g. 'u' or 'n'
            optimizer_plot_dimensions: [2]      # xy only (z axis off)
        connect:
            scanning_logic: scanning_probe_logic
            data_logic: scanning_data_logic
            optimize_logic: scanning_optimize_logic
logic:
   scanning_probe_logic:
        module.Class: 'scanning_probe_logic.ScanningProbeLogic'
        options:
            max_history_length: 20
            max_scan_update_interval: 2
            position_update_interval: 1
        connect:
            #scanner: scanner_dummy
            scanner: ni_scanner
    scanning_data_logic:
        module.Class: 'scanning_data_logic.ScanningDataLogic'
        options:
            max_history_length: 20
        connect:
            scan_logic: scanning_probe_logic
    scanning_optimize_logic:
        module.Class: 'scanning_optimize_logic.ScanningOptimizeLogic'
        connect:
            scan_logic: scanning_probe_logic
hardware:
    ni_scanner:
        module.Class: 'interfuse.ni_scanning_probe_interfuse.NiScanningProbeInterfuse'
        # to use without tilt correction
        # module.Class: 'interfuse.ni_scanning_probe_interfuse.NiScanningProbeInterfuseBare'
        connect:
            scan_hardware: 'ni_io'
            analog_output: 'ni_ao'
        options:
            ni_channel_mapping:
                x: 'ao0'
                y: 'ao1'
                z: 'ao2'
                APD1: 'PFI8'
                APD2: 'PFI9'
                AI0: 'ai0'
            position_ranges: # in m
                x: [-100e-6, 100e-6]
                y: [-100e-6, 100e-6]
                z: [-100e-6, 100e-6]
            frequency_ranges: #Aka values written/retrieved per second; Check with connected HW for sensible constraints.
                x: [1, 5000]
                y: [1, 5000]
                z: [1, 1000]
            resolution_ranges:
                x: [1, 10000]
                y: [1, 10000]
                z: [2, 1000]
            input_channel_units:
                APD1: 'c/s'
                APD2: 'c/s'
                AI0: 'V'
            backwards_line_resolution: 50 # optional
            move_velocity: 400e-6 #m/s; This speed is used for scanner movements and avoids jumps from position to position.
    ni_io:
        module.Class: 'ni_x_series.ni_x_series_finite_sampling_io.NIXSeriesFiniteSamplingIO'
        options:
            device_name: 'Dev1'
            input_channel_units:
                PFI8: 'c/s'
                PFI9: 'c/s'
                ai0: 'V'
                ai1: 'V'
            output_channel_units: # Specify used output channels
                'ao0': 'V'
                'ao1': 'V'
                'ao2': 'V'
                'ao3': 'V'
            adc_voltage_ranges:
                ai0: [-10, 10]  # optional
                ai1: [-10, 10]  # optional
            output_voltage_ranges:
                ao0: [-10, 10]
                ao1: [-10, 10]
                ao2: [-10, 10]
                ao3: [-10.0, 10.0]
            frame_size_limits: [1, 1e9]  # optional #TODO actual HW constraint?
            default_output_mode: 'JUMP_LIST' # optional, must be name of SamplingOutputMode
            read_write_timeout: 10  # optional
            sample_clock_output: '/Dev1/PFI11' # optional: routing of sample clock to a physical connection
    ni_ao:
        module.Class: 'ni_x_series.ni_x_series_analog_output.NIXSeriesAnalogOutput'
        options:
            device_name: 'Dev1'
            channels:
                ao0:
                    limits: [-10.0, 10.0]
                    keep_value: True
                ao1:
                    limits: [-10.0, 10.0]
                    keep_value: True
                ao2:
                    limits: [-10.0, 10.0]
                    keep_value: True

2. For ni_scanner, without unavailable AOs (and z axis)

gui:
    scanner_gui:
        module.Class: 'scanning.scannergui.ScannerGui'
        options:
            image_axes_padding: 0.02
            default_position_unit_prefix: null  # optional, use unit prefix characters, e.g. 'u' or 'n'
            optimizer_plot_dimensions: [2]      # xy only (z axis off)
        connect:
            scanning_logic: scanning_probe_logic
            data_logic: scanning_data_logic
            optimize_logic: scanning_optimize_logic
logic:
scanning_probe_logic:
        module.Class: 'scanning_probe_logic.ScanningProbeLogic'
        options:
            max_history_length: 20
            max_scan_update_interval: 2
            position_update_interval: 1
        connect:
            #scanner: scanner_dummy
            scanner: ni_scanner
    scanning_data_logic:
        module.Class: 'scanning_data_logic.ScanningDataLogic'
        options:
            max_history_length: 20
        connect:
            scan_logic: scanning_probe_logic
    scanning_optimize_logic:
        module.Class: 'scanning_optimize_logic.ScanningOptimizeLogic'
        connect:
            scan_logic: scanning_probe_logic
hardware:
    ni_scanner:
        module.Class: 'interfuse.ni_scanning_probe_interfuse.NiScanningProbeInterfuse'
        # to use without tilt correction
        # module.Class: 'interfuse.ni_scanning_probe_interfuse.NiScanningProbeInterfuseBare'
        connect:
            scan_hardware: 'ni_io'
            analog_output: 'ni_ao'
        options:
            ni_channel_mapping:
                x: 'ao0'
                y: 'ao1'
            #    z: 'ao2'
                APD1: 'PFI8'
                APD2: 'PFI9'
                AI0: 'ai0'
            position_ranges: # in m
                x: [-100e-6, 100e-6]
                y: [-100e-6, 100e-6]
            #    z: [-100e-6, 100e-6]
            frequency_ranges: #Aka values written/retrieved per second; Check with connected HW for sensible constraints.
                x: [1, 5000]
                y: [1, 5000]
            #    z: [1, 1000]
            resolution_ranges:
                x: [1, 10000]
                y: [1, 10000]
            #    z: [2, 1000]
            input_channel_units:
                APD1: 'c/s'
                APD2: 'c/s'
                AI0: 'V'
            backwards_line_resolution: 50 # optional
            move_velocity: 400e-6 #m/s; This speed is used for scanner movements and avoids jumps from position to position.
    ni_io:
        module.Class: 'ni_x_series.ni_x_series_finite_sampling_io.NIXSeriesFiniteSamplingIO'
        options:
            device_name: 'Dev1'
            input_channel_units:
                PFI8: 'c/s'
                PFI9: 'c/s'
                ai0: 'V'
                ai1: 'V'
            output_channel_units: # Specify used output channels
                'ao0': 'V'
                'ao1': 'V'
            #    'ao2': 'V'
            #    'ao3': 'V'
            adc_voltage_ranges:
                ai0: [-10, 10]  # optional
                ai1: [-10, 10]  # optional
            output_voltage_ranges:
                ao0: [-10, 10]
                ao1: [-10, 10]
            #    ao2: [-10, 10]
            #    ao3: [-10.0, 10.0]
            frame_size_limits: [1, 1e9]  # optional #TODO actual HW constraint?
            default_output_mode: 'JUMP_LIST' # optional, must be name of SamplingOutputMode
            read_write_timeout: 10  # optional
            sample_clock_output: '/Dev1/PFI11' # optional: routing of sample clock to a physical connection
    ni_ao:
        module.Class: 'ni_x_series.ni_x_series_analog_output.NIXSeriesAnalogOutput'
        options:
            device_name: 'Dev1'
            channels:
                ao0:
                    limits: [-10.0, 10.0]
                    keep_value: True
                ao1:
                    limits: [-10.0, 10.0]
                    keep_value: True
            #    ao2:
            #        limits: [-10.0, 10.0]
            #        keep_value: True

[syim70]

Hi @syim70,

For me the first error seems like you are trying to configure input channels as outputs. Try looking into the config to see if you configure the right channels for inputs and outputs, as I don't know, whether you could use output channels (ao) as input channels (ai).

The second error also seems like you are running an outdated version of the gui or logic files. The error TypeError: 'ConfigOption' object is not iterable occurs when a ConfigOption fails to correctly initialize as normally it should be converted to another type upon initialization, in this case it should be a Tuple. We removed the plot_dims ConfigOption in favor of a StatusVariable that can be set e.g. in the GUI. So the plot_dims variable should no longer be a ConfigOption in the first place. Try updating the files for the entire toolchain. The KeyError: 'x' is just a warning that occurs on first loading of the toolchain when defaults are initialized and once you successfully load the toolchain it should no longer be present.

Hi, TobiasSpohn,

First, I was not trying to configure a something special, but just used the default config file as it is, except the dimensions as [2]. So, I don't agree with your guess that the first error might come from the confusing the input and output channels.

I am not a code developer, so I hope using a stabilized code (the latest one, but not a pre-release). I am using a qudi-core (v.1.5.1) and qudi-iqo-modules (v.0.5.1). Last time, I found my misunderstanding of git, but now I believe I correctly pulled the "1d_scanner_optimze" branch. What you mean the "outdated version" is v.0.5.1? Should I pull some relevant branches altogether?

Thank you so much for your efforts.

[TobiasSpohn]

Hi,

For 1: In your config I don't spot any obvious errors, it seems fine. I personally use a very similar configuration. However, the error you posted above seems to me that you are trying to configure output channels where it should be input channels. Maybe your card does not support that many input or output channels?
Because this error message suggests that you are trying to configure invalid channels.

Invalid analog source channels encountered. Following sources will be ignored: ao2, ao3 Valid analog input channels are: ai0, ai1, ai2, ai3, ai4, ai5, ai6, ai7, ai8, ai9, ai10, ai11, ai12, ai13, ai14, ai15

It even lists the available channels below, you will have to try changing to the correct channels.

For 2.: If you are checking out a development branch, you are no longer working with a release (e.g. v0.5.1), but the development version of qudi. The main branch should still be stable, although some minor bugs might occur, which are mostly hotfixed pretty quickly. We are currently working on merging the 1d_scanner_optimize into main branch, however this is still a pre-release. A new release with these new features appended to, will still take some time after that.
Just from your error messages you posted above you have some error occuring:
for i_col, n_dim in enumerate(plot_dims): TypeError: 'ConfigOption' object is not iterable.
This error occurs when a ConfigOption (the options you specify in your config file) fails to initialize.
However, with the current state of 1d_scanner_optimize there is no ConfigOption plot_dims, as we removed it in favor of a graphical and cli way to change the plot_dims during runtime.
Looking at the branch in an earlier state (before we made the fixes) one can see that the ConfigOption is present in the code. This means you have not updated your (whole) file with the latest changes of 1d_scanner_optimize. Please make sure, especially when using a graphical program, e.g. the git utility in Pycharm, that you actually update everything from the remote branch and not keep some local changes.

[syim70]

Hi, TobiasSpohn,

I made a git pull of main branch as you suggested. The config files are same with the above posted on Nov.12.
The results are different, but the system fails again. As you mentioned above, my scanner has only two AOs, and that was the reason I raised this issue. I hope using two AOs for scanning actions (plus optimizer for 2-dim), and I don't want to use a scanning action or "optimizer" of z-axis. Please, let me know how to solve this problem. Thanks!

1. For ni_scanner, with a full list of AOs

Invalid analog source channels encountered. Following sources will be ignored:
  ao2, ao3
Valid analog input channels are:
  ai0, ai1, ai2, ai3, ai4, ai5, ai6, ai7, ai8, ai9, ai10, ai11, ai12, ai13, ai14, ai15
Exception during activation:
Traceback (most recent call last):
  File "...\lib\site-packages\qudi\core\module.py", line 301, in __activation_callback
    self.on_activate()
  File "...\qudi-iqo-modules\src\qudi\hardware\ni_x_series\ni_x_series_finite_sampling_io.py", line 290, in on_activate
    raise ValueError(
ValueError: The channels "ao3, ao2", specified in the config, were not recognized.

Traceback (most recent call last):
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 201, in activate_module
    self._modules[module_name].activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 502, in activate
    module.activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 502, in activate
    module.activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 502, in activate
    module.activate()
  [Previous line repeated 1 more time]
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 543, in activate
    raise RuntimeError(f'Failed to activate {self.module_base} module "{self.name}"!')
RuntimeError: Failed to activate hardware module "ni_io"!

2. For ni_scanner, without unavailable AOs (and z axis)

Exception during activation:
Traceback (most recent call last):
  File "...\lib\site-packages\qudi\core\module.py", line 301, in __activation_callback
    self.on_activate()
  File "...\qudi-iqo-modules\src\qudi\logic\scanning_optimize_logic.py", line 105, in on_activate
    self._set_default_scan_sequence()
  File "...\qudi-iqo-modules\src\qudi\logic\scanning_optimize_logic.py", line 451, in _set_default_scan_sequence
    possible_scan_sequences = self._allowed_sequences(self._optimizer_sequence_dimensions)
  File "...\qudi-iqo-modules\src\qudi\logic\scanning_optimize_logic.py", line 187, in _allowed_sequences
    raise NotImplementedError(
NotImplementedError: Requested optimization sequence (3) is greater than available scanner axes (2). This is currently not supported. Decrease 'optimizer_sequence_dimensions' in your config file.

Traceback (most recent call last):
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 201, in activate_module
    self._modules[module_name].activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 502, in activate
    module.activate()
  File "...\lib\site-packages\qudi\core\modulemanager.py", line 543, in activate
    raise RuntimeError(f'Failed to activate {self.module_base} module "{self.name}"!')
RuntimeError: Failed to activate logic module "scanning_optimize_logic"!

In addition, is it right to use optimizer_plot_dimensions in config file? It looks like changed to optimizer_sequence_dimensions in the new optimizer logic, but it is not in the example config in the setup_confocal_scanning docs.

new config?

scanner_gui:
        module.Class: 'scanning.scannergui.ScannerGui'
        options:
            image_axes_padding: 0.02
            default_position_unit_prefix: null  # optional, use unit prefix characters, e.g. 'u' or 'n'
            **optimizer_sequence_dimensions**: [2]      # xy only (z axis off)
        connect:
            scanning_logic: scanning_probe_logic
            data_logic: scanning_data_logic
            optimize_logic: scanning_optimize_logic

[TobiasSpohn]

Hi @syim70,

You no longer need the config option, it was deprecated and the dimension is now settable during runtime.
You can just use the default config.

    scanner_gui:
        module.Class: 'scanning.scannergui.ScannerGui'
        # options:
        #     default_position_unit_prefix: null  # optional, use unit prefix characters, e.g. 'u' or 'n'
        #     min_crosshair_size_fraction: 0.02   # minimum crosshair size as fraction of the displayed scan range
        connect:
            scanning_logic: scanning_probe_logic
            data_logic: scanning_data_logic
            optimize_logic: scanning_optimize_logic

For the error messages: The first config is not the right way to proceed, I believe.

The second one seems to initialize correctly, however you get the error that you are requesting a 3 dimensional optimization (so x,y and z) but only two axes where given. Thank you for spotting this bug, this is not intended behavior. I fixed it in #179. It should be merged soon into main, so you can pull it. If you want to test it out before, try checking out scanner_optimization_hotfix.