Nightly CI failing since Qiskit 2.1.2

[woodsp-ibm]

Some tests are failing with this:

ValueError: Non-Hermitian input observable: the input SparsePauliOp has non-zero imaginary part in its coefficients.

An issue was raised Qiskit/qiskit#14567 which I appears to be fixed for 2.2.0. Since CI passed with 2.1.1 I do not know what went into 2.1.2 that caused this failure. Before a failure like this was only seen in the tests against Qiskit main.

[ElePT]

Because of the pin in requirements-dev.txt, it looks like CI was not passing with 2.1.0 or 2.1.1 either (for different reasons). This might indicate that a bug was introduced in 2.1.0 and may not have been fixed yet. These are the error traces I see with 2.2.0b1:

==============================
Failed 10 tests - output below:
==============================

test.time_evolvers.test_trotter_qrte.TestTrotterQRTE.test_trotter_qrte_trotter_single_qubit_1
---------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/ddt.py", line 221, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/test/time_evolvers/test_trotter_qrte.py", line 65, in test_trotter_qrte_trotter_single_qubit
    np.testing.assert_array_almost_equal(

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/_utils/__init__.py", line 86, in wrapper
    return fun(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 1172, in assert_array_almost_equal
    assert_array_compare(compare, actual, desired, err_msg=err_msg,

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 921, in assert_array_compare
    raise AssertionError(msg)

    AssertionError:
Arrays are not almost equal to 6 decimals

Mismatched elements: 2 / 2 (100%)
Max absolute difference among violations: 0.74887246
Max relative difference among violations: 0.88995637
 ACTUAL: array([1.559437e-01-0.698456j, 7.945471e-19-0.698456j])
 DESIRED: array([0.291927-0.454649j, 0.708073-0.454649j])


test.time_evolvers.test_trotter_qrte.TestTrotterQRTE.test_trotter_qrte_qdrift_2
-------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/ddt.py", line 221, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/test/time_evolvers/test_trotter_qrte.py", line 184, in test_trotter_qrte_qdrift
    np.testing.assert_array_almost_equal(

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/_utils/__init__.py", line 86, in wrapper
    return fun(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 1172, in assert_array_almost_equal
    assert_array_compare(compare, actual, desired, err_msg=err_msg,

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 921, in assert_array_compare
    raise AssertionError(msg)

    AssertionError:
Arrays are not almost equal to 6 decimals

Mismatched elements: 2 / 2 (100%)
Max absolute difference among violations: 0.50573296
Max relative difference among violations: 0.74193692
 ACTUAL: array([1.559437e-01-0.698456j, 7.945471e-19-0.698456j])
 DESIRED: array([0.230718-0.694361j, 0.464631-0.498747j])


test.time_evolvers.test_trotter_qrte.TestTrotterQRTE.test_trotter_qrte_trotter_1
--------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/ddt.py", line 221, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/test/time_evolvers/test_trotter_qrte.py", line 102, in test_trotter_qrte_trotter
    np.testing.assert_array_almost_equal(

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/_utils/__init__.py", line 86, in wrapper
    return fun(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 1172, in assert_array_almost_equal
    assert_array_compare(compare, actual, desired, err_msg=err_msg,

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 921, in assert_array_compare
    raise AssertionError(msg)

    AssertionError:
Arrays are not almost equal to 6 decimals

Mismatched elements: 2 / 2 (100%)
Max absolute difference among violations: 0.83544228
Max relative difference among violations: 63.24100939
 ACTUAL: array([-4.526619e-01+0.630515j, -1.425611e-16+0.630515j])
 DESIRED: array([-0.99995 -0.000706j,  0.009957-0.000706j])


test.time_evolvers.variational.test_var_qrte.TestVarQRTE.test_time_dependent_hamiltonian
----------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/test/time_evolvers/variational/test_var_qrte.py", line 72, in test_time_dependent_hamiltonian
    self.assertTrue(np.allclose(final_expected_state[int(key)], expected_value, 1e-02))

      File "/Users/ept/.pyenv/versions/3.11.9/lib/python3.11/unittest/case.py", line 715, in assertTrue
    raise self.failureException(msg)

    AssertionError: False is not true


test.time_evolvers.variational.test_var_qite.TestVarQITE.test_run_d_1_time_dependent
------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/test/time_evolvers/variational/test_var_qite.py", line 282, in test_run_d_1_time_dependent
    self.assertTrue(np.allclose(evolved_value, state_expected[key], 1e-02))

      File "/Users/ept/.pyenv/versions/3.11.9/lib/python3.11/unittest/case.py", line 715, in assertTrue
    raise self.failureException(msg)

    AssertionError: False is not true


test.time_evolvers.test_trotter_qrte.TestTrotterQRTE.test_trotter_qrte_trotter_2
--------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/ddt.py", line 221, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/test/time_evolvers/test_trotter_qrte.py", line 102, in test_trotter_qrte_trotter
    np.testing.assert_array_almost_equal(

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/_utils/__init__.py", line 86, in wrapper
    return fun(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 1172, in assert_array_almost_equal
    assert_array_compare(compare, actual, desired, err_msg=err_msg,

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 921, in assert_array_compare
    raise AssertionError(msg)

    AssertionError:
Arrays are not almost equal to 6 decimals

Mismatched elements: 2 / 2 (100%)
Max absolute difference among violations: 0.37099384
Max relative difference among violations: 0.58908849
 ACTUAL: array([0.380387-0.293605j, 0.111406-0.86988j ])
 DESIRED: array([ 0.629499-0.018687j, -0.023146-0.776432j])


test.time_evolvers.variational.solvers.ode.test_ode_function.TestOdeFunctionGenerator.test_var_qte_ode_function_time_param
--------------------------------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/qiskit_algorithms/time_evolvers/variational/variational_principles/imaginary_mc_lachlan_principle.py", line 107, in evolution_gradient
    .result()
     ^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/qiskit/primitives/primitive_job.py", line 65, in result
    self._result = self._future.result()
                   ^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/.pyenv/versions/3.11.9/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/.pyenv/versions/3.11.9/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception

      File "/Users/ept/.pyenv/versions/3.11.9/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/qiskit_algorithms/gradients/lin_comb/lin_comb_estimator_gradient.py", line 122, in _run
    results = self._run_unique(
              ^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/qiskit_algorithms/gradients/lin_comb/lin_comb_estimator_gradient.py", line 216, in _run_unique
    results = run_estimator_job(self._estimator, pubs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/qiskit_algorithms/run_estimator_job.py", line 103, in run_estimator_job
    job = estimator.run(purged_pubs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/qiskit/primitives/statevector_estimator.py", line 140, in run
    coerced_pubs = [EstimatorPub.coerce(pub, precision) for pub in pubs]
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/qiskit/primitives/statevector_estimator.py", line 140, in <listcomp>
    coerced_pubs = [EstimatorPub.coerce(pub, precision) for pub in pubs]
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/qiskit/primitives/containers/estimator_pub.py", line 149, in coerce
    observables = ObservablesArray.coerce(pub[1])
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/qiskit/primitives/containers/observables_array.py", line 322, in coerce
    return cls(observables)
           ^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/qiskit/primitives/containers/observables_array.py", line 91, in __init__
    basis_obs = self.coerce_observable(obs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/qiskit/primitives/containers/observables_array.py", line 267, in coerce_observable
    observable = SparseObservable.from_sparse_pauli_op(observable)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

    TypeError: only 'SparsePauliOp' with complex-typed coefficients can be converted to 'SparseObservable'


The above exception was the direct cause of the following exception:


    Traceback (most recent call last):

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/test/time_evolvers/variational/solvers/ode/test_ode_function.py", line 127, in test_var_qte_ode_function_time_param
    qte_ode_function = ode_function_generator.var_qte_ode_function(time, param_dict.values())
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/qiskit_algorithms/time_evolvers/variational/solvers/ode/ode_function.py", line 36, in var_qte_ode_function
    ode_grad_res, _, _ = self._varqte_linear_solver.solve_lse(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/qiskit_algorithms/time_evolvers/variational/solvers/var_qte_linear_solver.py", line 125, in solve_lse
    evolution_grad_lse_rhs = self._var_principle.evolution_gradient(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/qiskit_algorithms/time_evolvers/variational/variational_principles/imaginary_mc_lachlan_principle.py", line 112, in evolution_gradient
    raise AlgorithmError("The gradient primitive job failed!") from exc

    qiskit_algorithms.exceptions.AlgorithmError: 'The gradient primitive job failed!'


test.time_evolvers.variational.test_var_qrte.TestVarQRTE.test_run_d_1_time_dependent
------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/test/time_evolvers/variational/test_var_qrte.py", line 253, in test_run_d_1_time_dependent
    np.testing.assert_almost_equal(float(parameter_value), thetas_expected[i], decimal=2)

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 627, in assert_almost_equal
    raise AssertionError(_build_err_msg())

    AssertionError:
Arrays are not almost equal to 2 decimals
 ACTUAL: 0.0
 DESIRED: 0.99


test.time_evolvers.test_trotter_qrte.TestTrotterQRTE.test_trotter_qrte_qdrift_1
-------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/ddt.py", line 221, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/test/time_evolvers/test_trotter_qrte.py", line 184, in test_trotter_qrte_qdrift
    np.testing.assert_array_almost_equal(

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/_utils/__init__.py", line 86, in wrapper
    return fun(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 1172, in assert_array_almost_equal
    assert_array_compare(compare, actual, desired, err_msg=err_msg,

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 921, in assert_array_compare
    raise AssertionError(msg)

    AssertionError:
Arrays are not almost equal to 6 decimals

Mismatched elements: 2 / 2 (100%)
Max absolute difference among violations: 0.50573296
Max relative difference among violations: 0.74193692
 ACTUAL: array([1.559437e-01-0.698456j, 7.945471e-19-0.698456j])
 DESIRED: array([0.230718-0.694361j, 0.464631-0.498747j])


test.time_evolvers.test_trotter_qrte.TestTrotterQRTE.test_trotter_qrte_trotter_single_qubit_2
---------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/ddt.py", line 221, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-algorithms/test/time_evolvers/test_trotter_qrte.py", line 65, in test_trotter_qrte_trotter_single_qubit
    np.testing.assert_array_almost_equal(

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/_utils/__init__.py", line 86, in wrapper
    return fun(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 1172, in assert_array_almost_equal
    assert_array_compare(compare, actual, desired, err_msg=err_msg,

      File "/Users/ept/qiskit_workspace/qiskit-serverless/.venv_serverless/lib/python3.11/site-packages/numpy/testing/_private/utils.py", line 921, in assert_array_compare
    raise AssertionError(msg)

    AssertionError:
Arrays are not almost equal to 6 decimals

Mismatched elements: 2 / 2 (100%)
Max absolute difference among violations: 0.24380729
Max relative difference among violations: 0.53625422
 ACTUAL: array([1.559437e-01-0.698456j, 7.945471e-19-0.698456j])
 DESIRED: array([0.291927-0.841471j, 0.      -0.454649j])

[woodsp-ibm]

It may well have been for the the same it got those versions pinned. I'll close this for now as your PR #245 addresses this.

[tnemoz]

I do not know what went into 2.1.2 that caused this failure.

Looking at the failures, it appears that this message only appears in the tests of AdaptVQE, the reason for which seems to be this check that I've added back in #197:

qiskit-algorithms/qiskit_algorithms/minimum_eigensolvers/adapt_vqe.py

Lines 241 to 244 in 26b4d59

	# We have to call simplify on it since Qiskit doesn't do so in versions 2.1.0 and 2.1.1, see
	# Qiskit/qiskit/issues/14567
	if get_qiskit_version_info() in ["2.1.0", "2.1.1"]:
	commutators = [obs.simplify() for obs in commutators]

When I wrote these checks, if I'm not mistaken, I think that Qiskit/qiskit#14567 was fixed, and thus I probably assumed that it was going to be in the 2.1.2 release, which is why I haven't included it. However, looking at Qiskit/qiskit#14714, as you mentioned, these changes are only to appear in the 2.2.0 version, hence the discrepancy. My bad :(

Thus, though pinning the qiskit version to be different from 2.1.2 is indeed required because of Qiskit/qiskit#14611 not being merged, these other failures could be avoided by adding the check for the 2.1.2 version like so

        # We have to call simplify on it since Qiskit doesn't do so in versions 2.1.X, see
        # Qiskit/qiskit/issues/14567
        if get_qiskit_version_info() in ["2.1.0", "2.1.1", "2.1.2"]:
            commutators = [obs.simplify() for obs in commutators]

This would also ensure that users don't run into this bug while using AdaptVQE with a 2.1.X version of Qiskit.