Jobs not being created in a session with 0.20.0

[mriedem]

Describe the bug

I have this code which used to work - it's a negative test to make sure the API returns an error when trying to submit a job whose max_execution_time is greater than the session's max_time:

def get_circuit():
    qc = QuantumCircuit(2)
    qc.h(0)
    qc.cx(0, 1)
    qc.measure_all()
    return qc


def run_job(service=None, session=None, wait=True, max_execution_time=None):
    """Runs a Sampler job

    :param service: Optional QiskitRuntimeService instance to use to run the job;
    not used if `session` is provided.
    :param session: Optional session to use for the job. If provided the
    session is left open.
    :param wait: If true, waits for the job to complete.
    :param max_execution_time: Maximum execution time in seconds. If a job exceeds
    this time limit, it is forcibly cancelled.
    :returns: RuntimeJob
    """
    options = Options(
        resilience_level=0,
        execution=dict(shots=1),
        max_execution_time=max_execution_time
    )
    qc = get_circuit()
    if session:
        sampler = Sampler(options=options, session=session)
        job = sampler.run(qc)
        LOG.info(f'created job: {job.job_id()}; session: {job.session_id}')
    else:
        backend = service.backend(os.getenv('TEST_DEVICE'))
        sampler = Sampler(backend=backend, options=options)
        job = sampler.run(qc)
        LOG.info(f'created job: {job.job_id()}')

    if wait:
        wait_for_final_state(job)
    return job


def test_session_max_time_less_than_job_max_execution_time_fails(service):
    max_execution_time = 300
    # We can't use the session fixture because we need to specify max_time in the Session
    session = Session(service=service,
                      backend='fake_backend1',  # the backend doesn't matter for this test
                      max_time=max_execution_time - 1)  # this should trigger a failure
    # https://internal-docs.quantum-computing.ibm.com/system-architecture/errors.html#c1213
    with pytest.raises(IBMRuntimeError, match=r'.*"code":1213.*'):
        run_job(service, session=session, max_execution_time=max_execution_time)

That's now failing with 0.20.0 but was working with 0.19.1:

_________ test_session_max_time_less_than_job_max_execution_time_fails _________

service = <QiskitRuntimeService>

    def test_session_max_time_less_than_job_max_execution_time_fails(service):

        max_execution_time = 300

        # We can't use the session fixture because we need to specify max_time in the Session

        session = Session(service=service,

                          backend='fake_backend1',  # the backend doesn't matter for this test

                          max_time=max_execution_time - 1)  # this should trigger a failure

        # https://internal-docs.quantum-computing.ibm.com/system-architecture/errors.html#c1213

        with pytest.raises(IBMRuntimeError, match=r'.*"code":1213.*'):

>           run_job(service, session=session, max_execution_time=max_execution_time)

E           Failed: DID NOT RAISE <class 'qiskit_ibm_runtime.exceptions.IBMRuntimeError'>

tests/test_sessions.py:99: Failed

------------------------------ Captured log setup ------------------------------

INFO     /home/travis/build/IBM-Q-Software/iqp-channel-e2e-tests/tests/conftest.py:conftest.py:28 getting service

INFO     /home/travis/build/IBM-Q-Software/iqp-channel-e2e-tests/tests/conftest.py:conftest.py:33 got service

----------------------------- Captured stderr call -----------------------------

base_primitive._run_primitive:INFO:2024-02-21 19:37:20,289: Submitting job using options {'optimization_level': 1, 'resilience_level': 0, 'max_execution_time': 300, 'transpilation': {'skip_transpilation': False, 'optimization_settings': {'level': 1}, 'coupling_map': None, 'basis_gates': None}, 'resilience': {'level': 0}, 'execution': {'shots': 1, 'init_qubits': True, 'noise_model': None, 'seed_simulator': None}, 'environment': {'log_level': 'WARNING', 'job_tags': []}, 'simulator': {}}

------------------------------ Captured log call -------------------------------

INFO     utils:utils.py:39 created job: cqb51w7ft0w0008hqewg; session: None

INFO     websocket:_logging.py:89 Websocket connected

INFO     utils:utils.py:61 job cqb51w7ft0w0008hqewg is in final state: JobStatus.DONE

When I looked at IQP job cqb51w7ft0w0008hqewg (in staging) it doesn't have a session associated with it.

Something must have regressed with #1372 where POST /sessions isn't being called in this case?

Steps to reproduce

see above

Expected behavior

The job should not get created because the API should reject it since the job's max_execution_time is greater than the session's max_time.

Suggested solutions

n/a

Additional Information

• qiskit-ibm-runtime version: 0.20.0
• Python version: 3.8
• Operating system: Ubuntu Focal LTS

[mriedem]

When I looked at IQP job cqb51w7ft0w0008hqewg (in staging) it doesn't have a session associated with it.

Looking at the IQP API logs in staging for that job it's created without a session:

Feb 21 19:37:21 sw-api-iqp-86f798d89-s7t4l sw-iqp-api 30 {"level":30,"time":1708544241326,"pid":1,"hostname":"sw-api-iqp-86f798d89-s7t4l","req":{"method":"POST","url":"/runtime/jobs","userAgent":"python-requests/2.31.0"},"reqId":"ff34d1ec-a812-46ca-bcef-7067acb5dbee","tracing":"d78ebc0daee2cf6f40a8e630cbee6c02","context":"CreateJobUseCase","id":"cqb51w7ft0w0008hqewg","sessionId":null,"msg":"Job created"}

[mriedem]

The docs here show creating a session only using the context manager method:

https://docs.quantum.ibm.com/api/qiskit-ibm-runtime/qiskit_ibm_runtime.Session

Is that the only way now?

[mriedem]

@kt474 pointed out in Slack that this no longer works because fake_backend1 is treated like a simulator which doesn't work with sessions now:

https://github.com/Qiskit/qiskit-ibm-runtime/blob/0.20.0/qiskit_ibm_runtime/session.py#L137

I tried using test_eagle as the backend and I got a different error for that:

qiskit_ibm_runtime.api.exceptions.RequestsApiError: '400 Client Error: Bad Request for url: https://api-dev.quantum.ibm.com/runtime/sessions. {"errors":[{"code":"bad_request","message":"Invalid session id to create a session","solution":"Verify the session ID is correct.","more_info":"https://docs.quantum-computing.ibm.com/errors"}]}'

[mriedem]

@kt474 fake_backend1 has been re-configured in staging to no longer show up as a simulator so the test is working again for me.

Given that, the code is working as designed, but it was a surprise.

Changing the behavior where jobs submitted to a session but against a simulator backend don't actually create the session isn't documented as far as I can tell.

It seems the 0.20.0 release note should be amended (if possible).

It also seems like there should be a mention somewhere in the docs related to sessions that simulator backends are not supported. The client won't error, it just won't submit jobs against a session since it's not supported in the API. It's doubly confusing because docs like this which are meant to tell users how to submit jobs against a session are using a simulator backend which won't actually create the session.

cc @jyu00

[jyu00]

There is actually documentation that says sessions don't work with simulators. But you are right that there is still a breaking change here - that sessions "created" for a simulator backend would no longer return a session ID, and that should have gone into the release note. Especially since it's a server side breaking change with no deprecation period.

It's doubly confusing because docs like this which are meant to tell users how to submit jobs against a session are using a simulator backend which won't actually create the session.

I believe the intent was to show the Session code syntax, and simulator was chosen for fast execution. But since we are retiring cloud simulators anyway, it makes sense to update the code to use a real backend instead. I opened a docs issue on this.

[kt474]

It seems the 0.20.0 release note should be amended (if possible).

I'll work with the docs team to update the release note

[kt474]

Closing this issue, release note has been updated - https://docs.quantum.ibm.com/api/qiskit-ibm-runtime/release-notes

Thank you @Eric-Arellano!