ValueError: the given item was not found in the heap

[fridex]

Describe the bug

When resolving this stack:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi-org"

[packages]
dask = "*"
flask = "*"
pandas = "*"
pyspark = "*"
pytest = "*"
scipy = "*"
tensorflow = "*"

[dev-packages]

[requires]
python_version = "3.6"

using stable recommendation type, I receive:

2020-11-05 09:03:52,501 thoth.adviser.run           ERROR: Adviser raised exception: the given item was not found in the heap
Traceback (most recent call last):
  File "/opt/app-root/src/thoth/adviser/run.py", line 76, in subprocess_run
    with_devel=with_devel, user_stack_scoring=user_stack_scoring
  File "/opt/app-root/src/thoth/adviser/resolver.py", line 1241, in resolve
    for state in self._do_resolve_states(with_devel=with_devel, user_stack_scoring=user_stack_scoring):
  File "/opt/app-root/src/thoth/adviser/resolver.py", line 1172, in _do_resolve_states
    with_devel=with_devel, user_stack_scoring=user_stack_scoring
  File "/opt/app-root/src/thoth/adviser/resolver.py", line 1121, in _do_resolve_states_raw
    state_returned = self._expand_state(state, unresolved_package_tuple)
  File "/opt/app-root/src/thoth/adviser/resolver.py", line 829, in _expand_state
    state=state, package_version=package_version, dependencies=list(chain(*dependencies.values())),
  File "/opt/app-root/src/thoth/adviser/resolver.py", line 1055, in _expand_state_add_dependencies
    return self._run_steps(state, package_version, all_dependencies, newly_added)
  File "/opt/app-root/src/thoth/adviser/resolver.py", line 429, in _run_steps
    self.beam.remove(cloned_state)
  File "/opt/app-root/src/thoth/adviser/beam.py", line 211, in remove
    self._heap.remove(state)
ValueError: the given item was not found in the heap

This might be related to thoth-station/thamos#566. It looks like we have a bug in state handling in beam.

Expected behavior

The resolution should succeed.

[fridex]

Hm... strange... I was not able to reproduce this locally with the same parameters. The stack is too huge to do it manually. Please report if you encounter this issue again.

[fridex]

Need a reproducer for this.

Future visitor: please provide a reproducer if you hit this issue.

[sesheta]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

[fridex]

No fix was addressed. Let's keep this open, please provide any reproducer to have this fixed.

/remove-lifecycle rotten

[sesheta]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

[sesheta]

@sesheta: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fridex]

/remove-lifecycle rotten
/reopen
/triage accepted

[sesheta]

@fridex: Reopened this issue.

In response to this:

/remove-lifecycle rotten
/reopen
/triage accepted

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[sesheta]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

[sesheta]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

[sesheta]

@sesheta: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[mayaCostantini]

Reopening this issue as it was encountered in integration tests from smaug-prod (2022-09-21 version 0.11.3):

Traceback (most recent call last):
  File "/opt/app-root/src/thoth/adviser/beam.py", line 221, in remove
    self._heap.remove(state)
ValueError: the given item was not found in the heap
2022-09-21 03:30:27,890 sentry_sdk.errors           WARNING: Rate-limited via x-sentry-rate-limits
2022-09-21 03:30:29,834 thoth.adviser.beam          ERROR: Encountered exception reported in https://github.com/thoth-station/adviser/issues/1541, ignoring...
Traceback (most recent call last):
  File "/opt/app-root/src/thoth/adviser/beam.py", line 221, in remove
    self._heap.remove(state)
ValueError: the given item was not found in the heap
2022-09-21 03:30:30,973 thoth.adviser.beam          ERROR: Encountered exception reported in https://github.com/thoth-station/adviser/issues/1541, ignoring...
Traceback (most recent call last):
  File "/opt/app-root/src/thoth/adviser/beam.py", line 221, in remove
    self._heap.remove(state)
ValueError: the given item was not found in the heap

[mayaCostantini]

/sig stack-guidance
/remove lifecycle-rotten
/priority critical-urgent

[sesheta]

Issues needing reporter input close after 60d,

If there is new input, reopen with /reopen.