fix watcher queue initialisation on concurrency

[chmouel]

Resolved an issue where pipelineRuns were being added to the queue instead of the running queue during the initialization of the queue manager. Introduced a new method, AddToPendingQueue, to add pipelineRuns directly to the pending queue without mixing them with the running queue.

Fixes #1790

See video below for a demo

recording-20241021-17h17.mp4

Changes

Submitter Checklist

• 📝 Please ensure your commit message is clear and informative. For guidance on crafting effective commit messages, refer to the How to write a git commit message guide. We prefer the commit message to be included in the PR body itself rather than a link to an external website (ie: Jira ticket).

• ♽ Before submitting a PR, run make test lint to avoid unnecessary CI processing. For an even more efficient workflow, consider installing pre-commit and running pre-commit install in the root of this repository.

• ✨ We use linters to maintain clean and consistent code. Please ensure you've run make lint before submitting a PR. Some linters offer a --fix mode, which can be executed with the command make fix-linters (ensure markdownlint and golangci-lint tools are installed first).

• 📖 If you're introducing a user-facing feature or changing existing behavior, please ensure it's properly documented.

• 🧪 While 100% coverage isn't a requirement, we encourage unit tests for any code changes where possible.

• 🎁 If feasible, please check if an end-to-end test can be added. See README for more details.

• 🔎 If there's any flakiness in the CI tests, don't necessarily ignore it. It's better to address the issue before merging, or provide a valid reason to bypass it if fixing isn't possible (e.g., token rate limitations).

[codecov]

Codecov Report

Attention: Patch coverage is 83.33333% with 5 lines in your changes missing coverage. Please review.

Project coverage is 64.95%. Comparing base (339f294) to head (052c102).
Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/sync/queue_manager.go	76.47%	2 Missing and 2 partials ⚠️
pkg/reconciler/queue_pipelineruns.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1791      +/-   ##
==========================================
+ Coverage   64.91%   64.95%   +0.04%     
==========================================
  Files         175      175              
  Lines       13363    13387      +24     
==========================================
+ Hits         8675     8696      +21     
- Misses       4108     4110       +2     
- Partials      580      581       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[chmouel]

Added e2e test, which seems to work properly!