UI - Namespace filter stopped working

[dkyanakiev]

Nomad version

Nomad v1.8.1
BuildDate 2024-06-19T06:43:57Z
Revision 5022543

Issue

The namespace filter has stopped working since the upgrade to.
It seems various components on the UI have stopped behaving the same after an upgrade to 1.8.1
When looking at the request - we can see that its stuck in some limbo between filtering with the selected namespace and using a * to filter.

Network activity on version 1.7.1

Network activity on the new version

Namespace filter:

If we inspect the response object from the requests I can see how the results that the
https://<url>/v1/jobs?meta=true&namespace=monitoring&index=13866023 gets contains jobs from various other namespaces
This is not the case with the old version - where things work as they should

Also found some errors in the logs we store altho while the behaviour in the UI remains the same it doesn't seem to always log the error but I'll drop them in case they help

2024-07-10T15:14:44.771Z [ERROR] http: request failed: method=GET path="/v1/jobs/statuses?filter=%28Status%20%3D%3D%20running%29%20and%20%28Type%20%3D%3D%20system%29&namespace=monitoring&per_page=25&index=30025075" error="event not processed by enough 'filter' and 'sink' nodes" code=500
--
  | 2024-07-10T15:14:36.404Z [ERROR] http: request failed: method=GET path="/v1/jobs/statuses?filter=%28Status%20%3D%3D%20running%29&namespace=monitoring&per_page=25&index=30026684" error="event not processed by enough 'filter' and 'sink' nodes" code=500
  | 2024-07-10T15:12:40.026Z [ERROR] http: request failed: method=GET path="/v1/jobs/statuses?filter=%28Status%20%3D%3D%20running%29&namespace=monitoring&per_page=25&index=30025944" error="event not processed by enough 'filter' and 'sink' nodes" code=500
  | 2024-07-10T15:12:38.373Z [ERROR] http: request failed: method=GET path="/v1/jobs/statuses?filter=&namespace=monitoring&per_page=25&index=30025944" error="event not processed by enough 'filter' and 'sink' nodes" code=500
  | 2024-07-10T15:09:28.498Z [ERROR] http: request failed: method=GET path="/v1/jobs/statuses?filter=%28Status%20%3D%3D%20running%29&namespace=monitoring&per_page=25&index=30025158" error="event not processed by enough 'filter' and 'sink' nodes" code=500

Reproduction steps

Reproduction is a bit complicated. I've stood up a small local cluster where the issue doesn't seem to be happening.
But in our actual clusters with a lot of workloads the issue persists. So i'm going to show some limited screenshots unfortunatelly.

Expected Result

Correctly display workloads for the selected namespace.

Actual Result

Ignores the filter entirely

Nomad Server logs (if appropriate)

Nomad Client logs (if appropriate)

[philrenaud]

Hi @dkyanakiev , thanks for raising this issue. There may be a bug here, but I wanted to clarify and get a little more info.

The behaviour you're seeing with the /statuses endpoint network activity is expected: we made some significant changes in #20452 that make 2 queries:

• First, a GET to /statuses with the queryParams (?filter=) you've set via the UI or URL. This returns a list of jobs by ID (and some more details)
• Then, using that list of jobs, a POST to /statuses with a body of those job IDs. This returns what we eventually show on the jobs index page.

In this way, your namespace filter should be observed.

This process repeats whenever a blocking query becomes unblocked (a new job appears, or the status of an allocation of a shown job changes, etc.), hence all the requests (the old version of the jobs page did not live update)

---

As to the potential for a bug here: my own network requests look something like this when I select from one of my namespace options:

So, similar to your own. In my case, the jobs list filters appropriately to only the neon namespaced job. If you aren't seeing this behaviour, that is likely a bug in 1.8.1 and I apologize.

An upcoming change in #23468 makes namespaces part of the filter queryParam rather than its own independent queryParam. Among other things, this makes it so you can filter on multiple namespaces at once, but it also changes the GET /statuses query parameter format slightly. Here is the equivalent that you should see in a soon-upcoming release:

The request URL is now v1/jobs/statuses?filter=%28Namespace%20%3D%3D%20%22neon%22%29&namespace=%2A&per_page=25, please forgive the escaped characters)

I think that, if your bug as described above persists, it should hopefully be fixed with this upcoming change.

If you're able to test the above format for GETting a /statuses request with namespace set within the filter string, and you can can verify the only returned jobs are those within that namespace, this should be indicative of the change working as a fix.

[dkyanakiev]

I see. Well yea it seems like the /statuses requests are just ignoring namespace selection (non filter one).

Since the results as I mentioned are from various namespaces

Since that MR you mentioned with the change on the filters is already merged when is it planned to be pushed as a hotfix?

[philrenaud]

You should see it included in an upcoming patch release coming out very soon!

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.