Skip to content

[1.x] Allow thresholds of slow queries, jobs and requests to be customised by regex pattern #340

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
May 9, 2024

Conversation

matheus-carvalho
Copy link
Contributor

I'm facing a problem with slow jobs and requests I've already know is slower than others, so it would be useful I could customise the thresholds individually, using regex patterns.

I think it would be useful to anyone who have resources known to be slow, like report jobs, huge-data endpoints or something else, and want to monitor them with a higher threshold, not just ignore them.

I guess this changes will not break any existing feature because we still with a 'default' threshold, which will be used until some individual customisation.

@matheus-carvalho
Copy link
Contributor Author

Many checks are broken but I can't understand the reason, even with a fresh clone of 1.x branch it was breaking locally

@jessarcher
Copy link
Member

Hi @matheus-carvalho,

Thanks for the PR!

I've fixed the failing test. The Livewire payload in the test was incomplete as it was previously inconsequential, but with these changes, the payload needs to be parsed to determine the path for the threshold.

I think this could be a useful feature, but at the moment, it seems like a breaking change for existing users. To move forward and consider this feature, it would need to support the existing config file format where the threshold is a single value rather than an associate array. I'm also not sure whether this new format should be the default in the config file or just something we document.

@timacdonald timacdonald marked this pull request as draft March 18, 2024 00:44
@matheus-carvalho
Copy link
Contributor Author

Hi @jessarcher,

Thank you for fixing the tests!

About the problem with the config file, what do you think about creating a previous check? We could test if the value is a array to continue this way, in case of negative we could fallback to the current way, just using the single value.

For this change I would need to change the two methods threshold in both Concerns HasThreshold and Thresholds, and a little adjustment at usage.blade.

If you think it's viable I could submit the changes.

@jessarcher
Copy link
Member

@matheus-carvalho Sounds reasonable.

Our other concern is introducing a concept that ties cards to individual recorders with the recorder method. Cards can leverage data from multiple recorders (e.g., the Usage card), so I think the threshold method might need to accept the recorder class or threshold config as an argument.

@matheus-carvalho
Copy link
Contributor Author

Thank you for your attention @jessarcher.

Alright, this would really be a problem, I'll implement the changes to fix these points and submit the commit.

Please let me know if there are something else worrying you.

@jessarcher
Copy link
Member

Thanks @matheus-carvalho!

I haven't run it locally yet to check out the UI additions, but the only other thing I'm wondering about is whether the "Top 10 users experiencing slow endpoints" view in the Usage card should have something to indicate when there are path-specific thresholds in play. I'm imagining a scenario where a user complains about something being "slow", but it doesn't show up on that card even though it's over the default threshold because it was under the path-specific threshold. The person viewing the dashboard may not be aware of the additional configuration.

It's also up to Taylor whether we want to take this feature on, but it looks useful to me!

@matheus-carvalho
Copy link
Contributor Author

Hello @jessarcher,

I have pushed some improvements to take care of the pointed concerns.

  • threshold methods now accepts a recorder argument.
  • check if the threshold config is an array to maintain compatibility.
  • add a information icon in the actions area of usage card to tell about customised thresholds.

Here are some images from my local tests so you can take a look at UI additions.

Information icon
Slow requests
Slow outgoing requests
Slow jobs
Slow queries

@matheus-carvalho
Copy link
Contributor Author

Hello,

excuse me but I just want to know if there are something else I could do for this PR.

@timacdonald
Copy link
Member

We'll review this one shortly and see if we can move it along.

@timacdonald timacdonald changed the title Allow thresholds of slow queries, jobs and requests to be customised by regex pattern [1.x] Allow thresholds of slow queries, jobs and requests to be customised by regex pattern May 6, 2024
@timacdonald timacdonald force-pushed the customise-thresholds branch from 9f4e0de to 90ae7b9 Compare May 6, 2024 23:48
Comment on lines 30 to 35
// @phpstan-ignore argument.templateType, argument.templateType
$custom = collect($config)
->except(['default'])
->first(fn ($threshold, $pattern) => preg_match($pattern, $value));

return $custom ?? $config['default'] ?? 1_000;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notice that we have a hard-coded 1_000ms here as the default threshold when no default is specified; this matches our config defaults.

When a user changes from a single threshold to an array, if they forget the default key that the app continues to work with this default.

Not sure if we wanna keep this or just document that they need to have the default key, but this felt okay to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if you want my opinion about this, but don't breaking the app even without a default key sounds good for me. Although, if you want help documenting the default key's mandatory, count on me.

@timacdonald timacdonald force-pushed the customise-thresholds branch 10 times, most recently from 63ea9f7 to 62bee6c Compare May 8, 2024 07:03
@timacdonald timacdonald force-pushed the customise-thresholds branch from 62bee6c to 71e1590 Compare May 8, 2024 07:06
<button title="{{ $message }}" @click="alert('{{ str_replace("\n", '\n', $message) }}')">
<button title="{{ $message }}" @click="alert(@js($message))">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Standardising use of the @js directive, which I've used in some of the changes in this PR.

@timacdonald
Copy link
Member

timacdonald commented May 9, 2024

Some UI shots here.

I've decided to keep this pretty simple and only change the UI based on the threshold config type. Array = per item threshold reporting. Integer = Global type threshold.

We could have done fancy stuff checking if there was more than one threshold value for the all the listed items - but I didn't like the idea that the UI would change if an entry came into the table that had a different threshold while you were watching it.

Slow requests

'threshold' => 1000,
Screenshot 2024-05-09 at 10 46 38
'threshold' => [
    'default' => 1000,
    '#pattern#' => 10,
],
Screenshot 2024-05-09 at 10 46 18

Application usage card

'threshold' => 1000,
Screenshot 2024-05-09 at 10 46 46
'threshold' => [
    'default' => 1000,
    '#pattern#' => 10,
],
Screenshot 2024-05-09 at 10 47 05 Screenshot 2024-05-09 at 10 47 11

Slow queries

'threshold' => 1000,
Screenshot 2024-05-09 at 10 50 18
'threshold' => [
    'default' => 1000,
    '#pattern#' => 10,
],
Screenshot 2024-05-09 at 10 49 59

Outgoing requests

'threshold' => 1000,
Screenshot 2024-05-09 at 10 52 30
'threshold' => [
    'default' => 1000,
    '#pattern#' => 10,
],
Screenshot 2024-05-09 at 10 53 06

Slow Jobs

'threshold' => 1000,
Screenshot 2024-05-09 at 10 55 15
'threshold' => [
    'default' => 1000,
    '#pattern#' => 10,
],
Screenshot 2024-05-09 at 10 55 48

@timacdonald timacdonald marked this pull request as ready for review May 9, 2024 01:05
Copy link
Member

@timacdonald timacdonald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fan of this feature. Thanks!

@taylorotwell taylorotwell merged commit 7190f4a into laravel:1.x May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants