-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use SC-generated paging links to page the audit message data #737
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Had a brief review. I'll need to review it a bit more and test it locally.
@@ -537,6 +564,7 @@ void LogError(IRestResponse response) | |||
static bool HasSucceeded(IRestResponse response) => successCodes.Any(x => response != null && x == response.StatusCode && response.ErrorException == null); | |||
|
|||
static IEnumerable<HttpStatusCode> successCodes = new[] { HttpStatusCode.OK, HttpStatusCode.Accepted, HttpStatusCode.NotModified, HttpStatusCode.NoContent }; | |||
static Regex linkExpression = new Regex(@"<([^>]+)>;\s?rel=""(\w+)"",?\s?", RegexOptions.Compiled); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't string operations simple enough for this purpose? Whenever I see a regex I cringe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately not because ,
is used as a delimiting character but can also be present in the URL, e.g. http://localhost:33333/api/messages?offsets=0,1,2,3; rel="next", http://localhost:33333/....
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd rather not have to maintain (and possibly debug) this regex. How complicated would the equivalent imperative code be?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be quite complex I believe. Not sure how to best find the commas that are actually delimiters and not part of the URL.
TryRebindMessageList(pagedResult); | ||
|
||
SearchBar.IsVisible = true; | ||
SearchBar.SetupPaging(pagedResult.TotalCount > 0 ? pagedResult.CurrentPage : 0, pagedResult.TotalCount, pagedResult.PageSize, pagedResult.Result, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is too verbose IMO. More on it down below.
endpoint: null, | ||
orderBy: orderBy, | ||
ascending: ascending); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the branching seem to be very similar and duplicated? could we get away with one? e.g.
endpoint = endpointNode ? null : endpointNode.Endpoint;
InitSearch(....)
so no branching.
TotalCount = pagedResult.TotalCount, | ||
Result = pagedResult.Result, | ||
}); | ||
SearchBar.SetupPaging(pagedResult.TotalCount > 0 ? pagedResult.CurrentPage : 0, pagedResult.TotalCount, pagedResult.PageSize, pagedResult.Result, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we simplify the SetupPaging method? too many arguments.
@@ -52,22 +51,22 @@ protected override void OnActivate() | |||
|
|||
public void GoToFirstPage() | |||
{ | |||
Parent.RefreshMessages(SelectedEndpoint, 1, SearchQuery); | |||
Parent.RefreshMessages(FirstLink, 1); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we still need the current page? Isn't the First/Prev/Last/Next link enough regardless of which page we're in? Same goes for usages below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to display the page index in the UI. The current SC unfortunately does not include the current page index in the response but will include in future. Then we'll able to drop these.
|
||
var result = GetPagedResult<StoredMessage>(request); | ||
if (result != null) | ||
if (link.StartsWith("http")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
StartWith("http", StringComparison.OrdinalIgnoreCase) ?
Why do we need to check this? The code also looks very similar. Could we do away with branching?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are two things that depend on the result of this comparison. I can replace the if
with ?
but I would end up with two ?
operators.
@@ -197,9 +198,9 @@ void AppendOrdering(IRestRequest request, string orderBy, bool ascending) | |||
request.AddParameter("direction", ascending ? "asc" : "desc", ParameterType.GetOrPost); | |||
} | |||
|
|||
void AppendPaging(IRestRequest request, int pageIndex) | |||
void AppendPaging(IRestRequest request, int pageSize) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is there anywhere where we change the page size? If it is a constant already and can't be changed, we can also remove the page sizes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It has a default value in SC (same as here, 50 messages). Do you say we should just rely on the SC default here? I think we can't do it because currently SC does not return the Page-Size
header. It will be added in future versions. So I think it is safer to enforce SI's default here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll defer to @HEskandari's review for now.
@HEskandari I've pushed fixes to the issues you pointed out and some refactorings aimed to reduce the complexity. |
Guys, can I ask you for another round of reviews? |
I can review and test it but only mid-week next week. |
@SzymonPobiega I did test it today, as promised. It seems to be working fine. I'm happy to merge this in. |
5fec6e9
to
92b572c
Compare
var ascending = order == ListSortDirection.Ascending; | ||
|
||
Model.ChangeSorting(orderBy, ascending); | ||
Model.RefreshMessages(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the reason for switching from a single operation to two here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change sorting is a local op, does not go to the server. The API now looks something like:
void Search(Endpoint endpoint, string searchQuery = null) //starts a brand new search from page 1
void NavigateToPage(string link, int pageIndex) //Continues a search using link returned from previous request
void RefreshMessages() //Starts a brand new search using values already present in the view model
else | ||
{ | ||
Parent.Search(SelectedEndpoint, SearchQuery); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How is this method invoked? I see no reference to it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That was probably some kind of a leftover. I'll remove it.
@@ -537,6 +564,7 @@ void LogError(IRestResponse response) | |||
static bool HasSucceeded(IRestResponse response) => successCodes.Any(x => response != null && x == response.StatusCode && response.ErrorException == null); | |||
|
|||
static IEnumerable<HttpStatusCode> successCodes = new[] { HttpStatusCode.OK, HttpStatusCode.Accepted, HttpStatusCode.NotModified, HttpStatusCode.NoContent }; | |||
static Regex linkExpression = new Regex(@"<([^>]+)>;\s?rel=""(\w+)"",?\s?", RegexOptions.Compiled); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd rather not have to maintain (and possibly debug) this regex. How complicated would the equivalent imperative code be?
} | ||
|
||
public PagedResult<StoredMessage> GetAuditMessages(Endpoint endpoint, string searchQuery = null, int pageIndex = 1, string orderBy = null, bool ascending = false) | ||
public PagedResult<StoredMessage> GetAuditMessages(string link) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the logic behind the two conditions in this method?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SC can return either absolute or relative URLs, depending on the version :(
{ | ||
var request = CreateMessagesRequest(); | ||
var request = CreateMessagesRequest(endpoint?.Name); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is now the only call to the CreateMessagesRequest
method, so the parameter should be made mandatory there.
|
||
public void RefreshMessages() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This method used to fork, depending on whether the selectedExplorerItem
was a ServiceControlExplorerItem
or an AuditEndpointExplorerItem
.
Is that no longer necessary?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is now handled in RefreshMessages
. Navigate uses link that SC returned in a previous search request so it does not care about initial search params.
92b572c
to
ba27778
Compare
@adamralph I've implemented your suggestions but I probably used |
FirstLink = pagedResult.FirstLink; | ||
LastLink = pagedResult.LastLink; | ||
PageSize = pagedResult.PageSize; | ||
SelfLink = selfLink; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the only reference to the SelfLink
property, i.e. it is never read, only written to. Can it be removed?
If so, the selfLink
parameter can also be removed from this method.
var ascending = order == ListSortDirection.Ascending; | ||
|
||
Model.ChangeSorting(orderBy, ascending); | ||
Model.RefreshMessages(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Continuing from https://github.com/Particular/ServiceInsight/pull/737/files#r133939640
Given this is the only call to ChangeSorting
, i.e. it is never called without then calling RefreshMessages
, would it be better to encaspulate that behind a RefreshMessages
overload with orderBy
and ascending
parameters?
@@ -537,6 +557,7 @@ void LogError(IRestResponse response) | |||
static bool HasSucceeded(IRestResponse response) => successCodes.Any(x => response != null && x == response.StatusCode && response.ErrorException == null); | |||
|
|||
static IEnumerable<HttpStatusCode> successCodes = new[] { HttpStatusCode.OK, HttpStatusCode.Accepted, HttpStatusCode.NotModified, HttpStatusCode.NoContent }; | |||
static Regex linkExpression = new Regex(@"<([^>]+)>;\s?rel=""(\w+)"",?\s?", RegexOptions.Compiled); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Continuing from https://github.com/Particular/ServiceInsight/pull/737/files#r133940544
OK, let's rewind: can you explain how the regex works?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, sure. The first part <([^>]+)>;
matches the url enclosed in <
and >
chars and a semicolon. Then there is an optional space and the rel part (rel=""(\w+)""
). In the end there is an optional comma and space which separate one link from the other.
At first I tried to split on the commas to separate links and then on the semicolon to separate URL from rel part but commas can be also present in the URL. I would have to split only on the commas which are not between <
and >
which cannot be done declaratively but requires me to go through the string, char by char, counting <
and >
.
I don't have a strong opinion and I am happy to rewrite it iteratively is you prefer this way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I made this comment on my last review at some point, but then got into a tangle so somehow it got removed:
In your previous comment, I see no mention of <
and >
. Could you please supply an example of the actual header value so I can understand better?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here's an example of the Link
header:
<http://localhost:33333/api/messages?per_page=1&page=24>; rel="last", <http://localhost:33333/api/messages?per_page=1&page=2>; rel="next"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does SC really include non-encoded commas in the URL's? They should be encoded to %2C
, then it would be a simple comma split. And semi-colons should be encoded to %3B
allowing a further semi-colon split.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not able to make sense of the regex and would much rather prefer an iterative approach.
Is it sufficient to split on the first ,
that follows a >;
? Or are there cases where there won't be any >
's?
Or what about splitting on <
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the regex really a problem here? I mean string splitting can easily get as complex as understanding a regex.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regex isn't a "problem", but none of the current maintainers feel comfortable in taking ownership of Regex. I'm happy to accept a Regex option for now so as to not block the TF, and then log an issue to change it to a non-regex approach.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The splitting would be based on the context (we can't just split at ;
because it may be a part of a URL. I am glad that the SI maintainers accepted this solution. I would love to see this simplified.
@@ -34,5 +34,8 @@ public class ServiceControlHeaders | |||
{ | |||
public const string ParticularVersion = "X-Particular-Version"; | |||
public const string TotalCount = "Total-Count"; | |||
public const string Link = "Link"; | |||
public const string PageSize = "Page-Size"; | |||
public const string PageNumber = "Page-Number"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We never use this, and instead we initialise the page number to 1 locally and then increment and decrement it locally when navigating. Is there any reason we don't read this from the response instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not yet implemented in SC. I added this and then forgot to remove when I realized we need to first introduce it to SC to make the logic in SI simpler.
|
||
public void RefreshMessages() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Continuing from https://github.com/Particular/ServiceInsight/pull/737/files#r133939821
I'm not really sure what you mean by "initial search params"", but that's probably because I don't know the difference between ServiceControlExplorerItem
and AuditEndpointExplorerItem
. 😁
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ServiceControlExplorerItem
is present there if you don't select any endpoint. If you do, you get the AuditEndpointExplorerItem
which contains the name of the endpoint to use to filter the audited messages.
ba27778
to
ae88240
Compare
@adamralph I force-pushed some changes you requested. |
var ascending = order == ListSortDirection.Ascending; | ||
|
||
Model.ChangeSorting(orderBy, ascending); | ||
Model.RefreshMessages(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Continuing from https://github.com/Particular/ServiceInsight/pull/737/files?diff=unified#r133977423
The changes have been made to MessageListViewModel
but not MessageListView
, so the project no longer compiles.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My bad. Will fix it in a second.
Marking as WIP until we confirm whether this fixes #736 or not. |
@adamralph why? Fixing of #736 (if I can confirm it) was only a side effect of this PR. |
Marked as WIP since we are investigating another approach |
Removed WIP. As discussed with @WilliamBZA removed the PageCount and replaced the absolute URI check with a more future proof one. Please review and merge. I will update the description with a few details around page count |
Fixed compilation. Fixed xaml binding for Search Fixed enabling/disabling refresh button
…on, default page size is only used in case SC doesn't return anything, NavigateToPage doesn't need to use the current page index anymore
edc938d
to
c29d866
Compare
Is this PR dependent on Particular/ServiceControl#1069? Or can it be reviewed and merged independently? |
@WilliamBZA this is independent and can be merged and reviewed. |
@Particular/serviceinsight-maintainers I've change the label to a feature as this has a material impact on the UX of the users. We also need at least one more approval before squashing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In addition to the inline comments, several things have been moved around within files, making this diff difficult to read. E.g. SearchBarViewModel.RefreshResult()
.
@HEskandari will reach out and help to clear up the diff before we go further with this.
@@ -562,6 +562,7 @@ II.2.12 <HandlesEvent />
 | |||
<s:Boolean x:Key="/Default/Environment/SettingsMigration/IsMigratorApplied/=JetBrains_002EReSharper_002EPsi_002ECSharp_002ECodeStyle_002ESettingsUpgrade_002EMigrateBlankLinesAroundFieldToBlankLinesAroundProperty/@EntryIndexedValue">True</s:Boolean> | |||
<s:Boolean x:Key="/Default/Environment/SettingsMigration/IsMigratorApplied/=JetBrains_002EReSharper_002EPsi_002ECSharp_002ECodeStyle_002ESettingsUpgrade_002EMigrateThisQualifierSettings/@EntryIndexedValue">True</s:Boolean> | |||
<s:Boolean x:Key="/Default/Environment/SettingsMigration/IsMigratorApplied/=JetBrains_002EReSharper_002EPsi_002EJavaScript_002ECodeStyle_002ESettingsUpgrade_002EJsCodeFormatterSettingsUpgrader/@EntryIndexedValue">True</s:Boolean> | |||
<s:Boolean x:Key="/Default/Environment/SettingsMigration/IsMigratorApplied/=JetBrains_002EReSharper_002EPsi_002EJavaScript_002ECodeStyle_002ESettingsUpgrade_002EJsParsFormattingSettingsUpgrader/@EntryIndexedValue">True</s:Boolean> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This appears to be unrelated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For people using resharper this settings migration will always happen
var orderBy = column.Tag as string; | ||
var ascending = order == ListSortDirection.Ascending; | ||
|
||
Model.RefreshMessages(orderBy, ascending); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd prefer to have this unrelated refactoring in a separate commit/PR.
public string Key { get; set; } | ||
|
||
public string Value { get; set; } | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change is unrelated and has been cherry-picked into #772.
@@ -69,7 +69,6 @@ | |||
<Button Content="3" cal:Message.Attach="[Event Click]=[Action GoToPreviousPage]" /> | |||
<TextBlock Text="Page" /> | |||
<TextBlock Text="{Binding CurrentPage}" /> | |||
<TextBlock Text="{Binding PageCount, StringFormat=of {0}}" /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There appears to be functional change here. E.g. we no longer display the page count, but it seems to be in the SC headers, so we should be able to preserve it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@danielmarbach thanks. I missed that. Is this a pre-existing problem? I.e. is the current method that SI uses to calculate the page count flawed in the same (or a similar) way?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@adamralph The paging doesn't work with multiple instances of ServiceControl. There is no way to know what the "last" page is when connecting to more than 1 instance.
|
||
if (pagedResult == null) | ||
{ | ||
return; | ||
} | ||
|
||
pagedResult.CurrentPage = 1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The current page is in the SC headers, so this seems redundant.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm pretty sure this is actually required. You can be on any page and then when you search something you'll be forwarded to the first page by SI. I will verify though
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Checked it is not needed. Removing it
What is the main issue with this PR? Can we maybe jump on a call and talk through it? |
4714796
to
d43ead5
Compare
I removed the endpoint properties based on #737 (comment) |
Replaced by #773 |
SC generates
Link
header which contains up to 4 links: next, prev, first and last. These are used, instead of generating the links in SI, to page through the data.The main benefit is making SI not dependent on the actual paging URL format of SC. The SI only needs to know how to initiate the search.
The actual goal is to be able to insert an intermediary between SI and SC. That intermediary could aggregate the results from multiple instances of SC without SI even knowing about its existence.
Also contains small refactorings to SC gateway.
Also solves #736
Page Count
As part of the TF work of extending SC with the multi instance mode for audits we discovered that the page count is not possible to determine upfront by doing a simple calculation based on the assumed page size. Since we went from a simplified paging approach in contrast to keep the offsets per SC instance each page can return up to
numberOfInstance * page_size
worth of data. Audits cannot be assumed to be uniformly distributed. Based on that we figured having a page size visible at all time makes no sense. The ability to page through the data is more important than knowing how many pages there are.Fixes #736