REGRESSION - Cannot do repo query: 'Present' query reports that historical query not supported #427

planetf1 · 2022-04-26T11:19:06Z

Testing 3.8.0-rc.0 against egeria 3.8 ('base' chart)

This occurs on EVERY SEARCH with the graph repository (default for the egeria-base chart)
ie broken.

planetf1 · 2022-04-26T12:23:24Z

This works ok with inmemory (presumably as historical queries are supported) - the failure is specifically for graph repo which is the default for the base chart (and an option for the notebooks)

davidradl · 2022-04-26T14:18:29Z

@planetf1 - it works for me using the graph repository current egeria master and the current main from egeria-react-ui. I have repository content.

davidradl · 2022-04-26T14:22:29Z

I do not understand why it failing for you. Maybe it is because you have not got any entities? does it work if you have entities? If I look for a type that does not have any entities it does not fail for me. sorry I cannot recreate this, I am not sure what I am missing.

davidradl · 2022-04-26T14:42:22Z

I have upgraded to the latest master and get the same results. If I switch to past and specify a date then it correctly gives the error popup you see.

davidradl · 2022-04-26T14:45:25Z

I see my view server in the server dropdown - I would have expected to see the view server in your server dropdown, if you had it set up the same as me. Could that be the cause of your issue? Or are you using a separate platform for the metadata server and the view server?

planetf1 · 2022-04-26T20:47:37Z

I have redeployed and get the same error. You are correct there are probably no entities yet.

The error on the ui container is:

response.data
{
  class: 'RexSearchResponse',
  relatedHTTPCode: 400,
  exceptionClassName: 'org.odpi.openmetadata.viewservices.rex.api.ffdc.RexViewServiceException',
  actionDescription: 'findEntities',
  exceptionErrorMessage: 'The repository explorer view service operation findEntities reported that a historical query cannot be issued',
  exceptionErrorMessageId: 'OMVS-REPOSITORY-EXPLORER-400-029',
  exceptionErrorMessageParameters: [
    'findEntities',
    'OMRS-METADATA-COLLECTION-501-001 OMRSMetadataInstanceStore method findEntitiesByPropertyValue for OMRS Connector org.odpi.openmetadata.adapters.repositoryservices.graphrepository.repositoryconnector.GraphOMRSMetadataCollection to repository type mds1 is not implemented'
  ],
  exceptionSystemAction: 'The system reported that the historical capability is not supported.',
  exceptionUserAction: 'Either use repositories that support historical queries or queries for the current entities.'
}

planetf1 · 2022-04-26T20:49:31Z

Whether the view server appears or not is down to the DINO configuration.

We have two standard charts - in the coco one the view server is in DINO's list. We also have an 'egeria-base' chart, and in this one the view server is not. So in that regard they are both working as designed (coded!)

planetf1 · 2022-04-26T20:52:46Z

I created a couple of assets and get the same error.

planetf1 · 2022-04-26T20:57:18Z

The only place I can see the unsupported function is when asOfTime is used ie (graph repo connector )

// findEntitiesByPropertyValue
    @Override
    public  List<EntityDetail> findEntitiesByPropertyValue(String                userId,
                                                           String                entityTypeGUID,
                                                           String                searchCriteria,
                                                           int                   fromEntityElement,
                                                           List<InstanceStatus>  limitResultsByStatus,
                                                           List<String>          limitResultsByClassification,
                                                           Date                  asOfTime,
                                                           String                sequencingProperty,
                                                           SequencingOrder       sequencingOrder,
                                                           int                   pageSize)
    throws
    InvalidParameterException,
    TypeErrorException,
    RepositoryErrorException,
    PropertyErrorException,
    PagingErrorException,
    FunctionNotSupportedException,
    UserNotAuthorizedException
    {

        final String methodName = "findEntitiesByPropertyValue";
        final String entityTypeGUIDParameterName = "entityTypeGUID";

        List<EntityDetail> entities = null;

        /*
         * Validate parameters
         */
        super.findEntitiesByPropertyValueParameterValidation(userId,
                                                             entityTypeGUID,
                                                             searchCriteria,
                                                             fromEntityElement,
                                                             limitResultsByStatus,
                                                             limitResultsByClassification,
                                                             asOfTime,
                                                             sequencingProperty,
                                                             sequencingOrder,
                                                             pageSize);


        if (asOfTime != null) {
            log.error("{} does not support asOfTime searches", methodName);

            super.reportUnsupportedOptionalFunction(methodName);
        }

As an aside the error

is arguably incorrect. The function is implemented, it just doesn't support one of the options. This information makes it to a log.error(), but not into the exception - so we should probably raise an issue on egeria base to improve the error. Though the react UI exception is accurate/complete.

However the question is why a historical query is being used when that slider is on the 'present' side

planetf1 · 2022-04-26T21:01:17Z

Checking the versions:

On quay.io:



Apr 13, 2022 |  
-- | --
  | latest was moved to SHA256 4b9b203c78cb from SHA256 c3f4c6258c9d | Wed, Apr 13, 2022 1:13 PM | Revert to SHA256 c3f4c6258c9d
  | 3.8.0-rc.0 was moved to SHA256 4b9b203c78cb fromSHA256 c3f4c6258c9d | Wed, Apr 13, 2022 1:13 PM | Revert to SHA256 c3f4c6258c9d

Apr 13, 2022	
latest was moved to SHA256 [4b9b203c78cb](https://quay.io/repository/odpi/egeria-react-ui/manifest/sha256:4b9b203c78cbfe196fd159153a48189f1301d37d7962697215110f41233a3b89)  from SHA256 [c3f4c6258c9d](https://quay.io/repository/odpi/egeria-react-ui/manifest/sha256:c3f4c6258c9de560957e1fa46c5c50eeedbfab0202ca8a58efb8dbd2d1f45313)
Wed, Apr 13, 2022 1:13 PM
Revert to SHA256 [c3f4c6258c9d](https://quay.io/repository/odpi/egeria-react-ui/manifest/sha256:c3f4c6258c9de560957e1fa46c5c50eeedbfab0202ca8a58efb8dbd2d1f45313)
3.8.0-rc.0 was moved to SHA256 [4b9b203c78cb](https://quay.io/repository/odpi/egeria-react-ui/manifest/sha256:4b9b203c78cbfe196fd159153a48189f1301d37d7962697215110f41233a3b89)  from SHA256 [c3f4c6258c9d](https://quay.io/repository/odpi/egeria-react-ui/manifest/sha256:c3f4c6258c9de560957e1fa46c5c50eeedbfab0202ca8a58efb8dbd2d1f45313)
Wed, Apr 13, 2022 1:13 PM
Revert to SHA256 [c3f4c6258c9d](https://quay.io/repository/odpi/egeria-react-ui/manifest/sha256:c3f4c6258c9de560957e1fa46c5c50eeedbfab0202ca8a58efb8dbd2d1f45313)

And from the pod on k8s

    Image:          quay.io/odpi/egeria-react-ui:3.8.0-rc.0
    Image ID:       quay.io/odpi/egeria-react-ui@sha256:4b9b203c78cbfe196fd159153a48189f1301d37d7962697215110f41233a3b89

ie they match, we are running the latest container build

And checking the source, the latest commit was indeed Wed Apr 13, so this all ties up - I think I'm running the latest code

planetf1 · 2022-04-26T21:06:12Z

Same behaviour on chrome+safari (latest versions of each)

planetf1 · 2022-04-26T21:07:07Z

My failing search is for '.*' , with my 'mds1' metadata server selected, and with everything else left as default

planetf1 · 2022-04-26T21:08:55Z

If you have access to k8s and want to reproduce:

helm repo update && helm search repo egeria  --devel
helm repo install base egeria/base --devel

planetf1 · 2022-04-26T21:26:17Z

Built and ran (prod) the UI from source, but pointed to my deployed view/mds servers (which are at 3.8 release version)

Same error occurs.

planetf1 · 2022-04-26T21:34:00Z

If it helps, here is chrome's 'copy as cURL' version - where present is selected

curl 'https://localhost:8091/servers/coco/rex/users/garygeeke/instances/entities/by-property-value' \
  -H 'Accept: application/json' \
  -H 'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8' \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: application/json' \
  -H 'Cookie: connect.sid=s%3ACAJn56RAC8CpMfDnhEZLyk9WXHgz3WxM.MUxkIZ6rx0FpyObRGGU4Px4Y2zJOPWhgKzb5BkNUKDE' \
  -H 'Origin: https://localhost:8091' \
  -H 'Referer: https://localhost:8091/coco/repository-explorer' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.41 Safari/537.36' \
  -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="101", "Google Chrome";v="101"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  --data-raw '{"serverName":"mds1","platformName":"platform","enterpriseOption":true,"searchText":".*","typeName":null,"classificationNames":[]}' \
  --compressed \
  --insecure

And this does fail with:

{"class":"RexSearchResponse","relatedHTTPCode":400,"exceptionClassName":"org.odpi.openmetadata.viewservices.rex.api.ffdc.RexViewServiceException","actionDescription":"findEntities","exceptionErrorMessage":"The repository explorer view service operation findEntities reported that a historical query cannot be issued","exceptionErrorMessageId":"OMVS-REPOSITORY-EXPLORER-400-029","exceptionErrorMessageParameters":["findEntities","OMRS-METADATA-COLLECTION-501-001 OMRSMetadataInstanceStore method findEntitiesByPropertyValue for OMRS Connector org.odpi.openmetadata.adapters.repositoryservices.graphrepository.repositoryconnector.GraphOMRSMetadataCollection to repository type mds1 is not implemented"],"exceptionSystemAction":"The system reported that the historical capability is not supported.","exceptionUserAction":"Either use repositories that support historical queries or queries for the current entities."}

When instead the toggle is set to historic the request is:


curl 'https://localhost:8091/servers/coco/rex/users/garygeeke/instances/entities/by-property-value' \
  -H 'Accept: application/json' \
  -H 'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8' \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: application/json' \
  -H 'Cookie: connect.sid=s%3ACAJn56RAC8CpMfDnhEZLyk9WXHgz3WxM.MUxkIZ6rx0FpyObRGGU4Px4Y2zJOPWhgKzb5BkNUKDE' \
  -H 'Origin: https://localhost:8091' \
  -H 'Referer: https://localhost:8091/coco/repository-explorer' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.41 Safari/537.36' \
  -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="101", "Google Chrome";v="101"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  --data-raw '{"serverName":"mds1","platformName":"platform","enterpriseOption":true,"searchText":".*","typeName":null,"classificationNames":[],"asOfTime":1650841200000}' \
  --compressed \
  --insecure

Only in this second case is 'asOfTime' specified -- as I'd expect.

This tends to suggest it is the back-end view server where the issue lies, rather than the react UI

(those two were taken from my locally running UI, but k8s backend)

planetf1 · 2022-04-26T21:41:31Z

I'm not familar with the rex/view service code, but a cursory glance seems to suggest that the asOfTime should be null, and the converted date also null, which is then passed to the graph repo, so shouldnt hit that condition .

davidradl · 2022-04-27T10:34:50Z

@planetf1 thanks for your investigation - there was a server fix I had that was not applied to master- I have raised pr odpi/egeria#6448. This pr should fix this.

planetf1 · 2022-04-27T10:35:46Z

Ok will retest and rebuild once merged

planetf1 · 2022-04-27T16:48:45Z

I have now retested. The behaviour has changed but is not fully fixed.

Using the 'egeria-base' helm chart again. Graph repo, just 2 entities in the repo.

When the UI is first launched the past/present toggle is green (to the right/present). A query works . CORRECT
Slide the toggle to the left. A query works. WRONG (note that the as of date is left as placeholder mm/dd/yyyy)
Use the date picker to select a valid date - I chose 04/27/2022 00:00. Search fails with capability not supported. CORRECT
Move toggle back to RHS/present (so no date shown). Search fails with capability not supported. WRONG

It appears as if the use of historical query is based on the mm/dd/yyyy being non-null, rather than the value of the toggle which isn't at all what one would expect.

So it's better but still incorrect.

davidradl · 2022-04-27T20:25:11Z

Just checking iI understand what you are saying.

query works
set to past, query works
set a past date , this fails
set toggle to present, this fails .

I will have a look tomorrow and make 2 more intuitive. 4. looks like a bug.

planetf1 · 2022-04-28T22:20:15Z

Testing of latest image (as of 23:00)

Load Repository Explorer - Historical query OFF, enterprise TICKED, mds1 selected (egeria-base chart)
Search (always for .*) -> works. CORRECT
Turn ON Historical query
(Minor) - Layout issues. Historical box is chopped on a smaller window - work in progress
Search works. WRONG? (Note - no date is selected, so we could argue this is ok, with blank date=present, but I think this selection is implying historical, and probably should be a form error)
Enter a date
Search fails with findEntities Error. CORRECT
Also add a time 00:00
Search fails. CORRECT
Turn off Historical Query
Search works. CORRECT
Turn on /off historical query 3 times ending with Historical query on
Search works. INCORRECT
Now regardless of position of historical query, searches continue to work regardless.
Reload web page
Regardless of position, searcg continues to work. INCORRECT
Log out, log back in again
Regardless of position, search continues to work. INCORRECT (appears as if context is preserved into a new login session:?)
Open in 'private window' -> continues as above - so context must be being kept server-side.
--
KILL POD
--
Historical query off
Search works - CORRECT
Historical query on
Search works - As above. Possible incorrect, but defensible!
Enter date/time for query
Search fails. CORRECT
Deselect enterprise
Search fails. CORRECT
Historical off
Search works. CORRECT
Historical on
Search works. INCORRECT

I suspect other combinations are possible. At some point after a few interactions it appears as if state is confused, additionally state is being kept between login sessions/reloads. So still looks quite buggy.

That being said, The basic interaction works. Historical searches are a new feature. Perhaps we just add something to the release notes to indicate it's incomplete, possibly refer to this issue, and go ahead with shipping the UI for 3.8 release.

#428 #427 fix css

davidradl · 2022-04-29T08:34:07Z

Great testing @planetf1 I appreciate it :-) I will look at it today

davidradl · 2022-04-29T13:59:46Z

I have it working - there is one quirk - the data is displaying as a full string in the date box nit as mm/dd/yyyy format. I have merged. Working through this display of the date.

planetf1 · 2022-04-29T16:22:52Z

The latest fix has addressed the date format, however the original problem has returned - See the grayed out screen behind — the slider is on the left, yet it’s doing a historical query

planetf1 · 2022-04-29T18:11:08Z

The latest change has fixed these issues - looks good for inmem & graph

planetf1 mentioned this issue Apr 26, 2022

[RELEASE] Release 3.8 odpi/egeria#6435

Closed

40 tasks

davidradl self-assigned this Apr 26, 2022

davidradl closed this as completed Apr 28, 2022

planetf1 reopened this Apr 28, 2022

davidradl added a commit that referenced this issue Apr 29, 2022

Merge pull request #430 from davidradl/git428

e72d7d7

#428 #427 fix css

planetf1 closed this as completed May 27, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

REGRESSION - Cannot do repo query: 'Present' query reports that historical query not supported #427

REGRESSION - Cannot do repo query: 'Present' query reports that historical query not supported #427

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

davidradl commented Apr 26, 2022 •

edited

Loading

davidradl commented Apr 26, 2022 •

edited

Loading

davidradl commented Apr 26, 2022

davidradl commented Apr 26, 2022 •

edited

Loading

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022 •

edited

Loading

planetf1 commented Apr 26, 2022

davidradl commented Apr 27, 2022

planetf1 commented Apr 27, 2022

planetf1 commented Apr 27, 2022

davidradl commented Apr 27, 2022

planetf1 commented Apr 28, 2022

davidradl commented Apr 29, 2022

davidradl commented Apr 29, 2022

planetf1 commented Apr 29, 2022

planetf1 commented Apr 29, 2022

REGRESSION - Cannot do repo query: 'Present' query reports that historical query not supported #427

REGRESSION - Cannot do repo query: 'Present' query reports that historical query not supported #427

Comments

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

davidradl commented Apr 26, 2022 • edited Loading

davidradl commented Apr 26, 2022 • edited Loading

davidradl commented Apr 26, 2022

davidradl commented Apr 26, 2022 • edited Loading

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022

planetf1 commented Apr 26, 2022 • edited Loading

planetf1 commented Apr 26, 2022

davidradl commented Apr 27, 2022

planetf1 commented Apr 27, 2022

planetf1 commented Apr 27, 2022

davidradl commented Apr 27, 2022

planetf1 commented Apr 28, 2022

davidradl commented Apr 29, 2022

davidradl commented Apr 29, 2022

planetf1 commented Apr 29, 2022

planetf1 commented Apr 29, 2022

davidradl commented Apr 26, 2022 •

edited

Loading

davidradl commented Apr 26, 2022 •

edited

Loading

davidradl commented Apr 26, 2022 •

edited

Loading

planetf1 commented Apr 26, 2022 •

edited

Loading