cas -541: update search endpoint #546

0xmovses · 2022-09-28T16:38:56Z

•adds method appendFiltersToResponse
•adds necessary fields onto search response
•extends SQL query
•adds constructDynamicSQL method for dynamic filter params
•moves community models sql queries into constants
•creates new generalised model function GetDefaultCommunities for both homepage SQL and default search SQL ( this felt like the right thing to do as they use similar SQL queries)

*Know Issue
if the search query contains a space, and its url encoded with a % I get a 400 back. This seems to be something to do with mux https://github.com/gorilla/mux/issues/77 I have tried the various fixes on the repo but so far none of the solutions working

linear · 2022-09-28T16:38:58Z

CAS-541

germanurrus · 2022-10-03T03:59:03Z

It looks like the default search is not filtering correctly:

The URL is:
http://localhost:5001/communities/search/defaultFeatured?start=0&count=10
The expected defaultFeatured should make filters to return only filters for the feature ones, also the all filter is gone.
I only have one feature community then dao should be 1 and the all filter should exist with one
Also when sending a filter that has 0 the result is the same, and the response brings the one feature community.
(Note: I've added a category to the Flow community manually and it's dao and also I have created a second community using dao)

{
    "filters": [
        {
            "text": "dao",
            "amount": 2
        },
        {
            "text": "social",
            "amount": 0
        },
        {
            "text": "protocol",
            "amount": 0
        },
        {
            "text": "creator",
            "amount": 0
        },
        {
            "text": "nft",
            "amount": 0
        },
        {
            "text": "collector",
            "amount": 0
        }
    ],
    "results": {
        "data": [
            {
                "id": 1,
                "name": "Flow",
                "category": "dao",
                "categoryCount": 2,
                "body": "Vote on Flow Validators",
                "timestamp": "",
                "compositeSignatures": null,
                "creatorAddr": ""
            }
        ],
        "start": 0,
        "count": 1,
        "totalRecords": 1,
        "next": -1
    }
}

germanurrus · 2022-10-03T04:23:12Z

also when sending text with space the front gets a 404:

0xmovses · 2022-10-03T12:14:34Z

The all field has been added

I've adjusted the SQL;
now when I do
http://localhost:5001/communities/search/defaultFeatured?start=0&count=10

I get :

{
    "filters": [
        {
            "text": "dao",
            "amount": 0
        },
        {
            "text": "social",
            "amount": 0
        },
        {
            "text": "protocol",
            "amount": 0
        },
        {
            "text": "creator",
            "amount": 1
        },
        {
            "text": "nft",
            "amount": 0
        },
        {
            "text": "collector",
            "amount": 0
        },
       {
            "text": "all",
            "amount":  1
        }
    ],
    "results": {
        "data": [
            {
                "id": 114,
                "name": "rich",
                "category": "creator",
                "categoryCount": 1,
                "body": "<html>something</html>",
                "timestamp": "",
                "compositeSignatures": null,
                "creatorAddr": ""
            }
        ],
        "start": 0,
        "count": 1,
        "totalRecords": 1,
        "next": -1
    }
}

URL encoded Special characters I am blocked on due to known issue with mux library. I've tried various solutions suggested on their issues page but none have worked so for.

germanurrus · 2022-10-03T15:55:39Z

The all field has been added

I've adjusted the SQL; now when I do http://localhost:5001/communities/search/defaultFeatured?start=0&count=10

I get :

{
    "filters": [
        {
            "text": "dao",
            "amount": 0
        },
        {
            "text": "social",
            "amount": 0
        },
        {
            "text": "protocol",
            "amount": 0
        },
        {
            "text": "creator",
            "amount": 1
        },
        {
            "text": "nft",
            "amount": 0
        },
        {
            "text": "collector",
            "amount": 0
        },
       {
            "text": "all",
            "amount":  1
        }
    ],
    "results": {
        "data": [
            {
                "id": 114,
                "name": "rich",
                "category": "creator",
                "categoryCount": 1,
                "body": "<html>something</html>",
                "timestamp": "",
                "compositeSignatures": null,
                "creatorAddr": ""
            }
        ],
        "start": 0,
        "count": 1,
        "totalRecords": 1,
        "next": -1
    }
}

URL encoded Special characters I am blocked on due to known issue with mux library. I've tried various solutions suggested on their issues page but none have worked so for.

For some reason this is not working on my side, I still get the same filters are before.

Just for clarification, then the defaultFeatured is sent as query search param, the frontend is expecting filters to be populated with the amount calculated based the featured communities returned.
When a text search is entered, filters should adjust on the result based on the search test.

The All filter is always present, with the total amount of communities returned based on the search text or the defaultFeatured.
When a filter is pressed, results changes but filters are the same and are kept based on the search text

0xmovses · 2022-10-03T18:39:30Z

I just pushed again and I think there was one change not pushed to the branch; could you try pulling again?

I just tested again and it seems to be working as expected with all filter always present, no duplicate dao filter counts etc. defaultFeatured only returns communities with is_featured column set to true.

I just did http://localhost:5001/communities/search/defaultFeatured?start=0&count=10

and get returned

{
    "filters": [
        {
            "text": "dao",
            "amount": 0
        },
        {
            "text": "social",
            "amount": 0
        },
        {
            "text": "protocol",
            "amount": 0
        },
        {
            "text": "creator",
            "amount": 2
        },
        {
            "text": "nft",
            "amount": 0
        },
        {
            "text": "collector",
            "amount": 0
        },
        {
            "text": "all",
            "amount": 2
        }
    ],
    "results": {
        "data": [
            {
                "id": 114,
                "name": "rich",
                "category": "creator",
                "categoryCount": 2,
                "body": "<html>something</html>",
                "timestamp": "",
                "compositeSignatures": null,
                "creatorAddr": ""
            },
            {
                "id": 115,
                "name": "rich",
                "category": "creator",
                "categoryCount": 2,
                "body": "<html>something</html>",
                "timestamp": "",
                "compositeSignatures": null,
                "creatorAddr": ""
            }
        ],
        "start": 0,
        "count": 2,
        "totalRecords": 2,
        "next": -1
    }
}

germanurrus · 2022-10-03T22:24:46Z

Filters for featured communities are not working, and filtering returns the same result for feature communities:

In the image above, there should not be any results since I'm selecting a pill with no results (maybe if the pill is 0 make it not clickable) but still the backend should return 0 results

When entering text with spaces, the url is not found and we get a 404 as a response:

Filters are not working on when results are returned on search, in the example below we should see only one result:

germanurrus

Make comments about three bugs found on filtering. It's possible to test using the frontend PR #542

jbluks · 2022-10-05T17:25:19Z

This PR will now be handled by this: #574

0xmovses and others added 5 commits September 27, 2022 22:12

bugfix search (#543)

e3cfd70

updated sql

71f59dc

updated scan func for search

3b157aa

added appending functions and parsing methods

31009ec

changed response json

3511ebd

0xmovses added 9 commits September 28, 2022 20:04

added featured default

1998241

changed default string param

ffd3c26

changed url param parse

240a190

updated sql

e98527e

updated community Struct

3715eae

updaed logic for filters and category count

41169ad

added constants for sql queries made default func generalised

0fbd03b

fixed bug with defaultFeatured

a1191b1

created constructDynamicSql func for filter search

8f9d9dc

0xmovses changed the title ~~cas -541: update search endpoint WIP~~ cas -541: update search endpoint Sep 30, 2022

0xmovses added 8 commits September 30, 2022 13:19

adjusted formatting of sql

28404a0

fixed defaultFeatured sql scan

52f2ec6

added case handle #UNFILTERED for no filters

0ec939c

refactored empty filters case

38acf4b

fixed bug

3ae70b8

added categories always filled in response

0a5b3d4

debugged defaultFeatured dao count

595c28c

removed logs

543dcff

adjusted SQL for defatult

4c38bd7

added all field

f1c587a

dbslone requested review from jbluks and germanurrus October 3, 2022 21:20

germanurrus suggested changes Oct 3, 2022

View reviewed changes

jbluks closed this Oct 5, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cas -541: update search endpoint #546

cas -541: update search endpoint #546

0xmovses commented Sep 28, 2022 •

edited

Loading

linear bot commented Sep 28, 2022

germanurrus commented Oct 3, 2022 •

edited

Loading

germanurrus commented Oct 3, 2022

0xmovses commented Oct 3, 2022 •

edited

Loading

germanurrus commented Oct 3, 2022

0xmovses commented Oct 3, 2022

germanurrus commented Oct 3, 2022

germanurrus left a comment •

edited

Loading

jbluks commented Oct 5, 2022

cas -541: update search endpoint #546

cas -541: update search endpoint #546

Conversation

0xmovses commented Sep 28, 2022 • edited Loading

linear bot commented Sep 28, 2022

germanurrus commented Oct 3, 2022 • edited Loading

germanurrus commented Oct 3, 2022

0xmovses commented Oct 3, 2022 • edited Loading

germanurrus commented Oct 3, 2022

0xmovses commented Oct 3, 2022

germanurrus commented Oct 3, 2022

germanurrus left a comment • edited Loading

Choose a reason for hiding this comment

jbluks commented Oct 5, 2022

0xmovses commented Sep 28, 2022 •

edited

Loading

germanurrus commented Oct 3, 2022 •

edited

Loading

0xmovses commented Oct 3, 2022 •

edited

Loading

germanurrus left a comment •

edited

Loading