Spike: Twitter advanced search documentation updates

[lacyg4]

Problem

Users of the Twitter endpoint want more granular search capabilities. Our API supports advanced search this but its not documented. Advanced search queries can be constructed here: https://x.com/search-advanced?lang=en

We currently do not document examples of advanced search in the swagger documentation or developer documentation so people understand the full functionality of the twitter search API.

Here are some examples of advanced search queries:

curl -X 'POST' \
  'http://localhost:8080/api/v1/data/twitter/tweets/recent' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "query": "Masa Moon (from:getmasafi) until:2022-01-01 since:2006-01-01",
  "count": 10
}'

curl -X 'POST' \
  'http://localhost:8080/api/v1/data/twitter/tweets/recent' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "query": "(Masa OR CoinList)(from:getmasafi) until:2022-01-01 since:2006-01-01",
  "count": 10
}'

These developer documentation can be found here: https://help.x.com/en/using-x/x-advanced-search

Customer feedback

https://masafinance.slack.com/archives/C078JCDS81E/p1719870998918199
https://masafinance.slack.com/archives/C073K1YP8CR/p1717494858189539

Acceptance criteria:

1. Documentation Structure:

   • Create a single, dedicated "Advanced Search" section in the API documentation
   • Implement a dropdown or tabbed interface for users to select different search types TBD

2. Search Type Documentation:

   • For each search type, provide:
     • A brief description of the search type
     • The correct syntax or format to use
     • At least one clear example of usage
     • Any relevant limitations or special considerations

3. Testing:

   • Conduct user testing to ensure the documentation is easy to understand and use
   • Verify all examples are correct and functional

Advance search to support:

• Exact phrase search
• Hashtag search
• Mention search (@ symbol)
• From user search
• To user search
• Language-specific search
• Date range search
• Retweets filtering
• Minimum likes filter
• Minimum retweets filter
• Keyword exclusion (NOT operator)
• OR operator for multiple terms
• Geo-location based search
• URL inclusion search
• Question tweets filter
• Safe search mode

Possible solutions

1. Document each case in swagger as an example (expensive)
2. Document each search query and have a couple of examples in dev docs

[teslashibe]

We may want to think about renaming the /data/twitter/tweets/recent endpoint to /data/twitter/tweets

[giovaroma]

@teslashibe updated the advance search option from Twitter and the acceptance criteria, please review the list of advance search options and let me know if this looks correct or we should remove some.

[teslashibe]

@giovaroma this looks fantastic - let move forward with this.

I have added this to a release in the roadmap here: masa-finance/roadmap#2 (cc @mudler @H34D @Luka-Loncar)

I changed this to "In review" on product board added this to no-status in the protocol board and it can be groomed on Monday

[giovaroma]

Wonderful, moving tickets down the pipeline @teslashibe

[mudler]

potentially a duplicate of #362 ?

[mudler]

Exact match documentation depends on #351

[Luka-Loncar]

Exact match and advanced search should be tested first. @nolanjacobson

[mudler]

This has been done and QA'd, follow-up #459