Subsocial squid

Indexer for work with Subsocial, Soonsocial and xSocial chains.

Quick running

# 1. Update Squid SDK and install dependencies
npm run update
npm ci
# 2. Compile typescript files
make build
# 3. Start target Postgres database and detach
make up

# 4. Set env variable for specifying target blockchain
export CHAIN=subsocial # if you need rung indexer for subsocial network
# or
export CHAIN=soonsocial # if you need rung indexer for soonsocial network

# 5. Start the processor
make process
# 6. The command above will block the terminal
#    being busy with fetching the chain data,
#    transforming and storing it in the target database.
#
#    To start the graphql server open the separate terminal
#    and run
make serve

Defined events in squid

  PostCreated
  PostDeleted # synthetic
  PostUpdated
  PostShared # synthetic
  PostMoved
  PostFollowed # synthetic
  PostUnfollowed # synthetic
  PostReactionCreated
  PostReactionUpdated
  PostReactionDeleted

  SpaceCreated
  SpaceUpdated
  SpaceFollowed
  SpaceUnfollowed
  SpaceOwnershipTransferAccepted

  AccountFollowed
  AccountUnfollowed
  ProfileUpdated

  CommentCreated # synthetic
  CommentDeleted # synthetic
  CommentUpdated # synthetic
  CommentShared # synthetic
  CommentReactionCreated # synthetic
  CommentReactionUpdated # synthetic
  CommentReactionDeleted # synthetic
  CommentReplyCreated # synthetic
  CommentReplyDeleted # synthetic
  CommentReplyUpdated # synthetic
  CommentReplyShared # synthetic
  CommentReplyReactionCreated # synthetic
  CommentReplyReactionUpdated # synthetic
  CommentReplyReactionDeleted # synthetic

Short info about indexer multi-chain structure

1. Chain configs/endpoints can be configured in config.ts file in appropriate folder of necessary chain:
   • Subsocial - src/chains/subsocial/config.ts
   • Soonsocial - src/chains/soonsocial/config.ts
   • xSocial - src/chains/xSocial/config.ts
2. Necessary events/calls/storage calls can be configured for each chain in appropriate file in /typegen folder.
3. Command make typegen uses custom shel script which generates types for each chain defined in /typegen folder.
4. Chain sensitive logic is implemented in sub-folder with appropriate name in /src/chains folder. It's required because each chain has it's own bunch of specs and as result it's own autogenerated types. Chain sensitive logic:
   • parsing events data
   • parsing calls data
   • making storage calls
5. Indexer can be deployed to Aquarium to 2 different squids with 2 separate deployment manifests:
   • Subsocial - ./squid-subsocial.yaml
   • Soonsocial - ./squid-soonsocial.yaml
   • xSocial - ./squid-xsocial.yaml

Search API

The Indexer GraphQL API has a search query (searchQuery) with various parameters for full text search and filtering.

Search query arguments:

• indexes?: <string> [ default: all ] - search within all indices, or a specific index (all | spaces | posts)
• limit?: <number> [ default: 10 ] - the number of search results per page.
• offset?: <number | null> [ default: 0 ] - the offset of the search results.
• q?: <string> [ default: * ] - the search query.
• spaceId?: <string> - filter the search results by the provided spaceId.
• tags?: <string[]> - filter the search results by the provided tags.

All arguments listed above can be used together in any combination, except for spaceId + indexes:spaces.

Search query results:

• err - the search result if an error occurred. It can be null if no error occurred.

  • reason: <string> - the text message of the error.
  • status: <number> - the status code of the error.

• hits - The list of search results.

  • _content - the source of the document.
    • name: <string> - the value of the name field (actual only for Space entity).
    • about: <string> - the value of the about field (actual only for Space entity).
    • username: <string> - the value of the username field (actual only for Space entity).
    • title: <string> - the value of the title field (actual only for Post entity).
    • body: <string> - the value of the body field (actual only for Post entity).
    • spaceId: <string> - the value of the spaceId field (actual only for Post entity).
    • tags: <string[]> - a list of the tags.
  • _id: <string> - the document ID (equal to the on-chain entity's ID).
  • _index: <string> - index particular document is located in.
  • _score: <number> - the search score of a particular document.

• total - the total metadata for a particular search request.

  • limit: <string> - number of search results per page that was used for this particular search request.
  • maxScore: <string> - the maximum score within the scope of this particular search's results.
  • offset: <string> - the page offset that was used for this particular search request.
  • totalResults: <string> - the total number of results matched to this particular search request.

More detailed information about the search API's schema structure can be found in the appropriate model file.

Analytics API

The Indexer GraphQL API has a analytics queries with various parameters.

Active users total number :: activeUsersTotalCount

Query

The API query returns the count of users who have created at least one post within a specified period.

query MyQuery($from: String!, $to: String!) {
    activeUsersTotalCount(from: $from, to: $to){
        account_count
    }
}

# Variables:
{
    from: "2023-04-13T09:38:00.027Z",
    to: "2023-07-13T09:38:00.027Z",
}

Active users total number with filters :: activeUsersTotalCountWithFilters

Query

The API query returns the count of users who have created specified number of posts within a specified period.

query MyQuery(
    $from: String!,
    $to: String!,
    $total_min_posts_number: Int!,
    $exclude_body: [String]
) {
    activeUsersTotalCountWithFilters(
        from: $from,
        to: $to,
        total_min_posts_number: $total_min_posts_number,
        exclude_body: $exclude_body
    ){
        account_count
    }
}

# Variables:
{
    from: "2023-04-13T09:38:00.027Z",
    to: "2023-07-13T09:38:00.027Z",
    total_min_posts_number: 3,
    exclude_body: ["Hi", "Hello there"]
}

User retention count :: userRetentionCount

Query

The API query returns the count of Accounts based on the following rules:

1. The first post created by an account must fall within a specific timeframe (full_query_range_from and full_query_range_to), which represents the query timeframe.
2. The total count of posts created by an account within the query timeframe must exceed a specified number (total_min_posts_number).
3. The account must create at least a specified number of messages (first_range_min_posts_number) within the first time range (first_range_from and first_range_to).
4. The account must create at least a specified number of messages (last_range_min_posts_number) within the second time range (last_range_from and last_range_to).
5. Posts that have the field body equal to any of the restricted words or sentences (exclude_body) must be excluded from the calculation."

query MyQuery(
  $full_query_range_from: String!,
  $full_query_range_to: String!,
  $first_range_from: String!,
  $first_range_to: String!,
  $last_range_from: String!,
  $last_range_to: String!,
  $total_min_posts_number: Int!,
  $first_range_min_posts_number: Int!,
  $last_range_min_posts_number: Int!,
  $exclude_body: [String!]!,
) {
  userRetentionCount(
    full_query_range_from: $full_query_range_from,
    full_query_range_to: $full_query_range_to,
    first_range_from: $first_range_from,
    first_range_to: $first_range_to,
    last_range_from: $last_range_from,
    last_range_to: $last_range_to,
    total_min_posts_number: $total_min_posts_number,
    first_range_min_posts_number: $first_range_min_posts_number,
    last_range_min_posts_number: $last_range_min_posts_number,
    exclude_body: $exclude_body
) {
    retention_count
  }
}

# Variables:

{
    "full_query_range_from": "2023-07-13T09:38:00.027Z",
    "full_query_range_to": "2023-07-17T15:31:16.677Z",
    "first_range_from": "2023-07-13T09:38:00.027Z",
    "first_range_to": "2023-07-15T09:38:00.027Z",
    "last_range_from": "2023-07-16T09:38:00.027Z",
    "last_range_to": "2023-07-17T15:31:16.677Z",
    "total_min_posts_number": 5,
    "first_range_min_posts_number": 3,
    "last_range_min_posts_number": 2,
    "exclude_body": ["Hello"]
}