Feat/142 collection stats #170

alko89 · 2023-04-12T17:30:16Z

Thank you for your contribution to the KodaDot Indexer.
👇 _ Let's make a quick check before the contribution.

PR type

Bugfix
Feature
Refactoring

What's new?

PR closes Resolver for stats of collection #142
This PR is required for Use new collection card on profile as we have on explorer nft-gallery#5032

Before submitting Pull Request, please make sure:

My contribution builds clean without any errors or warnings
I've merged recent default branch -- main and I've no conflicts
I've tried to respect high code quality standards
I've didn't break any original functionality
I've posted a screenshot of demonstrated change in this PR

Optional

I've run the indexer and it hasn't failed
I've changed schema and performed a migration
I found edge cases

Screenshot

My contribution is a resolver so I have pic from playground

alko89 · 2023-04-12T17:53:28Z

@vikiival lmk if this is ok, then I can also charge for kodadot/loligo#10

vikiival · 2023-04-12T18:12:30Z

Hello please check the PR that was opened by @Matehoo

feat: Add stats for collections rubick#236

src/mappings/index.ts

alko89 · 2023-04-12T19:33:57Z

Hello please check the PR that was opened by @Matehoo

Ohh didn't notice there should also be distribution, floor and ownerCount. Was wondering where this came from and noticed it mentioned in kodadot/loligo#10 😅 I'll add them tomorrow

vikiival · 2023-04-12T19:53:18Z

Hello please check the PR that was opened by @Matehoo

Ohh didn't notice there should also be distribution, floor and ownerCount. Was wondering where this came from and noticed it mentioned in kodadot/loligo#10 😅 I'll add them tomorrow

No worries, everything should be explained there. If something is not clear happy to explain

src/mappings/index.ts

Matehoo · 2023-04-17T11:56:41Z

@vikiival @alko89 What is the status on this PR?

vikiival · 2023-04-17T12:33:12Z

What is the status on this PR?

Feel free to do a review, if it's same as yours let's merge it

Matehoo

Some tiny cosmetic stuff, LGTM

src/mappings/utils/entity.ts

src/server-extension/resolvers/index.ts

.gitignore

vikiival · 2023-04-18T12:47:01Z

src/model/generated/nftEntity.model.ts

+ static createQueryBuilder(arg0: string) {
+ throw new Error('Method not implemented.')
+ }


was this generated by squid @alko89 ?

Matehoo · 2023-04-20T06:44:11Z

@vikiival @alko89 ETA on merge?

yangwao · 2023-04-20T11:00:07Z

@vikiival

roiLeo

Doesn't work for me on Rococo

Here is log output:

✔  success   [BURN] 3647975502-8 by bXjMVEdCzhjYbPc5rJk7NvPSDRiiNuCT3GdqohFp3wdpzJL6Z
FATAL sqd:processor QueryFailedError: syntax error at or near "undefined"
  at PostgresQueryRunner.query (.../snek/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:211:19)
  at processTicksAndRejections (node:internal/process/task_queues:96:5)
  at async DataSource.query (.../snek/node_modules/typeorm/data-source/DataSource.js:341:20)
  at async calculateCollectionOwnerCountAndDistribution (.../snek/lib/mappings/utils/entity.js:63:22)
  at async handleTokenBurn (.../snek/lib/mappings/index.js:188:42)
  at async HandlerRunner.processBlock (.../snek/node_modules/@subsquid/substrate-processor/lib/processor/handlerProcessor.js:505:25)
  at async .../snek/node_modules/@subsquid/typeorm-store/lib/database.js:157:13
  at async EntityManager.transaction (.../snek/node_modules/typeorm/entity-manager/EntityManager.js:72:28)
  at async FullTypeormDatabase.runTransaction (.../snek/node_modules/@subsquid/typeorm-store/lib/database.js:155:9)
  at async FullTypeormDatabase.transact (.../snek/node_modules/@subsquid/typeorm-store/lib/database.js:64:24)
  query:
          SELECT COUNT(DISTINCT current_owner) AS distribution,
                COUNT(current_owner) AS owner_count
          undefined
          FROM nft_entity
          WHERE collection_id = '3647975502-8'
          undefined

  driverError: {"length":99,"name":"error","severity":"ERROR","code":"42601","position":"103","file":"scan.l","line":"1176","routine":"scanner_yyerror","stack":"error: syntax error at or near \"undefined\"\n    at Parser.parseErrorMessage (.../snek/node_modules/pg-protocol/dist/parser.js:287:98)\n    at Parser.handlePacket (.../snek/node_modules/pg-protocol/dist/parser.js:126:29)\n    at Parser.parse (.../snek/node_modules/pg-protocol/dist/parser.js:39:38)\n    at Socket.<anonymous> (.../snek/node_modules/pg-protocol/dist/index.js:11:42)\n    at Socket.emit (node:events:513:28)\n    at addChunk (node:internal/streams/readable:315:12)\n    at readableAddChunk (node:internal/streams/readable:289:9)\n    at Socket.Readable.push (node:internal/streams/readable:228:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)"}
  length: 99
  severity: ERROR
  code: 42601
  position: 103
  file: scan.l
  line: 1176
  routine: scanner_yyerror
  blockHeight: 750082
  blockHash: 0x429b514c452a93b57db272bcc16ebe353c1edbf23a0181a6abcac9aed4f89de2
make: *** [process] Error 1

vikiival · 2023-04-20T11:46:37Z

I've run the indexer and it hasn't failed

@alko89 please make sure that it runs

roiLeo · 2023-04-20T11:58:26Z

src/mappings/utils/entity.ts

+ const query = `
+ SELECT COUNT(DISTINCT current_owner) AS distribution,
+ COUNT(current_owner) AS owner_count
+ ${
+ newOwner
+ && `
+ ,(SELECT max(CASE
+ WHEN current_owner = '${newOwner}' THEN 0
+ ELSE 1
+ END)
+ FROM nft_entity) AS adjustment
+ `
+} 
+ FROM nft_entity
+ WHERE collection_id = '${collectionId}'
+ ${newOwner && `AND current_owner != '${originalOwner}'`}
+ `;


This is difficult to read, I suggest this workaround (but not so pretty)

Suggested change

const query = `

SELECT COUNT(DISTINCT current_owner) AS distribution,

COUNT(current_owner) AS owner_count

${

newOwner

&& `

,(SELECT max(CASE

WHEN current_owner = '${newOwner}' THEN 0

ELSE 1

END)

FROM nft_entity) AS adjustment

`

}

FROM nft_entity

WHERE collection_id = '${collectionId}'

${newOwner && `AND current_owner != '${originalOwner}'`}

`;

let query;

if (newOwner && originalOwner) {

query = `SELECT

COUNT(DISTINCT current_owner) AS distribution,

COUNT(current_owner) AS owner_count,

(

SELECT max(

CASE

WHEN current_owner = '${newOwner}' THEN 0

ELSE 1

END

)

FROM nft_entity

) AS adjustment

FROM nft_entity

WHERE collection_id = '${collectionId}'

AND current_owner != '${originalOwner}'`;

}

query = `SELECT

COUNT(DISTINCT current_owner) AS distribution,

COUNT(current_owner) AS owner_count

FROM nft_entity

WHERE collection_id = '${collectionId}'`;

after this change build still not ok

FATAL sqd:processor AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value: (0, assert_1.default)(this.isV81)

pfff leeeet me fix that

new event version?

Fixed in #173

@roiLeo did you maybe forgot to rebuild? I've just run make process after building and didn't encounter the error. I applied your suggestion.

% make process ... ⬤ debug metadata: ipfs://ipfs/bafkreigzmr4zy7orconzhi2a6xofr7p6bvx2budsiowgi5kxu3qhcykns4 ✔ success [MINT] 2653871371-4 ☐ pending [LIST]: 1923312 ⬤ debug list: { "caller": "bXivCveEn3hMf4Cwt4q8uD7JQ2e58J9Mg1dbrgWvTNZZegR1p", "blockNumber": "1923312", "timestamp": "2023-04-12T02:44:48.181Z", "collectionId": "2653871371", "sn": "4", "price": "1000000" } ✔ success [LIST] 2653871371-4 by bXivCveEn3hMf4Cwt4q8uD7JQ2e58J9Mg1dbrgWvTNZZegR1p} for 1000000 ☐ pending [ASSET REGISTER]: 1931806 ✔ success [ASSET REGISTER]: by 1000021 is Calamari ☐ pending [BURN]: 1953949 ⬤ debug burn: { "caller": "bXmPf7DcVmFuHEmzH3UX8t6AUkfNQW8pnTeXGhFhqbfngjAak", "blockNumber": "1953949", "timestamp": "2023-04-20T15:05:36.153Z", "collectionId": "1", "sn": "95" } ✔ success [BURN] 1-95 by bXmPf7DcVmFuHEmzH3UX8t6AUkfNQW8pnTeXGhFhqbfngjAak ℹ info [CACHE UPDATE] PASSED TIME - 24296.799783333332 MINS ℹ info [METADATA CACHE UPDATE] PASSED TIME - 24296.799783333332 MINS ℹ info [MISSING METADATA] - NONE ✔ success [METADATA CACHE UPDATE] ✔ success [CACHE UPDATE] 19:11:42 INFO sqd:processor 1954261 / 1954262, rate: 75731 blocks/sec, mapping: 26 blocks/sec, 73 items/sec, ingest: 470 blocks/sec, eta: 0s 19:11:42 INFO sqd:processor 1954262 / 1954262, rate: 75727 blocks/sec, mapping: 26 blocks/sec, 73 items/sec, ingest: 445 blocks/sec, eta: 0s 19:11:43 INFO sqd:processor 1954263 / 1954263, rate: 73026 blocks/sec, mapping: 26 blocks/sec, 73 items/sec, ingest: 424 blocks/sec, eta: 0s 19:12:08 INFO sqd:processor 1954264 / 1954264, rate: 37387 blocks/sec, mapping: 26 blocks/sec, 73 items/sec, ingest: 405 blocks/sec, eta: 0s 19:12:33 INFO sqd:processor 1954265 / 1954265, rate: 25127 blocks/sec, mapping: 26 blocks/sec, 73 items/sec, ingest: 406 blocks/sec, eta: 0s

…-stats

Co-authored-by: roiLeo <medina.leo42@gmail.com>

…feat/142-collection-stats

roiLeo

✅ process wfm
sqd:processor 1956220 / 1956220, rate: 5786 blocks/sec, mapping: 5 blocks/sec, 13 items/sec, ingest: 22 blocks/sec, eta: 0s

but:

query MyQuery {
  collectionEntities(limit: 5, orderBy: distribution_DESC) {
    ownerCount
    supply
    volume
    highestSale
    floor
    distribution
  }
}

{
  "data": {
    "collectionEntities": [
      {
        "ownerCount": 0,
        "supply": 0,
        "volume": "0",
        "highestSale": "0",
        "floor": "0",
        "distribution": 0
      },
      {
        "ownerCount": 0,
        "supply": 0,
        "volume": "0",
        "highestSale": "0",
        "floor": "0",
        "distribution": 0
      },
      {
        "ownerCount": 0,
        "supply": 0,
        "volume": "0",
        "highestSale": "0",
        "floor": "0",
        "distribution": 0
      },
      {
        "ownerCount": 0,
        "supply": 0,
        "volume": "0",
        "highestSale": "0",
        "floor": "0",
        "distribution": 0
      },
      {
        "ownerCount": 0,
        "supply": 0,
        "volume": "0",
        "highestSale": "0",
        "floor": "0",
        "distribution": 0
      }
    ]
  }
}

same for ownerCount_DESC, works with volume.
Could you take a look?

alko89 · 2023-04-23T11:04:52Z

@roiLeo fixed, used event id instead of collection id in the query

roiLeo

Nice work!

✅ wfm

alko89 added 3 commits April 12, 2023 19:03

feat: Update collection entity and add migration

1239dbc

fix: migration

ef07e3c

fix: remove comment

925dd4c

vikiival reviewed Apr 12, 2023

View reviewed changes

src/mappings/index.ts Outdated Show resolved Hide resolved

fix: add floor, distribution and owner count

78a9ea2

vikiival reviewed Apr 15, 2023

View reviewed changes

src/mappings/index.ts Outdated Show resolved Hide resolved

fix: don't update ownerCount and distribution on mint

b48e6ec

Matehoo mentioned this pull request Apr 17, 2023

feat: Add volume sort to ExploreSort kodadot/nft-gallery#5699

Closed

18 tasks

Matehoo approved these changes Apr 17, 2023

View reviewed changes

src/mappings/utils/entity.ts Outdated Show resolved Hide resolved

src/server-extension/resolvers/index.ts Outdated Show resolved Hide resolved

src/server-extension/resolvers/index.ts Outdated Show resolved Hide resolved

fix: remove comment and shorten result query

d043805

yangwao requested a review from vikiival April 18, 2023 12:26

vikiival reviewed Apr 18, 2023

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

vikiival reviewed Apr 18, 2023

View reviewed changes

Apply suggestions from code review

fd8f80f

vikiival approved these changes Apr 18, 2023

View reviewed changes

yangwao mentioned this pull request Apr 20, 2023

Use new resolver for stats in collection view kodadot/nft-gallery#5737

Closed

roiLeo suggested changes Apr 20, 2023

View reviewed changes

roiLeo reviewed Apr 20, 2023

View reviewed changes

alko89 and others added 3 commits April 20, 2023 18:34

Merge remote-tracking branch 'upstream/main' into feat/142-collection…

cf7b2f7

…-stats

Update src/mappings/utils/entity.ts

e5f7530

Co-authored-by: roiLeo <medina.leo42@gmail.com>

Merge remote-tracking branch 'origin/feat/142-collection-stats' into …

e298076

…feat/142-collection-stats

roiLeo approved these changes Apr 21, 2023

View reviewed changes

fix: use correct id for query and fix floor

cec0410

roiLeo approved these changes Apr 24, 2023

View reviewed changes

vikiival merged commit 71ca4fd into kodadot:main Apr 24, 2023

This was referenced Aug 9, 2023

Use new collection card on profile as we have on explorer kodadot/nft-gallery#5032

Closed

Profile v0.9 kodadot/nft-gallery#6663

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat/142 collection stats #170

Feat/142 collection stats #170

alko89 commented Apr 12, 2023

alko89 commented Apr 12, 2023

vikiival commented Apr 12, 2023

alko89 commented Apr 12, 2023 •

edited

Loading

vikiival commented Apr 12, 2023

Matehoo commented Apr 17, 2023 •

edited

Loading

vikiival commented Apr 17, 2023

Matehoo left a comment

vikiival Apr 18, 2023

Matehoo commented Apr 20, 2023

yangwao commented Apr 20, 2023

roiLeo left a comment

vikiival commented Apr 20, 2023

roiLeo Apr 20, 2023

roiLeo Apr 20, 2023

vikiival Apr 20, 2023

roiLeo Apr 20, 2023

vikiival Apr 20, 2023

alko89 Apr 20, 2023 •

edited

Loading

roiLeo left a comment •

edited

Loading

alko89 commented Apr 23, 2023

roiLeo left a comment

Feat/142 collection stats #170

Feat/142 collection stats #170

Conversation

alko89 commented Apr 12, 2023

PR type

What's new?

Before submitting Pull Request, please make sure:

Optional

Screenshot

alko89 commented Apr 12, 2023

vikiival commented Apr 12, 2023

alko89 commented Apr 12, 2023 • edited Loading

vikiival commented Apr 12, 2023

Matehoo commented Apr 17, 2023 • edited Loading

vikiival commented Apr 17, 2023

Matehoo left a comment

Choose a reason for hiding this comment

vikiival Apr 18, 2023

Choose a reason for hiding this comment

Matehoo commented Apr 20, 2023

yangwao commented Apr 20, 2023

roiLeo left a comment

Choose a reason for hiding this comment

vikiival commented Apr 20, 2023

roiLeo Apr 20, 2023

Choose a reason for hiding this comment

roiLeo Apr 20, 2023

Choose a reason for hiding this comment

vikiival Apr 20, 2023

Choose a reason for hiding this comment

roiLeo Apr 20, 2023

Choose a reason for hiding this comment

vikiival Apr 20, 2023

Choose a reason for hiding this comment

alko89 Apr 20, 2023 • edited Loading

Choose a reason for hiding this comment

roiLeo left a comment • edited Loading

Choose a reason for hiding this comment

alko89 commented Apr 23, 2023

roiLeo left a comment

Choose a reason for hiding this comment

alko89 commented Apr 12, 2023 •

edited

Loading

Matehoo commented Apr 17, 2023 •

edited

Loading

alko89 Apr 20, 2023 •

edited

Loading

roiLeo left a comment •

edited

Loading