Pagination of table.fetch (getCollection) feature

[gregkysenko]

Hello! First of all, thank you guys for the library, we are using OneTable for up to three months now and it definitely made our Single table setup more stable.

I have a question: Currently I am using get collection feature to retrieve count of child enttities for my organisation e.g. :

        return this.table.fetch(
            ['Organisation', 'Label', 'Album', 'Track', 'Work', 'User', 'ComposerContract'],
            {
                PK: `orgId#${data.id}`,
            }
        );

I am wondering what are the limitations of this approach and how pagination would work in that case? Currently I don`t have that much child entities, but in the future I expect to have 1000+ Child entities per organisation, so I am wondering how fetch would handle such cases?

P.S. Can I fetch by GS1PK instead of PK?

[mobsense]

Glad you like it.

The fetch() API is not pageable due to calling groupByType. The end of fetch() does:

        let items = await this.queryItems(properties, params)
        return this.table.groupByType(items)

The queryItems is pageable, but groupByType does not currently return the paging next/prev tokens.

There could be 2 solutions:

1. Add the next/prev to the return values. However, since the return is a group {}, that would impact users who iterate over the properties of the return result. I suppose we could make them non-enumerable?
2. Add an option to fetch params to not group. Then you would just get back the items array with next/prev and you could group manually.

Yes, GSIs should work. Just pass {index: 'name'} in the params.

[gregkysenko]

Thank you for the answer! I think approach 2 is much easier & safer to implement, and it will cover my needs. Should I contribute and create a PR with this feature?

[mobsense]

Looking deeper, fetch shares the same OneParams type def -- so adding an option would require that to be split.

The actual logic for fetch is pretty simple. I've reduced it down here. It may be easier for you do just create your own function.

    async fetch(models, properties = {}, params = {}) {
        let where = []
        for (let model of models) {
            where.push(`\${${this.typeField}} = {${model}}`)
        }
        if (params.where) {
            params.where = `(${params.where}) and (${where.join(' or ')})`
        } else {
            params.where = where.join(' or ')
        }
        params.parse = true
        params.hidden = true

        return await this.queryItems(properties, params)
    }