Support for limiting (or expanding) response fields

[paultag]

It'd be really neat to request an endpoint and specify which fields I'd like back in the request, which may help keep size down if you know you only need a few fields.

I've not found a great way to do this yet, but I've got a few ideas around keeping a root serialize dict, and index into that for the values of a serialize call, defaulting to a list of default fields to include. Sounds a bit heavy-handed.

Anyone have better ideas or solutions?

[senko]

I can't think of a better way. I would try to keep it on one level (ie not make it possible to filter any sub- or related resources recursively - either they're in or out). That way, you could just pass them as fields option (still indexing into the dict of known values for validation).

[paultag]

Alrightly. I'll play around with this. I'll see what I come up with that might be generically useful

[paultag]

Took a pass. Looks like it wasn't that bad to nest stuff. I'll give this a go (building it out), but this seems like a pretty OK approach.

def get_fields(root, fields):
    subfields = defaultdict(list)
    concrete = []
    for field in fields:
        if '.' not in field:
            concrete.append(field)
            continue
        prefix, postfix = field.split(".", 1)
        subfields[prefix].append(postfix)
    ret = {x: root[x] for x in concrete}
    for key, fields in subfields.items():
        ret[key] = get_fields(root[key], fields)
    return {"fields": list(ret.items())}

[senko]

Neat!

You'd probably want to check for key existence in root (to avoid crashes if an invalid field name is supplied), but other than that, looks pretty solid to me.

My worry with recursive items is that this can result in poor performance if you don't select/prefetch the related items, but you can't hardcode the select/prefetch related because the fields are selected at runtime.

One option would be to just select/prefetch everything, but that gives non-optimal performance if related resources are not requested. Other would be to manually build select/prefetch list while recursing in get_fields. I'm not sure how messy that could become tho.

[paultag]

@senko yeah, you were totally right.

Anyway, this spiraled out. I've got prefetch and nested field limits workingish.

I've got a bunch of stuff downstream here, what should I do to get it into a shape where it might be useful up here?

Helpers / get_fields

A few rough edges, but with a bit of cleanup it might be nice to get generalized a bit.

[senko]

Thanks @paultag this looks very interesting. I think something like that, generalized, would be useful upstream. However, I'd prefer if this remains downstream for the moment, so you can see in retrospect (after some time) how well it worked in practice (from the architectural standpoint, not any bugs in the code that may sneak in). This is a way most of the other features landed in restless - I tried them on a couple of projects and they seemed to crystallize as a repeated pattern.