Communicate if data is sampled to the front end some how

[holdenk]

cc @rgbkrk is there a reasonable way to expose this in the current JSON format?

[rgbkrk]

The formal spec for Data Resource has a metadata field which can contain any number of new properties. We'll probably want to settle on our own format for conveying to the frontend the caveats about number of samples, that we're doing sampling, and how many (estimated) records aren't shown/available. We'd also want something similar if someone does a .head in Spark or Pandas to show that they're only listing N records.

@rufuspollock @pwalsh - what would be the best way to signify sampling or otherwise in data resource json? Totally ok with it being in metadata, we'll just at least semi-standardize it for spark, pandas, and other library <--> jupyter communication.

[rgbkrk]

It just occurred to me that we could indicate sampled data in another way, which is metadata on the displayed result.

display(
    JSON({'a': 'b'}),
    metadata={
        "application/json": {
            "expanded": True
        }
    }
)

We'd have to do the same for a data resource object, which would mean something like this:

display(
    df.sample(6),
    metadata={
        "application/vnd.dataresource+json": {
            'samples': 6
        }
    }
)

Either way it's a contract with the frontend for how we would want to indicate this, so I'd hope to version this somehow.

[rufuspollock]

@rgbkrk this is for situation where the data in the data resource was sampled from other place and you want to indicate the data was sampled is that right?

If so, i think this may be a reasonably common use case (i've certainly come across it myself) in which case it would be great if you could take a minute to open an issue on https://github.com/frictionlessdata/specs/issues so that other folks see this and we can come up with a simple pattern.

[rgbkrk]

Ok, cool. For the time being, if you pick something for ipython's display metadata field @holdenk we can start using it to prototype on the UI side.