feat: s3 preview client

[jroof88]

Summary of Changes

This commit implements a new preview client that gets preview data from S3. The preview client is good for folks who want to persist their preview data somewhere rather than relaying on an external API call that can both fail and take a while depending on the table the query is being run on.

In addition to the base client, I implemented a JSON version of the preview client that my organization is using. This fetches data from S3 in a JSON format and works very nicely with marshmellow serialization to the PreviewData format.

Tests

N/A. I didn't see tests for any other preview client so I decided not to implement any. The preview clients seem to be fairly custom so testing is probably overkill.

Documentation

Added a documentation file adjacent to the Apache Superset Preview Client docs 👍

CheckList

Make sure you have checked all steps below to ensure a timely review.

• PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"
  • In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.
• PR includes a summary of changes.
• PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
• In case of new functionality, my PR adds documentation that describes how to use it.
  • All the public functions and the classes in the PR contain docstrings that explain what it does

[jroof88]

@feng-tao Friendly ping for a review here. This is getting a bit stale.

[feng-tao]

sure, will take a look. also cc @dkunitsk @youngyjd as well

[feng-tao]

hey @jroof88 , I am fine with the pr. But curious: how do you persist preview data into s3 first? and why not trigger an API call or delta sql to fetch the data?

[jroof88]

@feng-tao

how do you persist preview data into s3 first?

Databricks job that scans our data lake, run a SELECT * FROM x LIMIT 10 on tables, and uploads results in JSON to S3

why not trigger an API call or delta sql to fetch the data?

I thought about this but we want to get something up quickly and the downsides were too great.

1. All of the network configuration would've been a headache. We'd have to someone run our Amundsen K8s cluster behind a NAT Gateway to get a stable IP on outbound requests that we could then allow list on Databricks since we have IP Allow List turned on for security
2. There is no concept of a cache or persistence in the Preview API layer so every time someone landed on a table we would emit a SELECT * FROM x LIMIT 10 from the same table. This seemed costly.
3. We want to leverage utilized partitions during our SELECT * FROM x LIMIT 10. There wouldn't really be an easy way to tell Amundsen how to query specific tables. On Databricks we can run a DESCRIBE x to know the partitions so we could get recent data. Most of our stuff is partitioned by date.

[feng-tao]

@jroof88 got it, thanks for the context!

[feng-tao]

@jroof88 is it ok to add your company as user list?

[jroof88]

@jroof88 is it ok to add your company as user list?

@feng-tao Checking with my team and I will get back to you