Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resources of graph data users can browse and download. #3

Open
thorwhalen opened this issue Dec 27, 2022 · 3 comments
Open

Resources of graph data users can browse and download. #3

thorwhalen opened this issue Dec 27, 2022 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@thorwhalen
Copy link
Collaborator

To make it easy for the user to start playing with our tools, would be nice to make it easy for them to * get source graph data, and

  • prepare this data (format it for cosmograph)

This issue regards the former.
(The Utils to transform data into the normalized form that cosmograph accepts issue is about the latter.)

The Importable test graph data issue is more about test data, that is, giving us quick access to data with specific properties meant to diagnose problems. We want this data to be small, or synthetic generated by code (not require any download of significant data)

But here, we're interested in making it easy and enticing users to explore interesting data using our tools. Sure, they can google around for data sources, and learn how to prepare these for cosmograph, but if we can already point them to some nice data source (awesome list of graph data?) and prepare a few of these for them in some gallery form, we'll make it easier for them to get hooked.

@rokotyan's comment to the Importable test graph data issue contains exactly that: Images of graphs and their links to it's (prepared) data source.

@thorwhalen
Copy link
Collaborator Author

How do we want to do this?

Eventually, we probably want to host this gallery ourselves in a cosmograph, with data stored in some blob storage system (S3 for example?).
Once our data transformers are stable, instead of prepared data we store ourselves, we'll also have the choice to only have to store transformer pipelines that produce the prepared data from their original source (we don't need to store the raw data ourselves, just provide it's url).

But for now, to get started, let's use github:
Here's a snippet of @rokotyan's comment:

image
pds-90.csv.zip

The markdown of which is:

![image](https://user-images.githubusercontent.com/755708/209574108-db2f551c-5f16-4e87-9c8d-543792d3322e.png)
[pds-90.csv.zip](https://github.com/cosmograph-org/py_cosmograph/files/10304428/pds-90.csv.zip)

The urls were created automatically by dragging image and csv.zip files to the browser, so perhaps we can just use that for now. Drag and drop images and files into a wiki in the current project, and then reuse the links to provide other access options (such as importable datasets).

@thorwhalen
Copy link
Collaborator Author

Created A gallery of graph data wiki for this purpose, where I put @rokotyan's examples for now.

Of course, this effort of providing easy access to graph data should be an open-ended ongoing effort. Let's leave this issue open as a reminder for awhile, and close it when we feel we have a sufficient current setup and a plan for future improvements.

@rokotyan
Copy link
Member

@thorwhalen Neat idea! We should probably have it in the main repo since it's not just for python.

We already store these files on our Digital Ocean, so far it works well and allows us to open them in Cosmograph using URL params without dealing with CORS restrictions, e.g.:
https://cosmograph.app/run/?data=https://cosmograph.app/data/blckhole.csv

@thorwhalen thorwhalen added the enhancement New feature or request label Dec 27, 2022
@thorwhalen thorwhalen self-assigned this Dec 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants