Example of packaging frictionless data json files with dataset

[jen-thomas]

I'm playing around with the best way to put the datapackage.json and tableschema.json files with the data files (resources) as part of a dataset publication and I was wondering if you could point me to a nice example please? I'm not sure if I've missed somewhere in the specs the best or recommended way to go about this. This is an example that I've done recently: https://zenodo.org/record/3897170

Not sure if this is the best way to go, or I should be doing it differently....I was slightly limited in this case by Zenodo that due to a bug it is not possible to upload .json files directly.

[lwinfree]

Hi @jen-thomas :-) I'm copying over our convo from Discord to record it here in case others have a similar question. Please reply here if you would like to continue this conversation. Thanks!

---

Lilly: here is an example in zenodo from one of our other collaborators, Phil: https://zenodo.org/record/2640873#.Xuy-VZ5KgWo
http://frictionlessdata.io/blog/2018/03/07/well-packaged-datasets/#download-your-data-package @ jen check this out too

---

Jen: Ah great, thanks @ Lilly ! So just alongside the data files is fine?
I thought I had read that somewhere in the past. It must have been a few months ago! Thanks for finding where it was.

Yes it is very useful!

Within the blog:
"
Your final Data Package file directory should look like this:

data/
dataresource1.csv
dataresource2.csv
datapackage.json

data/: All data files are contained in this folder. In our example, there is only one: data/gdp.csv .

"
In some cases (eg. Zenodo again - it's always the same example for me at the moment!) it isn't possible to have directories containing files, so this would either have to be flat or within a zip in this case. For a bit of context though, I try to avoid zips unless it is necessary because there are lots of data files or due to dataset size so that individual files can be downloaded.

---

Rufus: @ Jen, the subdirectory of data/ is just a convention - you can put all the files in the base directory

---

Jen: I was particularly thinking of someone harvesting datasets or re-using one. Having only really "produced" datasets that are described by FD, I don't have any experience of how this is done. I guess the structure of the resources would be captured from "path" within the datapackage but maybe it is just me worrying about something that isn't important?

---

Rufus: yes, exactly. which files in the directory are actually data files is precisely what the resources list in the data package does - it's basically the manifest ...

---

Carles: I think that one of Jen's concern is that if someone finds a file like: https://zenodo.org/record/2640873#.XuzDWHVfhhF (file rose-aroma-naturegenetics2018-treatment-group-mean-sem-report-datapackage.json): how does a crawler know that this is a frictionless tabular data package? I'm quite sure that on Github there is some issue to have $schema (with a version) so a crawler could know that "Ah, Frictionlessdata package schema, version 4, I can parse this"

---

Jen: Thanks @ carles for translating my question into FD language

---

Rufus: @ carles there's the profile field in the datapackage.json right?

---

Carles: there is also maybe the problem with the "path". In that package there is a resource with path: "rose-aroma-naturegenetics2018-treatment-group-mean-sem-report-table-example.csv". If someone found the JSON file without the CSV file next to it there is no way to find the csv file (no base_path, etc.) (and yes, I still have a ticket pending to write about "my" view of problems with "path" :slight_smile: )
checking

---

Rufus: that's the equivalent of $schema i think - btw i'm wondering if we should just switch profile to $schema
There is: https://specs.frictionlessdata.io/data-package/#metadata - if it's a upstream Frictionless Data Package I would expect the spec to have a recommended URL (with frictionlessdata.io in the URL, maybe resolving to the schema, I'm not very familiar with this or I might be having flashbacks on XML NS alike...)
if it's "tabular-data-package" it's a bit vague perhaps (schema.org tabular data package? or something else?)
oh, and with a schema version of course (not the datapackage version)

---

Jen: I think if I were looking for it, I would expect to have a permanent URL (using a DOI or something similar) with a version number for the schema

---

Rufus: see frictionlessdata/datapackage#689

---

Carles: I'm looking at schema.org and they seem to have (for JSON-LD)" @ context" and "@ type", without version (it surprised me that they didn't have version)

---

Rufus: i don't think you need version since you encode that in the url ...

---

Carles: Thanks! Watching (now) this repo as well to get notifications, I see some recent comments along these lines