API Guide should provide more examples of required JSON payload

[shaw2thefloor]

Im getting this error:
{"status":"ERROR","message":"Error parsing datasetVersion: org.glassfish.json.JsonStringImpl cannot be cast to javax.json.JsonArray"}'

I'm trying to create a Dataset on demo.dataverse.org against an existing dataverse, using python requests, with the following json fragment

{
    "datasetVersion": {
        "metadataBlocks": {
            "citation": {
                "displayName": "",
                "fields": [
                    {
                        "typeName": "title",
                        "multiple": false,
                        "typeClass": "primitive",
                        "value": "Title Monday "
                    },
                    {
                        "typeName": "datasetContact",
                        "multiple": true,
                        "typeClass": "compound",
                        "value": [
                            {
                                "datasetContactName": {
                                    "typeName": "datasetContactName",
                                    "multiple": false,
                                    "typeClass": "primitive",
                                    "value": "Shaw Felix"
                                },
                                "datasetContactAffiliation": {
                                    "typeName": "datasetContactAffiliation",
                                    "multiple": false,
                                    "typeClass": "primitive",
                                    "value": "tgac.ac.uk"
                                },
                                "datasetContactEmail": {
                                    "typeName": "datasetContactEmail",
                                    "multiple": false,
                                    "typeClass": "primitive",
                                    "value": "felix.shaw@tgac.ac.uk"
                                }
                            }
                        ]
                    },
                    {
                        "typeName": "dsDescription",
                        "multiple": true,
                        "typeClass": "compound",
                        "value": [
                            {
                                "dsDescriptionDate": {
                                    "typeName": "dsDescriptionDate",
                                    "multiple": false,
                                    "typeClass": "primitive",
                                    "value": "2017-05-25"
                                },
                                "dsDescriptionValue": {
                                    "typeName": "dsDescriptionValue",
                                    "multiple": false,
                                    "typeClass": "primitive",
                                    "value": "Test motorcade"
                                }
                            }
                        ]
                    },
                    {
                        "typeName": "subject",
                        "multiple": true,
                        "typeClass": "controlledVocabulary",
                        "value": [
                            "Astronomy and Astrophysics"
                        ]
                    },
                    {
                        "typeName": "author",
                        "multiple": true,
                        "typeClass": "compound",
                        "value": [
                            {
                                "authorName": {
                                    "typeName": "authorName",
                                    "multiple": false,
                                    "typeClass": "primitive",
                                    "value": "Shaw Felix"
                                },
                                "authorAffiliation": {
                                    "typeName": "authorAffiliation",
                                    "multiple": false,
                                    "typeClass": "primitive",
                                    "value": "tgac.ac.uk"
                                }
                            }
                        ]
                    },
                    {
                        "typeName": "depositor",
                        "multiple": false,
                        "typeClass": "primitive",
                        "value": "copo-project.org"
                    },
                    {
                        "typeName": "dateOfDeposit",
                        "multiple": false,
                        "typeClass": "primitive",
                        "value": "2017-05-25"
                    },
                    {
                        "typeName": "relatedMaterial",
                        "typeClass": "primitive",
                        "multiple": true,
                        "value": "abc"
                    }
                ]
            }
        }
    },
    "protocol": "",
    "identifier": "",
    "persistentUrl": "",
    "authority": ""
}

I'm pretty sure its the relatedMaterial bit, but it also does the same with dataSources and relatedDatasets.

Any help much appreciated.

F

[pdurbin]

@shaw2thefloor hi! I assume you meant to include another { at the start of your JSON.

Here's an example JSON file I use in integration tests:

• https://github.com/IQSS/dataverse/blob/v4.6.1/scripts/search/tests/data/dataset-finch1.json

Here's where I use it:

• https://github.com/IQSS/dataverse/blob/v4.6.1/scripts/search/tests/create-all-and-test#L10

I hope this helps. That error is a bit surprising so maybe you found a bug.

Also, I'm realizing that http://guides.dataverse.org/en/4.6.1/api/index.html doesn't say how to get help but this is certainly on topic at https://groups.google.com/forum/#!forum/dataverse-community and you're also welcome to email support@dataverse.org .

Please let me know if the JSON file above helps. Thanks!

[shaw2thefloor]

Hi @pdurbin. Thanks for the response. The json fragment you linked to which you use in your tests doesn't contain the problem field. Like I said at the bottom of my original post, I pretty sure its the relatedMaterial section causing the error. Also relatedDataset causes the same error. Without those fields, it works fine, however, my users will probably want to use most if not all of the Dublincore spec, so its necessary for all the fields to be submittable. Are you able to run your test with the following json fragment to see if you can reproduce the error on your side?

Thanks,
F

{
  "datasetVersion": {
    "metadataBlocks": {
      "citation": {
        "fields": [
          {
            "value": "Darwin's Finches",
            "typeClass": "primitive",
            "multiple": false,
            "typeName": "title"
          },
          {
            "value": [
              {
                "authorName": {
                  "value": "Finch, Fiona",
                  "typeClass": "primitive",
                  "multiple": false,
                  "typeName": "authorName"
                },
                "authorAffiliation": {
                  "value": "Birds Inc.",
                  "typeClass": "primitive",
                  "multiple": false,
                  "typeName": "authorAffiliation"
                }
              }
            ],
            "typeClass": "compound",
            "multiple": true,
            "typeName": "author"
          },
          {
            "value": [ 
                { "datasetContactEmail" : {
                    "typeClass": "primitive",
                    "multiple": false,
                    "typeName": "datasetContactEmail",
                    "value" : "finch@mailinator.com"
                }
            }],
            "typeClass": "compound",
            "multiple": true,
            "typeName": "datasetContact"
          },
          {
            "value": [ {
               "dsDescriptionValue":{
                "value":   "Darwin's finches (also known as the Galápagos finches) are a group of about fifteen species of passerine birds.",
                "multiple":false,
               "typeClass": "primitive",
               "typeName": "dsDescriptionValue"
            }}],
            "typeClass": "compound",
            "multiple": true,
            "typeName": "dsDescription"
          },
          {
            "value": [
              "Medicine, Health and Life Sciences"
            ],
            "typeClass": "controlledVocabulary",
            "multiple": true,
            "typeName": "subject"
          }, 
          {
            "typeName": "relatedMaterial",
            "typeClass": "primitive",
            "multiple": true,
            "value": "abc"
           }
        ],
        "displayName": "Citation Metadata"
      }
    }
  }
}

[pdurbin]

@shaw2thefloor because relatedMaterial has true for multiple, the value has to be a JSON array, like this:

          {
            "typeName": "relatedMaterial",
            "multiple": true,
            "typeClass": "primitive",
            "value": [
              "RelatedMaterial1",
              "RelatedMaterial2"
            ]
          },

Can you please give that a try?

This example comes from default_metadata_blocks-4.6.1.json, which I first posted at https://groups.google.com/d/msg/dataverse-community/qsY8swD9Hh0/Q9Tm4Ll-AgAJ and will attached below, adding ".txt" to the end so that GitHub Issues accepts it:

• default_metadata_blocks-4.6.1.json.txt

Even if this does work for you, I'd love to get some feedback on how we can make this easier for API users like yourself. Obviously, a more clear error message than Error parsing datasetVersion: org.glassfish.json.JsonStringImpl cannot be cast to javax.json.JsonArray would help!

Here are some related issues:

• Native API: What fields are required for dataset creation Native API: What fields are required for dataset creation #3060
• "Compact"/Make complete the native API output "Compact"/Make complete the native API output #3068
• Add API for simple Dataset Edits Add API for simple Dataset Edits #3599

[shaw2thefloor]

Ah ok thanks for that Phillip. As you said the error occurs when supplying an empty string for one of the multiple field types rather than an empty array. So for me, the most useful thing was finding the JSON examples on github. I spent a long time looking through the docs here http://guides.dataverse.org/en/4.6.1/api/native-api.html, but they mostly only document the urls and not the JSON required in the payload. I was working with these guys recently, and this is an example of what I found really useful REST documentation. http://developer.agaveapi.co Anyway, thanks for your help. Felix

[pdurbin]

@shaw2thefloor I agree that JSON examples would be helpful. #3192 is about a similar issue. How do you feel about changing the title of this issue to something like "API Guide should provide more examples of required JSON payload"?

Lately, I've been making an effort to make JSON examples downloadable from the API guide, as in 23fafb3, but the older APIs have been a bit neglected.

[shaw2thefloor]

hey @pdurbin. I can certainly do that. Thanks for your help. If there is anything else I can think of as I'm working with the API, I'll let you know.

[pdurbin]

@shaw2thefloor thanks. I just added another example in bffb3fe

[pdurbin]

The title of this issue is fairly broad. We should probably make a small chunk issue to add a cleaned up version of default_metadata_blocks-4.6.1.json.txt (test it first) from #3859 (comment) to the API Guide.

[pdurbin]

Yesterday in IRC I was chatting with @shirleyah about the JSON needed to create a new dataset and she reminded me that it's confusing that some fields are ignored such as the following:

"id": 18577,
"identifier": "QF0D5S",
"persistentUrl": "http://dx.doi.org/10.5072/FK2/QF0D5S",
"protocol": "doi",
"authority": "10.5072/FK2"

These fields are ignored because when you create a new dataset the dataset will get a new id and identifier. The protocol and authority will be whatever the Dataverse installation is configured to use. Basically, as the person creating the dataset, you don't have any control over the fields above so they should not be included in the JSON example that will be downloadable from the API Guide. This is what I meant in my last comment about how the example JSON mentioned above needs to be cleaned up.

@kcondon and I discussed how the fields above and others are ignored which let us to wonder if terms of use can be set via API. I have "Setting Terms of Use" listed under the "No functionality should be GUI-only. Make all functionality reachable via the API" issue I opened at #3440 so I don't believe it's possible. A separate issue should be created for this. This current issue should be only about documenting what's possible (with a JSON) example, using the native API.

@jggautier this relates to https://help.hmdc.harvard.edu/Ticket/Display.html?id=258964

@shaw2thefloor out of curiosity, are you still interested in this issue? Thanks again for opening it.

[pdurbin]

@shaw2thefloor two things... 😄

We haven't heard from you in a while. Are you still interested in this issue? 😄

As part of pull request #6107 I finally got around to making the more complete "all fields" example downloadable in the "create a dataset" section. Here's how it looks:

[poikilotherm]

This is related to the following issue:

• bklog: Deliverable - As a system integrator, I would appreciate a JSON Schema for validating my dataset JSON before uploading via API dataverse-pm#26