How to get framing results into struct

[fils]

I'm looking to use the framing aspect of this library to help me parse down schema.org JSON-LD into known chunks.

My example is at https://github.com/ESIPFed/snapHacks/blob/master/sh01-jsonldCrawl/simpleCrawler/main.go

I am using the JSON-LD at line 25
The frame defined at line 67

and it works! Two points...

1. The @context is huge... is this returned from the net somehow based on all of schame.org types or something?

2. The @graph is parsed out at line 83 and I am trying to figure out how to cast it to a []map[string]string which I think is the collection type. However, my assert fails.

Is there is a best practice for dealing with the results from the proc.Frame call at line 78?

The code is self contained, so it should download and run if you want to try it. Many of the functions are just placeholders for now.

Any guidance appreciated!

Thanks
Doug

[fils]

self answer, maybe not the best.. but it gets me from interface to json and then to struct...
There might be a quicker more pedantic way...

	graph := framedDoc["@graph"]
	// ld.PrintDocument("JSON-LD graph section", graph)  // debug print....
	jsonm, err := json.MarshalIndent(graph, "", " ")
	if err != nil {
		log.Println("Error trying to marshal data", err)
	}

	dss := make([]DataSetStruct, 0)
	json.Unmarshal(jsonm, &dss)

where

type DataSetStruct struct {
	Description string
	ID          string
	Type        string
	URL         string
}

[kazarena]

Hi Doug,

Apologies for the lack of replies, I'm travelling at the moment. Will get back to you within 2 days.

Stan

[kazarena]

@fils,

1. Yes, framing (as well as other JSON-LD operations) expand (in your case read: download and embed) the context. You may just ignore it, if it's not needed downstream. It may be a good idea to expose it as a (non-standard) option - I'll think about it.
2. The returned document would be of type []map[string]interface{}, not []map[string]string. And yes, you got it right, the simplest option to parse it into structs is to transform it to JSON and then back.

Regards,
Stan

[fils]

@kazarena Thanks! appreciate the info and confirmation. Thanks for a really nice Go based JSON-LD library too! Going to be using more and more of it it looks like.

[fils]

@kazarena I am revisiting this issue....

I am working on some code [1] that indexes tens of thousands of JSON-LD documents. I have a framing call I make in [2] and I think it is pulling down and embedding the context each time.

Is there a way to prevent the download each time or cache the download somehow? I ran the profiler and pprof and it seems this is a potential place I could make some gains (visual of cpu prof attached)

Maybe I just need to try and do this without framing, but the framing allows me to make a common schema I can map to my structs so nice.

Thanks!
Your package has been so wonderful to have!

[1] https://github.com/earthcubearchitecture-project418/crawler
[2] https://github.com/earthcubearchitecture-project418/crawler/blob/master/framing/spatialFrame.go

[kazarena]

@fils thanks for digging into it! I'll take a look.

[kazarena]

@fils I've looked at your code. I'm sorry, I still can't see how I can meaningfully change the library which is constrained by the current standard to address a very specific business case. The problem is: in order to introduce some caching or re-use of contexts (which are required regardless whether you want to see them in the framing output or not), the library would need to know what frames and contexts are the same. This goes beyond the library's responsibilities.

I would recommend exploring 2 options:

1. try using https://www.w3.org/TR/json-ld-api/#widl-JsonLdOptions-expandContext option and pre-construct the context you are using in your application
2. re-write the JsonLdProcessor.Frame() function in your code and re-use the objects that you know aren't changing between documents (i.e. the expanded frame and active context).

[fils]

@kazarena thanks for looking at it and I agree with your view there is nothing that should be done in the library. I really appreciate the suggestion and I give it go!

Again.. thanks for the time and review! truly appreciated and thanks for json-gold!

take care
Doug