Specify Consumer Discovery Process #274

mmccool · 2022-02-13T15:11:36Z

Better describe and constrain what a consumer has to do to implement discovery.
Resolves Issue #272

WIP: Do not merge until all checkboxes below are complete.

Update overview (Figure 1) to fix some typos (based on TD Link image, but reverting addition of TD Link path). BTW, this is useful: https://github.com/jgraph/drawio-desktop
Also added notes to README explaining requirements for SVG files, and notes on how to export from drawio in particular (specifically: embed fonts). This was because the SVG for the overview figure was broken (labels truncated), so I had to track down and figure out drawio, etc.
Add new section for Consumer Discovery Process
Add introductory text (copy/modify text from defunct PR Permit use of TD Links for Self-Description of Multiple Endpoints #269 )
Add assertions (see discussion in Issue Consumer Process for Discovery #272)
Clean up intro text to align with proposed "simple directory" solution for multiple Things on one host
Further update overview figure to indicate that expansion of contents of TD Directory is optional
~~Add~~ Change Terminology definitions for ~~D-Consumer and D-Client~~ Discoverer

Preview | Diff

mmccool · 2022-02-13T18:07:55Z

So, several points to discuss in our next meeting:

If Discovery is not mandatory for all Consumers, what entity do I make the assertions on? I have defined a "D-Consumer" as a "Consumer that supports Discovery". To discuss: whether or not an assertion in Architecture saying "All Consumers MUST support WoT Discovery" to get rid of this. I have tried to word things so that we can replace "D-Consumer" with "Consumer" if this happens. If we keep D-Consumer, a definition should go into Terminology (in Discovery or Architecture)
I have tried to keep the requirements for being a D-Consumer as low as possible. In particular, I allow non-HTTP URLs, including "file", and specifically call out Consumers that support only a single Direct URL and only "file" to fetch that URL. This still puts some VERY light constraints on Consumers to be D-Consumers, e.g. they have to understand the URL syntax for "file" at the very least.
D-Consumers can do what they want with Thing Links and Directories, including ignoring them.
I describe the process of using a Thing Link to refer to a Directory from other Directory (see Add a Directory to a Directory #271; ideally I want to resolve that issue too by defining a preferred mechanism).
I make it mandatory to keep track of which Thing Links or Directories have been "expanded" but prohibit re-expansions (to avoid infinite loops when directories refer to each other). To Do: what if a consumer wants to "refresh" the set of contents pulled from a directory, without starting over from Introductions?
How to update the figure to describe the process? I did fix some typos but I feel the current figure makes it looks like reading a directory is mandatory.

mmccool · 2022-02-13T18:12:33Z

We MAY want to be more particular about what it means for a Consumer to "support Discovery". In particular I waffled a lot about whether requiring the ability to read a TD from an HTTP URL should be included, but it seemed overly exclusive. CoAP consumers (without an HTTP client) may not be able to read from a Directory (which only has an HTTP API), but can do other things, such as use .well-known. I have an ed note about that.

mmccool · 2022-02-13T18:18:53Z

To be clear, Architecture only has the following assertion at this point, which makes Discovery optional:

arch-discovery-should-use-standard: Whenever possible, the Discovery mechanisms defined in [[!WOT-DISCOVERY]] SHOULD be used to distribute TDs.

So Discovery is NOT mandatory for all Consumers.

mmccool · 2022-02-14T12:53:07Z

@k-toumura makes a good point (see issue #272): the D-Consumer may not be a Consumer. There are clients, like Toumura's Node-RED implementation, that may use Discovery but not itself be a Consumer. So we need another term. I want to avoid "Discoverer" (we have used it for something else) so maybe "Discovery Client" (D-Client for short).

mmccool · 2022-02-14T14:54:31Z

@k-toumura I made some changes to the PR to address your point. I add an additional definition, "Discovering Client" or "D-Client", defined as "A client that supports Discovery". I still have Discoverying Consumers (D-Consumers), but these are a subclass of D-Clients that can also interact with the affordances exposed by a TD etc.

There is some overlap: a D-Client does need to interpret certain parts of TDs, e.g. the @type (to figure out if it is a Directory or Thing Link) and also it needs to extract the URL from the form of the "listing" affordance in Directories. It may also need to interpret some of the security information provided by Directory TDs. But it's true it's not a "full" Consumer. But a D-Consumer is always a D-Client.

I have rewritten all the assertionts to apply to D-Clients.

farshidtz · 2022-02-14T15:04:01Z

index.html

+	than at the exact URL provided by the Introduction.  Sets allows for single URLs to expand
+	into multiple Thing Descriptions without needing to use a Thing Description Directory.
+	Such sets are only appropriate for small static lists of Thing Descriptions.


Didn't we decide here to reduce mandatory requirements of a directory (#273) to allow returning a collection in favor of a Think Link?

That is leftover text in the Arch intro that I need to delete still. See comments below.

see updated version. I believe I have resolved this now.

farshidtz · 2022-02-14T15:47:39Z

index.html

+	</ul>
+    </section>
+    <section id="consumer-process" class="normative">
+        <h1>Discovering Clients and Consumers</h1>


I suggest keeping this under chapter 4. Architecture.

Will consider. The intention is that Architecture is a general overview, this is about clients, then we have introductions and exploration services. I think personally that detailed discussion of client and server features should be at the same level.
BTW the text under "Architecture" needs revision and I will update it soon. What's there is just a copy of what I had in the Thing Link PR and is not 100% appropriate. The diagram was updated to remove the Thing Link but maybe I should not have, since we DO need to resolve them, but only "describedby" relations.

I have not moved, but I have changed the name

farshidtz · 2022-02-14T15:50:51Z

index.html

+			A D-Client MUST support fetching a TD from least one URL provided as part of the Introduction process.</span>
+			In other words, not only must a D-Client accept URLs pointing at TDs, it must be
+			able to fetch that TD, for example by using a GET for an HTTP URL or by reading it from a file 
+			specified by way of a file URL.


Suggested change

specified by way of a file URL.

specified by way of a file URI.

https://en.wikipedia.org/wiki/File_URI_scheme

I was actually thinking of removing mention of file URLs/URIs completely and pointing out (see below) that Consumers that just read TDs from files are fine and ok, but should not be considered as implementing Discovery. Maybe I should also explicitly allow D-Consumers to ALSO allow reading of additional TDs from files and include them in the "Discovery" set... File URIs are a pain in the neck anyway.

File URI/URL might come to play when you have TDs that are referring to other TDs. Taking again the TD resulting for the composition process, if you are developing that locally you'll probably reference sub-parts using file URLs (since you have to put an URL in the href of links). However, most of the time these URLs will be relative so it might not really be a useful use case.

Thought I fixed this, let me checked (won't merge suggesting in case it screws up my other commits... but will take into account in my edits)

index.html

mmccool · 2022-02-14T16:17:05Z

Things noted during Discovery mtg:

SPARQL federation needs the URL of the SPARQL endpoint, not that of the TD for the TDD. So the last sentence in the section needs revision
"Discovering Clients and Consumers" is a confusing name, since "Discovering" can act as either a verb or an adjective. The latter is meant, but this is a relatively unusual English usage. So I think changing this name to "Clients and Consumers" is best.
There can be systems (Consumers) that just read a TD from a file, and that is fine. But we probably should not say these support Discovery (systems that only do this are not D-Clients). It should be pointed out that this is ok and suitable for some use cases. We could also disallow "file" URLs but I think I will settle for just removing mention of them.
A given client will have to limit its support to a finite set of protocols; it is not necessary (or possible, really) to support all possible protocols that can be embedded in URLs. I think this is covered under the assertion that a client MAY choose not to follow an Introduction URL but I will add this as a possible reason.

index.html

farshidtz · 2022-02-14T16:23:54Z

index.html

+	We will use the term <a>D-Consumer</a> to refer to a WoT Consumer supporting WoT Discovery,
+	and D-Client as a more general term relating to any client supporting WoT Discovery.


The distinction between the two terms is not clear. I'm really not sure we need the two terms.

pls read Toumura's comments, which I was trying to address. The other path we could take is calling things like Node-RED plugins scanning for TDs Consumers. They are, after all, able to do at least minimal interpretation of TDs and need to call specific affordances (listing) on Directories.

2 cents: probably having these two terms would help describe the discovery process in the Scripting API. We can use the term D-Client to indicate Servients runtimes that support the discovery and the D-Consumer to applications.

The components of a system can be named arbitrarily. I can argue that I need a Discovery Parser which parses the TD object only. I can also argue that a TD is too large to pass around and should be transported and consumed incrementally (we have assertions for these).

Right, the problem is that "Discoverer" sounds like "an entity capable of doing Discovery" which is not, however, its definition. We could

Use "Discoverer" rather than "D-Consumer" and "D-Client" and use another term for the current meaning (e.g. "Registerer")

Leave it as is.

Get rid of D-Client (since formally, since D-Clients need to process TDs, they are in fact D-Consumers). However, definition of Consumer implies interactions with Things, including endpoint Things (not just directories, so @k-toumura would prefer different terms).

Consensus: do 1, and also use this to resolve 3 (say a Discoverer MAY be a Consumer, but not necessarily).

farshidtz · 2022-02-14T16:32:25Z

index.html

+		<li><span class="rfc2119-assertion" id="consumer-fetch-links">A D-Client MAY fetch source TDs from the targets of the links in a Thing Link
+			described in its initial set of TDs and add them into the set of TD results.</span>
+			This only adds the results of fetching TDs from a Thing Link to the set of
+			results. These new results do not delete the original TD describing the Exploration mechanism. 
+		</li>
+		<li><span class="rfc2119-assertion" id="consumer-fetch-iteration">A D-Client MAY fetch additional TDs iteratively from any Thing Link or
+			Exploration mechanism described in its set of TDs and add them into the set of TD results.</span>
+			This only adds the results of fetching TDs from a Thing Link to the set of
+			results. These new results do not delete the original TD describing the Exploration mechanism. 
+		</li>


Is the second assertion referring to processing of nested links in TDs extracted from Thing Links? It is not very clear. There is another assertion two items below saying the same thing clearly.

I think there are some cut-and-paste errors here. Will try to address.

I agree with @farshidtz it can be improved.

farshidtz · 2022-02-14T16:51:21Z

index.html

+	The above process supports a simple way to let Directories reference other Directories without duplicating their TDs: 
+	a Directory wanting to reference other Directories should include a Thing Link with a "describedby" relation to the
+	TD of the other Directory service.  Then the above process would expand the Thing Link to obtain the actual TD of the Directory, 
+	and then (optionally) use the appropriate Directory affordance to access the contents of the linked Directory.


Some points:

Referencing a large TD is one of the documented use cases for Thing Link, but the TD of a directory isn't always large.

As mentioned in Add a Directory to a Directory #271 (comment), the directories aren't always self-describing, so the TD of a directory may not be available anywhere other than in the master directory.

The process via Thing Link is only possible if all those self-describing things have their TD available to the consumer. There could networking limitations and other security requirements making them less practical in most production settings. So recommending their use for federated directory linking may not be appropriate.

This often means an external reference, which is not suggested for consumers in another assertion above.

I now have assertions saying external references do not have to be followed; indeed, dereferencing external links is a privacy risk

I added a little text clarifying (I hope) that directories do not need to be self-describing, but will scan for other places where this might be implied

More comments after I read the discussion in the above issue...

mmccool · 2022-02-14T16:51:34Z

Fixed:

Change name of section (remove "Discovering" prefix, as it is unfortunately ambiguous grammar)
Clean up text in Architecture to remove mention of Thing Links for anything other than redirection
Clean up text talking about SPARQL federation to make it clear the URL needed is not the one pointing at the TD itself, but at the SPARQL endpoint given in the TD.

farshidtz

I added several inline comments.

I also suggest moving the Thing Link and Thing Directory type definitions along with the ontology from 6. Exploration Mechanisms to somewhere under chapter 4. Architecture, because there are now forward referenced.

mmccool · 2022-02-14T16:58:22Z

Introductions, Exploration, Directories etc. are mentioned in Architecture so that would be a good place to formally introduce these terms. We could also put formal definitions into Terminology. I definitely do need to talk about them in Clients. I also talk about the specific @type values that need to be checked in the new Clients section. Reversing the order of talking about Clients and Servers would resolve the issue too but would be less clear, I think (it would bury client behaviour down under a long laundry list of specific Introduction and Exploration mechanisms).

I could also just introduce an explicit forward reference (e.g. "defined in Section xxx") for ThingLink and ThingDirectory values for @type.

mmccool · 2022-02-14T17:02:57Z

Some of the use cases for Thing Links (mentioned down much farther in the text) are bugging me. I don't want to touch that part of the spec with this PR as it would cause conflicts, but at some point let's review those use cases.

farshidtz · 2022-02-14T17:26:41Z

I could also just introduce an explicit forward reference (e.g. "defined in Section xxx") for ThingLink and ThingDirectory values for @type.

That's also okay. But those types don't currently have a section for themselves that are directly refencable.

relu91

I tried to give my point of view about the current status which is already fine. Some additional points:

a D-Client may fail to fetch some of the discovered URLs, should it abort the process or continue? is it the process still successful or not?
I would also mention the fact that D-Client will probably need some "credential bootstrapping" to access TDs hosted at particular URLs. How should a D-Client behave if a fetch operation fails due to an Unauthorized response?

relu91 · 2022-02-16T09:54:24Z

index.html

+	We will use the term <a>D-Consumer</a> to refer to a WoT Consumer supporting WoT Discovery,
+	and D-Client as a more general term relating to any client supporting WoT Discovery.


2 cents: probably having these two terms would help describe the discovery process in the Scripting API. We can use the term D-Client to indicate Servients runtimes that support the discovery and the D-Consumer to applications.

relu91 · 2022-02-16T10:00:55Z

index.html

+	multiple TD links.
+	This is why in the following we allow any protocol to be used, even though
+	our Directory exploration mechanisms require HTTP.
+	</p>


I strongly agree with this ed note, will this stay thereafter publication?

Huh, I misread this at first as "disagree"; but you actually "agree" with "not limiting to HTTP". But as discussed below, I think limiting Discovery to HTTP will make things more interoperable. I can see both sides, but I think dealing with CoAP/CBOR etc. right now is not entirely feasible.

relu91 · 2022-02-16T10:14:10Z

index.html

+	As described in section <a href="#architecture"></a>, WoT Discovery has two phases, Introduction
+	and Exploration. In the following sections we will give details on various Introduction and Exploration
+	mechanisms. 
+	However, <em>all</em> Introduction mechanisms result one or more URLs which can be used to fetch TDs by
+	a D-Client. 
+	Some of these TDs, however, may link to
+	TDs stored elsewhere (Thing Links) or describe web services that manage sets of TDs (Thing Description Directories, or TDDs).


Hard to read text.

Suggested change

As described in section <a href="#architecture"></a>, WoT Discovery has two phases, Introduction

and Exploration. In the following sections we will give details on various Introduction and Exploration

mechanisms.

However, <em>all</em> Introduction mechanisms result one or more URLs which can be used to fetch TDs by

a D-Client.

Some of these TDs, however, may link to

TDs stored elsewhere (Thing Links) or describe web services that manage sets of TDs (Thing Description Directories, or TDDs).

As described in section <a href="#architecture"></a>, WoT Discovery has two phases, Introduction

and Exploration. In the following sections we will give details on various Introduction and Exploration

mechanisms.

Remember that <em>all</em> Introduction mechanisms result in one or more URLs that can be used to fetch TDs by a D-Client. Moreover, some of these TDs, may link to TDs stored elsewhere ( via Thing Links) or describe web services that manage sets of TDs (Thing Description Directories, or TDDs).

Or something similar. In particular, the sentence "However, all Introduction mechanisms feels disconnected.

Point noted, and will try to incorporate your changes, but I already merged another change from Farshid already here so can't apply your patch without some edits.

generally reorganized this part to try and make things flow better

relu91 · 2022-02-16T10:15:24Z

index.html

+	As described in section <a href="#architecture"></a>, WoT Discovery has two phases, Introduction
+	and Exploration. In the following sections we will give details on various Introduction and Exploration
+	mechanisms. 
+	However, <em>all</em> Introduction mechanisms result one or more URLs which can be used to fetch TDs by
+	a D-Client. 
+	Some of these TDs, however, may link to
+	TDs stored elsewhere (Thing Links) or describe web services that manage sets of TDs (Thing Description Directories, or TDDs).


(Thing Description Directories, or TDDs).

Isn't the acronym defined somewhere else? I think a reference should be enough

I will have to look at the order of the text. Generally I want to expand acronyms on first use, but I'm not sure if this is the first use or not.

Think I will leave this for now, then we need another pass to clean up terminology. I also use "Directory" in a lot of places where I should probably be using TDD.

relu91 · 2022-02-16T10:23:05Z

index.html

+			A D-Client MUST support at least one Introduction mechanism.</span>
+			The simplest Introduction mechanism, Direct, simply provides a single URL of a target TD.  
+			This assertion results in different minimal requirements depending on
+			which Introduction mechanism is selected out of the several available.
+			For example, when Direct is used as the sole Introduction mechanism,
+			at a minimum a D-Client must be able to accept a single URL pointing at a TD.


As a consequence of the editor's note above, we don't have any requirements for the URL scheme. Is a D-Client able to fetch TDs only from ftp URLs compliant? or not?

The Discovery spec should specify the minimum requirements for D-Clients. It's already pretty loose ("any one Introduction") but we had a similar discussion with the Profile group about forcing clients to support HTTP. A more realistic example than ftp is a Client that only supports CoAP. Can it follow CoAP links to get TDs? Yes, it won't work for Directories, but what about direct peer-to-peer access?

Let's discuss today. Directories though need to be specifically for HTTP, so any use cases that need to use Directories won't work for other protocols.

Pros of requiring HTTP: Discovery is more interoperable, avoiding fragmentation into different verticals supporting different protocols.
Cons: only relatively powerful clients can implement HTTP (although to be honest, any client that can parse a TD can probably implement HTTP without too much trouble, even if they use other protocols for other things).

So a Consumer only support CoAP would not be a Discoverer according to this spec. CoAP also tends to go along with CBOR but we don't formally define that for TDs...

relu91 · 2022-02-16T10:46:24Z

index.html

+			Note that even if only one Introduction mechanism is supported multiple results might be 
+			produced.  This assertion states that the overall output of the Introduction phase is
+			a single set of URLs.  
+			The word "set" is used here in the mathematical sense: the results are unordered.


Mathematically speaking the results should be also unique. In principle, this would imply what is asserted above.

Hmmm... true, duplicate removal is also implied by "set". I think that is ok and what we want, but maybe it could be emphasized here. I still think I want to explicitly state duplicate removal in an assertion and not depend on the mathematical definition of "set".

added "unordered and unique"

relu91 · 2022-02-16T10:51:53Z

index.html

+		<li><span class="rfc2119-assertion" id="consumer-fetch-links">A D-Client MAY fetch source TDs from the targets of the links in a Thing Link
+			described in its initial set of TDs and add them into the set of TD results.</span>
+			This only adds the results of fetching TDs from a Thing Link to the set of
+			results. These new results do not delete the original TD describing the Exploration mechanism. 


These new results do not delete the original TD describing the Exploration mechanism.

Is a ThinkLink an exploration mechanism or is this a copy&paste artifact?

Should change "original TD describing the Exploration mechanism" to "TD describing the Thing Link". We could also delete them when resolved but I'm worried about extra metadata being carried along in Thing Links.

copy-and-paste issue. Cleaned up in revision.

relu91 · 2022-02-16T10:52:52Z

index.html

+		<li><span class="rfc2119-assertion" id="consumer-fetch-links">A D-Client MAY fetch source TDs from the targets of the links in a Thing Link
+			described in its initial set of TDs and add them into the set of TD results.</span>
+			This only adds the results of fetching TDs from a Thing Link to the set of
+			results. These new results do not delete the original TD describing the Exploration mechanism. 
+		</li>
+		<li><span class="rfc2119-assertion" id="consumer-fetch-iteration">A D-Client MAY fetch additional TDs iteratively from any Thing Link or
+			Exploration mechanism described in its set of TDs and add them into the set of TD results.</span>
+			This only adds the results of fetching TDs from a Thing Link to the set of
+			results. These new results do not delete the original TD describing the Exploration mechanism. 
+		</li>


I agree with @farshidtz it can be improved.

relu91 · 2022-02-16T10:56:47Z

index.html

+			results. These new results do not delete the original TD describing the Exploration mechanism. 
+		</li>
+		<li><span class="rfc2119-assertion" id="consumer-termination">A D-Client MAY terminate fetching additional TDs at any point or for
+			any reason.</span>


I think we should have something that distinguishes a successful terminated discovery process from something that is not. Otherwise, D-Consumers will never be sure that what they got from the D-Client is really complete.

Not sure we CAN define termination since in theory a set of results could point out into the open web and have a recursive set of links that pull in more and more results. I think the application just has to decide when to stop in such cases. There will, however, be cases when there are no more links to resolve.

I actually have a specific assertion that lets a Discoverer stop for any reason (for instance, they could simply have hit a time limit or a memory limit). This will make discovery a bit inconsistent between implementations, but it would be anyway given the other variations allowed (e.g. types of intros).

relu91 · 2022-02-16T11:04:32Z

index.html

+			The D-Client can fetch additional TDs in any order from any Exploration mechanism and is not
+			required to use all of them.
+	        </li>
+		<li><span class="rfc2119-assertion" id="consumer-track">A D-Client MUST track (by id) which TDs describing links or Exploration mechanisms


Is id mandatory for ThingLinks and ThingDirectories? cause, if my mind is not failing me, it is not for generic TDs. For TMs loop cycle detection we are using the href of links as an ID.

Ack, good point. A Thing Link or a Thing Directory without an id would be a pain. I guess we could add "or href, if no id is available".

I changed this to just say that loops should be detected, not how. Could also use full-result hashes, but I think this is an implementation decision, not a requirement.

farshidtz · 2022-02-21T14:13:14Z

index.html

+	The WoT discovery process is designed using a two-phase approach.
+	In the first phase, one or more of a set of "Introduction" mechanisms may be used to 
+	generate a set of candidate URLs.  These URLs do not themselves contain metadata,


Is this the process of discovering a single Thing? Why more than one Intro mechanism is used? How do the clients decide between the candidate URLs?

WoT Discovery gives back a set of TDs. The application then needs to decide what to do with them (Consume them all, pick one and Consume it, etc). We do not define what clients do with the results of Discovery, just the Discovery process itself.

farshidtz · 2022-02-21T14:15:07Z

index.html

+	Note that the discovery process can produce a <em>set</em> of URLs following 
+	its Introduction phase, even if only one Introduction mechanism is used,
+	and the final output after the Exploration phase
+	can be a <em>set</em> of Thing Descriptions.


Same as last comment. If the process is to discover one Thing, why is there a "set of URLs" after Intro, even if only one mechanism is used?

Some Intro mechanisms can return multiple results (CoreRD, DID, DNS-SD, etc). Will discuss today.

farshidtz · 2022-02-21T14:29:31Z

index.html

+	are only available to clients that support this protocol.  However, use of HTTP (and Directories)
+	is not in fact a requirement of the WoT Discovery process: it is possible to discover multiple
+	TDs using just Introduction mechanisms.


Getting the TD is part of exploration. That is possible with or without a directory.

Ah, true. Wording needs some tweaking. Should say "without using Directories".

reworded in update

farshidtz · 2022-02-21T14:34:55Z

index.html

+	We will use the term <a>D-Consumer</a> to refer to a WoT Consumer supporting WoT Discovery,
+	and D-Client as a more general term relating to any client supporting WoT Discovery.


The components of a system can be named arbitrarily. I can argue that I need a Discovery Parser which parses the TD object only. I can also argue that a TD is too large to pass around and should be transported and consumed incrementally (we have assertions for these).

Co-authored-by: Farshid Tavakolizadeh <email@farshid.ws>

mmccool · 2022-02-21T15:44:42Z

I added several inline comments.

I also suggest moving the Thing Link and Thing Directory type definitions along with the ontology from 6. Exploration Mechanisms to somewhere under chapter 4. Architecture, because there are now forward referenced.

To do

mmccool · 2022-02-21T15:48:01Z

I think the last use case is not really "describedby" but "extends" or similar. I propose we remove it:

A device intends to publish an entire TD which contains private and public parts: publish one TD (Thing Link) with only the public information referencing another TD which contains the full description.

Comment from Farshid: "Not having extra metadata" should not be a consequence of ThingLink itself, but of the describedby relation."

McCool: think it is cleaner if ThingLink does one thing. Private/public decisions are an access control policy, and should be decided when the TD itself is delivered. However, my proposal is to delete the use case, but not disallow extra metadata at this time (and my other descriptions of the resolution process keep Thing Links around, just in case they DO have metadata...).

Consensus:

Delete use case
Don't disallow metadata in Thing Links

mmccool · 2022-02-21T16:10:00Z

Update: just as we were closing the meeting Farshid said he would be ok in adding an assertion that Thing Links using the "describedby" relation should not have affordances. So I'll add an assertion of that nature (such an assertion would not forbid Thing Links with other relation types from having affordances). I would personally, however, like to extend this to say "should not have affordances or other metadata". This would cover things like locations (or other random metadata using an extension), version info, security definitions, etc. My goal is to allow a Discoverer to discard a Thing Link (using a describedby relation, anyway) once it is resolved.

…covery into consumer-process

mmccool · 2022-03-07T03:54:39Z

Did the following:

Removed the third use case for Thing Links as discussed. However, as I was doing so I re-read the other two and realized they also implied "non-pure" Thing Links, i.e. a link combined with other metadata and affordances. Since that was my problem with the third use case, removing just it does not solve the problem. I commented it out for now but did not actually delete the content...
Rewrote section on Discoverer process, dealing with several of the issues noted in the discussion above.

mmccool · 2022-03-07T03:58:26Z

Haven't implemented #274 (comment) yet. It is probably still a good idea, but maybe we should handle it in a separate issue/PR, as we'll probably have to do some significant rewrites in that section, including removing/changing ALL the example use cases, which all seem to depend on partial TDs.

farshidtz

Thanks, I think it is much clearer now. I still have some concerns but they aren't in assertions and can be addresses in future.

index.html

farshidtz · 2022-03-07T14:34:30Z

index.html

+		<li><span class="rfc2119-assertion" id="discoverer-fetch-iteration">
+			A Discoverer MAY fetch additional TDs iteratively from any Thing Link or
+			Exploration mechanism described in its set of TDs and add them into the set of TD results.</span>
+			This only adds the results of fetching TDs from an Exploration mechanism to the set of
+			results. These new results do not delete the original TD describing the Exploration mechanism. 
+		</li>


Not sure what this assertion adds to the previous two.

discussion: can treat as an improvement and do in a second round

farshidtz · 2022-03-07T14:39:17Z

index.html

+		<li><span class="rfc2119-assertion" id="discoverer-any-order">
+			A Discoverer MAY fetch additional TDs by following links or fetching
+			additional TDs from Exploration mechanisms (e.g. TDDs) in any order.</span>


This assertion is similar to the one on line 402:

A Discoverer MAY fetch additional TDs from any Exploration mechanism
described in its initial set of TDs (including, in particular, Thing Description Directories)
and add them into the set of TD results.

defer to later improvements; maybe merge with others

farshidtz · 2022-03-07T14:59:33Z

index.html

+	a Directory wanting to reference other Directories should include a Thing Link with a "describedby" relation to the
+	TD of the other Directory service.  Then the above process would expand the Thing Link to obtain the actual TD of the Directory, 


I understand that this is an informative recommendation. But it has two problems:

Not all directories are self-describing. This basically says each directory should have a TD hosted somewhere, but not in another directory. This leaves us with self-describing directories or TDs served via an stand-alone web server.

It adds an addition fetch operation which in practice is only to get the base path. All other affordances are already known to a Discoverer which knew how to read the first directory.

Federated SPARQL requires depend on the URL of SPARQL which is not available in a Think Link but in the TD.

IMO, the directories should be allowed and even recommended to register their TDs into other directories. I don't understand the benefit of recommending the use of Thing Link with "describedby" relation. The Thing Link can be used (with another relation type) to register the TD partially if the whole object isn't needed. It is practical to register a Think Link for a directory, including the directory's base path, some metadata to describe this directory, and maybe the SPAQRQL endpoint.

can we turn the above into an issue and deal with it in another PR?

Co-authored-by: Farshid Tavakolizadeh <email@farshid.ws>

mmccool · 2022-03-07T16:03:03Z

Suggest we merge but create issues for the remaining problems and try to resolve them with focused PRs.

mmccool added 2 commits February 13, 2022 10:52

update overview image

931323d

add instructdions for images

85e86d1

mmccool requested review from farshidtz and k-toumura February 13, 2022 15:11

mmccool added 2 commits February 13, 2022 11:56

copy over arch text from PR#269

de92747

add consumer process constraints

1b713af

mmccool added 3 commits February 14, 2022 10:30

repair section structure

ca831bb

multiple minimum requirements

5a3b6e3

d-clients

343b5f2