cost/benefit discussion

[dret]

comment from john curran (comment 1/2, other one is #28):

1. You do an excellent job outlining some of the overheard associated with the establishment of a registry (noted in various portions of sections 3,4, and 5) but you don’t seem to take that to the logical conclusion of observing that the decision to go with a registry (rather than protocol specification) does come with overhead, and that the costs of having protocol registry need to be considered as a cost/benefit tradeoff.

[dret]

@jcurranz, when you say cost/benefit, what exactly did you have in mind? i can see this as a cost/benefit question for the spec writers (simpler self-contained spec, but the need to update frequently and probably in-place a la "living standard"), or the spec users (need to deal with registry changes and figure out how to write stable code even though the registry is changing)?

[dret]

maybe the best place to put this would to be early in section 4 ("When")? without going into too much detail, it could point out that section 3 ("Why") explains why one might want to use registries, but that other solutions can be picked as well. does that sounds reasonable?

[jcurranz]

Actually, you can attribute them to me (jcurranz), if you want my Github handle.

[jcurranz]

It's clear that a set of network identifiers for a global Internet are not something to be hardcoded into the spec of a protocol, and deserve a registry. It's equally clear that one can (and does) often avoid the use of a registry if you can easily enumerate the various values in the specification (e.g. message types, options, etc.) The point is simply that you don't want to make everything a registry, but do want to consider a registry when the cost of externalizing (and making more adaptable) the values is worth the overhead of initial registry setup and specification.

Perhaps include an example of when NOT to both with an IANA registry, due to simplicity or stability of the values?

[dret]

On 2016-02-15 23:21 , John Curran wrote:

Actually, you can attribute them to me (jcurranz), if you want my Github
handle.

sorry for guessing the wrong one, should be fixed now. thanks!

[dret]

On 2016-02-15 23:28 , John Curran wrote:

It's clear that a set of network identifiers for a global Internet are
not something to be hardcoded into the spec of a protocol, and deserve a
registry. It's equally clear that one can (and does) often avoid the use
of a registry if you can easily enumerate the various values in the
specification (e.g. message types, options, etc.) The point is simply
that you don't want to make everything a registry, but do want to
consider a registry when the cost of externalizing (and making more
adaptable) the values is worth the overhead of initial registry setup
and specification.

interestingly, this of course is influenced a lot by the actual cost.
IANA has registries running and makes establishing a new one pretty
easy. but then you have to live with the constraints of the IANA platform.

maybe some spec writers would like to use a different kind of registry,
but then they cannot use the existing IANA template. this is where the
HTML5 authors said "let's just use a wiki". i am wondering how often
this is happening, that specs establish registries, but non-IANA ones.
is that even an option for an IETF spec?

anyway, the new section 5.1 of
https://github.com/dret/I-D/blob/master/registries/draft-wilde-registries-02.txt
(Registry Operations) now talks a little bit about that, but from a
slightly different angle.

i am wondering whether your cost/benefit point would benefit from having
"no registry", "IANA registry", and "BYO registry" options, or whether
than would complicate matters too much.

[jcurranz]

Remember - the cost is more than the cost of IANA operations... you've got an maintenance and upkeep cost no matter what - the wiki looks like a bargain, but you've ultimate got someone who has to be in charge or you run the risk of the wiki being captured and your protocol suffering as a result.

[dret]

On 2016-02-15 23:43 , John Curran wrote:

Remember - the cost is more than the cost of IANA operations... you've
got an maintenance and upkeep cost no matter what - the wiki looks like
a bargain, but you've ultimate got someone who has to be in charge or
you run the risk of the wiki being captured and your protocol suffering
as a result.

yes, IANA registries aren't free. you still need the workflow to work,
and that needs resources and some stability over time. but setting up a
non-IANA registry from the scratch may be more work. that's unless you
pick some "anybody can enter anything and we don't really care that
much" model of a wiki, in which you trade some of the operational cost
for quality attributes of the registry.

i think we're getting somewhere here: there are cost/benefit trade-offs
along a variety of axes, and the IANA solution makes some choices for
you but still leaves others open. and a non-registry model also makes
some choices and leaves others open.

[jcurranz]

And the choice to have a registry at all (as opposed to simply enumerated values in the protocol spec) is another axis of cost/benefit tradeoff. Ideally, it would be recognized that specifying a registry is not a decision without cost (both initial and ongoing) and it would be done with that in mind due to the anticipated greater benefit.

[asbjornu]

While setting up a non-IANA registry might be more costly over time, I think it is much more approachable. Everyone and their dog knows how to install and edit a Wiki. But the IETF and IANA process is so tangled up in old formats and protocols that it feels like an insurmountable task to even figure out how to do it. I know this is a misconceptions, but I believe the fact that this misconception exists should be analyzed, understood and respected instead of being waved off and ignored.

Had the processes been modelled and designed by competent user experience designers and every step of every process was presented in a beautifully laid out HTML, CSS and illustrations, with a navigational hierarchy designed by information architects and text written by information strategists, I believe the situation would be very different.

[jcurranz]

Setting up an IANA registry is actually relatively straightforward - one writes an IANA Considerations Section with appropriate text and it happens. That aside, noting the option of a non-IANA registry is quite a reasonable thing to do, and has benefits in many scenarios. It should also be noted that a wiki-based registry carries the risk of hostile takeover (either by rogue admin or vox populi voting) with serious repercussions for protocol implementors if it occurs.

[dret]

On 2016-02-16 13:01, Asbjørn Ulsberg wrote:

While setting up a non-IANA registry might be more costly over time, I
think it is much more approachable. Everyone and their dog knows how to
install and edit a Wiki. But the IETF and IANA process is so tangled up
in old formats and protocols that it feels like an insurmountable task
to even figure out how to do it. I know this is a misconceptions, but I
believe the fact that this misconception exists should be analyzed,
understood and respected instead of being waved off and ignored.

it's not waved off and ignored. but it's complicated, and simple
solutions such as "let's just run one wiki with a nebulous process on
some server for that one registry" are not really a solution. IANA has
~2000 registries now. they have been around for a long time, and serve
many different needs. many of those needs would not be well-served by
just setting up a wiki. and like most of us using wikis have learned
over time, the initial perceived simplicity does have substantial
long-term implications, if the goal still is to achieve the results set
by RFC 7500.

that being said it is clear to everybody that IANA registries and the
process may look a bit old-fashioned. that's true because they are. but
if that's the only complaint, then there's not much to worry about,
imho. read the docs how to do it and get over it.

Had the processes been modelled and designed by competent user
experience designers and every step of every process was presented in a
beautifully laid out HTML, CSS and illustrations, with a navigational
hierarchy designed by information architects and text written by
information strategists, I believe the situation would be very different.

IETF always has heavily favored substance over form. UI/UX perfection
and instant gratification should not be the primary goal when thinking
about why, when, and how to use registries. they are not intended for
casual end users, they just need to work well and robustly for devs.

but: IETF is very open. if anybody feels like the substance needs to be
presented in a better way, then they are free to do so and contribute
that to the community. but keep in mind that from an economical
perspective, it really doesn't make sense to allow all 2000 registries
to morph into beautifully polished unicorns. that would very likely
cause huge maintenance headaches and costs further down the line.

[dret]

Setting up an IANA registry is actually relatively straightforward - one
writes an IANA Considerations Section
https://tools.ietf.org/html/rfc5226 with appropriate text and it
happens. That aside, noting the option of a non-IANA registry is quite a
reasonable thing to do, and has benefits in many scenarios.

i am currently writing about DNS as a non-IANA example (#29) and
probably also should add a non-IANA/non-DNS example. do you have an IETF
spec you could point me to that does that?

It should
also be noted that a wiki-based registry carries the risk of hostile
takeover (either by rogue admin or vox populi voting) with serious
repercussions for protocol implementors if it occurs.

a wiki carries more risks than you can shake a stick at. i have seen
most wikis go down the drain for a large variety of reasons, and very
few actually doing well over a long time. those that do well need a
lot of process around them, which in the end really doesn't make them
any different from other structured workflows.

this whole "let's use a registry wiki" idea was actually one motivation
to write this draft. instead of starting with a solution, i am hoping to
get people to think about why they want to use a registry, and how they
are seeing it being used and managed. a wiki might be one way of doing
it, for some set of constraints. but i am doubting that unless it is a
very structured wiki, it will be a good platform for most needs that we
see today.

[jcurranz]

i am currently writing about DNS as a non-IANA example

I'm slightly confused by the above statement... DNS is quite certainly an IANA registry; the IETF reserves the right to make reservations from the DNS for special/technical purposes and delegates authority for the remainder of the space to ICANN for administration.

When you mean non-IANA registry, is that a reference to the authority, the publication method, protocol used for access, or something else?

[jcurranz]

A wiki is a mechanism... As with most mechanisms, it serves as a representation of an authority model for conditions under which entries may be made. This may be a very loose and open authority model ("the OpenFizzBinConnector trade association allows any OpenFizzBinDevelopers to registry their OpenFizzBinDeviceCode in the following wiki...") or a more tightly administered authority model ("the OpenFizzBinConnector trade association will update the wiki with those OpenFizzBinDeviceCodes that have been approved by the Board in a meeting called for that purpose.")

In and of itself, the phrase "we'll use a wiki" doesn't identify anything regarding authority or policy for the registry.

[dret]

On 2016-02-16 13:40, John Curran wrote:

i am currently writing about DNS as a non-IANA example

I'm slightly confused by the above statement... DNS is quite certainly
an IANA registry; the IETF reserves the right to make reservations from
the DNS for special/technical purposes
http://www.iana.org/assignments/special-use-domain-names/special-use-domain-names.xhtml
and delegates authority for the remainder of the space to ICANN
https://tools.ietf.org/html/rfc2860 for administration.
When you mean non-IANA registry, is that a reference to the authority,
the publication method, protocol used for access, or something else?

i just meant "as opposed to the IANA registries run via RFC 7500 and RFC
5226". if you want to add a new namespace (different RR type) to DNS,
then that's a very different story from just setting up one more
IANA-hosted registry, and you probably have a good reason to choose DNS
as the platform for your registry. does that clear things up?

there are separate issues for the DNS example (#29) and some other
example, which is neither using the plain IANA platform nor DNS (#30).

[dret]

On 2016-02-16 13:46, John Curran wrote:

In and of itself, the phrase "we'll use a wiki" doesn't identify
anything regarding authority or policy for the registry.

+1. it's just a different platform. if you have zero authority or policy
constraints, it's probably a great platform. as soon as you start adding
those, the platform gets less and less suitable, until it is not really
reasonably called a wiki anymore.

[jcurranz]

On Feb 16, 2016, at 7:46 AM, Erik Wilde notifications@github.com wrote:

i just meant "as opposed to the IANA registries run via RFC 7500 and RFC
5226". if you want to add a new namespace (different RR type) to DNS,
then that's a very different story from just setting up one more
IANA-hosted registry, and you probably have a good reason to choose DNS
as the platform for your registry. does that clear things up?

Got it - that’s the use of DNS as mechanism for hosting some
other registry.

The DNS RR type’s are contained within an IANA registry.
The DNS space itself is an IANA registry.

there are separate issues for the DNS example (#29) and some other
example, which is neither using the plain IANA platform nor DNS (#30).

I would not focus so much on the particular IANA platform - it is subject
to change and in the end is simply a mechanism for implementation of
a specific registry policy.

For example, the current IANA web pages (with text and xml registry
delineations) may easily actually be maintained internally via a wiki
(I don’t really know either way) - if that wiki interface was suddenly
exposed publicly, it wouldn’t change a thing since 99.999% of those
going to pages would not have edit ability.

I think much of what you are trying to write about is the policy for update
of an IANA registry, and this is quite distinct from particular mechanism.

/John

[dret]

On 2016-02-16 13:53, John Curran wrote:

On Feb 16, 2016, at 7:46 AM, Erik Wilde notifications@github.com wrote:

i just meant "as opposed to the IANA registries run via RFC 7500 and RFC
5226". if you want to add a new namespace (different RR type) to DNS,
then that's a very different story from just setting up one more
IANA-hosted registry, and you probably have a good reason to choose DNS
as the platform for your registry. does that clear things up?
Got it - that’s the use of DNS as mechanism for hosting some
other registry.

yes, the DNS as a platform. a rather different one than the usual IANA
registries.

The DNS RR type’s are contained within an IANA registry.
The DNS space itself is an IANA registry.

yes, that's where the two things have some overlap, but that's not
relevant for the things i am discussing.

there are separate issues for the DNS example (#29) and some other
example, which is neither using the plain IANA platform nor DNS (#30).
I would not focus so much on the particular IANA platform - it is subject
to change and in the end is simply a mechanism for implementation of
a specific registry policy.

mostly true. but as a spec author thinking about establishing a
registry, it's the one platform i can easily choose, but that comes with
built-in constraints.

for example, if for some reason i want access to a complete history of
all changes that have been made over time, then the IANA platform does
not do that for me. and then i can ask myself: do i want to change my
policy and not have that history, or do i really need it and then have
to go through the effort of effectively establishing my own platform.

I think much of what you are trying to write about is the policy for update
of an IANA registry, and this is quite distinct from particular mechanism.

i am also talking about runtime access, for example. what if i needed an
API to get registry contents? or an API to get registry history? the
IANA platform has evolved as one implementation out of the policies set
by IETF. you could have many other possible implementations, all
satisfying the same policies. but they might have different
side-effects, which might or might not be interesting for spec writers.

for example, the RDF community would like to get access to registry
contents as linked data (i.e., in some RDF representation). that's not
something the current platform does. it could be added as a feature,
without disrupting much, but that's very much a platform issue. even
though you're right that in order to do that robustly, you would
probably have to add this on the policy level ("all/some data in a
registry must be available in RDF using the following vocabulary."), and
then the platform would have to follow suit.

anyway, this is getting a bit off-topic. but great discussion, and
things are definitely shifting in place a bit better than when i started
working on this. i may have to restructure the whole thing a bit at some
point in time.

[jcurranz]

for example, the RDF community would like to get access to registry
contents as linked data (i.e., in some RDF representation). that's not
something the current platform does. it could be added as a feature,
without disrupting much, but that's very much a platform issue. even
though you're right that in order to do that robustly, you would
probably have to add this on the policy level ("all/some data in a
registry must be available in RDF using the following vocabulary."), and
then the platform would have to follow suit.

You are correct. If an IANA Considerations section indicated that as a requirement, then the IANA would figure out how that gets implemented. The real questions would remain "Whom is the policy authority for the registry?" and "What is the initial policy for the registry?"

[asbjornu]

@jcurranz:

Setting up an IANA registry is actually relatively straightforward - one writes an IANA Considerations Section with appropriate text and it happens.

It's straigtforward for anyone familiar with IETF, IANA and writing RFC's. For everyone else, it looks like what it is: a very archaic description provided in a very archaic format (plain text) that is not in any way obviously created for human consumption, describing how to write a paragraph in what needs to be published as an RFC, which in itself is incredibly hard to do. Just the toolchain required to write an RFC may require weeks of deep study, trial and error, unless you accidentally find something like @martinthomson's I-D Template. And even then, it's far from anything resembling something that can be described as "straightforward".

@dret:

it's not waved off and ignored. but it's complicated, and simple solutions such as "let's just run one wiki with a nebulous process on some server for that one registry" are not really a solution.

I completely agree. "Let's just run a wiki" is a bad solution. But I completely understand why people choose to do that, given the current state of affairs.

IANA has ~2000 registries now. they have been around for a long time, and serve many different needs. many of those needs would not be well-served by just setting up a wiki. and like most of us using wikis have learned over time, the initial perceived simplicity does have substantial long-term implications, if the goal still is to achieve the results set by RFC 7500.

Absolutely. I do not oppose the birds-eye view of the current procedure. But I find the friction of every detail to be so high that it amounts to something that can easily be viewed as "impossible".

that being said it is clear to everybody that IANA registries and the process may look a bit old-fashioned. that's true because they are. but if that's the only complaint, then there's not much to worry about, imho. read the docs how to do it and get over it.

I've read the docs and been a contributor to several RFC's and still find it incredibly hard. Perhaps I'm just dim, but at least I find myself in good company. :-)

IETF always has heavily favored substance over form. UI/UX perfection and instant gratification should not be the primary goal when thinking about why, when, and how to use registries. they are not intended for casual end users, they just need to work well and robustly for devs.

I agree that UX shouldn't be the primary goal. But as it currently is, it doesn't seem to be a goal at all. If anything, it seems like having a high-friction, bad-UX, no-design process is almost intentional. I know it isn't, but that's how bad I think it is. Sorry for being blunt.

but: IETF is very open. if anybody feels like the substance needs to be presented in a better way, then they are free to do so and contribute that to the community. but keep in mind that from an economical perspective, it really doesn't make sense to allow all 2000 registries to morph into beautifully polished unicorns. that would very likely cause huge maintenance headaches and costs further down the line.

Perhaps. I know way too little about the tools involved in maintaining the existing registries, why the normative format of RFC's still can't be HTML, why it needs to be based on XML with a complex XSLT toolchain, etc. I'm new to most of this. But from a UX perspective, I think it's fair to say that there's a few things left to be desired. I'd love to help, but haven't even managed to publish my own I-D, so I don't know where to begin. That is also a very different discussion than that of this issue.