Skip to content
This repository has been archived by the owner on Feb 17, 2024. It is now read-only.

Trait merging #529

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Trait merging #529

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
325 changes: 278 additions & 47 deletions versions/raml-10/raml-10.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,7 @@ Throughout this specification, **Markdown** means [GitHub-Flavored Markdown](htt
- [Applying Resource Types and Traits](#applying-resource-types-and-traits)
- [Resource Type and Trait Parameters](#resource-type-and-trait-parameters)
- [Declaring HTTP Methods as Optional](#declaring-http-methods-as-optional)
- [Algorithm of Merging Traits and Methods](#algorithm-of-merging-traits-and-methods)
- [Resource Types and Traits Effect on Collections](#resource-types-and-traits-effect-on-collections)
- [Algorithm of Merging Traits and Resource Types](#algorithm-of-merging-traits-and-resource-types)
- [Security Schemes](#security-schemes)
- [Security Scheme Types](#security-scheme-types)
- [Security Scheme Declaration](#security-scheme-declaration)
Expand Down Expand Up @@ -2367,18 +2366,86 @@ resourceTypes:
# not required; same for the X-Chargeback header
```

### Algorithm of Merging Traits and Methods
### Algorithm of Merging Traits and Resource Types

Each RAML element has its branch of the RAML document. The high-level description of applying a trait to method is putting a traits branch under the methods branch. Actually, applying a trait to a method is a recursive procedure:
In order to apply trait to a method, trait YAML node and its sub-nodes are merged with method YAML node and its sub-nodes.
Everything described below for methods and traits also works in the same way for resource types and resources unless stated otherwise.

1. Method node properties are inspected and those that are undefined in trait node remain unchanged.
2. The method node receives all properties of trait node (excluding optional ones), which are undefined in the method node.
3. Properties defined in both method node and trait node (including optional ones) are treated as follows:
* Scalar properties remain unchanged.
* Collection properties are merged by value, as described later.
* Values of object properties are subjected to steps 1-3 of this procedure.
Algorithm has three phases:
1. YAML-level merging
2. Parameters application
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameters should be applied before merging, otherwise key parameters will not be correctly merged

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! In my tries to simplify the description I went too far. In practise, imho phases 1 and 2 should be performed simultaneously.

Applying parameters before merging, while possible, is probably going to complicate algorithm even more: the same trait / resource type can be used along the hierarchy with different parameters, so we'll have to save a copy for a trait/resource type and all of its dependencies (or at least the ones affected by parameter value) for each set of parameter values.

I'll try to refactor this description for a single pass of phases 1 and 2.

3. Reference patching

Generally, a method can have more than one trait, each having a sufficient hierarchy. Applying all traits is equivalent to building a stack of branches as follows:
#### Phase 1: YAML-level merging
At the first phase, YAML trees of methods, traits, resources and resource types are merged.

The following node merging procedure is applied for each YAML sub-node of trait and method, recursively:

##### Single trait or resource type merging algorithm

1. Method YAML MAP node MAPPINGs are inspected and those that have keys undefined in trait MAP node remain unchanged.
2. For each YAML MAPPING in trait node with key undefined in method node, the MAPPING is copied to the method node. This does not apply to the keys ending with "?" character (optional properties).
3. MAPPINGs with the keys defined in both method node and trait node (including optional ones) are treated as follows:
* MAPPING with value of SCALAR type remain unchanged.
* MAPPING with value of SEQUENCE type is merged by value, as described in the next section `Resource Types and Traits Effect on Collections`.
* For MAPPING with value of MAP type this procedure is repeated, recursively

##### Resource Types and Traits Effect on Collections

All collections or sequences that are affected by applied traits and resource types are merged. This example defines the enum values of a query parameter in both the trait and resource:

```yaml
#%RAML 1.0
title: Example API
version: v1
traits:
withQueryParameters:
queryParameters:
platform:
enum:
- win
- mac
/installer:
get:
is: [ withQueryParameters ]
queryParameters:
platform: #the actual enum is [ mac, unix, win ]
enum:
- mac
- unix
```

The enum value resulting from the merge that occurs is `[ mac, unix, win ]`.

In collections, a trait can appear as an "is" attribute value for a method, resource, trait, or resource type. Such lists can contain the same traits but different parameter sets and, thus, cannot be considered equal:

```yaml
#%RAML 1.0
title: Example API
version: v1
resourceTypes:
apiResource:
get:
is: [ { secured : { tokenName: access_token } } ]
traits:
secured:
queryParameters:
<<tokenName>>:
description: A valid <<tokenName>> is required
/servers:
get:
is: [ { secured : { tokenName: token } } ]
```

To resolve a collision arising from this inequality, priority is given to the trait in closest proximity to the target method or resource. In the previous example, the `tokenName` parameter value for the `GET:/servers` method is `token`, and the trait list consists of single trait occurrence: `[ {secured:{ tokenName:token}} ]`.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, we support applying same trate with different parameters raml-org/raml-js-parser-2#94




##### Applying several traits or resource types

Generally, a method can have more than one trait, each having a sufficient hierarchy.

Applying all traits is equivalent to building a stack of branches, where each branch is a sub-tree of a respective node YAML tree, and applying such branches according to `Single trait or resource type merging algorithm` in the following order:

* The top branch is the methods branch.
* Other branches are traits branches.
Expand All @@ -2388,6 +2455,8 @@ Generally, a method can have more than one trait, each having a sufficient hiera
* Queue(d+1) is obtained from Queue(d) by concatenating trait lists of its elements and canceling all but the first occurrence of each trait.
* Branch order is determined as follows: traits that have higher positions in the queue, have branches deeper in the stack.

##### Applying mixed traits and resource types

Finally, the resource can have its own traits, and a chain of resource types, for example resourceType1, resourceType2, ..., can be applied. Each resource type can potentially have its own traits and define the same method. The stack is constructed as follows:

1. Traits of method itself
Expand Down Expand Up @@ -2433,54 +2502,216 @@ Every explicit node wins over the ones that are declared in a resource type or t
application/json:
```

### Resource Types and Traits Effect on Collections
#### Phase 2: Parameters application

All collections or sequences that are affected by applied traits and resource types are merged. This example defines the enum values of a query parameter in both the trait and resource:
At the second phase of an algorithm, YAML tree is modified to apply the parameter values. The result of such application is also an YAML tree.

Nodes are being traversed in exactly the same order as in Phase 1. For each node containing parameter template (application), the template is being textually replaced with parameter value got from respective trait or resource type application in a method or resource. Changed nodes and text ranges inside the node are being associated with the unit, where the value is defined in for further use in phase 3.

If parameter application adds new trait reference to a method or new resource type to a resource, or make them resolvable, Phase 1 is repeated for such a trait or resource. Due to that, it is often more convenient to perform Phase 1 and Phase 2 simultaneously.

#### Phase 3: Reference patching

At the third phase, the whole RAML tree is traversed. For each node, which is defined by RAML language as a reference (like type reference), an attempt to resolve the reference according to the algorithm defined in `Reference resolving` section. If the reference is resolved, it is determined whether reference text needs to be patched in order to be accessed correctly from the main RAML unit (file), and patched if needed by an algorithm described in `Reference patching` section. The purpose is to have the valid RAML document after such patching.

In example, for the following files:

```yaml
#%RAML 1.0
title: Example API
version: v1
traits:
withQueryParameters:
queryParameters:
platform:
enum:
- win
- mac
/installer:
get:
is: [ withQueryParameters ]
queryParameters:
platform: #the actual enum is [ mac, unix, win ]
enum:
- mac
- unix
# main.raml
title: main
uses:
a: libA.raml

/users:
type: a.rtA
```

The enum value resulting from the merge that occurs is `[ mac, unix, win ]`.
```yaml
#%RAML 1.0 Library
# libA.raml

In collections, a trait can appear as an "is" attribute value for a method, resource, trait, or resource type. Such lists can contain the same traits but different parameter sets and, thus, cannot be considered equal:
types:
Monkey: number

resourceTypes:
rtA:
post:
headers:
hA2: Monkey
```

The result of Phase 2 looks like this:

```yaml
#%RAML 1.0
title: Example API
version: v1
resourceTypes:
apiResource:
get:
is: [ { secured : { tokenName: access_token } } ]
traits:
secured:
queryParameters:
<<tokenName>>:
description: A valid <<tokenName>> is required
/servers:
get:
is: [ { secured : { tokenName: token } } ]
# main.raml
uses:
a: libA.raml

/users:
post:
headers:
hA2: Monkey
```

To resolve a collision arising from this inequality, priority is given to the trait in closest proximity to the target method or resource. In the previous example, the `tokenName` parameter value for the `GET:/servers` method is `token`, and the trait list consists of single trait occurrence: `[ {secured:{ tokenName:token}} ]`.
But there is no `Monkey` type defined in `main.raml` file. So we need to patch `Monkey` text of the reference to become `a.Monkey` to make it a valid reference:

```yaml
#%RAML 1.0
# main.raml
uses:
a: libA.raml

/users:
post:
headers:
hA2: a.Monkey
```

Each reference is resolved and patched individually. So in case of type expressions, each participating type is resolved and patched individually.

This will be the main example for the next sections:

```yaml
#%RAML 1.0
# main.raml
title: type shadows
uses:
a: libA.raml

types:
Monkey: string

/users:
type:
a.rtA:
kind: Monkey
```

```yaml
#%RAML 1.0 Library
# libA.raml

uses:
b: libB.raml

types:
Monkey: number

resourceTypes:
rtA:
type:
b.rtB:
kindB: <<kind>>
kindB2: Monkey
post:
headers:
hA: <<kind>>
hA2: Monkey
```

```yaml
#%RAML 1.0 Library
# libB.raml

types:
Monkey: boolean

resourceTypes:
rtB:
post:
headers:
hB: <<kindB>>
hB2: Monkey
hB3: <<kindB>> | <<kindB2>> | Monkey
```

Phase 2 result for this example is:

```yaml
#%RAML 1.0
# main.raml
title: type shadows
uses:
a: libA.raml

types:
Monkey: string

/users:
post:
headers:
hA: Monkey
hA2: Monkey
hB: Monkey
hB2: Monkey
hB3: Monkey | Monkey | Monkey
```

##### Reference resolving

For each reference a queue of RAML units (files) is calculated and an algorithm should look for the reference target in each unit, in turn. The first matching reference name and kind is treated as a result. For the example above, we perform the resolving of each `Monkey` reference in Phase 2 results.

###### Static reference resolving

In case of reference value being NOT a result of template parameter substitution, the units queue is opposite to the units and order of node traversing in Phase 1 leading from initial resource or method node to the reference application node.

For the example above and `hA2: Monkey` reference phase 1 algorithm starts from `/users` resource node defined in unit `main.raml` and ends in `hA2: Monkey` node in `libA.raml` unit. So the final unit sequence for phase 3 is opposite to that order: [`libA.raml`,`main.raml`].
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a static reference should only be checked in the context where it was defined

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, this is a decision to not tot be taken lightly.

I personally like this approach, but I've seen several times real users were putting a reference into an fragment (or other included file), where such a reference was unresolved in that particular unit context, but was resolvable in the context of including file.

Are we going to still support that? If yes, then why we forbid similar behavior for dependency on traits?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe fragments should not define their own context unless they have libraries themselves, and act as regular non-fragment includes. If they have libraries then they define their own context and cannot access references from the parent context.
If we want fragments defining their own context always (no matter if they use libraries or not) then the user will need to use a non-fragment include in order to use the references from the parent file.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The concept of an included file either defining or not defining its own context basing on some circumstances will probably cause a lot of confusion for the users.

Basing functionality on a split of includes into "fragments" and "not fragments" will probably also make it hard.

I'd suggest one of the two approaches: we either allow resolving static references from all the contexts along the line, or we only allow current context and make an official exception in case of includes.

What do you think?


As a result `hA2: Monkey` reference target will be resolved to be pointing at `Monkey` type defined in `libA.raml` as this unit is the first one in the queue containing type called `Monkey`.

For the example above and `hB2: Monkey` reference phase 1 algorithm starts from `/users` resource node defined in unit `main.raml`, then goes to `rtA` resource type sub-nodes defined in `libA.raml` unit and ends in `hB2: Monkey` node in `libB.raml` unit. So the final unit sequence for phase 3 is opposite to that order: [`libB.raml`, `libA.raml`, `main.raml`].

As a result `hB2: Monkey` reference target will be resolved to be pointing at `Monkey` type defined in `libB.raml` as this unit is the first one in queue containing type called `Monkey`.

###### Parameter value reference resolving

In case of reference value being a result of template parameter substitution, the units queue is matching units and order of node traversing in Phase 1 leading from the node setting parameter value to the reference application node. If the node setting parameter value is in turn a result of other parameter application, that parameter value setting node is taken instead, recursively. If a single reference contains a result of more than a single parameter value substitution or a single reference crosses the textual boundaries of parameter value, such reference is resolved according to the static reference resolving rules.

For the example above and `hA: <<kind>>` reference defined in `libA.raml`, the node setting its value is `kind: Monkey` in `main.raml`. So the unit sequence for phase 3 is phase 1 path from `kind: Monkey` (`/users`) to `hA: <<kind>>`, and that is [`main.raml`, `libA.raml`].

As a result in this case, `kind: Monkey` will be resolved to `Monkey` type defined in `main.raml` as this unit is the first one in queue containing type called `Monkey`.

For the example above and `hB: <<kindB>>` reference defined in `libB.raml`, the node setting its value is `kindB: <<kind>>` in `libA.raml`, and its value is in turn set by `kind: Monkey` in `main.raml`. So the unit sequence for phase 3 is phase 1 path from `kind: Monkey` (`/users`) to `hB: <<kindB>>`, and that is [`main.raml`, `libA.raml`, `libB.raml`].

As a result in this case, `kind: Monkey` will be resolved to `Monkey` type defined in `main.raml` as this unit is the first one in queue containing type called `Monkey`.

##### Reference patching

After reference target is resolved, reference text must be patched to become a correct reference from the main unit to the previously found reference target.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this patching mechanism looks like an implementation detail and shouldn't be required as long as the references are correctly resolved.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, these patched references are going into JSON. Users are going to see them. Why do you think its implementation details?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK the JSON serialization is not part of the spec, otherwise it should be there.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree, JSON is not a part of the spec, so theoretically we may probably skip specifying how the result will look like.

But, taking into account we're trying to make parsers compatible and this JSON is something common between the parsers, why not to make clear what exactly to do with the references?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the problem I see is that in this section we provide information related to the json output, but we are not doing the same in all the other chapters. For me everything concerning the json output should be added to an appendix that describes the TCK serialization and is optional to implement.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I agree with @svacas about the patching mechanism. You shouldn't need it if you have already resolved references.
But I think the rules for visibility, scopes, etc are not very well defined. I can only find the Applying Libraries section in the spec.

Some questions about it:

  • Does every trait file open a new namespace and declaration scope?
  • Does these library namespaces have a name or are 'anonymous' namespaces?
  • I think it confuses an alias for a namespace (what we use to reference the library in the document) with the actual identifier of the namespace (that I think right now is anonymous)
  • Does the same rules for includes apply to uses statement to locate a library? Shouldn't they be all of the library references treated just as URIs?

For example, is this:

#%RAML 1.0
# libA.raml

types:
  Monkey: string

actually:

#%RAML 1.0
# file://libA.raml

types:
  file://libA#Monkey: string

And if that's the case, doing this:

#%RAML 1.0
#% main.raml

uses:
  a: libA.raml

/users:
  post:
    headers:
      hA2: a.Monkey

Just a shortcut for:

#%RAML 1.0
#% file://main.raml

/users:
  post:
    headers:
      hA2: file://libA#Monkey
  • If that's the case, is there a way to reference a symbol directly without a uses statement

Something like:

#%RAML 1.0
#% main.raml

/users:
  post:
    headers:
      hA2: http://raml.org/traits/libA#Monkey

It looks like RAML has a clear notion of modularity but lacks a well defined way of referencing the information in the modules.

Copy link
Author

@ddenisenko ddenisenko Jul 26, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These questions are more about the whole reference resolving system rather than about traits and resource types.

Does every trait file open a new namespace and declaration scope?

Every RAML file opens a new namespace and declaration scope. Fragments and non-fragment includes are in question at the moment.

Does these library namespaces have a name or are 'anonymous' namespaces?

Their are anonymous. There is not global library identifier/URL.

I think it confuses an alias for a namespace (what we use to reference the library in the document) with the actual identifier of the namespace (that I think right now is anonymous)

There is no confusion in this case because there is no such thing as the actual identifier of the namespace/unit in the spec. The only way of library identification is the path in uses instruction and thus identification is always context-dependent.

Does the same rules for includes apply to uses statement to locate a library? Shouldn't they be all of the library references treated just as URIs?

# notion in include paths does not work for RAML files if I am not missing something. This one is for schemas. uses instruction does not support # at all.

There is absolutely no way of referencing something directly without an alias set up by uses instruction, and that is the reason why I want the reference patching to be the part of the spec.

See, the whole trait merging algorithm is the algorithm of translation of one RAML tree (or RAML trees) into another one. The result of such translation should presumably be a valid RAML. While it is not mentioned in the spec, that result should be printed at some point of time, whether it will be JSON or some other form of output.

If I got it right, you want to cut off the last part of this translation. So, we do describe how exactly we modify the resulting tree by adding contents of a trait to a method, we describe how we modify the resulting tree by applying parameters, but we want to stop here and be silent regarding how we modify the tree to make the references valid/resolvable? Just mention that "references should be valid in the result" and that's it? Why so?

If there is no `uses` instruction in the main unit, which imports the unit, reference target is defined in, such an instruction must be automatically generated.

For the example above, and `hA2: Monkey` reference we found that its target is `Monkey` type defined in `libA.raml` unit. Correct reference from `main.raml` unit for this type will be `a.Monkey`, so we patch `hA2: Monkey` to become `hA2: a.Monkey`

For the example above, and `hB: Monkey` reference we found that its target is `Monkey` type defined in `main.raml` unit. Correct reference from `main.raml` unit for this type will be `Monkey`, so the change to `hB: Monkey` is not needed.

For the example above, and `hB2: Monkey` reference we found that its target is `Monkey` type defined in `libB.raml` unit. Correct reference from `main.raml` unit for this type is impossible to build as `main.raml` unit does not import `libB.raml` unit. So the `uses` instruction of `b: libB.raml` should be automatically generated, and the correct reference is then `b.Monkey`, so we patch `hB2: Monkey` to become `hB2: b.Monkey`

For `hB3: Monkey | Monkey | Monkey` we find individual target for each `Monkey` type reference in the expression and patch it accordingly to become `hB3: Monkey | a.Monkey | b.Monkey`.

The final patching result for the example above is:

```yaml
#%RAML 1.0
# main.raml
title: type shadows
uses:
a: libA.raml
b: libB.raml

types:
Monkey: string

/users:
post:
headers:
hA: Monkey
hA2: a.Monkey
hB: Monkey
hB2: b.Monkey
hB3: Monkey | a.Monkey | b.Monkey
```

## Security Schemes

Expand Down