as-set member validation behaves differently in single line and multiple line values

[briandfoy]

There's a difference in IRRD's behavior validating as-set object when the member line is a single line or multiple lines. It looks like spreading out an invalid member value over multiple lines lets an otherwise invalid value sneak through.

If I submit an as-set where I forget the commas in between members but all the values are on one line (so, no continuation lines), the submissions fails. I'd expect it to complain about whitespace, but there's a different error. However, it still fails like it should:

as-set: AS-BIFF-TEST-NO-COMMAS
descr: RADBNET-1779
members: AS-SOMETHING AS398465 AS398466 AS398467
tech-c: DUMMY-RADB
admin-c: DUMMY-RADB
notify: biff@example.com
mnt-by: MAINT-AS135602
changed: biff@example.com
source: RADB

$ irr_rpsl_submit -h irrd -p 8181 -D -j < objects.txt | jq -r .
{
  "request_meta": {
    "HTTP-Client-IP": "::ffff:192.168.96.10",
    "HTTP-User-Agent": "irr_rpsl_submit_v4"
  },
  "summary": {
    "objects_found": 1,
    "successful": 0,
    "successful_create": 0,
    "successful_modify": 0,
    "successful_delete": 0,
    "failed": 1,
    "failed_create": 0,
    "failed_modify": 0,
    "failed_delete": 0
  },
  "objects": [
    {
      "successful": false,
      "type": null,
      "object_class": "as-set",
      "rpsl_pk": "AS-BIFF-TEST-NO-COMMAS",
      "info_messages": [],
      "error_messages": [
        "Invalid AS number AS-SOMETHING AS398465 AS398466 AS398467: number part is not numeric",
        "Invalid set AS-SOMETHING AS398465 AS398466 AS398467: component AS-SOMETHING AS398465 AS398466 AS398467 is not a valid AS number nor a valid set name",
        "Invalid AS number AS-SOMETHING AS398465 AS398466 AS398467: number part is not numeric",
        "Invalid set AS-SOMETHING AS398465 AS398466 AS398467: component AS-SOMETHING AS398465 AS398466 AS398467 is not a valid AS number nor a valid set name"
      ],
      "new_object_text": null,
      "submitted_object_text": "as-set: AS-BIFF-TEST-NO-COMMAS\ndescr: RADBNET-1779\nmembers: AS-SOMETHING AS398465 AS398466 AS398467\ntech-c: DUMMY-RADB\nadmin-c: DUMMY-RADB\nnotify: biff@example.com\nmnt-by: MAINT-AS135602\nchanged: biff@example.com\nsource: RADB\n"
    }
  ]
}

However, if I change the object so that there is one line per member, still without commas, the submission succeeds:

as-set: AS-BIFF-TEST-NO-COMMAS
descr: RADBNET-1779
members: AS-SOMETHING
  AS398465
  AS398466
  AS398467
tech-c: DUMMY-RADB
admin-c: DUMMY-RADB
notify: biff@example.com
mnt-by: MAINT-AS135602
changed: biff@example.com
source: RADB

Now the submission succeeds:

$ irr_rpsl_submit -h irrd -p 8181 -D -j < objects.txt | jq -r .
{
  "request_meta": {
    "HTTP-Client-IP": "::ffff:192.168.96.10",
    "HTTP-User-Agent": "irr_rpsl_submit_v4"
  },
  "summary": {
    "objects_found": 1,
    "successful": 1,
    "successful_create": 0,
    "successful_modify": 1,
    "successful_delete": 0,
    "failed": 0,
    "failed_create": 0,
    "failed_modify": 0,
    "failed_delete": 0
  },
  "objects": [
    {
      "successful": true,
      "type": "modify",
      "object_class": "as-set",
      "rpsl_pk": "AS-BIFF-TEST-NO-COMMAS",
      "info_messages": [],
      "error_messages": [],
      "new_object_text": "as-set:         AS-BIFF-TEST-NO-COMMAS\ndescr:          odd things\nmembers:        AS-SOMETHING\n                AS398465\n                AS398466\n                AS398467\ntech-c:         DUMMY-RADB\nadmin-c:        DUMMY-RADB\nnotify:         biff@example.com\nmnt-by:         MAINT-AS135602\nchanged:        biff@example.com\nsource:         RADB\n",
      "submitted_object_text": "as-set: AS-BIFF-TEST-NO-COMMAS\ndescr: odd things\nmembers: AS-SOMETHING\n  AS398465\n  AS398466\n  AS398467\ntech-c: DUMMY-RADB\nadmin-c: DUMMY-RADB\nnotify: biff@example.com\nmnt-by: MAINT-AS135602\nchanged: biff@example.com\nsource: RADB\n"
    }
  ]
}

At first I thought that the newline might be an implicit comma in IRRD's parsing, but the object comes back with no commas. I would think if IRRD were thinking at that level, it would put it back together with explicit commas. I think this is invalid from a strict interpretation of RFC 2622:

$ whois -h irrd-radb -p 4343 AS-BIFF-TEST-NO-COMMAS
as-set:         AS-BIFF-TEST-NO-COMMAS
descr:          odd things
members:        AS-SOMETHING
                AS398465
                AS398466
                AS398467
tech-c:         DUMMY-RADB
admin-c:        DUMMY-RADB
notify:         biff@example.com
mnt-by:         MAINT-AS135602
changed:        biff@example.com
source:         RADB
last-modified:  2024-04-18T18:05:40Z

The middleware stuff we had gets this object and doesn't apply any special magic to the members field, which is then processed as a simple free form text field across multiple lines. This produces a single value instead of a list of values. We're basically relying on anything we get back from IRRD to be valid (but maybe not true).

If there are some commas, we still have a list, but that list is just the stuff between commas. With one comma, we end up with a two-item list:

as-set:         AS-BIFF-TEST-NO-COMMAS
descr:          odd things
members:        AS-SOMETHING
                AS398465
                AS398466,
                AS398467
tech-c:         DUMMY-RADB
admin-c:        DUMMY-RADB
notify:         biff@example.com
mnt-by:         MAINT-AS135602
changed:        biff@example.com
source:         RADB
last-modified:  2024-04-18T18:10:34Z

This still comes back exactly as submitted:

$ whois -h irrd-radb -p 4343 AS-BIFF-TEST-NO-COMMAS
as-set:         AS-BIFF-TEST-NO-COMMAS
descr:          odd things
members:        AS-SOMETHING
                AS398465
                AS398466,
                AS398467
tech-c:         DUMMY-RADB
admin-c:        DUMMY-RADB
notify:         biff@example.com
mnt-by:         MAINT-AS135602
changed:        biff@example.com
source:         RADB
last-modified:  2024-04-18T18:10:34Z

Expected behaviour

The big issue is that I expected the multiline form to act like the single line form. Specifically, the multiline example should have been rejected too.

I also expected that IRRD would validate the members field by looking at the values between \s*,\s* since it is already looking for at least one value that is another as-set. I know that the template shows it as a weak reference, but I'd also expect it would at least validate the format of each member. That's might be a different issue beyond the scope of this particular problem.

IRRd version you are running

$ whois -h irrd-radb -p 4343 '!v'
A22
IRRd -- version 4.4.2
C

[mxsasha]

Rereading RFC2622 on this, I think you are correct that IRRD should separate list elements on comma only, whether on one line or multiple (within one attribute) and it looks like that is precisely what it does.

For continuation, 2622 does not appear to be specific on whether or not a continuation line implies white space, which matters here, it just says

An attribute's value can be split over multiple lines

If continuation does not imply white space in the value, this:

members:        AS-SOMETHING
                AS398465
                AS398466,
                AS398467

Is a valid members attribute with entries AS-SOMETHINGAS398465AS398466 and AS398467. Being a weak reference, the format validation would also pass as AS-SOMETHINGAS398465AS398466 is a valid as-set name.

If we say that a continuation does imply a space in the value, the two entries are AS-SOMETHING AS398465 AS398466 and AS398467, and therefore invalid as spaces are not allowed in an ASN or as-set name.