doc updates

docs/admins/object-suppression.rst

@@ -2,24 +2,24 @@
 Object suppression overview
 ===========================
 
-IRRd supports three methods to suppress objects. The three methods are:
+IRRd supports three methods to suppress objects:
 
 * :doc:`RPKI </admins/rpki>`, where IRRd suppresses objects that are
  invalid according to
  `RFC6811 origin validation <https://tools.ietf.org/html/rfc6811>`_.
  In addition, IRRd will create pseudo-IRR objects representing ROAs.
 * The :doc:`scope filter </admins/scopefilter>`, which suppresses objects
- matching certain prefixes and/or AS numbers.
+ matching certain configured prefixes and/or AS numbers.
 * The :doc:`route object preference </admins/route-object-preference>`,
  which suppresses objects that overlap with other route objects from
  sources with a higher preference.
 
-Suppression means that IRRd may act as if the object was deleted, but
-it may become visible again later. Unlike normal deletions and creations,
+Suppression is a kind of pseudo delete/create: IRRd may act as if the
+object was deleted, but it may become visible again later. Unlike normal
+deletions and creations,
 this visibility change may have an external cause, e.g. a new ROA created
 by the address space holder, or a change in scope filter configuration.
-Essentially, suppression and the reverse are pseudo-deletes/creates
-generated by IRRd. This applies even to objects for which the instance
+Suppression can apply even to objects for which the instance
 is not authoritative.
 
 Periodic updates
@@ -29,13 +29,15 @@ relevant objects in the database. This resolves issues around temporary
 inconsistencies and, for RPKI, is also the moment when the local ROA
 storage is updated.
 
-For RPKI, this happens every ``rpki.roa_import_timer``. For the scope
-filter, this happens on startup and then after any configuration change.
+For RPKI, this happens every ``rpki.roa_import_timer``.
+
+The scope filter updates all statuses on startup and then after any
+configuration change.
 
 Receiving updates from mirrors
 ------------------------------
 When IRRd receives objects from mirrors, in full imports or over NRTM,
-the status of all objects is updated.
+the RPKI, scope filter and route preference status of all objects is updated.
 
 Authoritative changes
 ---------------------
@@ -51,9 +53,7 @@ To aid in debugging, it is possible to include suppressed objects in some
 query responses. On whois queries, you can use the ``!fno-rpki-filter``,
 ``!fno-scope-filter``, and/or ``!fno-route-preference-filter`` commands.
 The filter is then disabled only for ``!r`` queries and all RIPE style
-queries.
-
-In GraphQL ``rpslObjects`` queries, you can pass a specific list of
+queries. In GraphQL ``rpslObjects`` queries, you can pass a specific list of
 ``rpkiStatus``, ``scopeFilterStatus``, and/or ``routePreferenceStatus``
 to include in the response.
 
@@ -63,9 +63,11 @@ Object suppression is reflected in the journal so that mirrors from
 the IRRd instance follow with the suppression. This means that
 IRRd does not record an ADD for suppressed objects, and if an existing
 object changes from visible to suppressed, IRRd records a DEL.
+If an already suppressed object is deleted by the mirror source,
+IRRd does not record any journal entry.
 
 An example: your IRRd
-mirrors source DEMO the authoritative source, you keep a local journal, and
+mirrors source DEMO from the authoritative source, you keep a local journal, and
 a third party mirrors DEMO from you. When the authoritative source for
 DEMO sends an NRTM ADD for an RPKI invalid route, that update is not
 recorded in your IRRd's local journal. The third party that mirrors from
@@ -79,8 +81,6 @@ recorded in your local journal.
 
 Therefore, both the local state of your IRRd, and anyone mirroring from
 your IRRd, will be up to date with the RPKI status.
-This does not apply to excluded sources, whose objects are never seen
-as RPKI invalid.
 
 For route object preference, there are cases where IRRd records an ADD
 and then immediately a DEL for the same object.
@@ -95,18 +95,21 @@ suppressed.
 Following on the previous example: if along with the RPKI state,
 the object is out of scope according to the scope filter, IRRd will
 not generate an ADD in the journal at any point, and the object
-will not be visible in query responses.
+will never be visible in query responses.
 
 Important considerations when enabling
 --------------------------------------
 When you enable one of these features for the first time, the periodic
-task will take considerably longer than on subsequent runs. On the first run,
-a large number of objects in the database need to be updated, whereas this
-number is much smaller on subsequent runs.
+task will usually take considerably longer than on subsequent runs.
+On the first run, a large number of objects in the database may need to
+be updated, whereas this number is much smaller on subsequent runs.
 
 The same may occur after changing settings, if these change a large number
 of objects.
 
+While the import and validation is running, processing other
+database changes may be delayed.
+
 .. note::
  These large status changes may also generate a significant amount
  of local journal entries, which are used to generate NRTM responses
@@ -130,6 +133,11 @@ are visible, and to the periodic ROA update, the objects in the new source
 are not visible yet. This situation automatically resolves itself upon
 the next periodic ROA update, but may cause objects that should be marked
 RPKI-invalid to be included in responses in the mean time.
-This issue only occurs when RPKI-aware mode is enabled for the first time,
-and at the same time a new source is added. At other times, the RPKI
-status of new sources will be correct.
+
+Similarly: a case where two mirror imports with different route object
+preferences run at the precise same time, and both include a `route` object
+for there same prefix. IRRd will record both as visible, because neither
+mirror process was able to see the other's object. This also resolves
+on the next periodic full route object preference update.
+Generally, the timing of these processes is slightly different, making
+this a rare issue.

docs/admins/object-validation.rst

@@ -19,13 +19,13 @@ The general requirements for validation are:
 * Any objects submitted to IRRd directly (i.e. not from mirrors)
  should always be entirely valid. If they are not, the end user
  can fix their object and continue from there.
-* The NTTCOM database has some legacy objects which require more
+* Many databases have some legacy objects which require more
  leniency with an initial import, but we aim to be as restrictive
  as reasonably possible. It should not be possible to update invalid
- objects without correcting their issues.
+ authoritative objects without correcting their issues.
 * Mirrors contain wildly variant objects, so IRRd performs the minimal
  level of validation needed to correctly index and query them.
-* Under no condition may IRRd provide responses to any query, which
+* IRRd must never provide responses to any query, which
  are missing certain objects because indexed data could not be extracted
  from them, without logging errors about failing to import these objects.
 * If objects are received from mirrors that can not be accepted, e.g.
@@ -100,6 +100,7 @@ The ``rpki-ov-state`` attribute, which is used to indicate the
 :doc:`RPKI validation status </admins/rpki>`, is always discarded from all
 incoming objects. Where relevant, it is added to the output of queries.
 This applies to authoritative and non-authoritative sources.
+This attribute is not visible over NRTM and in exports.
 
 key-cert objects
 ^^^^^^^^^^^^^^^^
@@ -114,7 +115,7 @@ last-modified
 ^^^^^^^^^^^^^
 For authoritative objects, the ``last-modified`` attribute is set when
 the object is created or updated. Any existing ``last-modified`` values are
-discarded. This timestamp is not updated for changes in RPKI validation
+discarded. This timestamp is not updated for changes in object suppression
 status. This attribute is visible over NRTM and in exports.
 
 By default, this attribute is only added when an object is changed or

docs/admins/route-object-preference.rst

@@ -23,7 +23,7 @@ This is a number, where higher numbers have higher preference. See the
 :doc:`configuration documentation </admins/configuration>` for the
 exact syntax.
 
-You can exclude sources by not setting
+You can exclude sources by **not** setting
 ``sources.{name}.route_object_preference``.
 Objects from these sources are always seen as visible.
 
@@ -37,8 +37,6 @@ Validation
 ----------
 * In route object preference, each `route(6)` object is assigned a preference
  from its source's ``sources.{name}.route_object_preference`` setting.
-* Objects from sources without this setting are always visible and otherwise
- ignored in the validation process.
 * For the objects with a preference, the rule is: if there is an overlapping
  `route(6)` object with a higher preference, the lower preference object
  is suppressed.
@@ -47,20 +45,26 @@ Validation
 * If two overlapping objects have the same preference, both are visible
  (assuming there are no further overlaps).
 * Origin ASes are not considered.
+* Objects from sources without a preference setting are always visible and
+ otherwise completely ignored in the validation process. They are also not
+ included when considering suppression of other objects.
 
 For example, let's say the preferences are: TEST-H1 and TEST-H2 at
-priority 900, TEST-M at 200, and TEST-L at 100, and the following
-objects exist:
+priority 900, TEST-M at 200, TEST-L at 100, TEST-N with no preference,
+and the following objects exist:
 
 * A: TEST-H1 192.0.0.0/23 AS65530
 * B: TEST-H2 192.0.0.0/24 AS65530
 * C: TEST-L 192.0.0.0/23 AS65530
 * D: TEST-M 192.0.0.0/22 AS65530
 * E: TEST-M 192.0.1.0/24 AS65530
 * F: TEST-L 192.0.3.0/24 AS65530
+* G: TEST-N 192.0.0.0/22 AS65530
 
 In this case objects C, D, E and F will be suppressed.
 C, D and E all overlap directly with A and/or B, and A and B have a higher
 preference than all others. A and B have an identical preference so their
 objects will both be visible. Object F overlaps with object D, and object D
 has a higher preference, therefore object F is also suppressed.
+Object G remains visible and is otherwise ignored, as it has no
+preference set.

docs/admins/rpki.rst

@@ -29,8 +29,8 @@ Objects from these sources are always seen as ``not_found``.
 
 To disable the RPKI filter, set ``rpki_excluded`` for all sources
 to reset the state of all objects to ``not_found``. Once the periodic
-import has updated this, unset ``rpki.roa_source`` to disable the RPKI
-update process.
+import has updated the state of all objects,
+unset ``rpki.roa_source`` to disable the RPKI update process.
 
 Pseudo-IRR objects
 ------------------
@@ -44,10 +44,8 @@ other source with ``!s`` and ``-s``, and can be included in the
 
 Validation
 ----------
-By default, `route(6)` objects that conflict with a ROA are not included
-in any query response. This is determined using
-`RFC6811 origin validation <https://tools.ietf.org/html/rfc6811>`_ and
-applies to all query types.
+IRRd uses `RFC6811 origin validation <https://tools.ietf.org/html/rfc6811>`_.
+Objects that are RPKI invalid are suppressed.
 
 Query responses
 ---------------
@@ -89,9 +87,10 @@ Notifications are never sent for objects from non-authoritative sources.
 
 First import with RPKI-aware mode
 ---------------------------------
-Depending on ``rpki.notify_invalid_enabled``, many emails may be sent out
-as well. While the import and validation is running, processing other
-database changes may be delayed.
+Along with the concerns mentioned in
+:doc:`object suppression </admins/object-suppression>`, depending on
+``rpki.notify_invalid_enabled``, many emails may be sent out
+as well.
 
 Temporary inconsistencies
 -------------------------

docs/admins/scopefilter.rst

@@ -22,20 +22,19 @@ and/or ``scopefilter.asns``. See the
 exact syntax.
 
 As soon as this is enabled and you (re)start IRRd or send a SIGHUP,
-IRRd will check all RPSL in the database against the scope filter.
+IRRd will check all objects in the database against the scope filter.
 
 You can exclude sources by setting ``sources.{name}.scopefilter_excluded``.
 Objects from these sources are always seen as in scope.
 
 To disable the scope filter, set ``scopefilter_excluded`` for all sources
 to reset the state of all objects to in scope. Once the periodic
-import has updated this, unset ``scopefilter.prefixes`` and
-``scopefilter.asns`` to disable the update process.
+import has updated the status for all objects, unset ``scopefilter.prefixes``
+and ``scopefilter.asns`` to disable the update process.
 
 Validation
 ----------
-By default, RPSL objects that are out of scope are not included in
-in any query response.
+RPSL objects that are out of scope are suppressed:
 
 * A `route(6)` object is out of scope if the origin is out of scope,
  or the prefix overlaps with any out of scope prefix.

docs/users/queries/event-stream.rst

@@ -200,9 +200,7 @@ Filtering
 ---------
 Password hashes from `mntner` objects are removed in all output.
 
-When a server is in :doc:`RPKI-aware mode </admins/rpki>` or has
-the :doc:`scopefilter </admins/scopefilter>` enabled, IRR objects
-that are out of scope or RPKI invalid are omitted in the initial
-retrieval. Objects that become RPKI invalid or out of scope
-are included in the WebSocket stream as a deletion, with the
+:doc:`Suppressed objects </admins/object-suppression>` are omitted
+in the initial retrieval. Objects that change suppression status
+are included in the WebSocket stream as an add/delete, with the
 ``origin`` indicating this reason.