Ip allow categories by yaml

configs/ip_allow.schema.json

@@ -22,6 +22,10 @@
       "description": "A range of IP addresses in a single family.",
       "type": "string"
     },
+    "category": {
+      "description": "An IP category representing a set of IP ranges.",
+      "type": "string"
+    },
     "action": {
       "description": "Enforcement action.",
       "type": "string",
@@ -68,14 +72,35 @@
             }
           ]
         },
+        "ip_categories": {
+          "oneOf": [
+            {
+              "$ref": "#/definitions/category"
+            },
+            {
+              "type": "array",
+              "minItems": 1,
+              "items": {
+                "$ref": "#/definitions/category"
+              }
+            }
+          ]
+        },
         "action": {
           "$ref": "#/definitions/action"
         },
         "methods": {
           "$ref": "#/definitions/methods"
         }
       },
-      "required": [ "apply", "ip_addrs", "action" ]
+      "oneOf": [
+        {
+          "required": [ "apply", "ip_addrs", "action" ]
+        },
+        {
+          "required": [ "apply", "ip_categories", "action" ]
+        }
+      ]
     }
   }
 }

doc/admin-guide/files/ip_allow.yaml.en.rst

@@ -26,7 +26,7 @@ The :file:`ip_allow.yaml` file controls client access to |TS| and |TS| connectio
 This control is specified via rules. Each rule has:
 
 *  A direction (inbound or out).
-*  A range of IP address to which the rule applies.
+*  A range of IP addresses or an IP category to which the rule applies.
 *  An action, either accept or deny.
 *  A list of HTTP methods.
 
@@ -82,7 +82,21 @@ The keys in a rule are:
 
 ``ip_addrs``
    IP addresses to match for the rule to be applied. This can be either an address range or an
-   array of address ranges. This is a required key.
+   array of address ranges. Either this or ``ip_categories`` are required keys for a rule.
+
+``ip_categories``
+   A user defined string identifying a category of IP addresses relevant to a particular network.
+   For example, ``ACME_INTERNAL`` might represent the set of IP addresses for hosts within a
+   company's network. ``ACME_EXTERNAL`` might represet hosts belonging to the company's network, but
+   which are outside the company's firewall. ``ACME_ALL`` could be used to represent the set of both
+   of these categories. Multiple categories can be specified as an array of strings.
+
+   The set of IP ranges belonging to each category is specified via the separate ``ip_categories``
+   root level node. The :file:`ip_allow.yaml` parser also supports supplying the IP categories via
+   an external file specified with the :ts:cv:`proxy.config.cache.ip_categories.filename`
+   configuration.
+
+   Either this or ``ip_addrs`` are required keys for a rule.
 
 ``action``
    The action, which must be ``allow`` or ``deny``. This is a required key.
@@ -93,8 +107,9 @@ The keys in a rule are:
    keyword "ALL" means all methods, making the specification of any other method redundant. All
    methods comparisons are case insensitive. This is an optional key.
 
-An IP address range can be specified in several ways. A range is always IPv4 or IPv6, it is not
-allowed to have a range that contains addresses from different IP address families.
+An IP address range for ``ip_addrs`` or ``ip_categories`` can be specified in several ways. A range
+is always IPv4 or IPv6, it is not allowed to have a range that contains addresses from different IP
+address families.
 
 *  A single address, which specifies a range of size 1, e.g. "127.0.0.1".
 *  A minimum and maximum address separated by a dash, e.g. "10.1.0.0-10.1.255.255".
@@ -131,7 +146,7 @@ enables all methods for all outbound connections.
 Examples
 ========
 
-The following example enables all clients access.::
+The following example enables all clients access::
 
    apply: in
    ip_addrs: 0.0.0.0-255.255.255.255
@@ -222,7 +237,7 @@ This will match the IP address for the target servers on the outbound connection
 method is ``GET`` or ``HEAD`` the connection will be allowed, otherwise the connection will be
 denied.
 
-As a final example, here is the default configuration in compact form::
+For the purposes of illustration, here is the default configuration in compact form::
 
    ip_allow: [
      { apply: in, ip_addrs: 127.0.0.1, action: allow },
@@ -231,6 +246,47 @@ As a final example, here is the default configuration in compact form::
      { apply: in, ip_addrs: "::/0", action: deny, methods: [ PURGE, PUSH, DELETE, TRACE ] }
      ]
 
+The following example demonstrates how to use ``ip_categories``. In this example, the
+``ip_categories`` is ``ACME_INTERNAL`` which is presumably associated with trusted internal IP
+addresses and thus are allowed to ``POST`` and ``DELETE`` resources.
+
+Note this example demonstrates that it is OK to mix ``ip_categories`` and ``ip_addrs`` rules in a
+single :file:`ip_allow.yaml` file. In this case all other IPv4 addresses not matched on
+``ACME_INTERNAL`` match on ``0/0`` and can only perform ``GET`` and ``HEAD`` requests::
+
+     - apply: in
+       ip_categories: ACME_INTERNAL
+       action: allow
+       methods:
+         - GET
+         - HEAD
+         - POST
+         - DELETE
+     - apply: in
+       ip_addrs: 0/0
+       action: allow
+       methods:
+         - GET
+         - HEAD
+
+The set of IP addresses associated with ``ACME_INTERNAL`` can be specified
+using the ``ip_categories`` node like so::
+
+     ip_categories:
+       - name: ACME_INTERNAL
+         ip_addrs:
+           - 10.0.0.0/8
+           - 172.16.0.0/20
+           - 192.168.1.0/24
+
+     ip_allow:
+       - apply: in
+       # ...
+
+The ``ip_categories`` node will generally be at the start of the :file:`ip_allow.yaml` file.
+Alternatively, the same content with the ``ip_categories`` root node can exist in a separate file
+specified with the :ts:cv:`proxy.config.cache.ip_categories.filename` configuration.
+
 .. note::
 
    For ATS 9.0, this file is (almost) backwards compatible. If the first line is a single '#'

doc/admin-guide/files/records.yaml.en.rst

@@ -2090,6 +2090,24 @@ Security
    this value via a :ref:`host_sni_policy<override-host-sni-policy>` attribute.
 
 
+IP Allow
+========
+
+.. ts:cv:: CONFIG proxy.config.cache.ip_allow.filename STRING ip_allow.yaml
+   :reloadable:
+
+   Set the file path for the IP allow configuration file. For details of the use
+   of this file, see :file:`ip_allow.yaml`. If this is a relative path, |TS|
+   loads it relative to the ``SYSCONFDIR`` directory.
+
+.. ts:cv:: CONFIG proxy.config.cache.ip_categories.filename STRING NULL
+   :reloadable:
+
+   Set the file path for the IP allow categories definition file. For details of
+   the use of this file, see :file:`ip_allow.yaml`. If this is a relative path,
+   |TS| loads it relative to the ``SYSCONFDIR`` directory.
+
+
 Cache Control
 =============
 

doc/admin-guide/files/remap.config.en.rst

@@ -428,6 +428,13 @@ This will pass "1" and "2" to plugin1.so and "3" to plugin2.so
 
 .. _remap-config-named-filters:
 
+NextHop Selection Strategies
+============================
+
+You may configure Nexthop or Parent hierarchical caching rules by remap using the
+**@strategy** tag.  See :doc:`../configuration/hierarchical-caching.en` and :doc:`strategies.yaml.en`
+for configuration details and examples.
+
 Acl Filters
 ===========
 
@@ -455,10 +462,13 @@ Examples
 
     map http://foo.example.com/  http://foo.example.com/ @action=allow @src_ip=127.0.0.1 @method=post @method=get @method=head
 
+    map http://foo.example.com/  http://foo.example.com/ @action=allow @src_ip_category=ACME_INTERNAL @method=post @method=get @method=head
+
 Note that these Acl filters will return a 403 response if the resource is restricted.
 
 The difference between ``@src_ip`` and ``@in_ip`` is that the ``@src_ip`` is the client
 ip and the ``in_ip`` is the ip address the client is connecting to (the incoming address).
+``@src_ip_category`` functions like ``ip_category`` described in :file:`ip_allow.yaml`.
 
 Named Filters
 =============
@@ -516,13 +526,6 @@ would be ::
 
 Note this entirely disables IP Allow checks for those remap rules.
 
-NextHop Selection Strategies
-============================
-
-You may configure Nexthop or Parent hierarchical caching rules by remap using the
-**@strategy** tag.  See :doc:`../configuration/hierarchical-caching.en` and :doc:`strategies.yaml.en`
-for configuration details and examples.
-
 Including Additional Remap Files
 ================================
 

include/proxy/IPAllow.h

@@ -32,7 +32,6 @@
 
 #include <string>
 #include <string_view>
-#include <vector>
 
 #include "proxy/hdrs/HTTP.h"
 #include "iocore/eventsystem/ConfigProcessor.h"
@@ -89,6 +88,7 @@ class IpAllow : public ConfigInfo
   using self_type     = IpAllow; ///< Self reference type.
   using scoped_config = ConfigProcessor::scoped_config<self_type, self_type>;
   using IpMap         = swoc::IPSpace<Record const *>;
+  using IpCategories  = std::unordered_map<std::string, swoc::IPSpace<bool>>;
 
   // indicator for whether we should be checking the acl record for src ip or dest ip
   enum match_key_t { SRC_ADDR, DST_ADDR };
@@ -102,8 +102,38 @@ class IpAllow : public ConfigInfo
   static constexpr swoc::TextView OPT_METHOD{"method"};
   static constexpr swoc::TextView OPT_METHOD_ALL{"all"};
 
+  /*
+   * A YAML configuration file looks something like this:
+   *
+   * ip_categories:
+   *   - name: ACME_INTERNAL
+   *     ip_addrs:
+   *       - 10.0.0.0/8
+   *       - 172.16.0.0/20
+   *       - 192.168.1.0/24
+   *
+   * ip_allow:
+   *   - apply: in
+   *     ip_categories: ACME_INTERNAL
+   *     action: allow
+   *     methods:
+   *     - GET
+   *     - HEAD
+   *     - POST
+   *   - apply: in
+   *     ip_addrs: 127.0.0.1
+   *     action: allow
+   *     methods: ALL
+   *
+   */
   static const inline std::string YAML_TAG_ROOT{"ip_allow"};
+
+  static const inline std::string YAML_TAG_CATEGORY_ROOT{"ip_categories"};
+  static const inline std::string YAML_TAG_CATEGORY_NAME{"name"};
+  static const inline std::string YAML_TAG_CATEGORY_IP_ADDRS{"ip_addrs"};
+
   static const inline std::string YAML_TAG_IP_ADDRS{"ip_addrs"};
+  static const inline std::string YAML_TAG_IP_CATEGORIES{"ip_categories"};
   static const inline std::string YAML_TAG_APPLY{"apply"};
   static const inline std::string YAML_VALUE_APPLY_IN{"in"};
   static const inline std::string YAML_VALUE_APPLY_OUT{"out"};
@@ -163,7 +193,7 @@ class IpAllow : public ConfigInfo
     IpAllow *_config{nullptr}; ///< The backing configuration.
   };
 
-  explicit IpAllow(const char *config_var);
+  explicit IpAllow(const char *ip_allow_config_var, const char *categories_config_var);
 
   void Print() const;
 
@@ -201,6 +231,25 @@ class IpAllow : public ConfigInfo
 
   const swoc::file::path &get_config_file() const;
 
+  /**
+   * Check if an IP category contains a specific IP address.
+   *
+   * @param category The IP category to check.
+   * @param addr The IP address to check against the category.
+   * @return True if the category contains the address, false otherwise.
+   */
+  static bool ip_category_contains_addr(std::string const &category, swoc::IPAddr const &addr);
+
+  /** Indicate whether ip_allow.yaml has no rules associated with it.
+   *
+   * If there are no rules, then all traffic will be blocked. This is used
+   * during ATS configuration to verify that the user has provided a usable
+   * ip_allow.yaml file.
+   *
+   * @return True if there are no rules in ip_allow.yaml, false otherwise.
+   */
+  static bool has_no_rules();
+
 private:
   static size_t configid;               ///< Configuration ID for update management.
   static const Record ALLOW_ALL_RECORD; ///< Static record that allows all access.
@@ -209,17 +258,26 @@ class IpAllow : public ConfigInfo
   void DebugMap(IpMap const &map) const;
 
   swoc::Errata BuildTable();
-  swoc::Errata YAMLBuildTable(const std::string &);
+  swoc::Errata YAMLBuildTable(const std::string &content);
   swoc::Errata YAMLLoadEntry(const YAML::Node &);
   swoc::Errata YAMLLoadIPAddrRange(const YAML::Node &, IpMap *map, Record const *mark);
+  swoc::Errata YAMLLoadIPCategory(const YAML::Node &, IpMap *map, Record const *mark);
   swoc::Errata YAMLLoadMethod(const YAML::Node &node, Record &rec);
 
-  /// Copy @a src to the local arena and review a view of the copy.
+  swoc::Errata BuildCategories();
+  swoc::Errata YAMLBuildCategories(const std::string &content);
+  swoc::Errata YAMLLoadCategoryRoot(const YAML::Node &);
+  swoc::Errata YAMLLoadCategoryDefinition(const YAML::Node &);
+  swoc::Errata YAMLLoadCategoryIpRange(const YAML::Node &, swoc::IPSpace<bool> &space);
+
+  /// Copy @a src to the local arena and return a view of the copy.
   swoc::TextView localize(swoc::TextView src);
 
-  swoc::file::path config_file; ///< Path to configuration file.
+  swoc::file::path ip_allow_config_file;      ///< Path to ip_allow configuration file.
+  swoc::file::path ip_categories_config_file; ///< Path to ip_allow configuration file.
   IpMap _src_map;
   IpMap _dst_map;
+  IpCategories ip_category_map; ///< Map of IP categories to IP spaces.
   /// Storage for records.
   swoc::MemArena _arena;
 
@@ -364,5 +422,12 @@ IpAllow::makeAllowAllACL() -> ACL
 inline const swoc::file::path &
 IpAllow::get_config_file() const
 {
-  return config_file;
+  return ip_allow_config_file;
+}
+
+inline bool
+IpAllow::has_no_rules()
+{
+  auto const *self = IpAllow::acquire();
+  return self->_src_map.count() == 0 && self->_dst_map.count() == 0;
 }