Merge branch 'master' into docs_records_yaml

Author: brbzull0

doc/release-notes/upgrading.en.rst

@@ -34,6 +34,14 @@ version of ATS. Deprecated features should be avoided, with the expectation that
 removed in the next major release of ATS.
 
 * Removed Features
+  * HostDB no longer supports persistent storage for DNS resolution
+  * Removed support for the MMH crypto hash function
+
+  * Traffic Manager is no longer part of |TS|. Administrative tools now interact with |TS| directly by using the :ref:`jsonrpc-node`.
+
+    * traffic_ctl ``server``
+
+      As a part of the above feature removal the ``backtrace``, ``restart``, ``start``, ``status`` options are no longer available in this |TS| version.
 
 * Deprecated Features
 
@@ -45,8 +53,15 @@ The following features have been changed in this version of ATS.
 
 * Remap ACLs
 
-   Fixes with @action=allow to deny transactions that are not explicitly allowed.  Changed in-line ACLs to match before activated ACL rules.
-   For detail refer to: https://github.com/apache/trafficserver/pull/11033
+   Changed in-line ACLs to match before activated ACL rules.
+   For details refer to: https://github.com/apache/trafficserver/pull/11033 and :ref:`acl-filters`.
+
+
+
+* Administrative API (RPC)
+
+  Moved away from the binary serialization mechanism used to comunicate between |TS| and the tools to a JSON-RPC text based protocol. Underlying
+  Unix Domain Socket protocol remains the same. Check :ref:`jsonrpc-protocol` for more details.
 
 
 API Changes
@@ -56,6 +71,8 @@ The following APIs have changed, either in semantics, interfaces, or both.
 * Changed TS API
 
   * TSHttpTxnAborted
+  * TSMimeHdrPrint
+  * Enum values for hooks and events have been changed (ABI incompatible change)
 
 * Removed TS API
 
@@ -96,15 +113,6 @@ The following APIs have changed, either in semantics, interfaces, or both.
   * INKUDPPacketDestroy
   * INKUDPPacketGet
 
-* Removed Lua API
-
-  * ts.http.client_packet_tos_set
-  * ts.http.server_packet_tos_set
-  * ts.client_request.get_uri_params
-  * ts.client_request.set_uri_params
-  * ts.server_request.get_uri_params
-  * ts.server_request.set_uri_params
-
 
 Cache
 -----
@@ -121,14 +129,31 @@ Check :ref:`rec-config-to-yaml` and :file:`records.yaml`  for more details.
 The following :file:`records.yaml` changes have been made:
 
 - The records.yaml entry ``proxy.config.http.down_server.abort_threshold`` has been removed.
-- The records.yaml entry ``proxy.config.http.connect_attempts_max_retries_dead_server`` has been renamed to ``proxy.config.http.connect_attempts_max_retries_down_server``.
-- The entry ``proxy.config.http.connect.dead.policy`` has been renamed to ``proxy.config.http.connect.down.policy``.
+- The records.yaml entry ``proxy.config.http.connect_attempts_max_retries_dead_server`` has been renamed to :ts:cv:`proxy.config.http.connect_attempts_max_retries_down_server`.
+- The entry ``proxy.config.http.connect.dead.policy`` has been renamed to :ts:cv:`proxy.config.http.connect.down.policy`.
 - The records.yaml entry ``proxy.config.http.parent_proxy.connect_attempts_timeout`` and
   ``proxy.config.http.post_connect_attempts_timeout`` have been removed. Instead use
-  ``proxy.config.http.connect_attempts_timeout`` to control all connection to origin timeouts.
+  :ts:cv:`proxy.config.http.connect_attempts_timeout` to control all connection to origin timeouts.
 - The per server origin connection feature had a few configurations that were not used removed.
   ``proxy.config.http.per_server.connection.queue_size`` and ``proxy.config.http.per_server.connection.queue_delay``
   have been removed.
+- The default value for records.yaml entry ``proxy.config.ssl.client.verify.server.policy`` has been changed
+  from ``PERMISSIVE`` to ``STRICT``.
+- The records.yaml entry ``proxy.config.http.keepalive_internal_vc`` has been removed.  This entry
+  was previously undocumented.
+- The records.yaml entries ``proxy.config.http.parent_proxy.connect_attempts_timeout`` and
+  ``proxy.config.http.post_connect_attempts_timeout`` were previously referenced in default config
+  files, but they did not have any effect.  These have been removed from default configs files.
+- The default values for :ts:cv:`proxy.config.http.request_header_max_size`, :ts:cv:`proxy.config.http.response_header_max_size`, and
+  :ts:cv:`proxy.config.http.header_field_max_size` have been changed to 32KB.
+- The records.yaml entry :ts:cv:`proxy.config.http.server_ports` now also accepts the
+  ``allow-plain`` option
+- The records.yaml entry :ts:cv:`proxy.config.http.cache.max_open_write_retry_timeout` has been added to specify a timeout for starting a write to cache
+- The records.yaml entry :ts:cv:`proxy.config.net.per_client.max_connections_in` has
+  been added to limit the number of connections from a client IP. This works the
+  same as :ts:cv:`proxy.config.http.per_server.connection.max`
+- The records.yaml entry :ts:cv:`proxy.config.http.no_dns_just_forward_to_parent` is
+  not overridable
 - The default values for ``proxy.config.http.request_header_max_size``, ``proxy.config.http.response_header_max_size``, and
   ``proxy.config.http.header_field_max_size`` have been changed to 32KB.
 - The records.yaml entry ``proxy.config.output.logfile`` has been renamed to :ts:cv:`proxy.config.output.logfile.name`.
@@ -139,10 +164,11 @@ The following :file:`records.yaml` changes have been made:
 - The records.yaml entry ``proxy.config.ssl.TLSv1_3`` has been renamed to :ts:cv:`proxy.config.ssl.TLSv1_3.enabled`.
 - The records.yaml entry ``proxy.config.ssl.client.TLSv1_3`` has been renamed to :ts:cv:`proxy.config.ssl.client.TLSv1_3.enabled`.
 
-
 The following changes have been made to the :file:`sni.yaml` file:
 
 - ``disable_h2`` has been removed. Use ``http2`` with :code:`off` instead.
+- The ``ip_allow`` key can now take a reference to a file containing the ip
+  allow rules
 
 Plugins
 -------
@@ -162,16 +188,32 @@ The following plugins have been changed in this version of ATS.
 
 Lua Plugin
 ~~~~~~~~~~
-The following Http config constants have been renamed:
+* The following Http config constants have been renamed:
 
 TS_LUA_CONFIG_HTTP_CONNECT_ATTEMPTS_MAX_RETRIES_DEAD_SERVER has been renamed to TS_LUA_CONFIG_HTTP_CONNECT_ATTEMPTS_MAX_RETRIES_DOWN_SERVER.
 
 TS_LUA_CONFIG_HTTP_CONNECT_DEAD_POLICY has been renamed to TS_LUA_CONFIG_HTTP_CONNECT_DOWN_POLICY.
 
+* The following functions have been removed:
+
+  * ts.http.client_packet_tos_set
+  * ts.http.server_packet_tos_set
+  * ts.client_request.get_uri_params
+  * ts.client_request.set_uri_params
+  * ts.server_request.get_uri_params
+  * ts.server_request.set_uri_params
+
+
 Metrics
 -------
 
-The HTTP connection metric proxy.process.http.dead_server.no_requests has been renamed to proxy.process.http.down_server.no_requests.
+- The HTTP connection metric proxy.process.http.dead_server.no_requests has been renamed to proxy.process.http.down_server.no_requests.
+- The network metric ``proxy.process.net.calls_to_readfromnet_afterpoll`` has been removed
+- The network metric ``proxy.process.net.calls_to_writetonet_afterpoll`` has been removed
+- New cache metrics ``proxy.process.cache.stripes`` and
+  ``proxy.process.cache.volume_X.stripes`` that counts cache stripes
+- All metric names that ended in ``_stat`` have had that suffix dropped and no
+  longer end with ``_stat``
 
 Logging
 -------

doc/release-notes/whats-new.en.rst

@@ -31,6 +31,14 @@ total of <z> contributors have participated in this development cycle.
 New Features
 ------------
 
+* JSON-RPC based interface for administrative API
+
+   |TS| now exposes a JSON-RPC node to interact with external tools. Check :ref:`developer-guide-jsonrpc` for more details.
+
+* :file:`ip_allow.yaml` and :file:`remap.config` now support named IP ranges via IP
+  Categories. See the ``ip_categories`` key definition in :file:`ip_allow.yaml`
+  for information about their use and definitions.
+
 
 New or modified Configurations
 ------------------------------
@@ -42,6 +50,17 @@ The configuration settings :ts:cv: `proxy.config.http.parent_proxy.connect_attem
 All connect timeouts are controlled by :ts:cv: `proxy.config.http.connect_attempts_timeout`.
 
 
+ip_allow.yaml and remap.config ACL actions
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+There are two new sets of actions for HTTP request method filtering introduced in |TS| 10.x:
+
+- Both :file:`ip_allow.yaml` and :file:`remap.config` now support the ``set_allow`` and ``set_deny`` actions. These
+  actions both behave like ``allow`` and ``deny`` did for :file:`ip_allow.yaml` pre |TS| 10.x.
+- In addition, :file:`remap.config` now supports ``add_allow`` and ``add_deny`` actions. These behave like ``allow``
+  and ``deny`` actions did for :file:`remap.config` ACLs pre |TS| 10.x.
+
+The details about the motivation and behavior of these actions are documented in :ref:`acl-filters`.
 
 
 Logging and Metrics
@@ -50,6 +69,10 @@ Logging and Metrics
 Plugins
 -------
 
+* JSON-RPC
+
+   Remote clients, like :ref:`traffic_ctl_jsonrpc` have now bi-directional access to the plugin space. For more details check :ref:`jsonrpc_development`.
+
 Switch to C++20
 ^^^^^^^^^^^^^^^
 
@@ -67,3 +90,9 @@ Symbols With INKUDP Prefix
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 In the plugin API, all types and functions starting with the prefix INKUDP are removed.
+
+New plugin hook for request sink transformation
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A new hook, ``TS_HTTP_REQUEST_CLIENT_HOOK``, has been added. This provides the analoguas functionality of
+``TS_HTTP_RESPONSE_CLIENT_HOOK``, for request bodies.

src/records/RecordsConfig.cc

@@ -580,6 +580,8 @@ static const RecordElement RecordsConfig[] =
   ,
   {RECT_CONFIG, "proxy.config.http.cache.max_open_write_retries", RECD_INT, "1", RECU_DYNAMIC, RR_NULL, RECC_NULL, nullptr, RECA_NULL}
   ,
+  {RECT_CONFIG, "proxy.config.http.cache.max_open_write_retry_timeout", RECD_INT, "0", RECU_DYNAMIC, RR_NULL, RECC_NULL, nullptr, RECA_NULL}
+  ,
   //       #  open_write_fail_action has 3 options:
   //       #
   //       #  0 - default. disable cache and goto origin