Skip to content

Commit

Permalink
#1672 Custom Default Version Policy (#1673)
Browse files Browse the repository at this point in the history
* feature written, tests passed

* actualy passes almost all the test.

* resolve conflict, hopefully.

* please.

* let it cook.

* uses constants instead of string for version policies.

* conflict res

* swapped downstream method and version.

* #1731 Read the Docs configuration file v2 (#1733)

* fixing the documentation, using Release/20.0 as base branch

* using latest conf.py, created with sphinx-quickstart, fixing the warnings during documentation generation

* Update .readthedocs.yaml

* switching to threemammals.org for copyright

* adding requirements file, updating readthedocs.yaml, adding formats pdf / epub and config for requirements file

* fixing code block in websockets.rst

* ok, now it should be fine...

* Update kubernetes.rst: Review and fix markup code

* Update websockets.rst: Review and fix markup

* Update conf.py: Update release, author and copyright

---------

Co-authored-by: Raman Maksimchuk <dotnet044@gmail.com>

* * When using the QoS option "ExceptionsAllowedBeforeBreaking" the circuit breaker never opens the circuit.

* merge issue, PortFinder

* some code improvements, using httpresponsemessage status codes as a base for circuit breaker

* Adding more unit tests, and trying to mitigate the test issues with the method "GivenThereIsAPossiblyBrokenServiceRunningOn"

* fixing some test issues

* setting timeout value to 5000 to avoid side effects

* again timing issues

* timing issues again

* ok, first one ok

* Revert "ok, first one ok"

This reverts commit 2e4a673.

* inline method

* putting back logging for http request exception

* removing logger configuration, back to default

* adding a bit more tests to check the policy wrap

* Removing TimeoutStrategy from parameters, it's set by default to pessimistic, at least one policy will be returned, so using First() in circuit breaker and removing the branch Policy == null from delegating handler.

* Fix StyleCop warnings

* Format parameters

* Sort usings

* since we might have two policies wrapped,  timeout and circuit breaker, we can't use the name CircuitBreaker for polly qos provider, it's not right. Using PollyPolicyWrapper and AsnycPollyPolicy instead.

* modifying circuit breaker delegating handler name, usin Polly policies instead

* renaming CircuitBreakerFactory to PolicyWrapperFactory in tests

* DRY for FileConfiguration, using FileConfigurationFactory

* Add copy constructor

* Refactor setup

* Use expression body for method

* Fix acceptance test

* IDE1006 Naming rule violation: These words must begin with upper case characters

* CA1816 Change ReturnsErrorTests.Dispose() to call GC.SuppressFinalize(object)

* Sort usings

* Use expression body for method

* Return back named arguments

---------

Co-authored-by: raman-m <dotnet044@gmail.com>

* feature written, tests passed

* actualy passes almost all the test.

* resolve conflict, hopefully.

* missed this one.

* please.

* come on...

* let it build.

* let it cook.

* copied from main branch.

* conflict res

* resolving conflicts.

* another attempt.

* lf

* re-incorporate downstream version policy.

* renamed the version policies and added acceptance tests.

* trust the dotnet dev cert.

* accepts cert from dotnet.

* Fix compiling errors

* Refactor tests

* a bit of code cleanup, removing some usings

* a bit more cleanup in fileroute

* try and error with the tests

* "Yahoo!...", said @ibnuda :)

* FileRoute: let it go...
Binary copy! :LoL:

* FileRoute: let it cook...
Re-add sweet props

* `dotnet dev-certs` for the `build` job

* Recover `kubernetes.rst`

* docs/make.bat original version

* OcelotBuilderExtensions

* original src/Ocelot.Provider.Polly/v7/PollyPolicyWrapper.cs

* `IVersionPolicyCreator` XML docs

* Code review by @raman-m (part 1)

* RequestMapper : care about diff

* Code review by @raman-m (part 2)

* Fix Should_return_OK_status_and_multiline_indented_json_response_with_json_options_for_custom_builder

* Update configuration.rst

Add DownstreamVersionPolicy section

* Update docs

* Rename `DownstreamVersionPolicy` to `DownstreamHttpVersionPolicy`

* update docs after prop renaming

* Sort props

---------

Co-authored-by: Guillaume Gnaegi <58469901+ggnaegi@users.noreply.github.com>
Co-authored-by: Raman Maksimchuk <dotnet044@gmail.com>
  • Loading branch information
3 people authored Apr 19, 2024
1 parent 47a1379 commit 140f9b5
Show file tree
Hide file tree
Showing 44 changed files with 818 additions and 203 deletions.
4 changes: 2 additions & 2 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@ jobs:
resource_class: medium+
steps:
- checkout
- run: dotnet tool restore && dotnet cake
- run: dotnet dev-certs https && dotnet tool restore && dotnet cake
release:
docker:
- image: ocelot2/circleci-build:latest
steps:
- checkout
- run: dotnet tool restore && dotnet cake --target=Release
- run: dotnet dev-certs https && dotnet tool restore && dotnet cake --target=Release
workflows:
version: 2
main:
Expand Down
68 changes: 68 additions & 0 deletions docs/features/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Here is an example Route configuration. You don't need to set all of these thing
"DownstreamPathTemplate": "/",
"DownstreamHttpMethod": "",
"DownstreamHttpVersion": "",
"DownstreamHttpVersionPolicy": "",
"AddHeadersToRequest": {},
"AddClaimsToRequest": {},
"RouteClaimsRequirement": {},
Expand Down Expand Up @@ -395,11 +396,76 @@ Registering a callback
}
}
.. _config-http-version:

DownstreamHttpVersion
---------------------

Ocelot allows you to choose the HTTP version it will use to make the proxy request. It can be set as ``1.0``, ``1.1`` or ``2.0``.

* `HttpVersion Class <https://learn.microsoft.com/en-us/dotnet/api/system.net.httpversion>`_

.. _config-version-policy:

DownstreamHttpVersionPolicy [#f5]_
----------------------------------

This routing property enables the configuration of the ``VersionPolicy`` property within ``HttpRequestMessage`` objects for downstream HTTP requests.
For additional details, refer to the following documentation:

* `HttpRequestMessage.VersionPolicy Property <https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httprequestmessage.versionpolicy>`_
* `HttpVersionPolicy Enum <https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpversionpolicy>`_
* `HttpVersion Class <https://learn.microsoft.com/en-us/dotnet/api/system.net.httpversion>`_

The ``DownstreamHttpVersionPolicy`` option is intricately linked with the :ref:`config-http-version` setting.
Therefore, merely specifying ``DownstreamHttpVersion`` may sometimes be inadequate, particularly if your downstream services or Ocelot logs report HTTP connection errors such as ``PROTOCOL_ERROR``.
In these routes, selecting the precise ``DownstreamHttpVersionPolicy`` value is crucial for the ``HttpVersion`` policy to prevent such protocol errors.

HTTP/2 version policy
^^^^^^^^^^^^^^^^^^^^^

**Given** you aim to ensure a smooth HTTP/2 connection setup for the Ocelot app and downstream services with SSL enabled:

.. code-block:: json
{
"DownstreamScheme": "https",
"DownstreamHttpVersion": "2.0",
"DownstreamHttpVersionPolicy": "", // empty
"DangerousAcceptAnyServerCertificateValidator": true
}
**And** you configure global settings to use Kestrel with this snippet:

.. code-block:: csharp
var builder = WebApplication.CreateBuilder(args);
builder.WebHost.ConfigureKestrel(serverOptions =>
{
serverOptions.ConfigureEndpointDefaults(listenOptions =>
{
listenOptions.Protocols = HttpProtocols.Http2;
});
});
**When** all components are set to communicate exclusively via HTTP/2 without TLS (plain HTTP).

**Then** the downstream services may display error messages such as:

.. code-block::
HTTP/2 connection error (PROTOCOL_ERROR): Invalid HTTP/2 connection preface
To resolve the issue, ensure that ``HttpRequestMessage`` has its ``VersionPolicy`` set to ``RequestVersionOrHigher``.
Therefore, the ``DownstreamHttpVersionPolicy`` should be defined as follows:

.. code-block:: json
{
"DownstreamHttpVersion": "2.0",
"DownstreamHttpVersionPolicy": "RequestVersionOrHigher" // !
}
Dependency Injection
--------------------

Expand All @@ -425,8 +491,10 @@ You can find additional details in the dedicated :ref:`di-configuration-overview
.. [#f2] ":ref:`config-merging-tomemory`" subfeature is based on the ``MergeOcelotJson`` enumeration type with values: ``ToFile`` and ``ToMemory``. The 1st one is implicit by default, and the second one is exactly what you need when merging to memory. See more details on implementations in the `ConfigurationBuilderExtensions`_ class.
.. [#f3] :ref:`di-the-addocelot-method` adds default ASP.NET services to DI container. You could call another extended :ref:`di-addocelotusingbuilder-method` while configuring services to develop your own :ref:`di-custom-builder`. See more instructions in the ":ref:`di-addocelotusingbuilder-method`" section of :doc:`../features/dependencyinjection` feature.
.. [#f4] ":ref:`config-consul-key`" feature was requested in `issue 346 <https://github.com/ThreeMammals/Ocelot/issues/346>`_ as a part of version `7.0.0 <https://github.com/ThreeMammals/Ocelot/releases/tag/7.0.0>`_.
.. [#f5] ":ref:`config-version-policy`" feature was requested in `issue 1672 <https://github.com/ThreeMammals/Ocelot/issues/1672>`_ as a part of version `24.0`_.
.. _20.0: https://github.com/ThreeMammals/Ocelot/releases/tag/20.0.0
.. _23.2: https://github.com/ThreeMammals/Ocelot/releases/tag/23.2.0
.. _24.0: https://github.com/ThreeMammals/Ocelot/releases/tag/24.0.0
.. _ocelot.json: https://github.com/ThreeMammals/Ocelot/blob/main/test/Ocelot.ManualTest/ocelot.json
.. _ConfigurationBuilderExtensions: https://github.com/ThreeMammals/Ocelot/blob/develop/src/Ocelot/DependencyInjection/ConfigurationBuilderExtensions.cs
238 changes: 119 additions & 119 deletions docs/features/kubernetes.rst
Original file line number Diff line number Diff line change
@@ -1,119 +1,119 @@
.. |K8s Logo| image:: https://kubernetes.io/images/favicon.png
:alt: K8s Logo
:width: 40

|K8s Logo| Kubernetes [#f1]_ aka K8s
====================================

A part of feature: :doc:`../features/servicediscovery` [#f2]_

Ocelot will call the `K8s <https://kubernetes.io/>`_ endpoints API in a given namespace to get all of the endpoints for a pod and then load balance across them.
Ocelot used to use the services API to send requests to the `K8s <https://kubernetes.io/>`__ service but this was changed in `PR 1134 <https://github.com/ThreeMammals/Ocelot/pull/1134>`_ because the service did not load balance as expected.

Install
-------

The first thing you need to do is install the `NuGet package <https://www.nuget.org/packages/Ocelot.Provider.Kubernetes>`_ that provides **Kubernetes** [#f1]_ support in Ocelot:

.. code-block:: powershell
Install-Package Ocelot.Provider.Kubernetes
Then add the following to your ``ConfigureServices`` method:

.. code-block:: csharp
services.AddOcelot().AddKubernetes();
If you have services deployed in Kubernetes, you will normally use the naming service to access them.
Default ``usePodServiceAccount = true``, which means that Service Account using Pod to access the service of the K8s cluster needs to be Service Account based on RBAC authorization:

.. code-block:: csharp
public static class OcelotBuilderExtensions
{
public static IOcelotBuilder AddKubernetes(this IOcelotBuilder builder, bool usePodServiceAccount = true);
}
You can replicate a Permissive using RBAC role bindings (see `Permissive RBAC Permissions <https://kubernetes.io/docs/reference/access-authn-authz/rbac/#permissive-rbac-permissions>`_),
K8s API server and token will read from pod.

.. code-block:: bash
kubectl create clusterrolebinding permissive-binding --clusterrole=cluster-admin --user=admin --user=kubelet --group=system:serviceaccounts
Configuration
-------------

The following examples show how to set up a Route that will work in Kubernetes.
The most important thing is the **ServiceName** which is made up of the Kubernetes service name.
We also need to set up the **ServiceDiscoveryProvider** in **GlobalConfiguration**.

Kube default provider
^^^^^^^^^^^^^^^^^^^^^

The example here shows a typical configuration:

.. code-block:: json
"Routes": [
{
"ServiceName": "downstreamservice",
// ...
}
],
"GlobalConfiguration": {
"ServiceDiscoveryProvider": {
"Host": "192.168.0.13",
"Port": 443,
"Token": "txpc696iUhbVoudg164r93CxDTrKRVWG",
"Namespace": "Dev",
"Type": "Kube"
}
}
Service deployment in **Namespace** ``Dev``, **ServiceDiscoveryProvider** type is ``Kube``, you also can set :ref:`k8s-pollkube-provider` type.
Note: **Host**, **Port** and **Token** are no longer in use.

.. _k8s-pollkube-provider:

PollKube provider
^^^^^^^^^^^^^^^^^

You use Ocelot to poll Kubernetes for latest service information rather than per request.
If you want to poll Kubernetes for the latest services rather than per request (default behaviour) then you need to set the following configuration:

.. code-block:: json
"ServiceDiscoveryProvider": {
"Namespace": "dev",
"Type": "PollKube",
"PollingInterval": 100 // ms
}
The polling interval is in milliseconds and tells Ocelot how often to call Kubernetes for changes in service configuration.

Please note, there are tradeoffs here.
If you poll Kubernetes, it is possible Ocelot will not know if a service is down depending on your polling interval and you might get more errors than if you get the latest services per request.
This really depends on how volatile your services are.
We doubt it will matter for most people and polling may give a tiny performance improvement over calling Kubernetes per request.
There is no way for Ocelot to work these out for you.

Global vs Route levels
^^^^^^^^^^^^^^^^^^^^^^

If your downstream service resides in a different namespace, you can override the global setting at the Route-level by specifying a **ServiceNamespace**:

.. code-block:: json
"Routes": [
{
"ServiceName": "downstreamservice",
"ServiceNamespace": "downstream-namespace"
}
]
""""

.. [#f1] `Wikipedia <https://en.wikipedia.org/wiki/Kubernetes>`_ | `K8s Website <https://kubernetes.io/>`_ | `K8s Documentation <https://kubernetes.io/docs/>`_ | `K8s GitHub <https://github.com/kubernetes/kubernetes>`_
.. [#f2] This feature was requested as part of `issue 345 <https://github.com/ThreeMammals/Ocelot/issues/345>`_ to add support for `Kubernetes <https://kubernetes.io/>`_ :doc:`../features/servicediscovery` provider.
.. |K8s Logo| image:: https://kubernetes.io/images/favicon.png
:alt: K8s Logo
:width: 40

|K8s Logo| Kubernetes [#f1]_ aka K8s
====================================

A part of feature: :doc:`../features/servicediscovery` [#f2]_

Ocelot will call the `K8s <https://kubernetes.io/>`_ endpoints API in a given namespace to get all of the endpoints for a pod and then load balance across them.
Ocelot used to use the services API to send requests to the `K8s <https://kubernetes.io/>`__ service but this was changed in `PR 1134 <https://github.com/ThreeMammals/Ocelot/pull/1134>`_ because the service did not load balance as expected.

Install
-------

The first thing you need to do is install the `NuGet package <https://www.nuget.org/packages/Ocelot.Provider.Kubernetes>`_ that provides **Kubernetes** [#f1]_ support in Ocelot:

.. code-block:: powershell
Install-Package Ocelot.Provider.Kubernetes
Then add the following to your ``ConfigureServices`` method:

.. code-block:: csharp
services.AddOcelot().AddKubernetes();
If you have services deployed in Kubernetes, you will normally use the naming service to access them.
Default ``usePodServiceAccount = true``, which means that Service Account using Pod to access the service of the K8s cluster needs to be Service Account based on RBAC authorization:

.. code-block:: csharp
public static class OcelotBuilderExtensions
{
public static IOcelotBuilder AddKubernetes(this IOcelotBuilder builder, bool usePodServiceAccount = true);
}
You can replicate a Permissive using RBAC role bindings (see `Permissive RBAC Permissions <https://kubernetes.io/docs/reference/access-authn-authz/rbac/#permissive-rbac-permissions>`_),
K8s API server and token will read from pod.

.. code-block:: bash
kubectl create clusterrolebinding permissive-binding --clusterrole=cluster-admin --user=admin --user=kubelet --group=system:serviceaccounts
Configuration
-------------

The following examples show how to set up a Route that will work in Kubernetes.
The most important thing is the **ServiceName** which is made up of the Kubernetes service name.
We also need to set up the **ServiceDiscoveryProvider** in **GlobalConfiguration**.

Kube default provider
^^^^^^^^^^^^^^^^^^^^^

The example here shows a typical configuration:

.. code-block:: json
"Routes": [
{
"ServiceName": "downstreamservice",
// ...
}
],
"GlobalConfiguration": {
"ServiceDiscoveryProvider": {
"Host": "192.168.0.13",
"Port": 443,
"Token": "txpc696iUhbVoudg164r93CxDTrKRVWG",
"Namespace": "Dev",
"Type": "Kube"
}
}
Service deployment in **Namespace** ``Dev``, **ServiceDiscoveryProvider** type is ``Kube``, you also can set :ref:`k8s-pollkube-provider` type.
Note: **Host**, **Port** and **Token** are no longer in use.

.. _k8s-pollkube-provider:

PollKube provider
^^^^^^^^^^^^^^^^^

You use Ocelot to poll Kubernetes for latest service information rather than per request.
If you want to poll Kubernetes for the latest services rather than per request (default behaviour) then you need to set the following configuration:

.. code-block:: json
"ServiceDiscoveryProvider": {
"Namespace": "dev",
"Type": "PollKube",
"PollingInterval": 100 // ms
}
The polling interval is in milliseconds and tells Ocelot how often to call Kubernetes for changes in service configuration.

Please note, there are tradeoffs here.
If you poll Kubernetes, it is possible Ocelot will not know if a service is down depending on your polling interval and you might get more errors than if you get the latest services per request.
This really depends on how volatile your services are.
We doubt it will matter for most people and polling may give a tiny performance improvement over calling Kubernetes per request.
There is no way for Ocelot to work these out for you.

Global vs Route levels
^^^^^^^^^^^^^^^^^^^^^^

If your downstream service resides in a different namespace, you can override the global setting at the Route-level by specifying a **ServiceNamespace**:

.. code-block:: json
"Routes": [
{
"ServiceName": "downstreamservice",
"ServiceNamespace": "downstream-namespace"
}
]
""""

.. [#f1] `Wikipedia <https://en.wikipedia.org/wiki/Kubernetes>`_ | `K8s Website <https://kubernetes.io/>`_ | `K8s Documentation <https://kubernetes.io/docs/>`_ | `K8s GitHub <https://github.com/kubernetes/kubernetes>`_
.. [#f2] This feature was requested as part of `issue 345 <https://github.com/ThreeMammals/Ocelot/issues/345>`_ to add support for `Kubernetes <https://kubernetes.io/>`_ :doc:`../features/servicediscovery` provider.
Empty file modified docs/make.bat
100755 → 100644
Empty file.
8 changes: 8 additions & 0 deletions src/Ocelot/Configuration/Builder/DownstreamRouteBuilder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ public class DownstreamRouteBuilder
private SecurityOptions _securityOptions;
private string _downstreamHttpMethod;
private Version _downstreamHttpVersion;
private HttpVersionPolicy _downstreamHttpVersionPolicy;
private Dictionary<string, UpstreamHeaderTemplate> _upstreamHeaders;

public DownstreamRouteBuilder()
Expand Down Expand Up @@ -268,6 +269,12 @@ public DownstreamRouteBuilder WithUpstreamHeaders(Dictionary<string, UpstreamHea
return this;
}

public DownstreamRouteBuilder WithDownstreamHttpVersionPolicy(HttpVersionPolicy downstreamHttpVersionPolicy)
{
_downstreamHttpVersionPolicy = downstreamHttpVersionPolicy;
return this;
}

public DownstreamRoute Build()
{
return new DownstreamRoute(
Expand Down Expand Up @@ -305,6 +312,7 @@ public DownstreamRoute Build()
_securityOptions,
_downstreamHttpMethod,
_downstreamHttpVersion,
_downstreamHttpVersionPolicy,
_upstreamHeaders);
}
}
3 changes: 1 addition & 2 deletions src/Ocelot/Configuration/Builder/RouteBuilder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,7 @@ public Route Build()
_upstreamTemplatePattern,
_upstreamHost,
_aggregator,
_upstreamHeaders
);
_upstreamHeaders);
}
}
}
Loading

0 comments on commit 140f9b5

Please sign in to comment.