Query string parameter getting duplicated in downstream path

[sunilk3]

Expected Behavior

Query string parameter should not duplicate.

Upstream Request: contracts?projectNumber=45&startDate=2019-12-12&endDate=2019-12-12
Downstream Request: api/contracts?projectNumber=45&startDate=2019-12-12&endDate=2019-12-12

Upstream Request: contracts?$filter=ProjectNumber eq 45 and DateOfSale ge 2020-03-01T00:00:00z and DateOfSale le 2020-03-15T00:00:00z
Downstream Request: api/contracts?$filter=ProjectNumber eq 45 and DateOfSale ge 2020-03-01T00:00:00z and DateOfSale le 2020-03-15T00:00:00z

Actual Behavior

Query string parameter is duplicated. This works fine with web api endpoints even if query string parameter gets duplicated but fails on OData endpoints.

Upstream Request: contracts?projectNumber=45&startDate=2019-12-12&endDate=2019-12-12
Downstream Request: api/contracts?projectNumber=45&startDate=2019-12-12&endDate=2019-12-12&projectNumber=45

Upstream Request: contracts?$filter=ProjectNumber eq 45 and DateOfSale ge 2020-03-01T00:00:00z and DateOfSale le 2020-03-15T00:00:00z
Downstream Request: api/contracts?$filter=ProjectNumber eq 45 and DateOfSale ge 2020-03-01T00:00:00z and DateOfSale le 2020-03-15T00:00:00z&$filter=ProjectNumber eq 45 and DateOfSale ge 2020-03-01T00:00:00z and DateOfSale le 2020-03-15T00:00:00z

Please see console log below

info: Ocelot.RateLimit.Middleware.ClientRateLimitMiddleware[0]
      requestId: 0HLUK013URU17:00000001, previousRequestId: no previous request id, message: EndpointRateLimiting is not enabled for /api/contracts?{everythingelse}
info: Ocelot.Authentication.Middleware.AuthenticationMiddleware[0]
      requestId: 0HLUK013URU17:00000001, previousRequestId: no previous request id, message: No authentication needed for /contracts
info: Ocelot.Authorisation.Middleware.AuthorisationMiddleware[0]
      requestId: 0HLUK013URU17:00000001, previousRequestId: no previous request id, message: /api/contracts?{everythingelse} route does not require user to be authorised
info: Ocelot.Requester.Middleware.HttpRequesterMiddleware[0]
      requestId: 0HLUK013URU17:00000001, previousRequestId: no previous request id, message: 200 (OK) status code, request uri: http://localhost:45626/api/contracts?**projectNumber=45**&startDate=2019-12-12&endDate=2019-12-12**&projectNumber=45**
info: Ocelot.RateLimit.Middleware.ClientRateLimitMiddleware[0]
      requestId: 0HLUK013URU18:00000001, previousRequestId: no previous request id, message: EndpointRateLimiting is not enabled for /api/contracts?{everythingelse}
info: Ocelot.Authentication.Middleware.AuthenticationMiddleware[0]
      requestId: 0HLUK013URU18:00000001, previousRequestId: no previous request id, message: No authentication needed for /contracts
info: Ocelot.Authorisation.Middleware.AuthorisationMiddleware[0]
      requestId: 0HLUK013URU18:00000001, previousRequestId: no previous request id, message: /api/contracts?{everythingelse} route does not require user to be authorised
warn: Ocelot.Requester.Middleware.HttpRequesterMiddleware[0]
      requestId: 0HLUK013URU18:00000001, previousRequestId: no previous request id, message: 404 (Not Found) status code, request uri: http://localhost:45626/api/contracts?**$filter=ProjectNumber eq 45 and DateOfSale ge 2020-03-01T00:00:00z and DateOfSale le 2020-03-15T00:00:00z**&$**filter=ProjectNumber eq 45 and DateOfSale ge 2020-03-01T00:00:00z and DateOfSale le 2020-03-15T00:00:00z**

Steps to Reproduce the Problem

1. Create an api controller with two actions, one normal web api and other one as OData.
2. Define Reroute as below

{ 
      "DownstreamPathTemplate": "/api/contracts?{everythingelse}",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "localhost",
          "Port": 45626
        }
      ],
      "UpstreamPathTemplate": "/contracts?{everythingelse}",
      "UpstreamHttpMethod": [ "Get" ]
}

3. Fire get request on OData endpoint with $filter query string parameter and it return 404 not found while directly hitting same endpoint by removing duplicate query string it works fine.

Specifications

• Version: 14.1.0
• Platform: .NET Core 3.1 on Windows

More Details

While debugging I found following code block in DownstreamUrlCreatorMiddleware is causing query string duplication.

  if (string.IsNullOrEmpty(context.DownstreamRequest.Query))
  {
       context.DownstreamRequest.Query = GetQueryString(dsPath);
  }
  else
  {
      context.DownstreamRequest.Query += GetQueryString(dsPath).Replace('?', '&');
  }

Removing this code block solved the problem but i don't know what other negative impact it would cause.

[wast]

@TomPallister what is the purpose of this line exactly?
context.DownstreamRequest.Query += GetQueryString(dsPath).Replace('?', '&');

I feel like RemoveQueryStringParametersThatHaveBeenUsedInTemplate(context); should also be called to remove duplication.

Also duplicate of #952

[sunilk3]

@wast Removing this causes the following unit test to fail but resolves the issue I am facing.

DownstreamUrlCreatorMiddlewareTests.issue_473_should_not_remove_additional_query_string

[sunilk3]

Using DelegatingHandler I was able to workaround this problem.