Plus signs in url variables not encoded; indistinguishable from spaces [SPR-9831]

[spring-projects-issues]

Robert MacDonald opened SPR-9831 and commented

As discussed in the end of #10187, when + signs are included in url variables they are not encoded so that when the DispatcherServlet decodes it all pluses and spaces become spaces. The listed workaround for this requires the user to construct their own urls and properly encode them which effectively reproduces a number of classes and methods in Spring with the only effective difference being that plus signs are encoded to %2B during URI creation because Spring does not do this for us. Since the components performing the encoding are deep within Spring we cannot simple configure it differently or extend a class to change this so please fix it in Spring.

---

Affects: 3.1.2

1 votes, 7 watchers

[spring-projects-issues]

Rossen Stoyanchev commented

You can use UriComponentsBuilder to construct the URI with full control over encoding. That said I can't reproduce the problem you're talking about:

UriTemplate uriTemplate = new UriTemplate("http://www.example.org/?param=a+b");
System.out.println(uriTemplate.expand().toString());

Result:
http://www.example.org/?param=a%2Bb

[spring-projects-issues]

Robert MacDonald commented

My current defect scenario:

String name = "troublesome + parameter";
String url = "htt://localhost:8080/object/{name}");
Map<String, String> vars = new HashMap<String, String>();
vars.put("name", name);
ObjectResponse response = restTemplate.getForObject(url, ObjectResponse.class, vars);

What spring does inside getForObject:

public <T> T getForObject(String url, Class<T> responseType, Map<String, ?> urlVariables) throws RestClientException {
	AcceptHeaderRequestCallback requestCallback = new AcceptHeaderRequestCallback(responseType);
	HttpMessageConverterExtractor<T> responseExtractor =
			new HttpMessageConverterExtractor<T>(responseType, getMessageConverters(), logger);
	return execute(url, HttpMethod.GET, requestCallback, responseExtractor, urlVariables);
}

public <T> T execute(String url, HttpMethod method, RequestCallback requestCallback,
		ResponseExtractor<T> responseExtractor, Map<String, ?> urlVariables) throws RestClientException {

	UriTemplate uriTemplate = new HttpUrlTemplate(url);
	URI expanded = uriTemplate.expand(urlVariables);
	return doExecute(expanded, method, requestCallback, responseExtractor);
}

It sounds like you're saying that if I did your example code right before the restTemplate call then use the URI method it would work around this. Right?

String name = "troublesome + parameter";
String url = "htt://localhost:8080/object/{name}");
Map<String, String> vars = new HashMap<String, String>();
vars.put("name", name);
UriTemplate uriTemplate = new UriTemplate("http://www.example.org/?param=a+b");
ObjectResponse response = restTemplate.getForObject(uriTemplate.expand(vars), ObjectResponse.class, vars);

[spring-projects-issues]

Rossen Stoyanchev commented

No, I'm definitely not recommending that. I was merely taking a shortcut knowing that's that the RestTemplate does internally.

That said I see in your example the "+" is in the path while I assumed it was in the query string based on the SPR you referenced. A "+" is not a reserved character in the path and the DispatcherServlet should not be decoding it to space (see #10957).

[spring-projects-issues]

Anton Rukhlin commented

This problem is also present in 3.1.0.

[spring-projects-issues]

Rossen Stoyanchev commented

Anton Rukhlin, can you clarify? As I stated above "+" is not a reserved character when present in the path. I just tested to verify that when UrlPathHelper obtains the "lookupPath" from the request, the "+" remains "+".

[spring-projects-issues]

Anton Rukhlin commented

Sorry, let me provide more detail...

This code in version 3.1.0:

<spring:url value="/invoices">
    <spring:param name="ref" value="FHJI+1111" />
</spring:url>

Generates the following URL:
http://localhost:8080/invoices?ref=FHJI+1111

However, I am expecting it to generate the following URL:
http://localhost:8080/invoices?ref=FHJI%2B1111

In order words, I am expecting it to url-encode the plus sign as well. Because if it does not do it - there is no way to pass a plus sign as a part of a parameter.

I even wrote my own urlEncode function, which encodes the plus sign. But then, guess what, if I do this:

<spring:url value="/invoices">
    <spring:param name="ref" value="${myfn:urlEncode('FHJI+1111')}" />
</spring:url>

Then I get the following URL, because the percent sign is being encoded correctly:
http://localhost:8080/invoices?ref=FHJI%252B1111

I do understand that a plus sign in a path (e.g., http://localhost:8080/invoices+stuff/) is not supposed to be encoded.

[spring-projects-issues]

Rossen Stoyanchev commented

Okay, understood. The issues got mixed up a little, one of the examples above was with "+" in the path so I wasn't sure which issue you were talking about.

I just did a test with "http://localhost:8080/invoices?ref=FHJI+1111" in UrlTag. Both the latest in 3.1 and 3.2 result in "http://localhost:8080/invoices?ref=FHJI%2B1111", which is what I would expect as well.

Try upgrading to the latest 3.1.x release (3.1.4).

[spring-projects-issues]

Anton Rukhlin commented

thank you, Rossen. I will upgrade when possible and report back if there is an issue.

[spring-projects-issues]

Brian Clozel commented

Did you manage to reproduce this issue with 3.2.x?

[spring-projects-issues]

J. Pablo Fernández commented

In Spring 5.0.5 this bug is still present and it affects RestTemplate, UriTemplate, and UriComponentsBuilder.

If you make this request with RestTemplate:

String foo = "fo+o";
String bar = "ba r";
restTemplate.exchange("http://example.com/?foo={foo}&bar={bar}", HttpMethod.GET, null, foo, bar)

you'll get a GET request to the URL:

http://example.com/?foo=fo+o&bar=ba%20r

which means, inside Spring, the variable foo will contain "fo o" and the variable bar will contain "ba r".

I think RestTemplate using UriTemplate, right? You can see this bug in UriTemplate:

String foo = "fo+o";
String bar = "ba r";
UriTemplate uriTemplate = new UriTemplate("http://example.com/?foo={foo}&bar={bar}");
Map<String, String> vars = new HashMap<>();
vars.put("foo", foo);
vars.put("bar", bar);
URI uri = uriTemplate.expand(vars);
System.out.println(uri);
{/code}

That prints:

http://example.com/?foo=fo+o&bar=ba%20r

UriComponentsBuilder suffers from the same issue:

```java 
String foo = "fo+o";
String bar = "ba r";
UriComponentsBuilder ucb = UriComponentsBuilder.fromUriString("http://example.com/?foo={foo}&bar={bar}");
System.out.println(ucb.build().expand(foo, bar).toUri());
System.out.println(ucb.build().expand(foo, bar).toString());
System.out.println(ucb.build().expand(foo, bar).toUriString());
System.out.println(ucb.build().expand(foo, bar).encode().toUri());
System.out.println(ucb.build().expand(foo, bar).encode().toString());
System.out.println(ucb.build().expand(foo, bar).encode().toUriString());
System.out.println(ucb.buildAndExpand(foo, bar).toUri());
System.out.println(ucb.buildAndExpand(foo, bar).toString());
System.out.println(ucb.buildAndExpand(foo, bar).toUriString());
System.out.println(ucb.buildAndExpand(foo, bar).encode().toUri());
System.out.println(ucb.buildAndExpand(foo, bar).encode().toString());
System.out.println(ucb.buildAndExpand(foo, bar).encode().toUriString());

That prints:

http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba r
http://example.com/?foo=fo+o&bar=ba r
http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba r
http://example.com/?foo=fo+o&bar=ba r
http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba%20r

None of which are correct.

[spring-projects-issues]

Rossen Stoyanchev commented

An old ticket that's been closed a long time ago will get no further discussion. You need to create a new ticket.

[spring-projects-issues]

J. Pablo Fernández commented

I thought so, so, I created this one: #21399

[ceefour]

I also got this problem (also #21399), and my workaround is to use Apache URIBuilder instead: https://stackoverflow.com/a/50959115/122441

URIBuilder ub = new URIBuilder("http://example.com/query");
ub.setParameter("q", "foo+bar"); // foo%2Bbar
URI uri = ub.build();

I hope this is useful for other people having the same problem.

[jenslofgren]

I also got this problem (also #21399), and my workaround is to use Apache URIBuilder instead: https://stackoverflow.com/a/50959115/122441

URIBuilder ub = new URIBuilder("http://example.com/query");
ub.setParameter("q", "foo+bar"); // foo%2Bbar
URI uri = ub.build();

I hope this is useful for other people having the same problem.

Helped me a lot! Thanks!
#Greenwich.SR1 #2.1.5.RELEASE
/Jens

[rstoyanchev]

There are now also solutions to this in Spring. See https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#web-uri-encoding.