Reactive WebClient corrupts uploaded files [SPR-16246]

[spring-projects-issues]

Matúš Sekáč opened SPR-16246 and commented

When I try to upload some ordinary file through reactive WebClient, the file ends up corrupted. It happens with audio files as well as text files. I tried creating request body manually or via MultiPartBodyBuilder in newer version and the result was same. The behavior is same when using WebTestClient.

I also tried uploading to Apache web server and the files were still corrupted.

Sample test:

	@Test
	public void audioFile() throws IOException {
		ClassPathResource resource = new ClassPathResource(TEST_WAV_NAME);

		LinkedMultiValueMap<String, HttpEntity<?>> body = new LinkedMultiValueMap<>();
		HttpHeaders headers = new HttpHeaders();
		headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);
		HttpEntity<ClassPathResource> part = new HttpEntity<>(resource, headers);
		body.add("file", part);

		doTest(hash(resource), body);
	}

	private static long hash(AbstractFileResolvingResource resource) throws IOException {
		CRC32 crc = new CRC32();
		crc.update(Files.readAllBytes(resource.getFile().toPath()));
		return crc.getValue();
	}

	private void doTest(long hash, LinkedMultiValueMap<String, HttpEntity<?>> body) throws IOException {
		String result = WebClient.builder()
				.baseUrl("http://localhost:8080")
				.build()
				.post()
				.uri("/upload")
				.body(BodyInserters.fromMultipartData(body))
				.retrieve()
				.bodyToMono(String.class)
				.block();
		Assert.assertEquals(String.valueOf(hash), result);
	}

And corresponding controller:

@PostMapping(value = "/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public Mono<String> upload(@RequestPart("file") FilePart part) throws IOException {
     File tf = Files.createTempFile("spring-test", "temp").toFile();
     part.transferTo(tf);
     CRC32 crc = new CRC32();
     crc.update(Files.readAllBytes(tf.toPath()));
     return Mono.just(String.valueOf(crc.getValue()));
}

The whole sample project with tested files is attached.

---

Affects: 5.0 GA

Attachments:

• spring_reaxtiveweb_test.zip (2.34 MB)

Referenced from: commits b36af8a, 91d3e44

[spring-projects-issues]

Rossen Stoyanchev commented

The correct way to code the controller is something like this:

@PostMapping(value = "/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public Mono<String> upload(@RequestPart("file") FilePart part) throws IOException {

	File tf = Files.createTempFile("spring-test", "temp").toFile();
	Path path = tf.toPath();

	return part.transferTo(tf)
			.then(Mono.defer(() -> {
				CRC32 crc = new CRC32();
				try {
					byte[] bytes = Files.readAllBytes(path);
					System.out.println("length: " + bytes.length);
					crc.update(bytes);
				}
				catch (IOException e) {
					throw new IllegalStateException("");
				}
				long value = crc.getValue();
				System.out.println(value);
				return Mono.just(String.valueOf(value));
			}));
}

That said there does appear to be some issue. The (large) text file in the same project also doesn't upload correctly, but a smaller file does. Possibly an issue in the MultipartHttpMessageWriter based on the fact that it doesn't upload correctly to another server either. I will try to create a test based on that.

[spring-projects-issues]

Rossen Stoyanchev commented

This took a while but I managed to narrow it down. I created a test in our own tests (ignored for now). I suspect a Reactor Netty issue, so I've submitted a ticket there with a repro project.

[spring-projects-issues]

Rossen Stoyanchev commented

There is now a fix in Reactor Netty that I've been able to confirm using both our own test and the tests from the attached sample.