Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Large Request Header via Azure App Config and Blob Storage #4716

Closed
Intern426 opened this issue Jul 30, 2019 · 3 comments · Fixed by #4806
Closed

[BUG] Large Request Header via Azure App Config and Blob Storage #4716

Intern426 opened this issue Jul 30, 2019 · 3 comments · Fixed by #4806
Assignees
@alzimmermsft
Labels
App Configuration Azure.ApplicationModel.Configuration Azure.Core azure-core Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage Storage Service (Queues, Blobs, Files)
Milestone
Sprint 156

Comments

@Intern426
Copy link

Describe the bug
If the user attempts to access data from Azure App Configuration multiple times through the StorageAsyncClient from Azure Blob Storage, the request header gets too long because of repeating azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0

Exception or Stack Trace

84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationAsyncClient  - Listing ConfigurationSettings - SettingSelector(keys=BLOB*, labels=null, acceptDateTime=null, fields=ALL_FIELDS, range=null)
84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - --> GET https://bookinventory.azconfig.io/kv?key=BLOB%2a
84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Accept-Datetime:null
84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Date:Tue, 30 Jul 2019 21:12:31 GMT
84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Authorization:HMAC-SHA256 Credential=0-l1-s0:4QJGUxmhEpiHg9FhMzDd, SignedHeaders=Host;Date;x-ms-content-sha256, Signature=XifftriV5Sd3CXFU0ftkc/FYrcZDf8UoxGGnmL4Zlr0=
84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Content-Length:0
84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - x-ms-return-client-request-id:true
84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - x-ms-content-sha256:47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=
84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Host:bookinventory.azconfig.io
84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Content-Type:application/json
84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - x-ms-client-request-id:8bc8014c-fe10-4061-94f5-e49645c37990
84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - User-Agent:azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0 azsdk-java-application-configuration/1.0.0-SNAPSHOT 11.0.3; Windows 10 10.0
84709 [main] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Accept:application/vnd.microsoft.azconfig.kv+json
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - <-- 400 Bad Request https://bookinventory.azconfig.io/kv?key=BLOB%2a (22 ms, 250-byte body)
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Date:Tue, 30 Jul 2019 21:12:31 GMT
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Server:nginx/1.13.12
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Content-Length:250
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Content-Type:text/html
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Connection:close
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - <-- END HTTP
84774 [reactor-http-nio-4] WARN  com.azure.data.appconfiguration.ConfigurationAsyncClient  - Failed to list ConfigurationSetting - SettingSelector(keys=BLOB*, labels=null, acceptDateTime=null, fields=ALL_FIELDS, range=null)
com.azure.core.exception.HttpResponseException: Status code 400, "<html>
<head><title>400 Request Header Or Cookie Too Large</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<center>Request Header Or Cookie Too Large</center>
<hr><center>nginx/1.13.12</center>
</body>
</html>
"
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at com.azure.core.implementation.RestProxy.instantiateUnexpectedException(RestProxy.java:315)
	at com.azure.core.implementation.RestProxy.lambda$null$2(RestProxy.java:363)
	at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3848)
	at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:75)
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onComplete(MonoFlatMap.java:174)
	at reactor.core.publisher.MonoProcessor$NextInner.onComplete(MonoProcessor.java:592)
	at reactor.core.publisher.MonoProcessor.onNext(MonoProcessor.java:380)
	at reactor.core.publisher.MonoProcessor.onComplete(MonoProcessor.java:335)
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:141)
	at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1515)
	at reactor.core.publisher.MonoProcessor.subscribe(MonoProcessor.java:457)
	at reactor.core.publisher.MonoMap.subscribe(MonoMap.java:55)
	at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
	at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3848)
	at reactor.core.publisher.MonoProcessor.add(MonoProcessor.java:531)
	at reactor.core.publisher.MonoProcessor.subscribe(MonoProcessor.java:444)
	at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
	at reactor.core.publisher.MonoSwitchIfEmpty.subscribe(MonoSwitchIfEmpty.java:44)
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:150)
	at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1515)
	at reactor.core.publisher.MonoProcessor.onNext(MonoProcessor.java:389)
	at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onNext(FluxDoFinally.java:123)
	at reactor.core.publisher.FluxHandle$HandleSubscriber.onNext(FluxHandle.java:113)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onNext(FluxMapFuseable.java:287)
	at reactor.core.publisher.FluxUsing$UsingFuseableSubscriber.onNext(FluxUsing.java:350)
	at reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:113)
	at reactor.core.publisher.FluxPeekFuseable$PeekFuseableConditionalSubscriber.onNext(FluxPeekFuseable.java:486)
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1515)
	at reactor.core.publisher.MonoReduceSeed$ReduceSeedSubscriber.onComplete(MonoReduceSeed.java:156)
	at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:136)
	at reactor.netty.channel.FluxReceive.terminateReceiver(FluxReceive.java:378)
	at reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:202)
	at reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:343)
	at reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:325)
	at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:372)
	at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:511)
	at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:141)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1436)
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1203)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:591)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:508)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905)
	at java.base/java.lang.Thread.run(Thread.java:834)

To Reproduce

  1. Create an Azure App Configuration and Azure Blob Storage Account
  2. In App Configuration, create 3 settings.
  • One called BLOB_ACCOUNT_NAME to hold the blob storage account name
  • One called BLOB_KEY to hold the key for the Azure Blob Storage account
  • One called BLOB_URL to hold the url in the following format:
    https://<your-blob-account-name>.blob.core.windows.net
  1. Save your App Configuration connection string in the AZURE_APP_CONFIG variable

Code Snippet
https://github.com/Intern426/azure-sdk-for-java/blob/master/ConfigurationBugSample.java
@Intern426 Intern426 changed the title [BUG] Overflooded Request Header via Azure App Config and Blob Storage [BUG] Large Request Header via Azure App Config and Blob Storage Jul 30, 2019
@kurtzeborn kurtzeborn added Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage Storage Service (Queues, Blobs, Files) and removed triage labels Jul 30, 2019
@kurtzeborn
Copy link
Member

Thanks for reporting this issue. We'll need to have someone take a closer look to understand why this is happening.

CC: @jianghaolu, @alzimmermsft

@conniey conniey added the App Configuration Azure.ApplicationModel.Configuration label Jul 31, 2019
@joshfree joshfree added this to the Sprint 156 milestone Aug 1, 2019
@joshfree
Copy link
Member

joshfree commented Aug 1, 2019

@alzimmermsft can you please take a look at this customer reported issue today?

84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - <-- 400 Bad Request https://bookinventory.azconfig.io/kv?key=BLOB%2a (22 ms, 250-byte body)
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Date:Tue, 30 Jul 2019 21:12:31 GMT
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Server:nginx/1.13.12
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Content-Length:250
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Content-Type:text/html
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - Connection:close
84739 [reactor-http-nio-4] INFO  com.azure.data.appconfiguration.ConfigurationService.listKeyValues  - <-- END HTTP

@alzimmermsft alzimmermsft added the Azure.Core azure-core label Aug 1, 2019
@alzimmermsft
Copy link
Member

Thank you for reporting this issue.

It appears that the User-Agent policy is appending the same User-Agent to the header value for each call being made to the service. I will look into this further to find a root cause and determine how to mitigate this issue.

@alzimmermsft alzimmermsft mentioned this issue Aug 2, 2019
Merged
@github-actions github-actions bot locked and limited conversation to collaborators Apr 13, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@alzimmermsft alzimmermsft

Labels
App Configuration Azure.ApplicationModel.Configuration Azure.Core azure-core Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Milestone
Sprint 156
Development

Successfully merging a pull request may close this issue.

Fix Header Replication and Unit Test alzimmermsft/azure-sdk-for-java
5 participants
@joshfree @conniey @kurtzeborn @alzimmermsft @Intern426