SIGSEGV when indexing nullable value tuple with value #4703

leo-labs · 2020-05-24T20:02:04Z

NEST/Elasticsearch.Net version:
7.6.1/7.7

Elasticsearch version:
7.6.2

Runtime/OS info
dotnet 3.1.201 on Mac OS X 10.14.4 (BuildVersion: 18E226)

Description of the problem including expected versus actual behavior:

Indexing a document that contains a property that is a nullable value tuple such as (string info, int number)? with a value that is not null produces a SIGSEGV and the whole application crashes.

Steps to reproduce:

The issue can be reproduced with the following code:

using System;
using Nest;

public class ExampleDoc {
	public (string info, int number)? tupleNullable { get; set; }
}

public class Program {
	public static void Main() {
		var node = new Uri("http://localhost:9200");
		var client = new ElasticClient(new ConnectionSettings(node));
		
		var doc = new ExampleDoc {
			tupleNullable = ("somestring", 42),
		};

                var indexName= "index4";
                client.Indices.Create(indexName, index => index
                        .Map<ExampleDoc>(m => m
                        .AutoMap()
                ));

               client.Index(doc, i => i.Index(indexName));
	}
}

Expected behavior
The expected bahavior is to either throw an exception if this usecase is not supported or to successfully index the document

The text was updated successfully, but these errors were encountered:

russcam · 2020-05-25T06:08:06Z

Thanks for reporting @leo-labs. Looks like ValueTuple? needs special handling to be supported. On Windows, it looks like it enters into an infinite loop.

russcam · 2020-05-25T06:29:54Z

Stack trace on netcoreapp3.1 on Windows

Fatal error. Internal CLR error. (0x80131506)
   at System.Runtime.CompilerServices.ITuple.get_Length()
   at DynamicClass.Serialize(Byte[][], System.Object[], Elasticsearch.Net.Utf8Json.JsonWriter ByRef, System.ValueTuple`2<System.String,Int32>, Elasticsearch.Net.Utf8Json.IJsonFormatterResolver)
   at Elasticsearch.Net.Utf8Json.Resolvers.DynamicMethodAnonymousFormatter`1[[System.ValueTuple`2[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Int32, System.
Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Serialize(Elasticsearch.Net.Utf8Json.JsonWriter By
Ref, System.ValueTuple`2<System.__Canon,Int32>, Elasticsearch.Net.Utf8Json.IJsonFormatterResolver)
   at Elasticsearch.Net.Utf8Json.Formatters.StaticNullableFormatter`1[[System.ValueTuple`2[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Int32, System.Private
.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Serialize(Elasticsearch.Net.Utf8Json.JsonWriter ByRef, Sy
stem.Nullable`1<System.ValueTuple`2<System.__Canon,Int32>>, Elasticsearch.Net.Utf8Json.IJsonFormatterResolver)
   at DynamicClass.Serialize(Byte[][], System.Object[], Elasticsearch.Net.Utf8Json.JsonWriter ByRef, GitHubIssue4703.ExampleDoc, Elasticsearch.Net.Utf8Json.IJsonFormatterResolver)
   at Elasticsearch.Net.Utf8Json.Resolvers.DynamicMethodAnonymousFormatter`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Serialize(Elasticsearch.Net.Utf8Json.JsonW
riter ByRef, System.__Canon, Elasticsearch.Net.Utf8Json.IJsonFormatterResolver)
   at Elasticsearch.Net.Utf8Json.JsonSerializer.SerializeUnsafe[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon, Elasticsearch.Net.Utf8Json.IJsonFormatt
erResolver)
   at Elasticsearch.Net.Utf8Json.JsonSerializer.Serialize[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.IO.Stream, System.__Canon, Elasticsearch.Net.Utf8Json.
IJsonFormatterResolver)
   at Nest.DefaultHighLevelSerializer.Serialize[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon, System.IO.Stream, Elasticsearch.Net.SerializationFormat
ting)
   at Elasticsearch.Net.DiagnosticsSerializerProxy.Serialize[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon, System.IO.Stream, Elasticsearch.Net.Serial
izationFormatting)
   at Nest.IndexDescriptor`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Nest.IProxyRequest.WriteJson(Elasticsearch.Net.IElasticsearchSerializer, System.IO.Stream,
 Elasticsearch.Net.SerializationFormatting)
   at Nest.ProxyRequestFormatterBase`2[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, Public
KeyToken=7cec85d7bea7798e]].Serialize(Elasticsearch.Net.Utf8Json.JsonWriter ByRef, System.__Canon, Elasticsearch.Net.Utf8Json.IJsonFormatterResolver)
   at Elasticsearch.Net.Utf8Json.JsonSerializer.SerializeUnsafe[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon, Elasticsearch.Net.Utf8Json.IJsonFormatt
erResolver)
   at Elasticsearch.Net.Utf8Json.JsonSerializer.Serialize[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.IO.Stream, System.__Canon, Elasticsearch.Net.Utf8Json.
IJsonFormatterResolver)
   at Nest.DefaultHighLevelSerializer.Serialize[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon, System.IO.Stream, Elasticsearch.Net.SerializationFormat
ting)
   at Elasticsearch.Net.DiagnosticsSerializerProxy.Serialize[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon, System.IO.Stream, Elasticsearch.Net.Serial
izationFormatting)
   at Elasticsearch.Net.SerializableData`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Write(System.IO.Stream, Elasticsearch.Net.IConnectionConfigurationValues)
   at Elasticsearch.Net.InMemoryConnection.ReturnConnectionStatus[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Elasticsearch.Net.RequestData, Byte[], System.Nullabl
e`1<Int32>, System.String)
   at Elasticsearch.Net.InMemoryConnection.Request[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Elasticsearch.Net.RequestData)
   at Elasticsearch.Net.RequestPipeline.CallElasticsearch[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Elasticsearch.Net.RequestData)
   at Elasticsearch.Net.Transport`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Request[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=7cec85d7bea7798e]](Elasticsearch.Net.HttpMethod, System.String, Elasticsearch.Net.PostData, Elasticsearch.Net.IRequestParameters)
   at Elasticsearch.Net.ElasticLowLevelClient.DoRequest[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Elasticsearch.Net.HttpMethod, System.String, Elasticsearch.Net.
PostData, Elasticsearch.Net.IRequestParameters)
   at Nest.ElasticClient.DoRequest[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyT
oken=7cec85d7bea7798e]](System.__Canon, Elasticsearch.Net.IRequestParameters, System.Action`1<Elasticsearch.Net.IRequestConfiguration>)
   at Nest.ElasticClient.Index[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Nest.IIndexRequest`1<System.__Canon>)
   at Nest.ElasticClient.Index[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon, System.Func`2<Nest.IndexDescriptor`1<System.__Canon>,Nest.IIndexRequest`
1<System.__Canon>>)
   at GitHubIssue4703.Program.Main(System.String[])

russcam · 2020-05-25T07:02:05Z

I've added a test branch, fix/4703 with a unit test to reproduce, 61386f3. The unit test passes when run as part of the unit test suite e.g.

./build skipdocs

Will need to dig further.

leo-labs · 2020-05-25T15:50:34Z

Thanks for investigating!

On a different note, I would like to point out that the behavior with named value tuples is counterintuitive: If I have (string status, int code) myTuple auto-mapping does not preserve the item names but substitutes status with item1 and code with item2.

russcam · 2020-05-26T01:11:36Z

On a different note, I would like to point out that the behavior with named value tuples is counterintuitive: If I have (string status, int code) myTuple auto-mapping does not preserve the item names but substitutes status with item1 and code with item2.

The semantic names are syntactical sugar that Roslyn can understand, but ultimately, the IL generates Item* fields for ValueTuple<>. As an example

void Main()
{
	var example = new ExampleDoc
	{
		tupleNullable = ("somestring", 42)
	};
	
	var info = example.tupleNullable.Value.info;
}

public class ExampleDoc
{
	public (string info, int number)? tupleNullable { get; set; }
}

generates the following IL

IL_0000:  nop         
IL_0001:  newobj      UserQuery+ExampleDoc..ctor
IL_0006:  dup         
IL_0007:  ldstr       "somestring"
IL_000C:  ldc.i4.s    2A 
IL_000E:  newobj      System.ValueTuple<System.String,System.Int32>..ctor
IL_0013:  newobj      System.Nullable<System.ValueTuple<System.String,System.Int32>>..ctor
IL_0018:  callvirt    UserQuery+ExampleDoc.set_tupleNullable
IL_001D:  nop         
IL_001E:  stloc.0     
IL_001F:  ldloc.0     
IL_0020:  callvirt    UserQuery+ExampleDoc.get_tupleNullable
IL_0025:  stloc.2     
IL_0026:  ldloca.s    02 
IL_0028:  call        System.Nullable<System.ValueTuple<System.String,System.Int32>>.get_Value
IL_002D:  ldfld       System.ValueTuple<System.String,System.Int32>.Item1
IL_0032:  stloc.1     
IL_0033:  ret

You can see that assigning the value of example.tupleNullable.Value.info to the variable info generates ldfld System.ValueTuple<System.String,System.Int32>.Item1. If names are important in the emitted JSON, I would suggest using properties on the POCO instead.

This commit adds special serialization handling for nullable ValueTuples. Specialized handling is required as without, a nullable ValueTuple type ends up falling into CustomDyanmicObjectResolver inside of NestFormatterResolver, which will build a dynamic formatter using MetaType et.al, which enumerates interfaces and can cause a CLR error when calling `at System.Runtime.CompilerServices.ITuple.get_Length()`. Fixes #4703

…4715) This commit adds special serialization handling for nullable ValueTuples. Specialized handling is required as without, a nullable ValueTuple type ends up falling into CustomDyanmicObjectResolver inside of NestFormatterResolver, which will build a dynamic formatter using MetaType et.al, which enumerates interfaces and can cause a CLR error when calling `at System.Runtime.CompilerServices.ITuple.get_Length()`. Fixes #4703 Co-authored-by: Russ Cam <russ.cam@elastic.co>

…4716) This commit adds special serialization handling for nullable ValueTuples. Specialized handling is required as without, a nullable ValueTuple type ends up falling into CustomDyanmicObjectResolver inside of NestFormatterResolver, which will build a dynamic formatter using MetaType et.al, which enumerates interfaces and can cause a CLR error when calling `at System.Runtime.CompilerServices.ITuple.get_Length()`. Fixes #4703 Co-authored-by: Russ Cam <russ.cam@elastic.co>

russcam · 2020-05-27T09:34:01Z

This is now fixed and nullable ValueTuples supported in 7.7.1

xsoheilalizadeh · 2020-06-03T07:23:26Z

I'm facing this in ElasticSearch.Net 7.7.1, seems It's not able to serialize StringValues

russcam · 2020-06-03T07:57:06Z

@xsoheilalizadeh StringValues are not currently supported. It might be best to open a new issue for this to discuss 👍

xsoheilalizadeh · 2020-06-03T08:00:12Z

@russcam, I'm already working on implementing its formatter, will make the issue for that.

leo-labs added the Bug label May 24, 2020

russcam added a commit that referenced this issue May 25, 2020

Add unit test for #4703

61386f3

russcam self-assigned this May 25, 2020

russcam mentioned this issue May 26, 2020

Add special serialization handling for nullable ValueTuples #4713

Merged

russcam added a commit that referenced this issue May 27, 2020

Add unit test for #4703

78d9618

russcam closed this as completed in #4713 May 27, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SIGSEGV when indexing nullable value tuple with value #4703

SIGSEGV when indexing nullable value tuple with value #4703

leo-labs commented May 24, 2020 •

edited

Loading

russcam commented May 25, 2020

russcam commented May 25, 2020

russcam commented May 25, 2020

leo-labs commented May 25, 2020

russcam commented May 26, 2020

russcam commented May 27, 2020

xsoheilalizadeh commented Jun 3, 2020 •

edited

Loading

russcam commented Jun 3, 2020

xsoheilalizadeh commented Jun 3, 2020

SIGSEGV when indexing nullable value tuple with value #4703

SIGSEGV when indexing nullable value tuple with value #4703

Comments

leo-labs commented May 24, 2020 • edited Loading

russcam commented May 25, 2020

russcam commented May 25, 2020

russcam commented May 25, 2020

leo-labs commented May 25, 2020

russcam commented May 26, 2020

russcam commented May 27, 2020

xsoheilalizadeh commented Jun 3, 2020 • edited Loading

russcam commented Jun 3, 2020

xsoheilalizadeh commented Jun 3, 2020

leo-labs commented May 24, 2020 •

edited

Loading

xsoheilalizadeh commented Jun 3, 2020 •

edited

Loading