problem when serializing object #8

tamerh · 2016-10-27T10:51:21Z

Hello,

i am getting following exception while desalinizing my java object which is working with default java serialization. Problem is happening when i get my object from client putting the object seems ok.
it can be related with kryo but maybe you have an idea.Thanks in advance

at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.internal.strategy.GlobalKryoStrategy.readObject(GlobalKryoStrategy.java:18) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.internal.strategy.KryoStrategy.read(KryoStrategy.java:76) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.Serializer.read(Serializer.java:40) ~[subzero-all-0.6.jar:na] at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:46) ~[hazelcast-3.7.2.jar:3.7.2] at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:172) ~[hazelcast-3.7.2.jar:3.7.2] ... 8 common frames omitted Caused by: java.lang.UnsupportedOperationException: null at java.util.Collections$UnmodifiableCollection.add(Collections.java:1055) ~[na:1.8.0_65] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731) ~[subzero-all-0.6.jar:na] at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125) ~[subzero-all-0.6.jar:na]

The text was updated successfully, but these errors were encountered:

jerrinot · 2016-10-27T10:55:23Z

@tamerh: thanks for this report. it indeed looks like an issue in Kryo. Can you share the object you are trying to serialize?

jerrinot · 2016-10-27T10:56:49Z

are you using guava collections?

tamerh · 2016-10-27T11:07:53Z

thanks for the quick repyl @jerrinot

my object is like following. I am casting different object to the cachedObject and i suspect kryo does not like this. is it possible?

and no i am not using guava libiraries

`public class CacheObject implements Serializable {

private static final long serialVersionUID = 3575122621103004034L;
String domainId;
String                    refDomainId;
Object                    cachedObject;

public CacheObject() {
}

public CacheObject(String domainId, String refDomainId, Object cachedObject) {
    this.domainId = domainId;
    this.refDomainId = refDomainId;
    this.cachedObject = cachedObject;
}

public Object getCachedObject() {
    return cachedObject;
}

public String getDomainId() {
    return domainId;
}

public String getRefDomainId() {
    return refDomainId;
}

}`

jerrinot · 2016-10-27T11:10:09Z

maybe. I am not sure, unfortunately I don't know Kryo internals good enough to be able to tell.
Is the failure specific to any particular type used as cachedObject ?

tamerh · 2016-10-27T12:19:14Z

from the limited logs the suspicous part is a unmodifiable List. i am creating a unmodifiable string list via

Collections.unmodifiableList()

and then assign it to the List

do you think this should be declared or serialized in special way?

jerrinot · 2016-10-27T12:25:00Z

yeah, apparently unmodifiableList requires a special Kryo serializer. See this: https://github.com/magro/kryo-serializers

SubZero has support for custom Kryo serializers in the master branch, but it has not been released yet. Could you please try it? If that works for you then I could release a new version.

tamerh · 2016-10-27T12:30:27Z

Great! i am going to try the master. is this need any extra configuration?

jerrinot · 2016-10-27T12:40:07Z

in the simplest case you only need to create subzero-serializers.properties file with this format:

some.package.YourClass=other.package.KryoSerializer
some.package.OtherClass=other.package.OtherKryoSerializer

However in this case it appears to be little more complicated as the unmodifieable class are not public - they are always constructed via a factory method (Collections.unmodifiableList) so this is not usable.

I guess you will need to create own serialization subclass and do a programmatic registration. Currently this is not documented yet. This discussion could help you a bit.

This is going against the main idea of SubZero: It should be dead-simple and apparently it is not simple in your case. I'll try to improve it when I have time.

jerrinot · 2016-10-27T12:48:50Z

ok, I have an idea how to make it simpler.

It should be a simple as including the serializer class inside the subzero-serializers.properties So something like this:

de.javakaffee.kryoserializer.UnmodifiableCollectionsSerializer

Note there is no = sign.

The SubZero can figure out this is a serializer which does require special registration as documented at https://github.com/magro/kryo-serializers and call the registration method via reflection.

I'll try to implement it and push to master today evening (in Europe)

tamerh · 2016-10-27T13:00:48Z

sounds great @jerrinot . Just how can set the subzero-serializers.propertiesin my project? is there a enviroment variable for the file location?

jerrinot · 2016-10-27T13:04:18Z

the file has to be on your classpath. that's the only strategy as of now. I might add another strategies, perhaps an external file could be useful in some cases.

You can use the subzero.custom.serializers.config.filename system property to use a different filename. (but it still has to be on your classpath)

jerrinot · 2016-10-27T13:51:04Z

@tamerh: I just pushed a change into the master branch. It should be as simple as having subzero-serializers.properties on your classpath. This file has to have this line:

UnmodifiableCollectionsSerializer

Obviously the kryo-serializers have to be on your classpath. You can use these Maven coordinates:

<dependency>
    <groupId>de.javakaffee</groupId>
    <artifactId>kryo-serializers</artifactId>
    <version>0.41</version>
</dependency>

I noticed a bug in the UnmodifiableCollectionsSerializer - it does not work properly when the unmodified list is created by wrapping a list created by calling Arrays.asList(). I created an issue for this in the Kryo-Serializer project.

tamerh · 2016-10-27T15:34:57Z

although i have updated the master and clean mvn repo and configure subzero-serializers.properties
i am still getting the same error it seems new serialiazer is may not activated.i will make debug and make sure that i got your last commit and let you know. thanks @jerrinot

jerrinot · 2016-10-27T16:09:03Z

@tamerh: I pushed another update.
The issue with Arrays.asList() is not a bug in Kryo - it requires another serializer to be registered. It's now possible with the property file.

jerrinot · 2016-10-27T16:10:42Z

if you are going to debug it then this is a good starting point:

subzero/subzero-core/src/main/java/info/jerrinot/subzero/internal/PropertyUserSerializer.java

Line 53 in 3d844c3

while ((line = reader.readLine()) != null) {

tamerh · 2016-10-31T15:45:52Z

Hi @jerrinot today i have tried this again with proper setting and i got following exception any idea for the reason? Thanks
Caused by: java.lang.reflect.InvocationTargetException: null at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_65] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_65] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_65] at java.lang.reflect.Constructor.newInstance(Constructor.java:422) ~[na:1.8.0_65] at com.hazelcast.nio.ClassLoaderUtil.newInstance(ClassLoaderUtil.java:83) ~[hazelcast-3.7.2.jar:3.7.2] at com.hazelcast.nio.ClassLoaderUtil.newInstance(ClassLoaderUtil.java:74) ~[hazelcast-3.7.2.jar:3.7.2] at com.hazelcast.internal.serialization.impl.DefaultSerializationServiceBuilder.build(DefaultSerializationServiceBuilder.java:228) ~[hazelcast-3.7.2.jar:3.7.2] ... 12 common frames omitted Caused by: java.lang.IllegalStateException: Serializer UnmodifiableCollectionsSerializer does not have expected method 'registerSerializers()': at info.jerrinot.subzero.internal.PropertyUserSerializer.addNewSpecialSerializer(PropertyUserSerializer.java:90) ~[subzero-all-0.7-SNAPSHOT.jar:na] at info.jerrinot.subzero.internal.PropertyUserSerializer.readLineAndRegister(PropertyUserSerializer.java:73) ~[subzero-all-0.7-SNAPSHOT.jar:na] at info.jerrinot.subzero.internal.PropertyUserSerializer.initCustomSerializers(PropertyUserSerializer.java:54) ~[subzero-all-0.7-SNAPSHOT.jar:na] at info.jerrinot.subzero.internal.PropertyUserSerializer.<init>(PropertyUserSerializer.java:41) ~[subzero-all-0.7-SNAPSHOT.jar:na] at info.jerrinot.subzero.Serializer.<init>(Serializer.java:10) ~[subzero-all-0.7-SNAPSHOT.jar:na] ... 19 common frames omitted Caused by: java.lang.NoSuchMethodException: de.javakaffee.kryoserializers.UnmodifiableCollectionsSerializer.registerSerializers(info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo) at java.lang.Class.getMethod(Class.java:1786) ~[na:1.8.0_65] at info.jerrinot.subzero.internal.PropertyUserSerializer.addNewSpecialSerializer(PropertyUserSerializer.java:83) ~[subzero-all-0.7-SNAPSHOT.jar:na] ... 23 common frames omitted

jerrinot · 2016-10-31T15:54:18Z

@tamerh: can you check your are using the latest revision? this appears to be from an older version.

brunoml · 2016-11-11T14:15:33Z

@jerrinot Is there any release date for 0.7 version with this subzero-serializers.properties function? I have the same problem and could be fixed using my serializers.

jerrinot · 2016-11-11T14:17:18Z

@brunoml: I can do that this weekend. Can you please try the latest SNAPSHOT version and confirm it's working for you?

brunoml · 2016-11-11T14:18:57Z

I'll try the the latest SNAPSHOT version today and put the result here.

brunoml · 2016-11-11T21:30:46Z

@jerrinot I tried the master and subzero-serializers.properties worked for me but in my environment I already use Kryo and I'd like to pass all my configuration to Subzero (kryo.setDefaultSerializer(...) and kryo.setInstantiatorStrategy(...)) and the only way to do this is if accessing directly Kryo. I forked a version and made a push, I created tests and tried it, it worked. If you like it I can make a pull request.

jerrinot · 2016-11-14T18:44:52Z

I just released SubZero v0.7.
Many thanks to everyone for contributions!

tamerh closed this as completed Oct 27, 2016

tamerh reopened this Oct 27, 2016

jerrinot added this to the 0.7 milestone Oct 27, 2016

jerrinot added the enhancement label Oct 27, 2016

jerrinot closed this as completed Nov 14, 2016

chenghaojun mentioned this issue Feb 6, 2018

jstorm0.9.3升级2.1.0后反序列化异常 alibaba/jstorm#185

Closed

llaros mentioned this issue Sep 14, 2021

Custom Kryo serializer for unmodificable collections #46

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

problem when serializing object #8

problem when serializing object #8

tamerh commented Oct 27, 2016

jerrinot commented Oct 27, 2016

jerrinot commented Oct 27, 2016

tamerh commented Oct 27, 2016

jerrinot commented Oct 27, 2016

tamerh commented Oct 27, 2016

jerrinot commented Oct 27, 2016

tamerh commented Oct 27, 2016

jerrinot commented Oct 27, 2016

jerrinot commented Oct 27, 2016

tamerh commented Oct 27, 2016

jerrinot commented Oct 27, 2016

jerrinot commented Oct 27, 2016

tamerh commented Oct 27, 2016

jerrinot commented Oct 27, 2016 •

edited

Loading

jerrinot commented Oct 27, 2016

tamerh commented Oct 31, 2016

jerrinot commented Oct 31, 2016

brunoml commented Nov 11, 2016 •

edited

Loading

jerrinot commented Nov 11, 2016

brunoml commented Nov 11, 2016

brunoml commented Nov 11, 2016

jerrinot commented Nov 14, 2016

problem when serializing object #8

problem when serializing object #8

Comments

tamerh commented Oct 27, 2016

jerrinot commented Oct 27, 2016

jerrinot commented Oct 27, 2016

tamerh commented Oct 27, 2016

jerrinot commented Oct 27, 2016

tamerh commented Oct 27, 2016

jerrinot commented Oct 27, 2016

tamerh commented Oct 27, 2016

jerrinot commented Oct 27, 2016

jerrinot commented Oct 27, 2016

tamerh commented Oct 27, 2016

jerrinot commented Oct 27, 2016

jerrinot commented Oct 27, 2016

tamerh commented Oct 27, 2016

jerrinot commented Oct 27, 2016 • edited Loading

jerrinot commented Oct 27, 2016

tamerh commented Oct 31, 2016

jerrinot commented Oct 31, 2016

brunoml commented Nov 11, 2016 • edited Loading

jerrinot commented Nov 11, 2016

brunoml commented Nov 11, 2016

brunoml commented Nov 11, 2016

jerrinot commented Nov 14, 2016

jerrinot commented Oct 27, 2016 •

edited

Loading

brunoml commented Nov 11, 2016 •

edited

Loading