Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【JAVA】deserialize class not found USE withRefTracking(true).requireClassRegistration(false).withCompatibleMode(CompatibleMode.COMPATIBLE).buildThreadSafeFury() #1564

Closed
2 tasks done
weijiang157152688 opened this issue Apr 24, 2024 · 1 comment · Fixed by #1569
Labels
bug Something isn't working

Comments

@weijiang157152688
Copy link
Contributor

weijiang157152688 commented Apr 24, 2024

Search before asking

  • I had searched in the issues and found no similar issues.

Version

Version: 0.5.0 (built from source)
OS: macOS

Component(s)

Java

Minimal reproduce step

package alsc.ele.lpd.dispatch.matrix.service.tt;

import alsc.ele.lpd.dispatch.matrix.service.dto.AssignDetailInfoLog;
import alsc.ele.lpd.dispatch.matrix.service.dto.AssignType;
import alsc.ele.lpd.dispatch.matrix.service.util.FuryUtils;
import me.ele.lpd.dispatch.matrix.score.infra.config.CompressEnum;
import me.ele.lpd.dispatch.matrix.score.infra.utils.CompressObjectUtils;
import me.ele.lpd.dispatch.matrix.score.infra.utils.Lz4Util;
import org.xerial.snappy.Snappy;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.file.Files;

public class Test {


    public static void main(String[] args) throws IOException {

        // read
        read();

//        write();

    }

    public static void write() throws IOException {
        AssignDetailInfoLog assignDetailInfoLog = new AssignDetailInfoLog();
        assignDetailInfoLog.setType(AssignType.DEFAULT);
        assignDetailInfoLog.setId("dsa");
        byte[] data = FuryUtils.serialize(assignDetailInfoLog);
        saveToFile("test_file", data);
    }

    public static void read() throws IOException {

        // read
        byte[] data = readFileToBytes("test_file");
        FuryUtils.zeroCopyDeserialize(data);

    }

    public static void saveToFile(String fileName, byte[] data) {
        try (FileOutputStream outputStream = new FileOutputStream(fileName)) {
            outputStream.write(data);  // 写入byte[]
            System.out.println("Data has been written to " + fileName);
        } catch (IOException e) {
            e.printStackTrace();
        }

    }

    public static byte[] readFileToBytes(String filePath) {
        try {
            return Files.readAllBytes(new File(filePath).toPath());
        } catch (Exception e) {
            return null;
        }
    }
}

package alsc.ele.lpd.dispatch.matrix.service.dto;

import lombok.Data;

@Data
public class AssignDetailInfoLog {
    String id;
    AssignType  type;

}


package alsc.ele.lpd.dispatch.matrix.service.dto;

public enum AssignType {
    DEFAULT;
}


package alsc.ele.lpd.dispatch.matrix.service.util;

import org.apache.fury.Fury;
import org.apache.fury.ThreadSafeFury;
import org.apache.fury.config.CompatibleMode;
import org.apache.fury.logging.LoggerFactory;

import java.nio.ByteBuffer;

public class FuryUtils {
    private static final ThreadSafeFury fury;

    public FuryUtils() {
    }

    public static <T> byte[] serialize(T t) {
        return fury.serialize(t);
    }

    public static Object zeroCopyDeserialize(byte[] byteBuffer) {
        return fury.deserialize(byteBuffer);
    }

    public static Object zeroCopyDeserialize(ByteBuffer byteBuffer) {
        return fury.deserialize(byteBuffer);
    }

    static {
        fury = Fury.builder().withRefTracking(true).requireClassRegistration(false).withCompatibleMode(CompatibleMode.COMPATIBLE).buildThreadSafeFury();
        LoggerFactory.disableLogging();
    }
}

What did you expect to see?

deserialize

What did you see instead?

After deleting the AssignType file, an error occurred.

Deserialize failed, read objects are: [AssignDetailInfoLog(id=dsa), null]
at org.apache.fury.Fury.handleReadFailed(Fury.java:788)
at org.apache.fury.Fury.deserialize(Fury.java:750)
at org.apache.fury.Fury.deserialize(Fury.java:676)
at org.apache.fury.ThreadLocalFury.deserialize(ThreadLocalFury.java:130)
at alsc.ele.lpd.dispatch.matrix.service.util.FuryUtils.zeroCopyDeserialize(FuryUtils.java:21)
at alsc.ele.lpd.dispatch.matrix.service.tt.Test.main(Test.java:30)
Caused by: java.lang.IllegalStateException: Class alsc.ele.lpd.dispatch.matrix.service.dto.AssignType not found from classloaders [sun.misc.Launcher$AppClassLoader@18b4aac2, sun.misc.Launcher$AppClassLoader@18b4aac2]
at org.apache.fury.resolver.ClassResolver.loadClass(ClassResolver.java:1688)
at org.apache.fury.resolver.ClassResolver.loadBytesToClass(ClassResolver.java:1636)

Anything Else?

deserialize

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@weijiang157152688 weijiang157152688 added the bug Something isn't working label Apr 24, 2024
@chaokunyang
Copy link
Collaborator

I can run this successfully locally:
image

Here is my code:

package org.apache.fury;

import lombok.Data;
import org.apache.fury.config.CompatibleMode;
import org.apache.fury.logging.LoggerFactory;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.file.Files;

public class Test {
  @Data
  public static class AssignDetailInfoLog {
    String id;
    AssignType  type;

  }



  public enum AssignType {
    DEFAULT;
  }



  public static class FuryUtils {
    private static final ThreadSafeFury fury;

    public FuryUtils() {
    }

    public static <T> byte[] serialize(T t) {
      return fury.serialize(t);
    }

    public static Object zeroCopyDeserialize(byte[] byteBuffer) {
      return fury.deserialize(byteBuffer);
    }

    public static Object zeroCopyDeserialize(ByteBuffer byteBuffer) {
      return fury.deserialize(byteBuffer);
    }

    static {
      fury = Fury.builder().withRefTracking(true).requireClassRegistration(false).withCompatibleMode(CompatibleMode.COMPATIBLE).buildThreadSafeFury();
      LoggerFactory.disableLogging();
    }
  }

  public static void main(String[] args) throws IOException {

    // read
    read();

//        write();

  }

  public static void write() throws IOException {
    AssignDetailInfoLog assignDetailInfoLog = new AssignDetailInfoLog();
    assignDetailInfoLog.setType(AssignType.DEFAULT);
    assignDetailInfoLog.setId("dsa");
    byte[] data = FuryUtils.serialize(assignDetailInfoLog);
    saveToFile("test_file", data);
  }

  public static void read() throws IOException {
    write();
    // read
    byte[] data = readFileToBytes("test_file");
    FuryUtils.zeroCopyDeserialize(data);

  }

  public static void saveToFile(String fileName, byte[] data) {
    try (FileOutputStream outputStream = new FileOutputStream(fileName)) {
      outputStream.write(data);  // 写入byte[]
      System.out.println("Data has been written to " + fileName);
    } catch (IOException e) {
      e.printStackTrace();
    }

  }

  public static byte[] readFileToBytes(String filePath) {
    try {
      return Files.readAllBytes(new File(filePath).toPath());
    } catch (Exception e) {
      return null;
    }
  }
}

chaokunyang added a commit that referenced this issue Apr 26, 2024
## What does this PR do?

This PR support sdeserialize unexisted array/enum object.

For type compatible serialization, deserialization process may not have
the class in serialization peer. In such cases, we should support skip
object data of unexisted array/enum

## Related issues

Closes #1564


## Does this PR introduce any user-facing change?

<!--
If any user-facing interface changes, please [open an
issue](https://github.com/apache/incubator-fury/issues/new/choose)
describing the need to do so and update the document if necessary.
-->

- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?


## Benchmark

<!--
When the PR has an impact on performance (if you don't know whether the
PR will have an impact on performance, you can submit the PR first, and
if it will have impact on performance, the code reviewer will explain
it), be sure to attach a benchmark data here.
-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants