Skip to content

Conversation

@masteryhx
Copy link
Contributor

What is the purpose of the change

When forst starts to restore from reused checkpoint, the mapping entries include:

  1. remote uuid -> state handle (also remote uuid path in the state handle)
  2. remote sst -> remote uuid

When restored, the file is existed, but fails because it always try to find a sst in the local filesystem.
We should avoid reading NOT_OWNED file by local file system when restoring.

Brief change log

  • Judge whether the file is NOT_OWNED when trying to access local file system.

Verifying this change

  • Added ForStFlinkFileSystemTest#testNotOwnedFileStatus

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
  • The serializers: (yes / no / don't know)
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
  • The S3 file system connector: (yes / no / don't know)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@masteryhx
Copy link
Contributor Author

@AlexYinHan Kindly ping

@flinkbot
Copy link
Collaborator

flinkbot commented Apr 22, 2025

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@AlexYinHan
Copy link
Contributor

AlexYinHan commented Apr 25, 2025

@masteryhx Thanks for reporting and fixing this. LGTM. A NOT_OWNED file must be an SST file, so we can use file ownership to provide supplementary information for determining whether a file should be local.

@masteryhx masteryhx merged commit 95bf638 into apache:master Apr 28, 2025
yanand0909 pushed a commit to yanand0909/flink that referenced this pull request Jun 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants