Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some unittest on get_buffer #889

Merged
merged 7 commits into from
Jun 17, 2024
Merged

Add some unittest on get_buffer #889

merged 7 commits into from
Jun 17, 2024

Conversation

mgautierfr
Copy link
Collaborator

@mgautierfr mgautierfr commented May 30, 2024

This Pr was initially to fix #886. But it appears it was not a bug on our side.

However, it is better testing and it reveals some bug on Windows side.

@kelson42
Copy link
Contributor

Where is rhe codecov, it seems it has vanished!

@rgaudin
Copy link
Member

rgaudin commented May 30, 2024

I think it doesn't run on drafts

@mgautierfr mgautierfr changed the base branch from main to libzim_github_ci_windows June 10, 2024 15:45
@mgautierfr mgautierfr marked this pull request as ready for review June 10, 2024 16:02
src/buffer_reader.cpp Outdated Show resolved Hide resolved
src/fs_unix.cpp Show resolved Hide resolved
src/file_reader.cpp Outdated Show resolved Hide resolved
src/fs_windows.cpp Outdated Show resolved Hide resolved
src/fs_windows.cpp Outdated Show resolved Hide resolved
goto err;
}

if (size_read == 0) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe size_read != batch_to_read?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From ReadFile function, it is technically possible than size_read != batch_to_read. If we reach the end of file or (at least on linux, don't known on windows) if we have been interrupted by a signal.

The eof should not happen as we have check validity of the read before.

But I prefer to keep it. If it appear it is really something wrong (unexpected eof), we would have a step more in the loop before throwing a error (and mostly stopping the program).

src/fs_windows.cpp Outdated Show resolved Hide resolved
@mgautierfr mgautierfr force-pushed the libzim_github_ci_windows branch 3 times, most recently from 4d536ce to 0a1d2c3 Compare June 14, 2024 15:58
Base automatically changed from libzim_github_ci_windows to main June 15, 2024 14:39
Copy link
Collaborator

@veloman-yunkan veloman-yunkan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rebase and fix up. I am almost certain that then it will LGTM.

@mgautierfr mgautierfr force-pushed the test_reader branch 2 times, most recently from f1ad2e0 to 8f6d1ca Compare June 17, 2024 12:12
@mgautierfr
Copy link
Collaborator Author

Done

Copy link
Collaborator

@veloman-yunkan veloman-yunkan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me but not to my more picky self :)

Please merge after addressing the two concerns with the commit history

Comment on lines -40 to +50
virtual void read(char* dest, offset_t offset, zsize_t size) const = 0;
void read(char* dest, offset_t offset, zsize_t size) const {
if (can_read(offset, size)) {
if (size) {
// Do the actuall read only if we have a size to read
readImpl(dest, offset, size);
}
return;
}
throw std::runtime_error("Cannot read after the end of the reader");
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if the commit message is updated

Comment on lines +33 to +34
zsize_t size() const override;
offset_t offset() const override;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit better be moved right after the one introducing Reader::readImpl()

When building with the meson option `b_ndebug=true`
(never set on our CI/CD), `ASSERT` macro is removed and so we stop
testing that we are actually read in the reader range.

Now we introduce `readImpl` which do the actual read
(depending of implementation) and make `read` testing read range and call
`readImpl` when everything is ok.
Adding few `override` only on `readImpl` now trigger warnings
`inconsistent-missing-override`.

Better to correctly add `override` when needed.
`FileReader` expects that `readAt` throws an exception if something goes
wrong. So `readAt` must throw an exception instead of returning `-1`.
ReadFile take a 32bits size. So we cannot blindly pass a 64bits size.
Read by batch of 1GiB.
On Windows `long` is 32bits. It's better to use an explicit size.
@mgautierfr
Copy link
Collaborator Author

Looks good to me but not to my more picky self :)

Done. Hope picky @veloman-yunkan is happy with the change.

Please merge after addressing the two concerns with the commit history

I need a explicit approval to merge. Please approve (and merge directly while you have the button under your mouse cursor)

@kelson42 kelson42 self-requested a review June 17, 2024 13:05
Copy link
Contributor

@kelson42 kelson42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ;)

@kelson42 kelson42 merged commit d78acf0 into main Jun 17, 2024
34 checks passed
@kelson42 kelson42 deleted the test_reader branch June 17, 2024 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Content offset is ignored on macOS using LibZim 9.2.0
4 participants