Skip to content

Error when reading tfevents files written with TensorBoardLogger v0.1.21 #131

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
BatyLeo opened this issue Jul 18, 2023 · 3 comments · Fixed by #132
Closed

Error when reading tfevents files written with TensorBoardLogger v0.1.21 #131

BatyLeo opened this issue Jul 18, 2023 · 3 comments · Fixed by #132

Comments

@BatyLeo
Copy link

BatyLeo commented Jul 18, 2023

Hello, I updated TensorboardLogger to the latest 0.1.21 version, and the following minimum example doesn't work anymore:

using Logging
using TensorBoardLogger
using ValueHistories

logger = TBLogger("logs"; min_level=Logging.Info)

with_logger(logger) do
    @info "train" loss = 1.0
    @info "train" loss = 0.0
end

r = TBReader("logs1")
hist = MVHistory()

TensorBoardLogger.map_summaries(r) do tag, iter, val
    push!(hist, Symbol(tag), iter, val)
end

Here is the error:

ERROR: AssertionError: crc_data == crc_data_ck
Stacktrace:
 [1] read_event(f::IOStream)
   @ TensorBoardLogger ~/.julia/packages/TensorBoardLogger/LO5eP/src/Deserialization/deserialization.jl:49
 [2] iterate(it::TensorBoardLogger.TBEventFileIterator, state::Int64)
   @ TensorBoardLogger ~/.julia/packages/TensorBoardLogger/LO5eP/src/Deserialization/deserialization.jl:126
 [3] map_summaries(fun::var"#21#22", logdir::TBReader; purge::Bool, tags::Nothing, steps::Nothing, smart::Bool)
   @ TensorBoardLogger ~/.julia/packages/TensorBoardLogger/LO5eP/src/Deserialization/deserialization.jl:270
 [4] map_summaries(fun::Function, logdir::TBReader)
   @ TensorBoardLogger ~/.julia/packages/TensorBoardLogger/LO5eP/src/Deserialization/deserialization.jl:248

Reading files written with an older version still works.
Am I missing something ?

ps: by the way, it seems that version 0.1.20 cannot be installed from the general registry for some reason (only from the github repository url)

@JamieMair
Copy link
Contributor

JamieMair commented Jul 19, 2023

I can confirm that I am having the same error.

I have tracked down the commit which introduced this error (see 45f831e). It is caused by upgrading to the a newer protobuf version. I would guess that the masked_crc32c (this file) is perhaps faulty? I keep getting checksum errors for my own issues.

EDIT: MWE without other deps:

using Logging
using TensorBoardLogger

logger = TBLogger("logs", tb_increment)

with_logger(logger) do
    @info "train" loss = 1.0
    @info "train" loss = 0.0
end

r = TBReader("logs")

println("tag\titer\tval")
TensorBoardLogger.map_summaries(r) do tag, iter, val
    println("$tag\t$iter\t$val")
end

@JamieMair
Copy link
Contributor

@BatyLeo #132 will fix hopefully fix the issue. If you would like to try out the fix before it is merged, you can checkout the branch

pkg> rm TensorBoardLogging
pkg> add https://github.com/JamieMair/TensorBoardLogger.jl#fix-serialisation-bug

Just be sure to switch back to the original package once the PR is merged.

@BatyLeo
Copy link
Author

BatyLeo commented Jul 21, 2023

I just tested the changes, and it seems to work. Thank you for the quick fix !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants