rosbag throws rosbag::IOException at 2GB mark

[scott--]

After installing ros-indigo-desktop-full on my 32bit Ubuntu system, rosbag crashes when the bag file becomes about 2GB. It prints the following:

terminate called after throwing an instance of 'rosbag::BagIOException'
 what(): Error writing to file: writing 38 bytes, wrote 8 bytes

My machine is 32bit Ubuntu 14.04 and I installed indigo from the standard repositories.

To recreate on my system I can do the following:

roscore &
rosbag record -o garbage /garbage &
rostopic pub -r 100000 /garbage std_msgs/String "rubbish"

The same process worked fine in previous versions of ROS.

[dirk-thomas]

I have just tried to reproduce your use case and it does throw an exception on indigo as well as on Hydro. So this is clearly not regression but has been that way before.

You can do one of the following options:

• use the command line option --split to limit the maximum size of each rosbag file
• patch rosbag to be able to handle file sizes > 2 GB on 32bit systems
• use 64bit system (just for the completeness of this list)

Since there are straight forward workarounds and this is more an enhancement request then a regression the maintainers will likely not spend any time on this. Please consider providing a pull request if you would like to improve the behavior on 32bit systems.

[hashmuke]

I am facing the same problem with hydro, things are working fine with fuerte.

[trthanhquang]

I am using 64 bit system and have that issue when using the --split --size option, I cannot set the size to be larger than 4096MB because max_size(bytes) is in unit32.

Would it fix the issue by simply changing uint32 to uint64?

[dirk-thomas]

I don't think that the fix will be that easy but please go ahead and give it a try.

[flixr]

The CMakeLists.txt of rosbag has

# Support large bags (>2GB) on 32-bit systems
add_definitions(-D_FILE_OFFSET_BITS=64)

but it is missing in rosbag_storage.

[flixr]

Seems this got lost when splitting out rosbag_storage with #299
That is also why it was working in fuerte...

[dirk-thomas]

Thank you, @flixr.

@scott-- Can you please try the patch proposed in #554 on your 32bit system and confirm that it makes bagfiles > 2GB work for you?

[scott--]

After doing a git pull, and recompiling, rosbag works as expected.

Thanks.

[dirk-thomas]

Thank you for checking.