tf.GFile read file with more characters at the end of the last line

[jameslahm]

Environment information

python=3.7

Diagnostics

Diagnostics output

--- check: autoidentify
INFO: diagnose_tensorboard.py version 4725c70c7ed724e2d1b9ba5618d7c30b957ee8a4

--- check: general
INFO: sys.version_info: sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)
INFO: os.name: nt
INFO: os.uname(): N/A
INFO: sys.getwindowsversion(): sys.getwindowsversion(major=10, minor=0, build=18362, platform=2, service_pack='')

--- check: package_management
INFO: has conda-meta: True
INFO: $VIRTUAL_ENV: None

--- check: installed_packages
INFO: installed: tensorboard==2.0.0
WARNING: no installation among: ['tensorflow', 'tensorflow-gpu', 'tf-nightly', 'tf-nightly-2.0-preview', 'tf-nightly-gpu', 'tf-nightly-gpu-2.0-preview']
WARNING: no installation among: ['tensorflow-estimator', 'tensorflow-estimator-2.0-preview', 'tf-estimator-nightly']

--- check: tensorboard_python_version
Traceback (most recent call last):
  File "tensorboard.py", line 420, in main
    suggestions.extend(check())
  File "tensorboard.py", line 77, in wrapper
    result = fn()
  File "tensorboard.py", line 231, in tensorboard_python_version
    from tensorboard import version
ImportError: cannot import name 'version' from 'tensorboard' (C:\Users\13664\Desktop\test\tensorboard.py)

--- check: tensorflow_python_version
Traceback (most recent call last):
  File "tensorboard.py", line 420, in main
    suggestions.extend(check())
  File "tensorboard.py", line 77, in wrapper
    result = fn()
  File "tensorboard.py", line 237, in tensorflow_python_version
    import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'

--- check: tensorboard_binary_path
INFO: which tensorboard: b'F:\\Programs\\anaconda\\envs\\pytorch-nightly\\Scripts\\tensorboard.exe\r\nF:\\Programs\\anaconda\\envs\\pytorch\\Scripts\\tensorboard.exe\r\nF:\\Programs\\anaconda\\envs\\tensorflow\\Scripts\\tensorboard.exe\r\n'

--- check: readable_fqdn
INFO: socket.getfqdn(): 'DESKTOP-SBR4LHH.tsinghua.edu.cn'

--- check: stat_tensorboardinfo
INFO: directory: C:\Users\13664\AppData\Local\Temp\.tensorboard-info
INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=23362423067258806, st_dev=2856197527, st_nlink=1, st_uid=0, st_gid=0, st_size=4096, st_atime=1571050441, st_mtime=1571050441, st_ctime=1570974589)
INFO: mode: 0o40777

--- check: source_trees_without_genfiles
INFO: tensorboard_roots (1): ['F:\\Programs\\anaconda\\envs\\pytorch-nightly\\lib\\site-packages']; bad_roots (0): []

--- check: full_pip_freeze
INFO: pip freeze --all:
absl-py==0.8.1

av==6.2.0

backcall==0.1.0

certifi==2019.6.16

cffi==1.12.3

colorama==0.4.1

cycler==0.10.0

decorator==4.4.0

grpcio==1.24.1

ipykernel==5.1.2

ipython==7.8.0

ipython-genutils==0.2.0

jedi==0.15.1

jupyter-client==5.3.4

jupyter-core==4.6.0

kiwisolver==1.1.0

Markdown==3.1.1

matplotlib==3.1.1

numpy==1.17.1

olefile==0.46

parso==0.5.1

pickleshare==0.7.5

Pillow==6.2.0

pip==19.2.3

prompt-toolkit==2.0.10

protobuf==3.10.0

pycparser==2.19

Pygments==2.4.2

pyparsing==2.4.2

python-dateutil==2.8.0

pywin32==224

pyzmq==18.1.0

setuptools==41.4.0

six==1.12.0

tensorboard==2.0.0

torch==1.3.0.dev20191010

torchvision==0.5.0.dev20191013

tornado==6.0.3

traitlets==4.3.3

wcwidth==0.1.7

Werkzeug==0.16.0

wheel==0.33.6

wincertstore==0.2

Issue description

I try to read tensors.txt using tf.io.GFile,
The file:
tensors.txt
It has 100 lines and each line has 784 characters

here is the code:

from tensorboard.compat import tf

with tf.io.gfile.GFile("tensors.txt", 'r') as f:
    tensor = []
    for i,line in enumerate(f):
        line = line.rstrip('\n')
        if line:
            a=list(map(float, line.split('\t')))
            try:
                assert len(a)==784
            except:
                print(i)
            tensor.append(a)

Output:

99
833

The length of the last line should have been 784, but it has 833 characters and in fact, there are a lot more of zeros at the end of the last line

Reproduce

code:

from tensorboard.compat import tf

with tf.io.gfile.GFile("tensors.txt", 'r') as f:
    tensor = []
    for i,line in enumerate(f):
        line = line.rstrip('\n')
        if line:
            a=list(map(float, line.split('\t')))
            try:
                assert len(a)==784
            except:
                print(i)
            tensor.append(a)

file:
tensors.txt

[wchargin]

Hi @jameslahm! Thanks for the report. I can reproduce the bug in a new
virtualenv with tensorboard installed and tensorflow not installed
(so that we use the stub tf.io.gfile implementation).

Using the following simple test harness—

from tensorboard.compat import tf


def check(f):
  with f:
    for (i, line) in enumerate(f):
      print("len(line_%d) = %d" % (i, len(line)))


print("EXPECTED")
check(open("test.txt"))
print("ACTUAL")
check(tf.io.gfile.GFile("test.txt", "r"))

—we can reduce the problem by taking just a few lines from your file:

$ sed -n '1,2p;100p' tensors.txt >test.txt
$ wc -c test.txt
6400 test.txt
$ shasum -a 256 test.txt
0cd1bc80b9d4c22387bbabe38f72679a0fa8ca2ad1583594deae8a0acacb5c99  test.txt
$ python test.py
EXPECTED
len(line_0) = 2019
len(line_1) = 2204
len(line_2) = 2175
ACTUAL
len(line_0) = 2019
len(line_1) = 2204
len(line_2) = 2177

…and taking just the first three lines causes GFile to generate extra
lines out of nowhere:

$ sed -n '1,3p' tensors.txt >test.txt
$ wc -c test.txt
6166 test.txt
$ shasum -a 256 test.txt
317cda1cbbbbd0407a625d1c264263bc6bc8ccdb0172c81bcc07eb300bc38df8  test.txt
$ python test.py
EXPECTED
len(line_0) = 2019
len(line_1) = 2204
len(line_2) = 1940
ACTUAL
len(line_0) = 2019
len(line_1) = 2204
len(line_2) = 1940
len(line_3) = 2
len(line_4) = 1

This is pretty bad. We’ll definitely look into it. Thank you!

[wchargin]

One should also ask why reading the first three lines of your file
requires opening and closing the file descriptor 4 times and seeking 12
times, 8 of which are “seek to the current offset”…

Details

$ head -n 3 tensors.txt >test.txt
$ wc -c test.txt
6166 test.txt
$ shasum -a 256 test.txt
317cda1cbbbbd0407a625d1c264263bc6bc8ccdb0172c81bcc07eb300bc38df8  test.txt
$ cat hmm.py
import sys
from tensorboard.compat import tf

tf.__version__

sys.stderr.write("Start\n")
list(tf.io.gfile.GFile("test.txt", "r"))
$ strace -f -e trace=%desc python -u hmm.py 2>&1 | sed -n '/Start/,$p'
write(2, "Start\n", 6Start
)                  = 6
openat(AT_FDCWD, "test.txt", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0640, st_size=6166, ...}) = 0
ioctl(3, TCGETS, 0x7ffcce8fe5d0)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_SET)                   = 0
mmap(NULL, 16781312, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd664453000
read(3, "0\t0\t0\t0\t0\t0\t0\t0\t0\t2\t0\t80\t143\t98\t"..., 16777216) = 6166
mmap(NULL, 16781312, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd663452000
read(3, "", 16779216)                   = 0
close(3)                                = 0
openat(AT_FDCWD, "test.txt", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0640, st_size=6166, ...}) = 0
ioctl(3, TCGETS, 0x7ffcce8fe5d0)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 6163, SEEK_SET)                = 6163
mmap(NULL, 16781312, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd664453000
read(3, "0\r\n", 16777216)              = 3
mmap(NULL, 25169920, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd662c52000
read(3, "", 25165821)                   = 0
close(3)                                = 0
openat(AT_FDCWD, "test.txt", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0640, st_size=6166, ...}) = 0
ioctl(3, TCGETS, 0x7ffcce8fe5d0)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 6165, SEEK_SET)                = 6165
mmap(NULL, 16781312, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd664453000
read(3, "\n", 16777216)                 = 1
mmap(NULL, 16781312, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd663452000
read(3, "", 16777215)                   = 0
close(3)                                = 0
openat(AT_FDCWD, "test.txt", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0640, st_size=6166, ...}) = 0
ioctl(3, TCGETS, 0x7ffcce8fe5d0)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 6166, SEEK_SET)                = 6166
mmap(NULL, 16781312, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd664453000
read(3, "", 16777216)                   = 0
close(3)                                = 0
+++ exited with 0 +++

[jameslahm]

Thanks for the reply, expecting!

[davidsoergel]

The provided fix is for just this issue. I agree with @wchargin that there are additional changes we could make to this GFile implementation, but they are less critical (e.g. they're about clarity, performance, and aligning with the TF API, not correctness per se). Some of those will come in a followup PR.