Skip to content

Conversation

@wchargin
Copy link
Contributor

Summary:
Travis builds are failing on today’s tf-nightly and not on the other
environments (1.15.0rc3 and notf). The error is in the smoke test:

+python -c
import tensorboard as tb
tb.summary.v1.scalar_pb('test', 42)
tb.summary.scalar('test v2', 1337)
from tensorboard.plugins.beholder import Beholder, BeholderHook
from tensorboard.plugins.mesh import summary
2019-10-24 19:50:45.339101: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2019-10-24 19:50:45.339143: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
*** Error in `python': free(): invalid pointer: 0x00000000013e6340 ***
Aborted (core dumped)
+cleanup
+[ -n set ]
+rm -rf /tmp/tmp.E6thqwE4O7

I can’t reproduce this locally on a virtualenv with these packages:

pip: tb-nightly==2.1.0a20191024
pip: tf-nightly==2.1.0.dev20191024
python: tb.__version__ == 2.1.0a20191024
python: tf.__git_version__ == v1.12.1-16634-g0e90b26
python: tf.__version__ == 2.1.0-dev20191024

Downgrading nightly in the hope that this will unblock PRs, and will try
to understand the underlying issue later.

Test Plan:
See what Travis thinks.

wchargin-branch: pin-20191023-nightly

Summary:
Travis builds are failing on today’s `tf-nightly` and not on the other
environments (1.15.0rc3 and notf). The error is in the smoke test:

```
+python -c
import tensorboard as tb
tb.summary.v1.scalar_pb('test', 42)
tb.summary.scalar('test v2', 1337)
from tensorboard.plugins.beholder import Beholder, BeholderHook
from tensorboard.plugins.mesh import summary
2019-10-24 19:50:45.339101: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2019-10-24 19:50:45.339143: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
*** Error in `python': free(): invalid pointer: 0x00000000013e6340 ***
Aborted (core dumped)
+cleanup
+[ -n set ]
+rm -rf /tmp/tmp.E6thqwE4O7
```

I can’t reproduce this locally on a virtualenv with these packages:

```
pip: tb-nightly==2.1.0a20191024
pip: tf-nightly==2.1.0.dev20191024
python: tb.__version__ == 2.1.0a20191024
python: tf.__git_version__ == v1.12.1-16634-g0e90b26
python: tf.__version__ == 2.1.0-dev20191024
```

Downgrading nightly in the hope that this will unblock PRs, and will try
to understand the underlying issue later.

Test Plan:
See what Travis thinks.

wchargin-branch: pin-20191023-nightly
@wchargin wchargin merged commit 9b188c1 into master Oct 24, 2019
@wchargin wchargin deleted the wchargin-pin-20191023-nightly branch October 24, 2019 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants