TorchText implementation

[swapnull7]

Logs when running the torchtext example

downloading ptb.train.txt
I1209 11:56:18.214439 140266512029440 utils.py:64] Downloading file ptb.train.txt to .data/penn-treebank/ptb.train.txt.
ptb.train.txt: 5.10MB [00:00, 53.9MB/s]
I1209 11:56:18.340973 140266512029440 utils.py:72] File .data/penn-treebank/ptb.train.txt downloaded.
downloading ptb.valid.txt
I1209 11:56:18.418945 140266512029440 utils.py:64] Downloading file ptb.valid.txt to .data/penn-treebank/ptb.valid.txt.
ptb.valid.txt: 400kB [00:00, 11.5MB/s]
I1209 11:56:18.454811 140266512029440 utils.py:72] File .data/penn-treebank/ptb.valid.txt downloaded.
downloading ptb.test.txt
I1209 11:56:18.528911 140266512029440 utils.py:64] Downloading file ptb.test.txt to .data/penn-treebank/ptb.test.txt.
ptb.test.txt: 450kB [00:00, 12.7MB/s]
I1209 11:56:18.565306 140266512029440 utils.py:72] File .data/penn-treebank/ptb.test.txt downloaded.
2019-12-09 11:56:20.154461: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2019-12-09 11:56:21.090438: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.7335
pciBusID: 0000:02:00.0
2019-12-09 11:56:21.091663: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 1 with properties:
name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.7335
pciBusID: 0000:03:00.0
2019-12-09 11:56:21.131080: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-09 11:56:21.273498: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-12-09 11:56:21.368069: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2019-12-09 11:56:21.408129: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2019-12-09 11:56:21.573502: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2019-12-09 11:56:21.659324: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2019-12-09 11:56:22.034719: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2019-12-09 11:56:22.039558: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0, 1
2019-12-09 11:56:22.049416: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-12-09 11:56:22.183587: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3298065000 Hz
2019-12-09 11:56:22.185531: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55e4717fd5b0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2019-12-09 11:56:22.185575: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2019-12-09 11:56:22.433129: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55e4718909f0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2019-12-09 11:56:22.433170: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1080, Compute Capability 6.1
2019-12-09 11:56:22.433181: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): GeForce GTX 1080, Compute Capability 6.1
2019-12-09 11:56:22.436357: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.7335
pciBusID: 0000:02:00.0
2019-12-09 11:56:22.437230: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 1 with properties:
name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.7335
pciBusID: 0000:03:00.0
2019-12-09 11:56:22.437291: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-09 11:56:22.437317: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-12-09 11:56:22.437338: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2019-12-09 11:56:22.437358: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2019-12-09 11:56:22.437379: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2019-12-09 11:56:22.437398: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2019-12-09 11:56:22.437418: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2019-12-09 11:56:22.440758: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0, 1
2019-12-09 11:56:22.459071: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-09 11:56:22.495038: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-12-09 11:56:22.495066: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0 1
2019-12-09 11:56:22.495077: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N Y
2019-12-09 11:56:22.495086: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 1:   Y N
2019-12-09 11:56:22.523980: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7604 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:02:00.0, compute capability: 6.1)
2019-12-09 11:56:22.525612: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 7606 MB memory) -> physical GPU (device: 1, name: GeForce GTX 1080, pci bus id: 0000:03:00.0, compute capability: 6.1)
2019-12-09 11:56:30.951525: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
Epoch: 0 Train Perplexity: nan
Epoch: 0 Valid Perplexity: nan
Epoch: 1 Train Perplexity: nan
Epoch: 1 Valid Perplexity: nan
Epoch: 2 Train Perplexity: nan
Epoch: 2 Valid Perplexity: nan
Epoch: 3 Train Perplexity: nan
Epoch: 3 Valid Perplexity: nan