AttributeError: '_VariantDataset' object has no attribute 'output_shapes' #179

loretoparisi · 2019-07-16T12:22:41Z

Hi guys,
I have successfully trained a G2P seq2seq model for the IPA dictionary in the first release of this repo. I'm now trying to reproduce the same results with an updated IPA dictionary, and using this new codebase, but I'm getting this error:

Traceback (most recent call last):
  File "/usr/local/bin/g2p-seq2seq", line 11, in <module>
    load_entry_point('g2p-seq2seq==6.2.2a0', 'console_scripts', 'g2p-seq2seq')()
  File "/usr/local/lib/python3.7/site-packages/g2p_seq2seq-6.2.2a0-py3.7.egg/g2p_seq2seq/app.py", line 113, in main
    g2p_model.train()
  File "/usr/local/lib/python3.7/site-packages/g2p_seq2seq-6.2.2a0-py3.7.egg/g2p_seq2seq/g2p.py", line 259, in train
    execute_schedule(self.exp, self.params)
  File "/usr/local/lib/python3.7/site-packages/g2p_seq2seq-6.2.2a0-py3.7.egg/g2p_seq2seq/g2p.py", line 589, in execute_schedule
    getattr(exp, params.schedule)()
  File "/usr/local/lib/python3.7/site-packages/tensor2tensor-1.7.0-py3.7.egg/tensor2tensor/utils/trainer_lib.py", line 309, in continuous_train_and_eval
    self._eval_spec)
  File "/usr/local/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/training.py", line 473, in train_and_evaluate
    return executor.run()
  File "/usr/local/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/training.py", line 613, in run
    return self.run_local()
  File "/usr/local/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/training.py", line 714, in run_local
    saving_listeners=saving_listeners)
  File "/usr/local/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 367, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "/usr/local/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1158, in _train_model
    return self._train_model_default(input_fn, hooks, saving_listeners)
  File "/usr/local/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1185, in _train_model_default
    input_fn, ModeKeys.TRAIN))
  File "/usr/local/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1022, in _get_features_and_labels_from_input_fn
    self._call_input_fn(input_fn, mode))
  File "/usr/local/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1113, in _call_input_fn
    return input_fn(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/tensor2tensor-1.7.0-py3.7.egg/tensor2tensor/data_generators/problem.py", line 733, in estimator_input_fn
    dataset_kwargs=dataset_kwargs)
  File "/usr/local/lib/python3.7/site-packages/tensor2tensor-1.7.0-py3.7.egg/tensor2tensor/data_generators/problem.py", line 895, in input_fn
    batching_scheme["batch_sizes"])
  File "/usr/local/lib/python3.7/site-packages/tensor2tensor-1.7.0-py3.7.egg/tensor2tensor/utils/data_reader.py", line 103, in bucket_by_sequence_length
    window_size_fn))
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 1853, in apply
    return DatasetV1Adapter(super(DatasetV1, self).apply(transformation_func))
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 1290, in apply
    dataset = transformation_func(self)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/data/experimental/ops/grouping.py", line 121, in _apply_fn
    window_size_func)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/data/experimental/ops/grouping.py", line 377, in __init__
    self._make_reduce_func(reduce_func, input_dataset)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/data/experimental/ops/grouping.py", line 424, in _make_reduce_func
    input_structure=input_structure)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 2555, in __init__
    self._function = wrapper_fn._get_concrete_function_internal()
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1355, in _get_concrete_function_internal
    *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1349, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1652, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1545, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 715, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 2549, in wrapper_fn
    ret = _wrapper_helper(*args)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 2489, in _wrapper_helper
    ret = func(*nested_args)
  File "/usr/local/lib/python3.7/site-packages/tensor2tensor-1.7.0-py3.7.egg/tensor2tensor/utils/data_reader.py", line 99, in batching_fn
    return padded_batch(grouped_dataset, batch_size, padded_shapes)
  File "/usr/local/lib/python3.7/site-packages/tensor2tensor-1.7.0-py3.7.egg/tensor2tensor/utils/data_reader.py", line 110, in padded_batch
    for name, shape in dataset.output_shapes.items()])
AttributeError: '_VariantDataset' object has no attribute 'output_shapes'

The IPA dictionary I'm using was the same I have used in the old training (that worked but was a year ago) and it is available here
https://github.com/loretoparisi/docker/tree/master/g2p-seq2seq/data/dict/cmudict-ipa

homan			howˈmʌn
homann			howˈmʌn
homans			howˈmʌnz
homart			howˈmɑˌɹt
hombre			hɑˈmbɹʌ

I have used default options: g2p-seq2seq --train data/cmuipadict.txt --model_dir models/ and I'm running:

ip-192-168-22-157:g2p-seq2seq loretoparisi$ python3 -c "import tensorflow as tf; print(tf.__version__)"
1.14.0

The text was updated successfully, but these errors were encountered:

nshmyrev · 2019-07-16T12:32:06Z

This is due to tesnor2tensor version incompatibility.

loretoparisi · 2019-07-16T12:34:48Z

@nshmyrev okay, but what I have to do to fix it? I have installed using pip3 install setupy!

nshmyrev · 2019-07-16T12:36:12Z

Readme says tensorflow 1.8 and tensor2tensor 1.6, then it will work.

loretoparisi · 2019-07-16T12:38:29Z

@nshmyrev so I have to downgrade to TF 1.8? According to the README it's at least 1.8 (so I'm getting confused here). It would be worth to specify in setup.py a specific version of TF, because by defaults it's 1.14.0 currently on pip3:

  Could not find a version that satisfies the requirement tensorflow==1.8.0 (from -r requirements.txt (line 1)) (from versions: 1.13.0rc1, 1.13.0rc2, 1.13.1, 1.13.2, 1.14.0rc0, 1.14.0rc1, 1.14.0, 2.0.0a0, 2.0.0b0, 2.0.0b1)
No matching distribution found for tensorflow==1.8.0 (from -r requirements.txt (line 1))

loretoparisi · 2019-07-16T12:51:58Z

@nshmyrev I have tried

setuptools>=41.0.0
tensorflow==2.0.0b1
tensor2tensor==1.6.6

but it does not work.

ksoky · 2019-11-26T13:15:11Z

@nshmyrev I have tried
setuptools>=41.0.0
tensorflow==2.0.0b1
tensor2tensor==1.6.6
but it does not work.

Dear Everyone,
Do you have this solution yet?
I have tried many versions of tensor2tensor and TensorFlow, but it still has the same problem.
and for tensorflow==2.0, it has a problem without module 'tensorflow.contrib'.

nshmyrev · 2019-11-26T13:19:20Z

Guys, give me a week, I'll update it to much more stable pytorch.

joazoa · 2019-12-24T11:10:17Z

Do you have any updates on this ?

raikarsagar · 2020-07-15T06:40:26Z

Any updates on pytorch migration?

nshmyrev · 2020-07-15T06:51:45Z

Try https://github.com/hajix/G2P

nshmyrev closed this as completed Jul 16, 2019

loretoparisi mentioned this issue Jul 16, 2019

New release to support Python 3.7.0+ #177

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AttributeError: '_VariantDataset' object has no attribute 'output_shapes' #179

AttributeError: '_VariantDataset' object has no attribute 'output_shapes' #179

loretoparisi commented Jul 16, 2019 •

edited

Loading

nshmyrev commented Jul 16, 2019

loretoparisi commented Jul 16, 2019

nshmyrev commented Jul 16, 2019

loretoparisi commented Jul 16, 2019 •

edited

Loading

loretoparisi commented Jul 16, 2019

ksoky commented Nov 26, 2019 •

edited

Loading

nshmyrev commented Nov 26, 2019

joazoa commented Dec 24, 2019 •

edited

Loading

raikarsagar commented Jul 15, 2020

nshmyrev commented Jul 15, 2020

AttributeError: '_VariantDataset' object has no attribute 'output_shapes' #179

AttributeError: '_VariantDataset' object has no attribute 'output_shapes' #179

Comments

loretoparisi commented Jul 16, 2019 • edited Loading

nshmyrev commented Jul 16, 2019

loretoparisi commented Jul 16, 2019

nshmyrev commented Jul 16, 2019

loretoparisi commented Jul 16, 2019 • edited Loading

loretoparisi commented Jul 16, 2019

ksoky commented Nov 26, 2019 • edited Loading

nshmyrev commented Nov 26, 2019

joazoa commented Dec 24, 2019 • edited Loading

raikarsagar commented Jul 15, 2020

nshmyrev commented Jul 15, 2020

loretoparisi commented Jul 16, 2019 •

edited

Loading

loretoparisi commented Jul 16, 2019 •

edited

Loading

ksoky commented Nov 26, 2019 •

edited

Loading

joazoa commented Dec 24, 2019 •

edited

Loading