Fix #484: Remove dataset from entity

[dhermes]

Also:

• Dropping dataset and kind from Entity constructor.
• Dropping Entity.from_path factory.
• Removing Connection.dataset() quasi-factory.
• helpers.entity_from_protobuf() no longer accepts / needs dataset.
• Making Entity.key a data attribute instead of a getter/setter method.
• Removing Entity.dataset() (since no longer stored).
• Making Entity.kind() and Entity.exclude_from_indexes() into @property's.
• Removing Entity._must_dataset (no longer needed).
• Adding optional connection argument to Entity.save() and Entity.reload().
• Making Entity.save() and Entity.reload() return nothing.
• Updating regression/datastore.py for changes.

Fix #484

NOTE: Has #483 as diffbase.

[coveralls]

Coverage remained the same when pulling 0dbf10c on dhermes:remove-dataset-from-entity into b79d862 on GoogleCloudPlatform:master.

[dhermes]

@tseaver I am re-working this now, having just merged #483.

Can you take a stab at #485?

[dhermes]

@tseaver Rebase completed. PTAL.

[coveralls]

Coverage remained the same when pulling 1dab892 on dhermes:remove-dataset-from-entity into a02a73e on GoogleCloudPlatform:master.

[tseaver]

Looks OK. No impact on regression tests?

[dhermes]

The impact is the code looks nicer every time we throw something away 👍

LGTY?

[tseaver]

Yes -- I just didn't see any changes in the regression side.

[dhermes]

They're there, at the very bottom.

[tseaver]

We have regression failures right now.

[dhermes]

Where? If you're referring to build 808, that was a transitive failure (we should add retries so flakes stop occurring so often).

UPDATE: Don't see any failures on PR builds: https://travis-ci.org/GoogleCloudPlatform/gcloud-python/pull_requests (latest is 819)

[tseaver]

Like so:

======================================================================
FAIL: test_empty_kind (datastore.TestDatastoreSave)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/tseaver/projects/agendaless/Google/src/gcloud-python/regression/datastore.py", line 141, in test_empty_kind
    self.assertEqual(posts, [])
AssertionError: Lists differ: [<Entity[{'kind': u'Post', 'id': 123456789[522 chars]se}>] != []

First list contains 2 additional elements.
First extra element 0:
<Entity[{'kind': u'Post', 'id': 123456789L}] {u'title': u'How to make the perfect pizza in your grill', u'publishedAt': datetime.datetime(2001, 1, 1, 0, 0, tzinfo=<UTC>), u'wordCount': 400L, u'author': u'Silvano', u'rating': 5.0, u'tags': [u'pizza', u'grill'], u'isDraft': False}>

+ []
- [<Entity[{'kind': u'Post', 'id': 123456789L}] {u'title': u'How to make the perfect pizza in your grill', u'publishedAt': datetime.datetime(2001, 1, 1, 0, 0, tzinfo=<UTC>), u'wordCount': 400L, u'author': u'Silvano', u'rating': 5.0, u'tags': [u'pizza', u'grill'], u'isDraft': False}>,
-  <Entity[{'kind': u'Post', 'id': 5216192952270848L}] {u'title': u'How to make the perfect homemade pasta', u'publishedAt': datetime.datetime(2001, 1, 1, 0, 0, tzinfo=<UTC>), u'wordCount': 450L, u'author': u'Silva

[dhermes]

The test is test_empty_kind, so you have some entities from a previously failed test that didn't get deleted.

I made regression/clear_datastore.py for cases like that but filed #453 when I realized I couldn't keep up with it as the API changed massively.

[tseaver]

Ugh:

$ .tox/regression/bin/python regression/clear_datastore.py 
This command will remove all entities for the following kinds:
- Character
- Company
- Kind
- Person
- Post
Is this OK [y/n]? y
Traceback (most recent call last):
File "regression/clear_datastore.py", line 107, in <module>
    remove_all_entities()
File "regression/clear_datastore.py", line 103, in remove_all_entities
    remove_kind(dataset, kind)
File "regression/clear_datastore.py", line 66, in remove_kind
    with dataset.transaction():
AttributeError: 'Dataset' object has no attribute 'transaction'

[dhermes]

Yeah, I will fix the utility after we get stable again. In the meantime:

from gcloud import datastore
from gcloud.datastore.query import Query

datastore.set_default_connection()
datastore.set_default_dataset(dataset_id='foo')

q = Query(kind='Post')

for post in q.fetch():
    post.key.delete()

[tseaver]

That works, but test_empty_kind still fails for me.

[dhermes]

It could be that the tests don't run in a deterministic way? But each case cleans up after itself transactionally so it shouldn't matter.

It should only fail if there are non-deleted keys. Are you using the same dataset ID for tests and in the snippet above? Do you have other failures occurring before that test gets run (hence failed deletes)?

[tseaver]

No other failures; I'm running by doing:

$ .tox/regression/bin/python
>>> import regression.datastore # to set up the connection / dataset
>>> from gcloud.datastore.query import Query
>>> q = Query(kind='Post')
>>> list(q.fetch())
[]

but the regression test still fails.

[dhermes]

Can you print the failure message? (Is it same as above?)

[tseaver]

The 'id' fields in the keys are different; everything else is the same.

[dhermes]

I can't replicate locally or on Travis. Are you running from master (22c8b19) or elsewhere? Is it possible that Transaction is swallowing some failures and some self.case_entities_to_delete.append statements aren't being executed? (Potentially related to #457.)

[tseaver]

Uhoh: list(query.fetch()) returned empty, while list(query.fetch(limit=100)) returned 3 items.

[dhermes]

It's good to have found that though.

[tseaver]

#491

So, now I've deleted them, and my regression tests are passing.

[dhermes]

w00t.

Gonna supply some code in your issue which illustrates this regression.