Problem with Django 1.7 migrations?

[jannon]

I just ran my first migration on a model that has an hstore field in it. I had changed the default of a datetime field within the hstore field (because of encode/django-rest-framework#2687) and the migration blew up with:

ValueError: Cannot alter field batches.Collection.end_date into app.MyModel.end_date - they do not properly define db_type (are you using PostGIS 1.5 or badly-written custom fields?)

So I took a look at what the migration was doing:

migrations.AlterField(
            model_name='mymodel',
            name='end_date',
            field=django_hstore.virtual.VirtualField(default=datetime.date(2015, 1, 1)),
        )

It seems like the django_hstore.virtual.VirtualField fields shouldn't be included in the migration. I wanted to check and see if others had this issue or if I was just missing something before I looked in to how we might avoid this (other than manually editing the migration every time)

[nemesifier]

Oh no again.
I tried reporting this problem last year:
https://code.djangoproject.com/ticket/23159
I also sent a pull request: https://github.com/django/django/pull/3013/files
But I was adviced to just set db_type as None in the virtual fields by @andrewgodwin and the issue was set as won't fix.
With some more tricks I was able to make the basic cases just work fine, but the solutions I found might not work in all cases, like yours.

So, I don't really know how to proceed. Apart from just deleting part of the migration manually, which obviously sucks but will work if in a hurry, we could do two things:

• insist to reopen the issue and find a way to tell the django migration framework to ignore virtual fields
• try to find another workaround on this particular issue

Federico

[jannon]

Bah! Okay, yeah I doubt that we'd get them to reopen the issue, especially given Django1.8's new HStoreField. Until I get a chance to see how 1.8 works, or until the DRF bug gets fixed, I think I'll just manually edit the migrations

[nemesifier]

Ok @jannon.

[timgraham]

Can you paste the model field before and after?

[nemesifier]

@jannon could you answer to @timgraham's question please?

[jannon]

Before:

class MyModel(models.Model):
    hstore_field = hstore.DictionaryField(schema=[
        {
            'name': 'end_date',
            'class': 'DateField',
            'kwargs': {'null': True, 'blank': True}
        }
    ], null=True, blank=True)

After:

class MyModel(models.Model):
    hstore_field = hstore.DictionaryField(schema=[
        {
            'name': 'end_date',
            'class': 'DateField',
            'kwargs': {'null': True, 'blank': True, 'default': date(2015, 1, 1)}
        }
    ], null=True, blank=True)

[timgraham]

I think the migration that's generated is correct, but Django doesn't seem to have good support for ignoring virtual fields in the schema editor. You might be able to workaround it by adding a rel attribute to django_hstore.virtual.VirtualField with the appropriate sub-attributes so that you reach the no-op branch in the schema editor.

This might be properly fixed if Django gains support for composite fields in 1.9, but it seems like a ticket is justified (a separate one from what was originally reported).

[nemesifier]

thx for the info @timgraham, is there any link where we can find more information? A mailing list thread, a ticket, or anything else?

[timgraham]

More information about what? Composite field DEP is here if that's what you meant: django/deps#12

[nemesifier]

Exactly, thank you.

[nemesifier]

@timgraham & @jannon I pushed a possible fix for the ValueError in e3ff9bf, the problem is I don't have a clue of how to make a failing test case ... 👎 any hints?

[nemesifier]

After some manual testing it turned out the code of e3ff9bf was not effective.

Thanks to @apollo13's help I come up with 618f1ff which correctly eliminates the ValueError, the problem is I still can't figure out how to test this automatically.

[timgraham]

You might take inspiration from Django's test suite: https://github.com/django/django/blob/master/tests/schema/tests.py

[nemesifier]

It should be fixed.

Thanks to everyone who helped out: @jannon for reporting, @timgraham and @apollo13 for helping me with suggestions.

I opened a new thread on django-developers here: https://groups.google.com/forum/#!topic/django-developers/_FmJRK3sJGs

I received another suggestion by @MarkusH which I'll try out before closing this issue.

[jannon]

Federico,
This works great!
It'll be nice if the the virtual=True suggestion will even help clean things up a bit

[nemesifier]

I mark this closed for the moment, but I'll keep following the mailing list for suggestions.