db:migrate:with_data not picking up schema changes

[jasonivers]

I have two migrations, one a schema update to add a column to a table, the other a data migration to set the right data for existing records for that field. The schema migration has an id/timestamp/whatever of 20201111204508, and the data migration has an id of 20201111204520. The schema migration logs out to the screen first (as it should), but the data migration fails because it doesn't see the new column in the table (or attribute on the model). If I then, after all migrations finish, just run rails data:migrate, the same migration runs just fine.

Do I need to call reset_column_information on the model in the data migration before updating the new field, or is it something else?

[saiidalhalawi]

I've faced same issue, and this seems to be caused by below part.

https://github.com/ilyakatz/data-migrate/blob/master/tasks/databases.rake#L33-L38

As far as I can see, schema_migrations versions don't be executed if both schema_migrations and data_migrations are picked.

[pjmartorell]

@saiidalhalawi I don't think this is the issue, if there are pending schema_migrations and data_migrations they are sorted and run by order of timestamp, no issues there. The issue is that, when the model is loaded, its schema is being cached and new created columns are not visible by Rails except you call reset_column_information.

[pjmartorell]

@jasonivers this scenario is described in Rails 4.0 docs and for what I understand is an expected behaviour:

For example, imagine that Alice creates a migration that selectively updates the description field on certain products. She runs the migration, commits the code, and then begins working on the next feature, which is to add a new column fuzz to the products table.
She creates two migrations for this new feature, one which adds the new column, and a second which selectively updates the fuzz column based on other product attributes.
These migrations run just fine, but when Bob comes back from his vacation and calls rake db:migrate to run all the outstanding migrations, he gets a subtle bug: The descriptions have defaults, and the fuzz column is present, but fuzz is nil on all products.
The solution is again to use Product.reset_column_information before referencing the Product model in a migration, ensuring the Active Record’s knowledge of the table structure is current before manipulating data in those records.

[sully7]

Is this related to #201 by chance? Currently both current_schema_version and current_data_version are both set to the value of ActiveRecord::Migrator.current_version

[valenzuelaomar]

splitting the schema and data migration commands fixes the issue. run db:migrate first, then db:migrate:with_data after