ContentFields migration error (DataMigrationManager)

[nuno-tiago-reis]

Hello, I've recently upgraded to OrchardCore 1.0.0-rc1-10607 (from beta 1.0.0-beta3-72452) and when initializing the following error occurs:

fail: OrchardCore.Data.Migration.DataMigrationManager[0]
      Could not run migrations automatically on 'OrchardCore.ContentFields'
System.InvalidOperationException: Nullable object must have a value.
   at OrchardCore.Data.Migration.DataMigrationManager.UpdateAsync(String featureId)
   at OrchardCore.Data.Migration.DataMigrationManager.UpdateAllFeaturesAsync() in C:\projects\orchardcore\src\OrchardCore\OrchardCore.Data\Migration\DataMigrationManager.cs:line 364

This doesn't seem to break anything but I thought I should report it anyway.

[deanmarcussen]

I will try and repro this.

It maybe related to the ContentFields never previously having a migration

[deanmarcussen]

@nuno-tiago-reis I can partly repro this, but only by editing the database to cause a fault with the database migration record.

Theres' a couple of related issues around this #3421 and #4743 one of which is probably caused by mysql, the other not so sure.

Can I ask a couple of questions?

• Are you running mysql or postgresql?
• Did you have any other errors?
• Is it possible the first startup during your migration failed partway through?
• Do you have your old database still from before the backup, and be willing to email it?
• Could you share the database migration record from your current setup? (it's normally the top record in the document database with type OrchardCore.Data.Migration.Records.DataMigrationRecord, OrchardCore.Data

for info, the error can be caused at line 173, in the DataMigrationManager as Version could be null. but I would like to find out why it is null, before fixing, as it maybe a bad fix.

if (dataMigrationRecord != null)
{
    current = dataMigrationRecord.Version.Value;
}

[nuno-tiago-reis]

Sorry, I've been away for a few days and only saw your questions today.

I'm running MySql, the only other error is listed below, regarding the Alias.
I tried creating a fresh installation and the error occurred as well.
I've attached the data export from the Orchard schema.

[2019-12-09 11:41:40 EROR] OrchardCore.Data.Migration.DataMigrationManager
Error while running migration version 0 for '"OrchardCore.Alias"'.
{"ActionId": "faa2d655-1ffd-451c-bde3-405ec5f4987c", "ActionName": "OrchardCore.Setup.Controllers.SetupController.IndexPOST (OrchardCore.Setup)", "RequestId": "0HLRSJHRUR2TH:00000021", "RequestPath": "/", "SpanId": "|4ea2bf81-429e981792f9b34d.", "TraceId": "4ea2bf81-429e981792f9b34d", "ParentId": ""}
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> MySql.Data.MySqlClient.MySqlException (0x80004005): Specified key was too long; max key length is 3072 bytes
 ---> MySql.Data.MySqlClient.MySqlException (0x80004005): Specified key was too long; max key length is 3072 bytes
   at MySqlConnector.Core.ResultSet.ReadResultSetHeaderAsync(IOBehavior ioBehavior) in C:\projects\mysqlconnector\src\MySqlConnector\Core\ResultSet.cs:line 49
   at MySql.Data.MySqlClient.MySqlDataReader.ActivateResultSet() in C:\projects\mysqlconnector\src\MySqlConnector\MySql.Data.MySqlClient\MySqlDataReader.cs:line 130
   at MySql.Data.MySqlClient.MySqlDataReader.CreateAsync(CommandListPosition commandListPosition, ICommandPayloadCreator payloadCreator, IDictionary`2 cachedProcedures, IMySqlCommand command, CommandBehavior behavior, IOBehavior ioBehavior, CancellationToken cancellationToken) in C:\projects\mysqlconnector\src\MySqlConnector\MySql.Data.MySqlClient\MySqlDataReader.cs:line 391
   at MySqlConnector.Core.CommandExecutor.ExecuteReaderAsync(IReadOnlyList`1 commands, ICommandPayloadCreator payloadCreator, CommandBehavior behavior, IOBehavior ioBehavior, CancellationToken cancellationToken) in C:\projects\mysqlconnector\src\MySqlConnector\Core\CommandExecutor.cs:line 62
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteNonQueryAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in C:\projects\mysqlconnector\src\MySqlConnector\MySql.Data.MySqlClient\MySqlCommand.cs:line 226
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteNonQuery() in C:\projects\mysqlconnector\src\MySqlConnector\MySql.Data.MySqlClient\MySqlCommand.cs:line 74
   at Dapper.SqlMapper.ExecuteCommand(IDbConnection cnn, CommandDefinition& command, Action`2 paramReader) in C:\projects\dapper\Dapper\SqlMapper.cs:line 2797
   at Dapper.SqlMapper.ExecuteImpl(IDbConnection cnn, CommandDefinition& command) in C:\projects\dapper\Dapper\SqlMapper.cs:line 568
   at YesSql.Sql.SchemaBuilder.Execute(IEnumerable`1 statements)
   at YesSql.Sql.SchemaBuilder.AlterTable(String name, Action`1 table)
   at OrchardCore.Alias.Migrations.Create() in C:\projects\orchardcore\src\OrchardCore.Modules\OrchardCore.Alias\Migrations.cs:line 36
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at OrchardCore.Data.Migration.DataMigrationManager.UpdateAsync(String featureId) in C:\projects\orchardcore\src\OrchardCore\OrchardCore.Data\Migration\DataMigrationManager.cs:line 187

[2019-12-09 11:41:42 EROR] OrchardCore.Data.Migration.DataMigrationManager
Could not run migrations automatically on '"OrchardCore.Alias"'
{"ActionId": "faa2d655-1ffd-451c-bde3-405ec5f4987c", "ActionName": "OrchardCore.Setup.Controllers.SetupController.IndexPOST (OrchardCore.Setup)", "RequestId": "0HLRSJHRUR2TH:00000021", "RequestPath": "/", "SpanId": "|4ea2bf81-429e981792f9b34d.", "TraceId": "4ea2bf81-429e981792f9b34d", "ParentId": ""}
System.InvalidOperationException: Nullable object must have a value.
   at OrchardCore.Data.Migration.DataMigrationManager.UpdateAsync(String featureId)
   at OrchardCore.Data.Migration.DataMigrationManager.UpdateAllFeaturesAsync() in C:\projects\orchardcore\src\OrchardCore\OrchardCore.Data\Migration\DataMigrationManager.cs:line 358

Orchard.zip

Let me know if you need anything else.

[deanmarcussen]

ok, so to me it looks like the failure of one migration is causing the failure somewhere else.

and the problem is MySql being limited to the length of characters that can go into an index.

This covers probably the issue https://stackoverflow.com/questions/8746207/1071-specified-key-was-too-long-max-key-length-is-1000-bytes

that MySql won't take an index length of the 1000 characters that the alias part now has (with utf8 encoding)

I have a fix, looks like the easiest way is to not index those columns on MySql

[sebastienros]

Could we instead detect that the index will be too long, and internally (yessql) use a constant hash (one that doesn't change from machine to machine) as the actual name? Or at least keep the first N letters and complete with a hash?

The YesSql dialects would have the max length for these index names, and we could compute the length of the prefix based on that and the length of the hash (of the full string).

[deanmarcussen]

Yes, we should do that for the index/prefix name length issue.

This issue is different. Caused by the length of the string being indexed. Eg alias is 1000 characters. The index for MySQL won’t support 1000 characters so the dialect to create the index needs to be create index alias (250) (or whatever it is keeps it under 3096 bytes in utf8 collation for all columns in the index.

We can do it in YesSql but need to add some kind of calculation to the create index dialect to figure out total bytes n split it down to 3096.

Apparently with MySQL indexes should only include the start of the columns data to stay within those limits