Load Issue on Seq 2024.3.12250

[Jaben]

Describe the bug
On Startup:

[12:57:29 INF] Seq 2024.3.12250 running on OS Ubuntu 22.04.4 LTS
[12:57:29 INF] Opening event store at /data/Stream/stream.flare
[12:57:29 INF] Ingestion enabled
[12:57:29 INF] Opening metastore /data/Documents/metastore.flare
[12:57:29 INF] Applying migration Disable legacy event schema compatibility if possible
[12:57:29 INF] Closing metastore
[12:57:29 INF] Closing event store
Application startup exception: Autofac.Core.DependencyResolutionException: An exception was thrown while activating λ:Seq.Cluster.IClusterNode -> λ:Seq.Server.StorageSubsystem.
---> System.ArgumentNullException: Value cannot be null. (Parameter 's')
at System.DateTime.Parse(String s)
at Seq.Data.Storage.Migrations.DisableLegacyEventSchemaCompatibilityIfPossible.Apply(DocumentSession session)
at Quince.Storage.Native.NativeDocumentStore.Migrate(MigrationList migrations)
at Seq.Server.StorageSubsystem.Initialize(IComponentContext ctx, SeqConfig config, StorageLayout layout, PerformanceCounters sharedCounters)
at Seq.Server.SeqServerModule.b__9_10(IComponentContext c)
at Autofac.RegistrationExtensions.<>c__DisplayClass41_01.<Register>b__0(IComponentContext c, IEnumerable1 p)
at Autofac.Builder.RegistrationBuilder.<>c__DisplayClass0_01.<ForDelegate>b__0(IComponentContext c, IEnumerable1 p)
at Autofac.Core.Activators.Delegate.DelegateActivator.ActivateInstance(IComponentContext context, IEnumerable1 parameters) at Autofac.Core.Activators.Delegate.DelegateActivator.<ConfigurePipeline>b__2_0(ResolveRequestContext context, Action1 next)
at Autofac.Core.Resolving.Middleware.DelegateMiddleware.Execute(ResolveRequestContext context, Action1 next) at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.<BuildPipeline>b__1(ResolveRequestContext context) at Autofac.Core.Resolving.Middleware.DisposalTrackingMiddleware.Execute(ResolveRequestContext context, Action1 next)
at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.b__1(ResolveRequestContext context)
at Autofac.Core.Resolving.Middleware.ActivatorErrorHandlingMiddleware.Execute(ResolveRequestContext context, Action1 next) --- End of inner exception stack trace --- at Autofac.Core.Resolving.Middleware.ActivatorErrorHandlingMiddleware.Execute(ResolveRequestContext context, Action1 next)
at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.b__1(ResolveRequestContext context)
at Autofac.Core.Pipeline.ResolvePipeline.Invoke(ResolveRequestContext context)
at Autofac.Core.Resolving.Middleware.RegistrationPipelineInvokeMiddleware.Execute(ResolveRequestContext context, Action1 next) at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.<BuildPipeline>b__1(ResolveRequestContext context) at Autofac.Core.Resolving.Middleware.SharingMiddleware.Execute(ResolveRequestContext context, Action1 next)
at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.b__1(ResolveRequestContext context)
at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.b__1(ResolveRequestContext context)
at Autofac.Core.Resolving.Middleware.CircularDependencyDetectorMiddleware.Execute(ResolveRequestContext context, Action1 next) at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.<BuildPipeline>b__1(ResolveRequestContext context) at Autofac.Extensions.DependencyInjection.KeyedServiceMiddleware.Execute(ResolveRequestContext context, Action1 next)
at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.b__1(ResolveRequestContext context)
at Autofac.Core.Pipeline.ResolvePipeline.Invoke(ResolveRequestContext context)
at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, ResolveRequest& request)
at Autofac.Core.Resolving.ResolveOperation.ExecuteOperation(ResolveRequest& request)
at Autofac.Core.Resolving.ResolveOperation.Execute(ResolveRequest& request)
at Autofac.Core.Lifetime.LifetimeScope.ResolveComponent(ResolveRequest& request)
at Autofac.Core.Container.ResolveComponent(ResolveRequest& request)
at Autofac.Core.Container.Autofac.IComponentContext.ResolveComponent(ResolveRequest& request)
at Autofac.ResolutionExtensions.TryResolveService(IComponentContext context, Service service, IEnumerable1 parameters, Object& instance) at Autofac.ResolutionExtensions.ResolveOptionalService(IComponentContext context, Service service, IEnumerable1 parameters)
at Autofac.ResolutionExtensions.ResolveOptional(IComponentContext context, Type serviceType, IEnumerable`1 parameters)
at Autofac.ResolutionExtensions.ResolveOptional(IComponentContext context, Type serviceType)
at Autofac.Extensions.DependencyInjection.AutofacServiceProvider.GetService(Type serviceType)
at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider)
at Microsoft.Extensions.Internal.ActivatorUtilities.CreateInstance(IServiceProvider provider, Type instanceType, Object[] parameters)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.ReflectionMiddlewareBinder.CreateMiddleware(RequestDelegate next)
at Microsoft.AspNetCore.Builder.ApplicationBuilder.Build()
at Microsoft.AspNetCore.Hosting.WebHost.BuildApplication()

To Reproduce
Startup Seq 2024.3.12250

Expected behavior
Not fail on startup.

Screenshots

Environment (please complete the following information):
Docker running in OS Ubuntu 22.04.4 LTS -- Host is Ubuntu 20.

[Jaben]

My solution for now is to keep running 2024.2 -- which still works.

[nblumhardt]

Thanks for the report, @Jaben. We've pulled together a fix that should address this, although we're not sure of the sequence of events — backups, restores, upgrades, etc. — that put the configuration in the problematic state. The issue could have been in any version since late 5.1.x so it might have to remain a mystery (if this is a new instance please let us know, though).

I'm not sure of an ETA yet but expect it'll be within the next few weeks.

[Jaben]

I can zip the metadata and email it to you. What files should I include?