-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tag table with backwards compatibility #68
Merged
Aaronontheweb
merged 27 commits into
akkadotnet:dev
from
to11mtm:tag-table-with-backwards-compat
Feb 3, 2023
Merged
Changes from 9 commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
660f637
index creation app example
to11mtm fec6cf5
WIP Tag Table with backwards compatibility
to11mtm 8b65724
fix incomplete rename refactor, re-add case to guarantee manifest is …
to11mtm adc8ce4
fix: code for journal queries was not always respecting isdeleted
to11mtm 6936c1e
Added WIP Tag-join-via UUID Support for users who may want faster wri…
to11mtm 627d159
Try refactoring to make older compiler happy about expressions.
to11mtm bc52579
Try Unwinding ternarys to make compiler happy.
to11mtm 3778099
WIP Allow proper handling of EventManifest
to11mtm 4c2dc03
Merge branch 'dev' into tag-table-with-backwards-compat
Aaronontheweb 34570b9
Rename Class1.cs to JournalIndexHelper.cs
Aaronontheweb d41e3c1
Merge branch 'dev' into tag-table-with-backwards-compat
Arkatufus f5e59b3
Move dangling projects into src folder
Arkatufus 56777fa
Cleanup property names
Arkatufus f67450d
Merge branch 'dev' into tag-table-with-backwards-compat
Arkatufus d4e2984
Fix merge issues
Arkatufus 060da94
Fix table column name compatibility problem
Arkatufus 5a4595b
Fix logical delete compatibility problem
Arkatufus 67bd601
Merge branch 'dev' into tag-table-with-backwards-compat
Arkatufus 9e32549
Merge branch 'dev' into tag-table-with-backwards-compat
Arkatufus 095b0dd
post-merge cleanup
Arkatufus 969116b
Convert unit tests to use TestKit
Arkatufus bfe4f86
Fix unit tests
Arkatufus 3455e3c
Merge branch 'dev' into tag-table-with-backwards-compat
Arkatufus 1db2523
Merge branch 'dev' into tag-table-with-backwards-compat
Arkatufus d7a283d
Add SqlServer SQL script migration spec
Arkatufus b2a3329
Add PostgreSql and MySql SQL script spec
Arkatufus c59c980
Remove bit flag
Arkatufus File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFramework>netcoreapp3.1</TargetFramework> | ||
| <ImplicitUsings>enable</ImplicitUsings> | ||
| <Nullable>enable</Nullable> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="Akka.Streams" Version="1.4.21" /> | ||
| <PackageReference Include="linq2db" Version="3.4.1" /> | ||
| <PackageReference Include="Reactive.Streams" Version="1.0.2" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
25 changes: 25 additions & 0 deletions
25
Akka.Persistence.Linq2Db.IndexHelperApp/Akka.Persistence.Linq2Db.IndexHelperApp.csproj
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <OutputType>Exe</OutputType> | ||
| <TargetFramework>netcoreapp3.1</TargetFramework> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\Akka.Persistence.Linq2Db.IndexHelperLib\Akka.Persistence.Linq2Db.IndexHelperLib.csproj" /> | ||
| <ProjectReference Include="..\src\Akka.Persistence.Sql.Linq2Db.Tests\Akka.Persistence.Sql.Linq2Db.Tests.csproj" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="CommandLineParser" Version="2.8.0" /> | ||
| <PackageReference Include="FluentMigrator.Runner" Version="3.3.1" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <None Remove="example.hocon" /> | ||
| <Content Include="example.hocon"> | ||
| <CopyToOutputDirectory>Always</CopyToOutputDirectory> | ||
| </Content> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| using CommandLine; | ||
|
|
||
| namespace Akka.Persistence.Linq2Db.IndexHelperApp | ||
| { | ||
| public class Options | ||
| { | ||
| [Option('f',"file", Required=true, HelpText = "Specify the HOCON file to use")] | ||
| public string File { get; set; } | ||
|
|
||
| [Option('p',"path", Required = true, HelpText = "The Path to the Akka.Persistence.Linq2Db Config in the HOCON.")] | ||
| public string HoconPath { get; set; } | ||
|
|
||
| [Option("OrderingIdx", Required = true, Group = "IndexType", HelpText = "Generates the SQL Text for an Ordering index")] | ||
| public bool GenerateOrdering { get; set; } | ||
|
|
||
| [Option("PidSeqNoIdx", Required = true, Group = "IndexType", HelpText = "Generates the SQL Text for an index on PersistenceID and SeqNo")] | ||
| public bool GeneratePidSeqNo { get; set; } | ||
|
|
||
| [Option("TimeStampIdx", Required = true, Group = "IndexType", HelpText = "Generates the SQL Text for a Timestamp Index")] | ||
| public bool GenerateTimestamp { get; set; } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| using System; | ||
| using System.IO; | ||
| using Akka.Configuration; | ||
| using Akka.Persistence.Linq2Db.IndexHelperLib; | ||
| using Akka.Persistence.Sql.Linq2Db.Config; | ||
| using Akka.Persistence.Sql.Linq2Db.Tests; | ||
| using CommandLine; | ||
| using FluentMigrator.Expressions; | ||
| using FluentMigrator.Runner.Generators; | ||
| using FluentMigrator.Runner.Generators.Generic; | ||
| using FluentMigrator.Runner.Generators.MySql; | ||
| using FluentMigrator.Runner.Generators.Oracle; | ||
| using FluentMigrator.Runner.Generators.Postgres; | ||
| using FluentMigrator.Runner.Generators.Postgres92; | ||
| using FluentMigrator.Runner.Generators.SQLite; | ||
| using FluentMigrator.Runner.Generators.SqlServer; | ||
| using FluentMigrator.Runner.Processors.Postgres; | ||
| using LinqToDB; | ||
| using Microsoft.Extensions.Options; | ||
| using Newtonsoft.Json; | ||
|
|
||
| namespace Akka.Persistence.Linq2Db.IndexHelperApp | ||
| { | ||
| class Program | ||
| { | ||
|
|
||
| static void Main(string[] args) | ||
| { | ||
| Parser.Default.ParseArguments<Options>(args) | ||
| .WithParsed(opts => | ||
| { | ||
| //var str = Linq2DbJournalDefaultSpecConfig.customConfig("testGen", | ||
| // "journalTbl", "metadataTbl", ProviderName.SqlServer, | ||
| // "connStr"); | ||
| var conf = | ||
| ConfigurationFactory.ParseString( | ||
| File.ReadAllText(opts.File)); | ||
|
|
||
| var journalConf = | ||
| new Akka.Persistence.Sql.Linq2Db.Config.JournalConfig( | ||
| conf.GetConfig( | ||
| opts.HoconPath | ||
| //"akka.persistence.journal.linq2db.testGen" | ||
| ) | ||
| .WithFallback(Akka.Persistence.Sql.Linq2Db | ||
| .Journal | ||
| .Linq2DbWriteJournal.DefaultConfiguration)); | ||
| var generator = getGenerator(journalConf.ProviderName); | ||
| var helper = new JournalIndexHelper(); | ||
| CreateIndexExpression expr = null; | ||
| GeneratePerOptions(opts, helper, journalConf, generator); | ||
| }); | ||
| } | ||
|
|
||
| private static void GeneratePerOptions(Options opts, JournalIndexHelper helper, | ||
| JournalConfig journalConf, GenericGenerator generator) | ||
| { | ||
| CreateIndexExpression expr; | ||
| if (opts.GeneratePidSeqNo) | ||
| { | ||
| expr = new CreateIndexExpression() | ||
| { | ||
| Index = helper.JournalOrdering(journalConf.TableConfig.TableName, | ||
| journalConf.TableConfig.ColumnNames.Ordering, | ||
| journalConf.TableConfig.SchemaName) | ||
| }; | ||
| GenerateWithHeaderAndFooter(generator, expr, "Ordering"); | ||
| } | ||
|
|
||
| if (opts.GeneratePidSeqNo) | ||
| { | ||
| expr = new CreateIndexExpression() | ||
| { | ||
| Index = helper.DefaultJournalIndex( | ||
| journalConf.TableConfig.TableName, | ||
| journalConf.TableConfig.ColumnNames.PersistenceId, | ||
| journalConf.TableConfig.ColumnNames.SequenceNumber, | ||
| journalConf.TableConfig.SchemaName) | ||
| }; | ||
| GenerateWithHeaderAndFooter(generator, expr, "PidAndSequenceNo"); | ||
| } | ||
|
|
||
| if (opts.GenerateTimestamp) | ||
| { | ||
| expr = new CreateIndexExpression() | ||
| { | ||
| Index = helper.JournalTimestamp(journalConf.TableConfig.TableName, | ||
| journalConf.TableConfig.ColumnNames.Created, | ||
| journalConf.TableConfig.SchemaName) | ||
| }; | ||
| GenerateWithHeaderAndFooter(generator, expr, "Timestamp"); | ||
| } | ||
| } | ||
|
|
||
| private static void GenerateWithHeaderAndFooter(GenericGenerator generator, | ||
| CreateIndexExpression expr, string indexType) | ||
| { | ||
| Console.WriteLine("-------"); | ||
| Console.WriteLine($"----{indexType} Index Create Below"); | ||
| Console.WriteLine(generator.Generate(expr)); | ||
| Console.WriteLine($"----{indexType} Index Create Above"); | ||
| Console.WriteLine("-------"); | ||
| } | ||
|
|
||
| static GenericGenerator getGenerator(string dbArg) | ||
| { | ||
| if (dbArg.StartsWith("sqlserver", | ||
| StringComparison.InvariantCultureIgnoreCase)) | ||
| { | ||
| return new SqlServer2008Generator(); | ||
| } | ||
| else if (dbArg.Contains("sqlite", | ||
| StringComparison.InvariantCultureIgnoreCase)) | ||
| { | ||
| return new SQLiteGenerator(); | ||
| } | ||
| else if (dbArg.Contains("postgres", | ||
| StringComparison.InvariantCultureIgnoreCase)) | ||
| { | ||
| return new Postgres92Generator( | ||
| new PostgresQuoter(new PostgresOptions()), | ||
| new OptionsWrapper<GeneratorOptions>( | ||
| new GeneratorOptions())); | ||
| } | ||
| else if (dbArg.Contains("mysql", | ||
| StringComparison.InvariantCultureIgnoreCase)) | ||
| { | ||
| return new MySql5Generator(); | ||
| } | ||
| else if (dbArg.Contains("oracle", | ||
| StringComparison.InvariantCultureIgnoreCase)) | ||
| { | ||
| return new OracleGenerator(); | ||
| } | ||
| else | ||
| { | ||
| throw new Exception("IDK what to do with this!"); | ||
| } | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| akka.persistence.journal.linq2db{ | ||
| testGen { | ||
| class = "Akka.Persistence.Sql.Linq2Db.Journal.Linq2DbWriteJournal, Akka.Persistence.Sql.Linq2Db" | ||
| provider-name = "SqlServer" | ||
| connection-string = "connStr" | ||
| tables{ | ||
| journal{ | ||
| auto-init = true | ||
| warn-on-auto-init-fail = false | ||
| table-name = "journalTbl" | ||
| metadata-table-name = "metadataTbl" | ||
| } | ||
| } | ||
| } | ||
| } |
15 changes: 15 additions & 0 deletions
15
Akka.Persistence.Linq2Db.IndexHelperLib/Akka.Persistence.Linq2Db.IndexHelperLib.csproj
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFramework>netstandard2.0</TargetFramework> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\src\Akka.Persistence.Sql.Linq2Db\Akka.Persistence.Sql.Linq2Db.csproj" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="FluentMigrator" Version="3.3.1" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| using System; | ||
| using FluentMigrator.Model; | ||
|
|
||
| namespace Akka.Persistence.Linq2Db.IndexHelperLib | ||
| { | ||
| public class JournalIndexHelper | ||
| { | ||
| public IndexDefinition DefaultJournalIndex(string tableName, string persistenceIdCol, string sequenceNoCol, string schemaName = null) | ||
| { | ||
| var idx = beginCreateIndex(tableName, schemaName, $"UX_{tableName}_PID_SEQNO"); | ||
| //short name for easy compat with all dbs. (*cough* oracle *cough*) | ||
| idx.Columns.Add(new IndexColumnDefinition(){ Name = persistenceIdCol }); | ||
| idx.Columns.Add(new IndexColumnDefinition(){Name = sequenceNoCol, Direction = Direction.Ascending}); | ||
| idx.IsUnique = true; | ||
| return idx; | ||
| } | ||
|
|
||
| public IndexDefinition JournalOrdering(string tableName, | ||
| string orderingCol, string schemaName = null) | ||
| { | ||
| var idx = beginCreateIndex(tableName, schemaName,$"IX_{tableName}_Ordering"); | ||
| idx.Columns.Add(new IndexColumnDefinition(){Name = orderingCol}); | ||
| //Should it be? | ||
| //idx.IsUnique = true; | ||
| return idx; | ||
| } | ||
|
|
||
| public IndexDefinition JournalTimestamp(string tableName, | ||
| string timestampCol, string schemaName = null) | ||
| { | ||
| var idx = beginCreateIndex(tableName, schemaName, | ||
| $"IX_{tableName}_TimeStamp"); | ||
| idx.Columns.Add(new IndexColumnDefinition(){Name = timestampCol}); | ||
| //Not unique by any stretch. | ||
| return idx; | ||
| } | ||
|
|
||
| private static IndexDefinition beginCreateIndex(string tableName, string schemaName, string indexName) | ||
| { | ||
| var idx = new IndexDefinition(); | ||
| if (string.IsNullOrWhiteSpace(schemaName) == false) | ||
| { | ||
| idx.SchemaName = schemaName; | ||
| } | ||
| idx.TableName = tableName; | ||
| idx.Name = indexName; | ||
| return idx; | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM