Skip to content
This repository has been archived by the owner on Oct 17, 2023. It is now read-only.

Adaptor: Add MySQL #515

Merged
merged 101 commits into from
May 26, 2022
Merged
Show file tree
Hide file tree
Changes from 89 commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
4f9a4e2
Very initial work on MySQL, copying Postgresql
atomicules Nov 11, 2021
5d03c81
Make MySQL Client test pass
atomicules Nov 11, 2021
ff84001
Add some initial README notes on developing MySQL
atomicules Nov 11, 2021
51c5bb0
Change default URI / DSN, strip prefix if there is one
atomicules Dec 8, 2021
31c316f
Initial work on adding reader tests (run, but fail)
atomicules Dec 8, 2021
465684b
Update types for MySQL, insert data for reader tests
atomicules Dec 10, 2021
93b3a24
Reader tests only failing complex type
atomicules Dec 10, 2021
185d75e
Some progress on TestReadComplex
atomicules Dec 13, 2021
93c0a79
Fix id and colinteger test
atomicules Dec 13, 2021
9666419
Remove timestamp test
atomicules Dec 13, 2021
a376b5f
TestReadComplex works for Blob
atomicules Dec 14, 2021
5a99bec
Use os/exec instead of io.Copy
atomicules Dec 15, 2021
5cb5620
TestReadComplex - binary column passing
atomicules Dec 15, 2021
b6c8bcc
Reference commit - Bit value passing for TestReadComplex
atomicules Dec 15, 2021
17c896e
Reference commit - Trying to test Point values
atomicules Dec 15, 2021
88d4b2e
Reference commit - go-geom instead of orb
atomicules Dec 15, 2021
7fc4970
Reference commit - Working test for colpoint
atomicules Dec 15, 2021
4f81e87
Hard wrap some text in README
atomicules Dec 15, 2021
c469e0d
All TestReadComplex tests pass (including Spatial)
atomicules Dec 15, 2021
f93ab50
Formatting / commenting / layout changes
atomicules Dec 16, 2021
43cf53a
Add writer tests. Run, but fail.
atomicules Dec 16, 2021
1a54d44
TestInsert passing
atomicules Dec 16, 2021
db0064c
TestReadComplex - Some comment/naming changes only
atomicules Dec 17, 2021
86553b8
Reference commit - TestComplexInsert
atomicules Dec 17, 2021
d3c30e4
WIP - Wrap geometry in `ST_GeomFromText` at insert
atomicules Jan 10, 2022
d73d7ad
Bump Binary length for test
atomicules Jan 11, 2022
6b38eb8
Reference commit - Wrap geometry placeholders
atomicules Jan 11, 2022
6e9e7df
Add note to README about install MySQL 5.7 for testing
atomicules Jan 11, 2022
51dc73a
Whitespace change only. Indent switch/case
atomicules Jan 11, 2022
18c9523
TestComplexInsert passes
atomicules Jan 11, 2022
b8cd2fe
Tidy up (comment out) some of debug logging
atomicules Jan 11, 2022
e93c042
TestUpdate passes
atomicules Jan 11, 2022
2f6c6d2
TestDelete passes
atomicules Jan 12, 2022
666f6b0
Add missed change from `public` for TestInsert
atomicules Jan 12, 2022
97e3a99
TestComplexUpdate passes
atomicules Jan 12, 2022
47f6f59
TestComplexDelete passes
atomicules Jan 12, 2022
68fee06
TestComplexDeleteWithoutAllPrimarykeys passes
atomicules Jan 12, 2022
b5cfdcb
I like my switch/cases indented
atomicules Jan 12, 2022
b120441
Fix TestReadComplex due to change in binary column
atomicules Jan 12, 2022
e009932
Add mysql_test
atomicules Jan 12, 2022
562f065
Initial work on switching to go-mysql-org/go-mysql
atomicules Jan 17, 2022
400073f
TestReadComplex pass - Need uint64 for year
atomicules Jan 17, 2022
63cbb1a
TestInit passes - Need the `mysql://` prefix
atomicules Jan 17, 2022
7d588d9
Whitespace / comment change only
atomicules Jan 18, 2022
18373cb
Writer tests pass - handle scanning time values
atomicules Jan 18, 2022
07a7661
Handle writing time.Time values
atomicules Jan 20, 2022
69f31b2
Whitespace only changes
atomicules Jan 20, 2022
c834bef
Skip more system/default databases/tables.
atomicules Jan 20, 2022
85c6819
Comment changes only
atomicules Jan 28, 2022
9618d84
Disable Foreign Key Checks on write
atomicules Jan 28, 2022
ea0f7c5
Disable foreign key checks once per client
atomicules Jan 28, 2022
5f92603
Use forked go-mysql-org/go-mysl, TLS in driver
atomicules Jan 28, 2022
fc5842a
Reference commit for setting custom TLS config
atomicules Feb 8, 2022
f45cbc4
Revert "Reference commit for setting custom TLS config"
atomicules Feb 8, 2022
091baa6
Pretty much add/copy in files per Postgresl
atomicules Feb 8, 2022
ecae3f5
Whitespace changes only
atomicules Feb 15, 2022
36596b3
Reference commit for MySQL tailing
atomicules Feb 15, 2022
95a9f44
Review feedback - Avoid GetEvent blocking done
atomicules Feb 15, 2022
91aad5b
Reference commit for MySQL tailing, processEvent
atomicules Feb 15, 2022
511093c
Reference commit - parseEventData builds
atomicules Feb 21, 2022
1e0b5b2
Reference commit - loop through eventRows one level up
atomicules Feb 22, 2022
f24f2ec
Move fetching column info up one level
atomicules Feb 22, 2022
7962e9f
Enable tailer test for MySQL - runs, doesn't pass
atomicules Feb 22, 2022
ac00362
Use `QueryRow` for querying master status
atomicules Feb 22, 2022
0cb9b03
Tailing tests pass! Whooooooooop!
atomicules Feb 22, 2022
029c43e
go mod updates
atomicules Feb 22, 2022
ae8e249
Remove accidentally committed temp file
atomicules Feb 22, 2022
686e967
Apple M1 related update
atomicules Apr 19, 2022
5a9dfcc
Read actual serverID instead of hard-coding.
atomicules Apr 19, 2022
c4e227c
Actually enable the MySQL tailer
atomicules Apr 19, 2022
d42b060
Support specifying CA cert and server name
atomicules Apr 20, 2022
0db78f5
Make WithCustomTLS pass through if no TLS options
atomicules Apr 21, 2022
78e0293
Tidy up debug logging
atomicules Apr 21, 2022
122afa8
More logging / debugging related changes
atomicules Apr 21, 2022
4e1cbc2
Error handing / TODOs in the test files
atomicules Apr 21, 2022
0444743
Tidy up some easy TODOs, debug logging, error handling
atomicules Apr 21, 2022
b7399e4
Update README with some usage instructions
atomicules Apr 21, 2022
49aa767
gofmt changes only
atomicules Apr 21, 2022
ff78972
Stupid logging error / mistake
atomicules Apr 21, 2022
6c8549d
Pull in latest forked version of mysql driver
atomicules Apr 21, 2022
f117c7c
Initial README And Changelog changes to suit PRing.
atomicules Apr 21, 2022
786b686
Only tested/developed with mysql as both sink and source
atomicules Apr 21, 2022
07c7cee
Fix golangci-lint composite literal uses unkeyed fields
atomicules Apr 21, 2022
cbdb864
Fix golangci-lint lostcancel
atomicules Apr 21, 2022
7ef88a4
Fix golangci-lint type assertions in switch cases
atomicules Apr 21, 2022
cef843b
Comment changes only. Add TODOs
atomicules Apr 22, 2022
e394857
Mention need to create sink table structure first
atomicules Apr 22, 2022
7777280
Update references to go-mysql-org/go-mysql
atomicules May 5, 2022
69a00e3
Markup in adaptor README
atomicules May 5, 2022
7ac0ec4
Add some error handling that was TODO, tidy up some comments
atomicules May 9, 2022
9b6a8c0
Remove irrelevant Json array stuff
atomicules May 9, 2022
6b8575a
golangci-lint fixes
atomicules May 9, 2022
03bf8f2
Revert "Remove irrelevant Json array stuff"
atomicules May 10, 2022
8285344
Indentation only
atomicules May 10, 2022
866e648
Add Json column test
atomicules May 10, 2022
9d6d829
Remove `case []interface{}` from MySQL writer
atomicules May 10, 2022
2982e76
Indentation (again)
atomicules May 10, 2022
c3fabd2
Handle certain column types in reader, not reader_test
atomicules May 10, 2022
f678a8b
Merge branch 'master' into dev-add-mysql
atomicules May 25, 2022
a7309f1
Move MySQL development notes to separate file
atomicules May 26, 2022
0c803e7
Update Changelog for MySQL adaptor
atomicules May 26, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
## v1.0.0 [TBD]
## v1.1.0 [TBD]

### Changes

- Adds MySQL Adaptor. _More info to be provided_

## v1.0.0 [2021-12-01]

This release does not introduce any new functionalities, but catches up transporter with modern golang.
It also introduces a new standard way of running adaptor tests.
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ Below is a list of each adaptor and its support of the feature:
| file | | X |
| mongodb | X | X |
| postgresql | | X |
| mysql | | X |
| rabbitmq | X | |
| rethinkdb | | X |
+---------------+-------------+----------------+
Expand All @@ -64,6 +65,7 @@ Each adaptor has its own README page with details on configuration and capabilit
* [postgresql](./adaptor/postgres)
* [rabbitmq](./adaptor/rabbitmq)
* [rethinkdb](./adaptor/rethinkdb)
* [mysql](./adaptor/mysql)

Native Functions
----------------
Expand Down
1 change: 1 addition & 0 deletions adaptor/all/all.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
_ "github.com/compose/transporter/adaptor/file"
_ "github.com/compose/transporter/adaptor/mongodb"
_ "github.com/compose/transporter/adaptor/postgres"
_ "github.com/compose/transporter/adaptor/mysql"
_ "github.com/compose/transporter/adaptor/rabbitmq"
_ "github.com/compose/transporter/adaptor/rethinkdb"
)
Loading