Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental: Mysql Protocol support for VStream #6675

Closed
rohit-nayak-ps opened this issue Sep 5, 2020 · 1 comment
Closed

Experimental: Mysql Protocol support for VStream #6675

rohit-nayak-ps opened this issue Sep 5, 2020 · 1 comment
Milestone

Comments

@rohit-nayak-ps
Copy link
Contributor

Note: This is a quick experimental POC and NOT for production use

We add a new vstream command in vtgate that uses the VStream API to stream data
from a table.

To try it out:

  • run local_example upto (and not including) the 401_cleanup.sh
  • start the mysql client pointing to vtgate
    mysql -u root --host=127.0.0.1 -P 15306 --quick
mysql> set workload=olap;
Query OK, 0 rows affected (0.00 sec)

mysql> vstream * from product \G
*************************** 1. row ***************************
         op: +
        sku: SKU-1001
description: Monitor
      price: 100
*************************** 2. row ***************************
         op: +
        sku: SKU-1002
description: Keyboard
      price: 30

/* In another mysql client run "update product set price=200 where sku = 'SKU-1001';", gives: */
*************************** 3. row ***************************
         op: *
        sku: SKU-1001
description: Monitor
      price: 200

/* In another mysql client run "delete from product where sku = 'SKU-1001';", gives: */
*************************** 4. row ***************************
         op: -
        sku: SKU-1001
description: Monitor
      price: 200

Notes

  • The first column contains + for inserts, - for deletes and * for updates
  • Without any where clause all binlog events for that table are streamed
  • We can get just new event using "vstream * from product where pos > 'current';"
  • The pos in the where clause can be any gtid to start streaming from
  • There is a limit clause to only stream a few rows "vstream * from product limit 10;"
  • Currently data is streamed from all shards in the keyspace
  • Only one table can be streamed at a time
  • This is separate from the "stream * from " support which uses the message subsystem
@rohit-nayak-ps
Copy link
Contributor Author

The functionality detailed above has been implemented in #6670.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants