Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mysql listener: idle mysql database causes rogue listener #98

Open
burleyb opened this issue Feb 25, 2019 · 1 comment
Open

mysql listener: idle mysql database causes rogue listener #98

burleyb opened this issue Feb 25, 2019 · 1 comment

Comments

@burleyb
Copy link
Contributor

burleyb commented Feb 25, 2019

Description:
While working on a dev database with no changes happening in the db, the listener checkpoint gets stuck on a binlog file that doesn't exist anymore. This causes a rogue that makes it hard to develop because you have to reset the checkpoint every hour or so to restart the listener.

How to reproduce:

  1. If I do show master logs; I get this:
    Log_name File_size
    mysql-bin-changelog.238847 578
    mysql-bin-changelog.238848 578
    mysql-bin-changelog.238849 578
    mysql-bin-changelog.238850 521

  2. I set the checkpoint to start at the earliest log position that exists. mysql-bin-changelog.238847::0. Then restart the bot.

  3. If I make a change to a table that is being listened to, the listener works as expected and catches up to the last log location. The checkpoint points to a location in the last file mysql-bin-changelog.238850

  4. After an amount of time, the listener goes into rogue status with the error.
    Error: ER_MASTER_FATAL_ERROR_READING_BINLOG: Could not find first log file name in binary log index file at Binlog.Sequence._packetToError (/var/task/index.js:84094:14) at Binlog.Sequence.ErrorPacket (/var/task/index.js:84143:17) at Protocol._parsePacket (/var/task/index.js:79237:23) at Parser.write (/var/task/index.js:78556:12) at Protocol.write (/var/task/index.js:78997:16) at Socket. (/var/task/index.js:76656:28) at emitOne (events.js:116:13) at Socket.emit (events.js:211:7) at addChunk (_stream_readable.js:263:12) at readableAddChunk (_stream_readable.js:250:11) -------------------- at Protocol._enqueue (/var/task/index.js:79103:48) at Immediate._start [as _onImmediate] (/var/task/index.js:20906:29) at runCallback (timers.js:794:20) at tryOnImmediate (timers.js:752:5) at processImmediate [as _immediateCallback] (timers.js:729:5) code: 'ER_MASTER_FATAL_ERROR_READING_BINLOG', errno: 1236, sqlMessage: 'Could not find first log file name in binary log index file', sqlState: 'HY000'

  5. Doing show master logs; indicates that the checkpoint location points to a file that has already been rotated out and deleted.

Expected behavior:
Our install is mysql setup on RDS with a standard configuration. It rotates files pretty quickly, even though there is no activity. I realize that I can change mysql parameters to not rotate and delete as quickly, and that this wouldn't be an issue on production. But I would expect the listener not to get stuck regardless of inactivity... Among other issues, it wastes a lot of time during development to have to reset the checkpoint.

@seriouscoderone
Copy link
Contributor

Sorry for the delayed response. We will work on this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants