Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update async replication example and add semisync replication example #547

Merged
merged 2 commits into from
Nov 29, 2023

Conversation

an3l
Copy link
Contributor

@an3l an3l commented Nov 27, 2023

Overview

The PR consists of 2 commits :

  1. update of async replication yaml file
  2. add semi-sync example

Update of docker compose example for async replication

When run compose-replication.yml docker-compose file, it doesn't start healthy service for primary

Primary service health failure
$ docker compose -f official-docker-compose.yml up
[+] Running 3/2
✔ Network official-docker-files_default      Created                                                                                                                                                                                    0.0s 
✔ Container official-docker-files-master-1   Created                                                                                                                                                                                    0.1s 
✔ Container official-docker-files-replica-1  Created                                                                                                                                                                                    0.1s 
Attaching to official-docker-files-master-1, official-docker-files-replica-1
official-docker-files-master-1   | 2023-11-27 12:22:28+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.0.2+maria~ubu2204 started.
official-docker-files-master-1   | 2023-11-27 12:22:29+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
official-docker-files-master-1   | 2023-11-27 12:22:29+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.0.2+maria~ubu2204 started.
official-docker-files-master-1   | 2023-11-27 12:22:29+00:00 [Note] [Entrypoint]: Initializing database files
official-docker-files-master-1   | 
official-docker-files-master-1   | 
official-docker-files-master-1   | PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
official-docker-files-master-1   | To do so, start the server, then issue the following command:
official-docker-files-master-1   | 
official-docker-files-master-1   | '/usr/bin/mariadb-secure-installation'
official-docker-files-master-1   | 
official-docker-files-master-1   | which will also give you the option of removing the test
official-docker-files-master-1   | databases and anonymous user created by default.  This is
official-docker-files-master-1   | strongly recommended for production servers.
official-docker-files-master-1   | 
official-docker-files-master-1   | See the MariaDB Knowledgebase at https://mariadb.com/kb
official-docker-files-master-1   | 
official-docker-files-master-1   | Please report any problems at https://mariadb.org/jira
official-docker-files-master-1   | 
official-docker-files-master-1   | The latest information about MariaDB is available at https://mariadb.org/.
official-docker-files-master-1   | 
official-docker-files-master-1   | Consider joining MariaDB's strong and vibrant community:
official-docker-files-master-1   | https://mariadb.org/get-involved/
official-docker-files-master-1   | 
official-docker-files-master-1   | 2023-11-27 12:22:31+00:00 [Note] [Entrypoint]: Database files initialized
official-docker-files-master-1   | 2023-11-27 12:22:31+00:00 [Note] [Entrypoint]: Starting temporary server
official-docker-files-master-1   | 2023-11-27 12:22:31+00:00 [Note] [Entrypoint]: Waiting for server startup
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] Starting MariaDB 11.0.2-MariaDB-1:11.0.2+maria~ubu2204-log source revision 0005f2f06c8e1aea4915887decad67885108a929 as process 102
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] InnoDB: Number of transaction pools: 1
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] InnoDB: Completed initialization of buffer pool
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] InnoDB: Opened 3 undo tablespaces
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active.
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] InnoDB: log sequence number 47289; transaction id 14
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] Plugin 'FEEDBACK' is disabled.
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] Plugin 'wsrep-provider' is disabled.
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Warning] 'user' entry 'root@2ae8fe424a9b' ignored in --skip-name-resolve mode.
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Warning] 'proxies_priv' entry '@% root@2ae8fe424a9b' ignored in --skip-name-resolve mode.
official-docker-files-master-1   | 2023-11-27 12:22:31 0 [Note] mariadbd: ready for connections.
official-docker-files-master-1   | Version: '11.0.2-MariaDB-1:11.0.2+maria~ubu2204-log'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
official-docker-files-master-1   | 2023-11-27 12:22:32+00:00 [Note] [Entrypoint]: Temporary server started.
official-docker-files-master-1   | 2023-11-27 12:22:33+00:00 [Note] [Entrypoint]: Creating database testdb
official-docker-files-master-1   | 2023-11-27 12:22:33+00:00 [Note] [Entrypoint]: Creating user testuser
official-docker-files-master-1   | 2023-11-27 12:22:33+00:00 [Note] [Entrypoint]: Giving user testuser access to schema testdb
official-docker-files-master-1   | 2023-11-27 12:22:33+00:00 [Note] [Entrypoint]: Creating user repl
official-docker-files-master-1   | 2023-11-27 12:22:33+00:00 [Note] [Entrypoint]: Securing system users (equivalent to running mysql_secure_installation)
official-docker-files-master-1   | 
official-docker-files-master-1   | 2023-11-27 12:22:33+00:00 [Note] [Entrypoint]: Stopping temporary server
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] mariadbd (initiated by: unknown): Normal shutdown
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: FTS optimize thread exiting.
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Starting shutdown...
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Buffer pool(s) dump completed at 231127 12:22:33
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Shutdown completed; log sequence number 47289; transaction id 15
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] mariadbd: Shutdown complete
official-docker-files-master-1   | 
official-docker-files-master-1   | 2023-11-27 12:22:33+00:00 [Note] [Entrypoint]: Temporary server stopped
official-docker-files-master-1   | 
official-docker-files-master-1   | 2023-11-27 12:22:33+00:00 [Note] [Entrypoint]: MariaDB init process done. Ready for start up.
official-docker-files-master-1   | 
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] Starting MariaDB 11.0.2-MariaDB-1:11.0.2+maria~ubu2204-log source revision 0005f2f06c8e1aea4915887decad67885108a929 as process 1
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Number of transaction pools: 1
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Completed initialization of buffer pool
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Opened 3 undo tablespaces
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active.
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: log sequence number 47289; transaction id 14
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] Plugin 'FEEDBACK' is disabled.
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] Plugin 'wsrep-provider' is disabled.
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] InnoDB: Buffer pool(s) load completed at 231127 12:22:33
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] Server socket created on IP: '0.0.0.0'.
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] Server socket created on IP: '::'.
official-docker-files-master-1   | 2023-11-27 12:22:33 0 [Note] mariadbd: ready for connections.
official-docker-files-master-1   | Version: '11.0.2-MariaDB-1:11.0.2+maria~ubu2204-log'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
official-docker-files-master-1   | 2023-11-27 12:22:39 4 [Warning] Access denied for user 'root'@'127.0.0.1' (using password: NO)
official-docker-files-master-1   | 2023-11-27 12:22:39 5 [Warning] Access denied for user 'root'@'localhost' (using password: NO)
official-docker-files-master-1   | 2023-11-27 12:22:49 6 [Warning] Access denied for user 'root'@'127.0.0.1' (using password: NO)
official-docker-files-master-1   | 2023-11-27 12:22:49 7 [Warning] Access denied for user 'root'@'localhost' (using password: NO)
official-docker-files-master-1   | 2023-11-27 12:22:59 8 [Warning] Access denied for user 'root'@'127.0.0.1' (using password: NO)
official-docker-files-master-1   | 2023-11-27 12:22:59 9 [Warning] Access denied for user 'root'@'localhost' (using password: NO)
dependency failed to start: container official-docker-files-master-1 is unhealthy

Add semi-sync example

  • Add new compose file supporting semi-sync replication

@grooverdan
Copy link
Member

Pull a 11.0.4 version should fix the "Access denied for user" from healthcheck, it make a new user.

Copy link
Member

@grooverdan grooverdan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm very glad to see the use of yml based syntax to improve the readabilty, remove duplication etc.

Just some small fixes to keep examples as minimal as possible. And a few questions about why things where included.

examples/.env Outdated
@@ -0,0 +1,7 @@
MARIADB_ROOT_PASSWORD=secret
MYSQL_INITDB_SKIP_TZINFO=Y
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is MARIADB_INITDB_SKIP_TZINFO, but is there really a need to exclude tz init? Generally like to keep examples small.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack will remove.

MARIADB_REPLICATION_PASSWORD=replicationpass
PRIMARY_name='mariadb-primary'
REPLICATION_name_1='mariadb-replica-1'
REPLICATION_name_2='mariadb-replica-2'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The REPLICATION_* are only used once. Can we just omit them and get the default configuration.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are here giving the exact name of replicas, I personally prefer to specify the name of replicas.

x-common-attributes: &common-attributes
image: mariadb:lts
env_file:
- .env
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in preparation for non-replication based examples here, recommend explicit naming about env_replication as a env_file.

What's the gain on having it as an env file? It it just showcasing the hiding of passwords out of the main compose file as best practices?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The gain as you said for password and having it in place and not repeating the variables for different containers.
Note this is docker-compose file and it has to be used env_file with specifiyng .env.

test: ["CMD", "healthcheck.sh", "--connect", "--replication_io", "--replication_sql", "--replication_seconds_behind_master=1", "--replication"]
interval: 10s
timeout: 5s
retries: 3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is start_period worth including and works correctly?10s?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it works, have added, thanks. There is also start_interval, but I wouldn't added it here.

- *common-replication
container_name: ${REPLICATION_name_1}
command: --server-id=2 --log-basename=mariadb --rpl_semi_sync_slave_enabled
environment:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can environment and depends_on go into common-replication? (in the hope that environment is additive)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we can not move environment but we can do depends_onand that I have added. Only thing that could work is that we specify MARIADB_MASTER_HOST=${PRIMARY_name} and MARIADB_HEALTHCHECK_GRANTS=REPLICA MONITOR in .env but I think this way is more readable.

interval: 10s
timeout: 5s
retries: 3
logging:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is logging needed here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As default login setting for all services set explicitly. I'm ok if you want to exclude it.

retries: 3
logging:
driver: journald
networks:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't a dedicated network created for the compose file? Is there something gained by this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but with auto-generated name, I don't prefer this. It is gained when we create the new front-end service, or new service for backend, intuitively we know what is missing and what to add and where.

@grooverdan grooverdan merged commit 28edc90 into master Nov 29, 2023
22 checks passed
@grooverdan grooverdan deleted the anel-replication-example branch November 29, 2023 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants