[Solved] Docker container mysql 8 fail to start with ‘./slave57_binlog.~rec~’ not found (OS errno 13 – Permission denied)

Avinash Pawar Asks: Docker container mysql 8 fail to start with ‘./slave57_binlog.~rec~’ not found (OS errno 13 – Permission denied)
I have a master and slave setup on docker containers running.

master57 – master container – 5.7.34 Percona Server

slave57 – slave container – 5.7.34 Percona Server

Code:
root@docker-host:~# docker ps
CONTAINER ID   IMAGE            COMMAND                  CREATED        STATUS          PORTS                                       NAMES
bd90196665fc   percona:ps-5.7   "/docker-entrypoint.…"   19 hours ago   Up 3 seconds    0.0.0.0:3311->3306/tcp, :::3311->3306/tcp   slave57
bd0300578bef   percona:ps-5.7   "/docker-entrypoint.…"   20 hours ago   Up 26 minutes   0.0.0.0:3310->3306/tcp, :::3310->3306/tcp   master57

The slave starts and connects successfully to the master without any issues. These instances have their data directories hosted on the docker host. Following are my docker run commands

Code:
docker run -d --name master57 --mount type=bind,src=/mysql_data/mysql_config/master57.cnf,dst=/etc/mysql/my.cnf --mount type=bind,src=/mysql_data/mysql_data/master57,dst=/var/lib/mysql --mount type=bind,src=/mysql_data/mysql_logs/master57_logs/master57_error.log,dst=/var/log/mysql/master57_error.log -p 3310:3306 -e MYSQL_ROOT_PASSWORD=oracle percona:ps-5.7

docker run -d --name slave57 --mount type=bind,src=/mysql_data/mysql_config/slave57.cnf,dst=/etc/mysql/my.cnf --mount type=bind,src=/mysql_data/mysql_data/slave57,dst=/var/lib/mysql --mount type=bind,src=/mysql_data/mysql_logs/slave57_logs/slave57_error.log,dst=/var/log/mysql/slave57_error.log -p 3311:3306 -e MYSQL_ROOT_PASSWORD=oracle percona:ps-5.7

What I am trying to do is upgrade the slave to 8.0.13. I had tested the upgrade procedure on another instance and I am following the same procedure.

Code:
Stop the 5.7 version slave
Start new container with new image for higher version (8.0.13-4).
Run mysql_upgrade
Restart the higher version container.

 docker run -d --name slave8 --mount type=bind,src=/mysql_data/mysql_config/slave57.cnf,dst=/etc/mysql/my.cnf --mount type=bind,src=/mysql_data/mysql_data/slave57,dst=/var/lib/mysql -p 3311:3306 -e MYSQL_ROOT_PASSWORD=oracle percona:ps-8.0.13-4

When I try to create a new container I receive the following errors

Code:
2021-07-27T05:51:12.084554Z 0 [ERROR] [MY-010846] [Server] MYSQL_BIN_LOG::open_purge_index_file failed to open register file.
    2021-07-27T05:51:12.084713Z 0 [ERROR] [MY-010817] [Server] MYSQL_BIN_LOG::open_index_file failed to sync the index file.
    2021-07-27T05:51:12.085008Z 0 [ERROR] [MY-010119] [Server] Aborting


root@docker-host:~# docker logs slave8
mysqld: File './slave57_binlog.~rec~' not found (OS errno 13 - Permission denied)
2021-07-27T05:51:12.067542Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
2021-07-27T05:51:12.067915Z 0 [Warning] [MY-010091] [Server] Can't create test file /var/lib/mysql/mysqld_tmp_file_case_insensitive_test.lower-test
2021-07-27T05:51:12.068055Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.13-4) starting as process 1
2021-07-27T05:51:12.069935Z 0 [Warning] [MY-010091] [Server] Can't create test file /var/lib/mysql/mysqld_tmp_file_case_insensitive_test.lower-test
2021-07-27T05:51:12.069942Z 0 [Warning] [MY-010159] [Server] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
**2021-07-27T05:51:12.084554Z 0 [ERROR] [MY-010846] [Server] MYSQL_BIN_LOG::open_purge_index_file failed to open register file.
2021-07-27T05:51:12.084713Z 0 [ERROR] [MY-010817] [Server] MYSQL_BIN_LOG::open_index_file failed to sync the index file.
2021-07-27T05:51:12.085008Z 0 [ERROR] [MY-010119] [Server] Aborting**
2021-07-27T05:51:12.085195Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.13-4)  Percona Server (GPL), Release 14, Revision 3558242.

I checked the file and directory permissions inside the docker container and on the docker host, they appear to be in order. I am out of thought here being new to docker.

The docker container (slave57) mysql user and docker host mysql user have same user ids and group ids.

Code:
root@docker-host:~# docker exec -ti -u 0 slave57 bash
[root@bd90196665fc /]# id mysql
uid=999(mysql) gid=999(mysql) groups=999(mysql)
[root@bd90196665fc /]# exit
exit
root@docker-host:~# id mysql
uid=999(mysql) gid=999(mysql) groups=999(mysql)
root@docker-host:~#

Following is the config file for slave57

Code:
cat /mysql_data/mysql_config/slave57.cnf
   
[mysqld_safe]
socket          = /var/lib/mysql/slave57.sock
nice            = 0

[mysqld]
skip-name-resolve
user            = mysql
socket          = /var/lib/mysql/slave57.sock
port            = 3306
basedir         = /usr
datadir         = /var/lib/mysql
tmpdir          = /tmp
innodb_flush_log_at_trx_commit = 2
ignore-db-dir = lost+found

bind-address    = 0.0.0.0
log-bin = slave57_binlog
server-id       = 102

max_connections                 = 2000
max_connect_errors              = 10000
table_open_cache                = 15000
table_definition_cache          = 15000
sort_buffer_size                = 2M
read_buffer_size                = 1M
read_rnd_buffer_size            = 1M
myisam_sort_buffer_size         = 2M
thread_cache_size               = 400
thread_handling                 = pool-of-threads
open-files-limit                = 60000
max_heap_table_size             = 1G
tmp_table_size                  = 1G
log_bin_trust_function_creators = 1
event_scheduler                 = 1
connect_timeout                 = 60

userstat                        = ON
thread_statistics               = ON

#InnoDB Engine Fine Tunning
innodb_file_per_table           = 1
innodb_buffer_pool_size         = 2G
innodb_log_buffer_size          = 300M
innodb_flush_log_at_trx_commit  = 1
innodb_lock_wait_timeout        = 200
innodb_thread_concurrency       = 16
innodb_flush_method             = O_DIRECT

symbolic-links=0

Ten-tools.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your response here to help other visitors like you. Thank you, Ten-tools.