Data Backup

1. Function Description

As a distributed database product, TcaplusDB not only provides read and write access to data but also provides data security and reliability. This file is aimed at introducing the data backup capability of TcaplusDB.

2. Application Scenarios

TcaplusDB provides the following backup capabilities

3. Instructions

3.1 Using the Tcaplus Cluster Installation Script

1. If a gluster storage machine is prepared, the backup function is automatically enabled during the environment installation to support transactional data recovery scenarios

2. If a gluster storage machine is not prepared, the backup function is automatically enabled during the environment installation. However, since there is no remote backup storage, the data is only backed up locally in the standby DB, and the transactional data recovery scenario is not supported (manual recovery is allowed)

3.2 Other Situations

1. If the backup function is not enabled in the environment (on the tcapcenter machine and server, the crontab - l | grep mysql_tcaplus_backup_script of the tcaplus account has no result)

2. Enable the backup function manually

2.1. ssh to the tcapcenter machine using the tcaplus account

ssh tcaplus@TCAPCENTER_IP -p36000

2.2. Enter the backup script directory

cd /data/tcaplus/tcaplus_service/bin/tcaplus_backup_script/backup_script_for_overseas/mysql_tcaplus_backup_script

2.3. Copy the configuration file template

cp config_info.txt.template config_info.txt

2.4. Calculate the password ciphertext of the current account tcaplus. Assume that the password is tcaplus123

/data/tcaplus/tcaplus_service/bin/encrypt_tool/encrypt_tool tcaplus123
qhcFSG6f2CqHdcc1T7LGMA==

2.5. Modify the configuration file config_info to enter the password ciphertext

vim config_info.txt

# Enter the calculated ciphertext in 2.4 into: svr_password
svr_password="qhcFSG6f2CqHdcc1T7LGMA=="

2.6. If backup remote storage supported by gluster, cos, and s3 is not available in the current environment, skip to 2.10

2.7. Gluster Backup Mode

2.7.1. By default, the Gluster storage cluster has been installed, and provides the Gluster access node: 10.125.32.21 and volume name: glusterfs_storage

2.7.2. Modify the tcapcenter configuration file

vim /data/tcaplus/tcaplus_service/cfg/tcapcenter/tcapcenter_cfg.xml

# Add a few lines of configuration in <BackupApi> (Fill the GlusterVolumeName and GlusterNodeIP as required)
BackupType="1"
GlusterVolumeName="glusterfs_storage"     
GlusterNodeIP="10.125.32.21"
BackupMapPath="/data/gluster_dbbak"

2.7.3. Restart tcapcenter

cd /data/tcaplus/tcaplus_service/bin/; ./tcapcenter_stop.sh; sleep 10; ./tcapcenter_start.sh

2.7.4. Edit the backup configuration file again

vim config_info.txt

# Set need_gluster_backup to ON
need_gluster_backup="ON"
# Fill gluster_backup_path into the BackupMapPath of 2.7.2
BackupMapPath="/data/gluster_dbbak"

2.7.5. Skip to Step 2.10. Now the gluster backup configuration is complete

2.8. Cos Backup Mode

2.8.1. Modify the tcapcenter configuration file

vim /data/tcaplus/tcaplus_service/cfg/tcapcenter/tcapcenter_cfg.xml

# Add a few lines of configuration in <BackupApi> (Enter as required)
# cos_ENDPOINT is the domain name information, including three parts. Do not change the last two parts. The first part is the region domain name of cos, which can be queried from the cos management console
# cos_ACCESS_KEY_ID is the Tencent Cloud API KEYID
# cos_ACCESS_KEY_SECRET is the Tencent Cloud API KEYSECRET
# cos_BUCKET_NAME is the COS bucket name
BackupType="3"
cos_ENDPOINT="cos.ap-singapore.myqcloud.com 123456 123456"
cos_ACCESS_KEY_ID="AAAAAAAABBBBBBBBB "
cos_ACCESS_KEY_SECRET="BBBBBBBBBAAAAAAAA "
cos_BUCKET_NAME=" /tcaplusdb-sg-999999999/ "

2.8.2. Restart tcapcenter

cd /data/tcaplus/tcaplus_service/bin/; ./tcapcenter_stop.sh; sleep 10; ./tcapcenter_start.sh

2.8.3. Edit the backup configuration file again

vim config_info.txt

# Set need_cos_backup to ON
need_cos_backup="ON"

2.8.4. Modify a File (L5GetRoute1)

su - tcaplus
cd /data/tcaplus/l5_agent/bin
mv L5GetRoute1 L5GetRoute1.old

2.8.5. vim L5GetRoute1 (The following is the file content, which is pasted directly into it)

#! /bin/bash

    cos_url=$(grep cos_ENDPOINT /data/tcaplus/tcaplus_service/cfg/tcapcenter/tcapcenter_cfg.xml | awk -F'=' '{print $2}' | sed -r 's#"##g' | awk '{print $1}')

    if [ x"$cos_url" = "x" ]; then
        echo "A B :169.254.0.47" # default ip: by https://cloud.tencent.com/document/product/436/31315#.E5.86.85.E7.BD.91.E4.B8.8E.E5.A4.96.E7.BD.91.E8.AE.BF.E9.97.AE
            exit 0
            fi

            real_cos_ip=$(nslookup $cos_url | grep 'Address:' | tail -1 | awk '{print $2}')
            echo "A B :$real_cos_ip"
            exit 0

2.8.6. Add executable authority

chmod +x L5GetRoute1

2.8.7. Skip to step 2.10. Now the cos backup configuration is completed

2.9. S3 Backup Mode

2.9.1. Modify the tcapcenter configuration file

vim /data/tcaplus/tcaplus_service/cfg/tcapcenter/tcapcenter_cfg.xml

# Add a few lines of configuration in <BackupApi> (Enter as required)
# AwsS3StorageBucketUrl is the bucket name of S3 
# AwsAccessKeyId is the AWS API KEYID
# AwsSecretAccessKey is the AWS API Secret ACCESSKEY
BackupType="4"
AwsS3StorageBucketUrl="s3://apexdev-tcapbackup"
AwsAccessKeyId="EEEEEEEMAAAAAAAA"
AwsSecretAccessKey="GGGGGG2BBBBBBBBBBBBB"

2.9.2. Restart tcapcenter

cd /data/tcaplus/tcaplus_service/bin/; ./tcapcenter_stop.sh; sleep 10; ./tcapcenter_start.sh

2.9.3. Edit the backup configuration file again

vim config_info.txt

# Set need_s3_backup to ON
need_s3_backup="ON"
# Fill s3_backup_bucket into the S3 storage bucket name
s3_backup_bucket="s3://apexdev-tcapbackup"

2.9.4. Manually create S3 authentication files and write accesskey and secret information to Tcapcenter and tcapsvr that has been mounted

mkdir -p ~/.aws/, vim credentials

[default]
aws_access_key_id = EEEEEEEMAAAAAAAA
aws_secret_access_key = GGGGGG2BBBBBBBBBBBBB

2.9.5. Now the s3 backup configuration is complete

2.10. Configure a periodic backup task on the crontab (crontab -e)

# Add a line in editing status
# 5,20,35,50 * * * * /bin/bash /data/tcaplus/tcaplus_service/bin/tcaplus_backup_script/backup_script_for_overseas/mysql_tcaplus_backup_script/main.sh > /tmp/mysql_tcaplus_backup_script.tmplog

4. Notations

4.1 Backup Data Used in Data Recovery Scenarios

Only when the overseas backup mode (backup_mode="1") is enabled in the backup configuration (/data/tcaplus_service/bin/tcaplus_backup_script/backup_script_for_oversea/mysql_tcaplus_backup_script/config_info. txt) and any remote backup storage of cluster, cos, and s3 is used can the backed up data be used in the data recovery scenario. Note: In the domestic IDC environment, the backup mode is set to "0".

4.2 How to View the Details of Daily Backup Files

Entry: Operation and Maintenance Platform -- More -- Backup Management, here provides a variety of backup query methods:

1. Conventional cold backup (TCAPLUS_FULL), used to query details about periodic full backup

2. binlog stream backup (TCAPLUS_INCR), used to query details about periodic incremental backup

3. Self-service backup (OSDATA), used to query details about the user's self-service backup

4. The cluster backup status summary helps view the backup failure of the standby DB in the cluster

5. Viewing and editing backup expiration policies can set the TTL of backup files stored in Gluster, Cos, and S3 remote storage (personalized configuration can be performed for clusters, services, game areas, and tables)