专注于 JetBrains IDEA 全家桶,永久激活,教程
提供 JetBrains 全家桶激活码、注册码、破解补丁下载及详细激活教程,等工具的永久激活

使用GoldenGate完成MySQL到MySQL的同步

(一)基础环境配置

  源库 目标库
操作系统版本 CentOS Linux release 7.4 CentOS Linux release 7.4
IP地址 192.168.10.11 192.168.10.12
数据库版本 社区版 5.7.24 社区版 5.7.24
需要同步的数据库 testdb(所有表) testdb_repl
GoldenGate版本 OGG-12.2.0.2 OGG-12.2.0.2

说明:

1、oracle GoldenGate下载地址:https://edelivery.oracle.com/ 免费注册一个账号即可登入,直接搜索“goldengate for non oracle database”,找到想要的版本下载即可。

92_1.png

2、经过测试,使用OGG-11.2.0.1同步数据库会报错:

ERROR OGG-00146 VAM function VAMRead returned unexpected result: error 600 – VAM Client Report .

更换为OGG-12.2.0.2后解决,故本文采用的Ogg版本是12.2.0.2。

(二)MySQL数据库初始信息

  源库 目标库
MySQL安装位置(basedir) /usr/local/mysql /usr/local/mysql
数据存放位置    (datadir) /mysql/data /mysql/data
二进制日志位置 (log_bin) /mysql/binlog /mysql/binlog

(三)MySQL数据库开启二进制日志

要使用ogg抽取数据,必须开启二进制日志。在mysql的配置文件/etc/my.cnf中配置二进制日志,重启数据库生效。

[mysqld]

# binlog配置
server_id = 1
binlog_format=ROW
log_bin=/mysql/binlog/master-bin
sync_binlog=1 
expire_logs_days=1

(四)源端和目标端安装配置GoldenGate(源端和目标端都要执行)

(4、1)源端和目标端创建安装ogg,实际上ogg并不需要安装,解压就可以使用

创建ogg安装路径

[root@masterdb ~]# mkdir /ogg

将安装包拷贝到该路径下解压:

[root@masterdb ogg]# ls
ggs_Linux_x64_MySQL_64bit.tar
[root@masterdb ogg]# tar -xvf ggs_Linux_x64_MySQL_64bit.tar 
./
./mysql_checklist.sql
...略
./libicuuc.so.56.1
[root@masterdb ogg]# ls
bcpfmt.tpl              emsclnt                        libggnnzitp.so        mysql.txt
bcrypt.txt              extract                        libggparam.so         notices.txt
cachefiledump           freeBSD.txt                    libggperf.so          oggerr
checkprm                ggcmd                          libggrepo.so          prvtclkm.plb
chkpt_mysql_create.sql  ggMessage.dat                  libicudata.so.56      replicat
convchk                 ggparam.dat                    libicudata.so.56.1    retrace
convprm                 ggsci                          libicui18n.so.56      reverse
db2cntl.tpl             ggs_Linux_x64_MySQL_64bit.tar  libicui18n.so.56.1    server
defgen                  help.txt                       libicuuc.so.56        sqlldr.tpl
demo_mysql_create.sql   keygen                         libicuuc.so.56.1      tcperrs
demo_mysql_insert.sql   lib                            libmysqlclient.so.18  ucharset.h
demo_mysql_load.sql     lib12                          libxerces-c-3.1.so    UserExitExamples
demo_mysql_misc.sql     libantlr3c.so                  logdump               usrdecs.h
dirout                  libdb-6.1.so                   mgr                   zlib.txt
dirwww                  libgglog.so                    mysql_checklist.sql

(4、2)使用ggsci创建ogg相关路径

[root@masterdb ogg]# pwd
/ogg
[root@masterdb ogg]# ./ggsci

Oracle GoldenGate Command Interpreter for MySQL
Version 12.2.0.2.2 OGGCORE_12.2.0.2.0_PLATFORMS_170630.0419
Linux, x64, 64bit (optimized), MySQL Enterprise on Jun 30 2017 06:32:17
Operating system character set identified as UTF-8.

Copyright (C) 1995, 2017, Oracle and/or its affiliates. All rights reserved.

GGSCI (masterdb) 1> create subdirs

Creating subdirectories under current directory /ogg

Parameter files                /ogg/dirprm: created
Report files                   /ogg/dirrpt: created
Checkpoint files               /ogg/dirchk: created
Process status files           /ogg/dirpcs: created
SQL script files               /ogg/dirsql: created
Database definitions files     /ogg/dirdef: created
Extract data files             /ogg/dirdat: created
Temporary files                /ogg/dirtmp: created
Credential store files         /ogg/dircrd: created
Masterkey wallet files         /ogg/dirwlt: created
Dump files                     /ogg/dirdmp: created

(五)源端配置GoldenGate(源端执行)

(5、1)创建数据库用户ogg,该用户用来同步数据

mysql> GRANT ALL PRIVILEGES ON `testdb`.* TO 'ogg'@'%'IDENTIFIED BY 'ogg';

(5、2)配置管理进程mgr

GGSCI (masterdb) 1> edit param mgr
port 7809  
dynamicportlist 7840-7939  
purgeoldextracts /ogg/dirdat/*,usecheckpoints, minkeepdays 1 

GGSCI (masterdb) 2> start mgr
Manager started.

GGSCI (masterdb) 3> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING

(5、3)配置捕获进程exta

GGSCI (masterdb) 5> edit param exta
extract exta  
sourcedb testdb@192.168.10.11:3306 userid ogg password ogg  
exttrail /ogg/dirdat/ea
discardfile  /ogg/dirrpt/exta.dsc,append  
TranLogOptions AltLogDest /mysql/binlog/master-bin.index 
table testdb.*; 

GGSCI (masterdb) 10> add extract exta,tranlog,begin now  
EXTRACT added.

GGSCI (masterdb) 11> add exttrail /ogg/dirdat/ea,extract exta  
EXTTRAIL added.

(5、4)配置投递进程dmpa

GGSCI (masterdb) 15> edit param dmpa 
extract dmpa 
passthru  
sourcedb testdb@192.168.10.11:3306 userid ogg password ogg  
rmthost  192.168.10.12,mgrport 7809,compress  
rmttrail  /ogg/dirdat/da  
dynamicresolution  
numfiles 3000  
table testdb.*; 

GGSCI (masterdb) 16> add extract dmpa ,exttrailsource /ogg/dirdat/ea  
EXTRACT added.

GGSCI (masterdb) 17> add rmttrail /ogg/dirdat/da,extract dmpa 
RMTTRAIL added.

(5、5)创建表定义文件,传送到目标端

(5、5.1)创建表定义文件

GGSCI (masterdb) 23> edit param defgen  
defsfile ./dirdef/testdb.def  
sourcedb testdb@192.168.10.11:3306 userid ogg, password ogg  
table testdb.*;  

GGSCI (masterdb) 26> exit

(5、5.2)生成表定义

[root@masterdb ogg]# ./defgen paramfile ./dirprm/defgen.prm  

***********************************************************************
        Oracle GoldenGate Table Definition Generator for MySQL
      Version 12.2.0.2.2 OGGCORE_12.2.0.2.0_PLATFORMS_170630.0419
 Linux, x64, 64bit (optimized), MySQL Enterprise on Jun 30 2017 07:27:58

Copyright (C) 1995, 2017, Oracle and/or its affiliates. All rights reserved.

                    Starting at 2020-02-08 18:15:40
***********************************************************************

Operating System Version:
Linux
Version #1 SMP Tue Aug 22 21:09:27 UTC 2017, Release 3.10.0-693.el7.x86_64
Node: masterdb
Machine: x86_64
                         soft limit   hard limit
Address Space Size   :    unlimited    unlimited
Heap Size            :    unlimited    unlimited
File Size            :    unlimited    unlimited
CPU Time             :    unlimited    unlimited

Process id: 2510

***********************************************************************
**            Running with the following parameters                  **
***********************************************************************
defsfile ./dirdef/testdb.def
sourcedb testdb@192.168.10.11:3306 userid ogg, password ***
table testdb.*;
Expanding wildcard table specification testdb.*:

Retrieving definition for testdb.test01.
Retrieving definition for testdb.test02.
Retrieving definition for testdb.test03.

Definitions generated for 3 tables in ./dirdef/testdb.def.

(5、5.3)传送到目标端

[root@masterdb ogg]# scp /ogg/dirdef/testdb.def 192.168.10.12:/ogg/dirdef/
root@192.168.10.12's password: 
testdb.def                            100% 1785     1.9MB/s   00:00

(六)目标端配置GoldenGate(目标端执行)

(6、1)创建数据库用户ogg,该用户用来同步数据

mysql> GRANT ALL PRIVILEGES ON `testdb_repl`.* TO 'ogg'@'%'IDENTIFIED BY 'ogg';

(6、2)配置管理进程mgr

GGSCI (slavedb) 3> edit param mgr
port 7809  
dynamicportlist 7840-7939  
purgeoldextracts /ogg/dirdat/*,usecheckpoints, minkeepdays 1 

GGSCI (slavedb) 5> start mgr
Manager started.

GGSCI (slavedb) 6> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING

(6、3)配置检查表

GGSCI (slavedb) 9> dblogin sourcedb testdb_repl@192.168.10.12:3306 userid ogg password ogg 
Successfully logged into database.

GGSCI (slavedb DBLOGIN as ogg) 10> add checkpointtable testdb_repl.checkpoint  

Successfully created checkpoint table testdb_repl.checkpoint.

GGSCI (slavedb DBLOGIN as ogg) 11> edit params ./GLOBALS 
checkpointtable testdb_repl.checkpoint

(6、4)配置应用进程repa

GGSCI (slavedb DBLOGIN as ogg) 13> edit params repa

replicat repa 
targetdb testdb_repl@192.168.10.12:3306 userid ogg password ogg  
handlecollisions  
sourcedefs /ogg/dirdef/testdb.def  
discardfile /ogg/dirrpt/repa.dsc,purge  
map testdb.*, target testdb_repl.*;
 

GGSCI (slavedb DBLOGIN as ogg) 16> add replicat repa,exttrail /ogg/dirdat/da,checkpointtable testdb_repl.checkpoint
REPLICAT added.

(七)导入初始化数据

将源库需要同步的表导出,然后导入到目标数据库

# 源库导出,并传到目标数据库服务器上
[root@masterdb ~]# mysqldump -uroot -p123456 -h192.168.10.11  --single-transaction --flush-logs --master-data=2 testdb  > testdb.sql
mysqldump: [Warning] Using a password on the command line interface can be insecure.

[root@masterdb ~]# ls
anaconda-ks.cfg  testdb.sql  wgetlog-02-05-20-22:27.log  wgetlog-02-05-20-22:28.log  wget.sh

[root@masterdb ~]# scp testdb.sql root@192.168.10.12:/root
root@192.168.10.12's password: 
testdb.sql                                                                                100%   15MB  68.9MB/s   00:00
# 目标数据库执行导入
[root@slavedb ~]# mysql -uroot -p123456 testdb_repl < testdb.sql 
mysql: [Warning] Using a password on the command line interface can be insecure.

(八)开启ogg同步进程

(8、1)源端开启捕获进程(源端执行)

由于数据库数据一直在变化,所以对于导出的testdb.sql文件,我们需要记录二进制日志文件的log_file和log_pos

[root@slavedb ~]# cat testdb.sql |grep "CHANGE MASTER TO MASTER_LOG_FILE"
-- CHANGE MASTER TO MASTER_LOG_FILE='master-bin.000005', MASTER_LOG_POS=154;

然后从导出时的binlog开始抽取

GGSCI (masterdb) 5> alter extract exta,vam,lognum 5,logpos 154
EXTRACT altered.

GGSCI (masterdb) 6> start exta

Sending START request to MANAGER ...
EXTRACT EXTA starting

(8、2)开启投递进程(源端执行)

GGSCI (masterdb) 19> start dmpa

Sending START request to MANAGER ...
EXTRACT DMPA starting

(8、3)开启应用进程(目标端执行)

GGSCI (slavedb DBLOGIN as ogg) 18> start repa

Sending START request to MANAGER ...
REPLICAT REPA starting

(九)确认同步状态

(9、1)使用info all查看进程是否运行正常

所有进程状态为“running”代表正常

源端:

GGSCI (masterdb) 8> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
EXTRACT     RUNNING     DMPA        00:00:00      00:00:08    
EXTRACT     RUNNING     EXTA        00:00:00      00:00:04

目标端:

GGSCI (slavedb DBLOGIN as ogg) 23> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
REPLICAT    RUNNING     REPA        01:14:40      00:00:00

(9、2)源端手动插入数据,查看目标端是否同步

源端执行:

insert into test03 values(1,'a');

目标端查看数据是否同步过来:

mysql> use testdb_repl ;
Database changed

mysql> select * from test03;
+-----+------+
| id3 | name |
+-----+------+
|   1 | a    |
+-----+------+
1 row in set (0.04 sec)

【完】

文章永久链接:https://tech.souyunku.com/35365

未经允许不得转载:搜云库技术团队 » 使用GoldenGate完成MySQL到MySQL的同步

JetBrains 全家桶,激活、破解、教程

提供 JetBrains 全家桶激活码、注册码、破解补丁下载及详细激活教程,支持 IntelliJ IDEA、PyCharm、WebStorm 等工具的永久激活。无论是破解教程,还是最新激活码,均可免费获得,帮助开发者解决常见激活问题,确保轻松破解并快速使用 JetBrains 软件。获取免费的破解补丁和激活码,快速解决激活难题,全面覆盖 2024/2025 版本!

联系我们联系我们