A Strategy of Data Synchronization in Distributed System with Read Separating from Write

一种面向分布式读写分离系统的数据同步策略

School of Computer Science, Northwestern Polytechnical University, Xi'an 710072, China

Received: 26 February 2019

Abstract

Read separating from write is a strategy that NewSQL adopts to incorporate the advantages of traditional relation database and NoSQL database. Under this architecture, baseline data is split into multiple partitions stored at distributed physical nodes, while delta data is stored at single transaction node. For reducing the pressure of transaction node and improving the query performance, delta data needs to be synchronized into storage nodes. The current strategies trigger the procedure of data synchronization per partition, meaning that unchanged partitions will also participate in data synchronization, which consumes extra network cost, local IO and space resources. For improving the efficiency of data synchronization meanwhile mitigating space utilization, the fine-grained data synchronization strategy is proposed, whose main idea includes that fine-grained logical partitions upon original coarse-grained partitions is established, providing more correct synchronized unit; the delta data sensing strategy is introduced, which records the mapping between changed partitions and its delta data; instead of partition driven, the data synchronization through the delta-broadcasting mechanism is driven, constraining that only changed partitions can participate in data synchronization. The fine-grained data synchronization strategy on Oceanbase is implemented, which is a distributed database with read separating from write, and the results show that our strategy is better than other strategies in efficiency of data synchronizing and space utilization.

摘要

读写分离是NewSQL数据库兼容传统关系型数据库以及NoSQL数据库各自优势的一种常用策略。这种架构下，基线数据被分割为多个分区分布存储于不同存储节点，更改数据存储于单个事务节点，为减轻事务节点压力以及提升查询效率，需要将更改数据定期同步到存储节点。当前策略以分区粒度进行数据同步，导致无更改数据的分区同样参与同步操作，消耗额外网络代价、本地IO代价、内存空间以及磁盘空间。为提升同步效率，降低空间消耗，提出一种细粒度数据同步策略，在原始分区之上建立细粒度逻辑分区，提供更精确的同步单位；引入更改感知策略，记录被更改的分区以及对应的更改数据；利用更改发布机制驱动同步的进行，限制参与同步的分区为发生改变的分区。在分布式读写分离系统Oceanbase上验证细粒度数据同步策略，结果表明其同步效率和空间占用量均优于其他策略。