Working of Certification based Replication in Galera Cluster

galera-cluster
In the earlier articles, we have covered the basics of Galera Cluster for MySQL and MariaDB and another article about MariaDB Galera Cluster in particular. To recall the overview of Galera Cluster – It is a synchronous multi-master cluster that uses the InnoDB storage engine (XtraDB also for MariaDB). It is actually the Galera replication plugin that extends the wsrep API of the underlying DBMS, MySQL or MariaDB. Galera Cluster uses Certification based synchronous replication in the multi master server setup. In this article we will look into the technical aspects of Galera Cluster Certification based replication functionality.
Certification based Database Replication
Database replication allows setting up of synchronized master-slave or master-master database clusters. Since data is synchronized across the individual databases also called nodes, this setup is fault proof with failover since a failed node is replaced by other nodes until the former’s recovery, thus ensuring high availability. The type of database replication can be either Synchronous or Asynchronous. In synchronous replication the transactions are committed at all nodes concurrently and in asynchronous replication, target nodes receive new transaction set from source node after a negligible lag from original transaction. Synchronous replication ensures that all nodes are in same state (transactions being committed at same time) and thus there are high availability 24/7 with consistent replicas and no data loss during individual node crashes. Another advantage is improved performance because clients can always perform READ/WRITEs at any nodes irrespective of where the transaction originated.
The disadvantage with Synchronous replication is the increased lock time and possible conflicts arose due to parallel transaction commits at all nodes. This can affect performance when number of nodes and transactions increases. Asynchronous transaction solves this problem by allowing each node to independently commit transactions that they have received from the source node. This prevents concurrent locks and ensures more availability with large number of nodes and transactions. The performance issues with synchronous replication are solved by Certification based synchronous replication in Galera Cluster. This replication is based on:
1. Group communication model that define a pattern for inter-node communication.
2. Write-sets that groups or bundles a set of WRITES as a single transaction message to be applied to individual nodes.
3. Database state machine process that treats READ/WRITE transactions on one database as its state at a given time and then broadcasts the transaction to other nodes to change their state to that of the source node’s state.
4. Transaction reordering to reorder transactions that are either not certified or not committed due to a node failure. Transactions are re-ordered so that they are not lost.
The Certification process uses a global coordinated certification scheme in which transaction from the source node is broadcasted to other nodes using global total ordering of concurrent transactions to achieve global consistency. Certification based replication works with databases having the following features:
1. Transactional database with COMMIT and ROLLBACK capability.
2. Atomic changes capable database that accepts entire transaction to be applied for COMMIT else no COMMIT at all.
3. Global ordering capable database that can undergo global ordering of replication events or transactions.
Working of Certification based Replication
When a transaction (series of changes) occurs at a database node and it issues a COMMIT, before the commit actually takes place, all the changes or WRITES/UPDATES/ALTERS occurred at the database node along with the modified rows’ PRIMARY KEYS are collected as a write-set. The source node then broadcasts this write-set to all other nodes. Each node in the cluster, including the originating node then performs a deterministic certification test on the write-set using the PRIMARY KEYS in the write-set and the actual PRIMARY KEY values in the nodes. This test is to determine the key constraint integrity of the write-set. If the test fails, the originating node drops the write-set and the cluster rolls back the original transaction. If the certification test succeeds, the transaction commits and write-sets are applied to all nodes in the cluster, thus making the replication.
In Galera Cluster, each transaction is assigned a global ordinal sequence number. During the deterministic certification test on the write-set, the cluster checks the current transaction with the last successful transaction. If any transactions would have occurred in between these 2 globally ordered transactions, primary key conflicts will occur and the test fails. Upon a successful deterministic certification check, all replicas apply the transaction in the same order. Thus all nodes reach on a consensus about the outcome of the transaction and replication happens. On this, the originating node notifies the client about the successful transaction commit.
Galera Cluster Replication Plugin and wsrep API
The core of Galera replication is the Galera Replication Plugin that implements the write-set replication functionality. The DBMS (MySQL or MariaDB) that is used to setup the Galera Cluster uses the Galera Replication Plugin that implements an API called the Write Set Replication API or wsrep API. It is implemented as a replication plugin interface in the Galera Replication Plugin. Thus the Galera Replication Plugin is the replication or wsrep provider. It consists of 3 components:
1. Certification layer that prepares the write-sets and performs the certification tests.
2. Replication layer that manages the replication process and global ordering.
3. Group communication framework provides plugin architecture for the group communication systems in the Cluster.
 
The wsrep API considers the content including data and schema of a database as a state. When a client performs WRITE/UPDATE/ALTERs on the database, it is considered as a transaction which is nothing but the changes happened to the database represented as a series of atomic changes. To keep a consistent state across all nodes, the wsrep API uses a Global Transaction ID (GTID) for each transaction write-set. The GTID allows to identify state changes and to compare two states, the current and a previous one. In the Galera Cluster all nodes need to have the same state. The synchronization and replication needed to maintain consistency in state is performed using the GTID serial order. The GTID consists of 2 parts: a State UUID that identifies the state and sequence of changes happened, and an Ordinal Sequence Number (seqno) used to denote the position of the change in the sequence.
Eg: 45eec521-2f34-11e0-0800-2a36050b826b:94530586304
The Galera Cluster Replication Plugin uses the wsrep hooks defined in the DBMS to call the wsrep API. A function called dlopen() acts as a bridge between the wsrep provider (the Galera Replication Plugin and wsrep API) and the wsrep hooks in the DBMS. The state change or atomic changes grouped as a transaction write-set with a GTID is the key for implementing replication. Below are the steps performed to implement replication using the wsrep API.
1. At any node a change occurs, causing a state change.
2. The database invokes the replication provider using the wsrep hooks to create the write-set from the changes.
3. dlopen() connects the wsrep provider functions with the wsrep hooks in the database.
4. The Galera Replication Plugin performs the write-set certification and replication in the cluster.

Read more


Is Cloud Database Good for Your Business?

You have heard lot about Cloud now-a-days – cloud computing, cloud storage, cloud database, etc. but have you thought about how cloud technology can influence your business, the positive ones and negative sides? Also it is worth discussing how you can utilize cloud for the development of your business, overcoming any disadvantages associated with it. This article is trying to define what cloud database implies for your business and its advantages and disadvantages. Also it is trying to provoke discussions about the suitability of cloud database for you.
What is a Cloud Database?
A cloud database is a DBMS system deployed and running on a cloud computing platform. The underlying infrastructure, hardware, software installation, setup etc are provided by the cloud provider. Customers are provided with access to the cloud database they have purchased or subscribed through a service model that allows management of the database, requesting more computing, memory and network resources. This service model is called Database As A Service (DBaaS). The requested or demanded resources are immediately made available to the customer’s database, also called DB Instance. So cloud databases provide high availability, scalability, rapid provisioning and automated DBA.
Advantages of Cloud Databases
A cloud database offers the much needed advantages to your business in terms of cost, time, effort, expertise, planning, etc. The cloud vendor is a business that has costly implementations of distributed and distributable infrastructure, platform and resources that are used to provide you sophisticated database service. You along with other organizations consume his broad pool of resources as per demand and these results in advantages for you.
1. Reduction in IT expenditure: Businesses can avoid the initial investment in setting up the infrastructure and then the operational and administrative costs associated with maintaining an in-house database and networking system. What are not needed when you use a cloud database are – the hardware and platform software including OS, server rooms including furnishing and air-conditioning, salaries and other perks needed to sustain a proactive and smart DBA team, networking costs and periodical costs needed for hardware/software/infrastructure upgrades. What you need to pay is only the consumption or utilization costs for resources that you need at the moment, scalable to foreseeable future, like CPUs, memory, storage, bandwidth and the DBMS engine optionally.
2. Go with the latest technology: Technologies change with new innovations and releases. Database industry is highly competitive, especially due to the ever growing and unavoidable demand of databases for any organization. Whether it is the enterprise giants or an open source global community that is behind the database product, the product is evolving continuously with minor releases and patches to major version releases and even new product families that incorporate the latest in technology in terms of computing power, speed, scalability, performance and reliability. Contrary to an in-house database deployment where you need to spend money, time and effort to incorporate a new database technology, the cloud provider is doing this for you and you need to worry only about how to enhance your application to best utilize the capabilities of the upgraded or new version of the database.
3. Scalability: For an in-house database, you need money, time and effort to scale up the database beyond certain thresholds. This need for scalability can occur not only with a business growth, but when you accommodate additional operative or service projects in your organization or during a peak season when you experience a spike in traffic and incoming requests. With a cloud database provider and his distributed deployments, rich with pre-configured or raw resources, your only task is to demand provisioning of additional resources needed for your requirement. The rapid provisioning technologies associated with any cloud database will immediately bring them at your disposal for utilization.
4. Centralized management and reliability: Since the core technology infrastructure and platform is managed by the cloud vendor and automated programs perform the DBA tasks for your database(s), cloud offers significant benefits in terms of effort and expertise demanding maintenance and operational requirements. Your data is accessible for your applications through the distributed or replicated cloud database instances. The failover mechanisms ensure that there is no single point of failure for your cloud database and you have replicas or stand-by databases synchronized with the primary database to take charge in case of failures. This ensures reliability of operations and services for your organization.
Disadvantages with Cloud Databases
However there are some disadvantages that you need to know and consider when going for a cloud database. These may not be significant enough for you to hold off from choosing the cloud, but you can use this information to plan your consumptions, configurations and above all adapt policies that make use of the advantages of cloud more than letting you affected with the disadvantages.
1. Security: A cloud system, like any other network information system, is vulnerable to threats. While there are unlikely any internal threats or misconfigurations that are common with in-house deployments, there can be hackers and intruders who try to intrude into the system through application vulnerabilities or any other system loopholes. Moreover, cloud system hosts databases and other resources in shared or virtualized server environments in case of shared products. In such cases, your database may be in the same machine or network with other businesses. An improper security configuration or application vulnerability of another customer can compromise the server in which your database is hosted.
However this is less likely or probable due to the logical and platform based isolation the cloud vendor is providing for his customers. Latest containerization and virtualization technologies provide separate secure environments for each customer. In a business view point, you can mitigate or nullify this risk by securing your application, database configurations and setups that you can control, like strong passwords and hashes, using secure communication options instead of cheaper raw format data transmission etc. Also using a Private or Hybrid Cloud for sensitive and confidential data and a Public Cloud for operational and project data is a good decision to overcome these security concerns, especially for medium to big businesses.
2. Vendor Lock-In and limited control: Various cloud technologies differ in configuration at various levels of implementation like OS level, containerization, virtualization, replication, synchronization, optimization etc. So in a case when you want to migrate your database to another vendor, there can be compatibility or migration issues surface up due to these differences. This typically occurs when you attempt to migrate the entire database filespace, including the physical files, schema, configuration, implementation setup like replication parameters etc. An experienced DBA can easily migrate your database through a step by step or incremental approach with his planning and skills. Also, since cloud provider is providing you a virtual environment for your databases, access to the underlying OS, shell access or other firmware level routines are not provided to customers. Though these are not needed in most cases since you are getting the most optimized configuration, sometimes this can prevent vertical configuration scalability. Dedicated or Elastic Cloud based database instances are exceptions and grant you complete access.
3. Need of planning to select optimal packages : Though not a disadvantage in the first glimpse, this is a crucial factor to be considered and worked out before purchasing your first cloud database instance. Cloud database or cloud computing vendors provide a wide variety of packages, categorized and organized in hierarchies of resource availability and performance. The cost scale ranges from the simple package – a couple of GB memory, a dual core or quad core Virtual CPU and minimum bandwidth to enterprise level packages that are optimized for various options like General, Storage, Computing, Speed etc. While the temptation is always to save cost in the beginning, before going too much into the operational realm, you need to realize the optimum resource requirements and configuration your business needs for operation and scaling. Such a package must be selected to avoid unnecessary downtimes and DOS during crucial periods. Another factor is the selection between Single Availability Zone versus Multi Availability Zone (Multi-AZ) packages. While single zone packages are better for small businesses, most SMBs and higher businesses need Multi-AZ packages that have synchronized and standby database instances deployed in multiple geo-locations to ensure high availability and failover 24/7. Obviously these packages come at higher costs, but again significantly less than a privately owned multi location cloud.
The Verdict
As clearer from above points, cloud database’s advantages outweigh its disadvantages or have provisions and methods to make your database run on a carefully planned and selected cloud package. The savings in cost, time and money you will make along with the opportunity to dedicate your focus to development and growth rather than infrastructure and implementation, will benefit your organization in the near future itself. It is a matter of blending the cloud culture with that of your IT needs, where the difference lies whether cloud is suitable for your business or not.

Read more