High-Availability at MySQL Central

This year’s MySQL Central at Oracle Open World was an exhilarating experience. In contrast to the previous year’s MySQL Connect events, MySQL have now got their own Central at the main Oracle Open World. In the previous years, we were always short on time and trying to get a lot of sessions into just two days was just to much. This time I could both present sessions, attend sessions by other users, and also to talk to people in the MySQL community: something that I really enjoy and also find very valuable to see where we should be heading.

This year, the “MySQL Fabric Team” representation on MySQL Central was me and Narayanan Venkateswaran, which is heading the sharding solution in MySQL Fabric. Together with the conference, we also released MySQL Fabric 1.5.2 as the GA release of MySQL Fabric 1.5 containing a few new features:

  • Server Provisioning support was developed and added to MySQL Fabric 1.5 by Alfranio Correia. This allow us to integrate OpenStack, some other cloud solutions, or in general any asset management system, with MySQL Fabric making it possible to automatically provision new machines and incorporate them into the MySQL Fabric farm. When talking to people and seeing the existing deployments we realized this is critical for operations, so we decided on implementing support for it.
  • In addition, Narayanan extended range sharding using DATETIME and string sharding keys. Before 1.5 there were only support in range sharding for integral keys, where hash sharding supported arbitrary keys, but now you can use range sharding to shard based on datetimes or columns that hold strings such as CHAR, VARCHAR, or BLOB.
  • MySQL-RPC support was added by Geert Vanderkelen so that it is now possible to use the MySQL protocol to talk to the MySQL Fabric node. This is critical to be able to implement Fabric support in connectors that do not have easy access to an HTTP and XML library, such as the C or C++ connector.
  • A Fabric-aware Connector/C was developed by Rafal Somla, Igor Solodovnikov, and Bogdan Degtyariov in the connector team and was distributed as a labs release. The C connector is base for many other connectors such as MySQL/Ruby connector and the Perl connector DBD::mysql, and this has been requested by several people, so we decided to prioritize implementing this. You can find more information in the blog Using Connector/C with Fabric.
  • A Fabric-aware Connector/.NET was developed by Roberto Ezequiel Garcia Ballesteros and Fernando Gonzalez Sanchez and increase our connector support for MySQL Fabric.
  • MySQL Workbench 6.2 have also been released with some support for Fabric so that you can view the structure of a MySQL Fabric farm and the status of the groups and the servers in the farm.

We had several sessions on MySQL Fabric-related topics:

There were a lot of interest in MySQL Fabric and you also had a lot of good questions and comments that will help us going forward. With release 1.5 of MySQL Fabric in GA, we now have a lot of support for managing a farm:

  • High-availability support with automatic fail-over of servers.
  • Range and hash sharding with support for datetime, strings, and integers.
  • Sharding operations to move and split shards with minimal locking time.
  • Integration with OpenStack and new interface to support adding extensions for integrating with other cloud or asset management systems.
  • Easy-to-use command-line interface for managing a farm of MySQL server.
  • Support for custom failure detectors so that you can use your own failure detector with MySQL Fabric.
  • Clear and easy to use APIs in XML-RPC and MySQL-RPC (RPC over MySQL protcol) so that you can write your own applications to automate whatever tasks you need done.
  • Fabric-aware connectors for Java, PHP, .NET, C, and Python.

However, it does not stop there. From the questions and the people I talked to on MySQL Central, it seems like the most important topics are:

  • Ease of use. Many questions were about how to do various tasks, for example: how to add new shards to a hash-sharded system to balance the load, how to repair and bring back servers into the farm, how to provision new slaves, and what changes to make to the application to use MySQL Fabric. Technically, this is already possible, but it can be easier and more intuitive to use. Ease of use is a continuous work we do and something that we always need to improve both in how to use the tools but also through providing good documentation and information on the all the features we have.
  • High-Availability of the Fabric system itself is a topic that came up quite frequently. We gave a solution for this in the “High-Availability on Different Levels” session above, but we need to ensure that we have good documentation for this so that is it clear what needs to be done. And this is, of course, always possible to improve on through introducing solutions that are easier to deploy and use.
  • Integrating with other cloud system than OpenStack was also raised a few times, so it seems like having a broader support for various cloud solutions (and probably some asset management systems that are not cloud based) is important to do.
  • Multi-cast for a sharded system was often requested directly or indirectly. This can be useful for reporting application where you need to collect data from all shards and aggregate information about it.

Does this give an accurate picture of what you consider the most needed items going forward? Do you have any other feature that you want to mention explicitly?

About mats.kindahl@oracle.com

Mats is team lead and head architect for the MySQL High-Availability Team, which maintain and release MySQL Fabric. Mats has been working for MySQL since 2004 and has implemented a number of server features, mainly in replication. Prior to joining MySQL, Mats worked with implementing C and C++ compilers and as a researcher developing algorithms for verification of distributed systems.

3 thoughts on “High-Availability at MySQL Central

  1. Does MySQL Fabric support a query with aggregation functions across shards? For example, I transform my original employee table into two shards hashed by employee id and I would like to sum the total number of employee of the two shards with MySQL Fabric.
    If I issue “select count(*) from employee” as a single command to MySQL Fabric, can MySQL Fabric split the command into two shards, count the number employee of each shard and return back the aggregated result?

    1. Currently, neither the connectors nor the router support shard aggregation to do what you describe above. To do that, you would have to issue a request in the application to the Fabric node and ask for all the shards and then dispatch the query to the shards.

      There are several levels of support for shard aggregation that we are considering. The first level is a simple dispatch to all and compute the union, but more complex queries, especially involving aggregation functions such as MAX or SUM will require more advanced support.

Leave a Reply

Your email address will not be published. Required fields are marked *

Please enter. * Time limit is exhausted. Please reload CAPTCHA.