{"action":"created","issue":{"url":"https://api.github.com/repos/openebs/openebs/issues/803","repository_url":"https://api.github.com/repos/openebs/openebs","labels_url":"https://api.github.com/repos/openebs/openebs/issues/803/labels{/name}","comments_url":"https://api.github.com/repos/openebs/openebs/issues/803/comments","events_url":"https://api.github.com/repos/openebs/openebs/issues/803/events","html_url":"https://github.com/openebs/openebs/pull/803","id":269477624,"number":803,"title":"Include deployment YAMLs for percona synchronous multi-master replication cluster (Galera)","user":{"login":"ksatchit","id":21166217,"avatar_url":"https://avatars0.githubusercontent.com/u/21166217?v=4","gravatar_id":"","url":"https://api.github.com/users/ksatchit","html_url":"https://github.com/ksatchit","followers_url":"https://api.github.com/users/ksatchit/followers","following_url":"https://api.github.com/users/ksatchit/following{/other_user}","gists_url":"https://api.github.com/users/ksatchit/gists{/gist_id}","starred_url":"https://api.github.com/users/ksatchit/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ksatchit/subscriptions","organizations_url":"https://api.github.com/users/ksatchit/orgs","repos_url":"https://api.github.com/users/ksatchit/repos","events_url":"https://api.github.com/users/ksatchit/events{/privacy}","received_events_url":"https://api.github.com/users/ksatchit/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"ksatchit","id":21166217,"avatar_url":"https://avatars0.githubusercontent.com/u/21166217?v=4","gravatar_id":"","url":"https://api.github.com/users/ksatchit","html_url":"https://github.com/ksatchit","followers_url":"https://api.github.com/users/ksatchit/followers","following_url":"https://api.github.com/users/ksatchit/following{/other_user}","gists_url":"https://api.github.com/users/ksatchit/gists{/gist_id}","starred_url":"https://api.github.com/users/ksatchit/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ksatchit/subscriptions","organizations_url":"https://api.github.com/users/ksatchit/orgs","repos_url":"https://api.github.com/users/ksatchit/repos","events_url":"https://api.github.com/users/ksatchit/events{/privacy}","received_events_url":"https://api.github.com/users/ksatchit/received_events","type":"User","site_admin":false},"assignees":[{"login":"ksatchit","id":21166217,"avatar_url":"https://avatars0.githubusercontent.com/u/21166217?v=4","gravatar_id":"","url":"https://api.github.com/users/ksatchit","html_url":"https://github.com/ksatchit","followers_url":"https://api.github.com/users/ksatchit/followers","following_url":"https://api.github.com/users/ksatchit/following{/other_user}","gists_url":"https://api.github.com/users/ksatchit/gists{/gist_id}","starred_url":"https://api.github.com/users/ksatchit/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ksatchit/subscriptions","organizations_url":"https://api.github.com/users/ksatchit/orgs","repos_url":"https://api.github.com/users/ksatchit/repos","events_url":"https://api.github.com/users/ksatchit/events{/privacy}","received_events_url":"https://api.github.com/users/ksatchit/received_events","type":"User","site_admin":false}],"milestone":null,"comments":2,"created_at":"2017-10-30T04:18:07Z","updated_at":"2017-11-02T02:14:10Z","closed_at":null,"author_association":"OWNER","pull_request":{"url":"https://api.github.com/repos/openebs/openebs/pulls/803","html_url":"https://github.com/openebs/openebs/pull/803","diff_url":"https://github.com/openebs/openebs/pull/803.diff","patch_url":"https://github.com/openebs/openebs/pull/803.patch"},"body":"<!--  Thanks for sending a pull request!  Here are some tips for you -->\r\n\r\n**What this PR does / why we need it**:\r\n\r\nAdded the cluster service and deployment specification YAMLs for the synchronous-multi master percona xtradb-mysql cluster (Galera), with the xtradb nodes using OpenEBS storage as the mysql datadir.\r\n\r\n**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #460 \r\n\r\n**Special notes for your reviewer**:\r\n\r\nGalera Cluster\r\n----------------\r\nPercona XtraDB Cluster is an active/active high availability and high scalability open source solution for MySQL clustering. It integrates Percona Server and Percona XtraBackup with the Codership Galera library of MySQL high availability solutions in a single package\r\n\r\nThis PR provides a set of simple k8s deployment specification YAMLs to setup this cluster. Together, it constitutes :\r\n\r\n- A cluster service YAML which can be used for client connections (pxc-cluster) \r\n- The node deployment and service spec YAMLs to setup a 3-node replication cluster (pxc-node)\r\n\r\nThe images used in these pods is  capttofu/percona_xtradb_cluster_5_6:beta. When the deployment is created, it results starting corresponding container, run an entrypoint script that installs the MySQL system tables, set up users, and build up a list of servers that is used with the galera parameter ```wsrep_cluster_address```. This is a list of running nodes that galera uses for election of a node to obtain ```SST (Single State Transfer)``` from.\r\n\r\nUsage guidelines\r\n-------------------\r\n\r\n- The galera cluster is recommended to be created with at least 3 nodes/replicas. See here for details : \r\nhttps://www.percona.com/blog/2015/06/23/percona-xtradb-cluster-pxc-how-many-nodes-do-you-need/\r\n\r\n- It is important to first deploy the service/pod for primary node and wait for it to be processed before starting the secondary/other nodes. Deploying all YAMLs together can cause the pods to be restarted repeatedly. As per kubernetes documentation, the reason is : \r\n\r\n  \"If there is a node in wsrep_clsuter_address without a backing galera node there will be nothing to obtain SST from which will cause the node to shut itself down and the container in question to exit and relaunch\"\r\n\r\nA statefulset for the same will be attempted under a different issue (currently seeing some issues w/ dynamic provisioning of vols , see https://github.com/kubernetes/kubernetes/issues/23828)  \r\n\r\nSteps & Outputs\r\n-------------------\r\n\r\n1. Apply the pxc-cluster service YAML\r\n\r\n ```kubectl apply -f pxc-cluster-service.yaml```\r\n\r\n2. Apply the pxc-node1 service & deployment YAML\r\n\r\n ```kubectl apply -f pxc-node1.yaml```\r\n\r\n3. Check that the PVCs are created and the pxc-node1 enters \"Running\" status \r\n\r\n```\r\nkarthik@MayaMaster:~/galera-deployment$ kubectl get pods\r\nNAME                                                             READY     STATUS    RESTARTS   AGE\r\nmaya-apiserver-2245240594-r7mj7                                  1/1       Running   0          2d\r\nopenebs-provisioner-4230626287-nr6h4                             1/1       Running   0          2d\r\npvc-23de7dd2-bd1c-11e7-9be8-000c298ff5fc-ctrl-1918293941-x2g56   1/1       Running   0          8m\r\npvc-23de7dd2-bd1c-11e7-9be8-000c298ff5fc-rep-2730456691-5x5cf    1/1       Running   0          8m\r\npvc-23de7dd2-bd1c-11e7-9be8-000c298ff5fc-rep-2730456691-q780h    1/1       Running   0          8m\r\npxc-node1-2878682957-3vsv3                                       1/1       Running   0          8m\r\n```\r\n\r\n4.  Apply the pxc-node2  and pxc-node3 spec YAMLs \r\n\r\n```\r\nkubectl apply -f pxc-node2yaml\r\nkubectl apply -f pxc-node3.yaml\r\n```\r\n\r\n5. Verify that the node2 and node3 pods are able to instantiate successfully\r\n\r\n```\r\nkarthik@MayaMaster:~/galera-deployment$ kubectl get pods\r\nNAME                                                             READY     STATUS    RESTARTS   AGE\r\nmaya-apiserver-2245240594-r7mj7                                  1/1       Running   0          2d\r\nopenebs-provisioner-4230626287-nr6h4                             1/1       Running   0          2d\r\npvc-235b15a5-bd1f-11e7-9be8-000c298ff5fc-ctrl-2525793473-nv88z   1/1       Running   0          12m\r\npvc-235b15a5-bd1f-11e7-9be8-000c298ff5fc-rep-144100677-2rm8b     1/1       Running   0          12m\r\npvc-235b15a5-bd1f-11e7-9be8-000c298ff5fc-rep-144100677-gmn51     1/1       Running   0          12m\r\npvc-82885a9c-bd1e-11e7-9be8-000c298ff5fc-ctrl-2555717164-sspqn   1/1       Running   0          16m\r\npvc-82885a9c-bd1e-11e7-9be8-000c298ff5fc-rep-3501200001-p9778    1/1       Running   0          16m\r\npvc-82885a9c-bd1e-11e7-9be8-000c298ff5fc-rep-3501200001-x3nxs    1/1       Running   0          16m\r\npvc-cc94c5eb-bd1f-11e7-9be8-000c298ff5fc-ctrl-15702123-tn460     1/1       Running   0          7m\r\npvc-cc94c5eb-bd1f-11e7-9be8-000c298ff5fc-rep-4137665767-0lhjb    1/1       Running   0          7m\r\npvc-cc94c5eb-bd1f-11e7-9be8-000c298ff5fc-rep-4137665767-h8r6j    1/1       Running   0          7m\r\npxc-node1-2984138107-zjf22                                       1/1       Running   0          16m\r\npxc-node2-1007987438-q831l                                       1/1       Running   0          12m\r\npxc-node3-82203929-mh5p9                                         1/1       Running   0          7m\r\n```\r\n\r\nTest replication in the Galera Cluster\r\n----------------------------------------\r\n\r\n1. Check the replication cluster size on any of the nodes\r\n\r\n```\r\nmysql> show status like 'wsrep_cluster_size';\r\n+--------------------+-------+\r\n| Variable_name      | Value |\r\n+--------------------+-------+\r\n| wsrep_cluster_size | 3     |\r\n+--------------------+-------+\r\n1 row in set (0.01 sec)\r\n```\r\n\r\n2. On the pxc-node1, create a test database with some content\r\n\r\n```\r\nkarthik@MayaMaster:~/galera-deployment$ kubectl exec -it pxc-node1-2984138107-zjf22 /bin/bash\r\nroot@pxc-node1-2984138107-zjf22:/# mysql -uroot -p -h pxc-cluster\r\nEnter password:\r\nWelcome to the MySQL monitor.  Commands end with ; or \\g.\r\nYour MySQL connection id is 5\r\nServer version: 5.6.24-72.2-56-log Percona XtraDB Cluster (GPL), Release rel72.2, Revision 43abf03, WSREP version 25.11, wsrep_25.11\r\n\r\nCopyright (c) 2009-2015 Percona LLC and/or its affiliates\r\nCopyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.\r\n\r\nOracle is a registered trademark of Oracle Corporation and/or its\r\naffiliates. Other names may be trademarks of their respective\r\nowners.\r\n\r\nType 'help;' or '\\h' for help. Type '\\c' to clear the current input statement.\r\n\r\nmysql> create database testdb;\r\nQuery OK, 1 row affected (0.10 sec)\r\n\r\nmysql> use testdb;\r\nDatabase changed\r\n\r\nmysql> CREATE TABLE Hardware (Name VARCHAR(20),HWtype VARCHAR(20),Model VARCHAR(20));\r\nQuery OK, 0 rows affected (0.11 sec)\r\n\r\nmysql> INSERT INTO Hardware (Name,HWtype,Model) VALUES ('TestBox','Server','DellR820');\r\nQuery OK, 1 row affected (0.06 sec)\r\n\r\nmysql> select * from Hardware;\r\n+---------+--------+----------+\r\n| Name    | HWtype | Model    |\r\n+---------+--------+----------+\r\n| TestBox | Server | DellR820 |\r\n+---------+--------+----------+\r\n1 row in set (0.00 sec)\r\n\r\nmysql> exit\r\nBye\r\n```\r\n3. Verify that this data is synced on the other nodes. Say, node2\r\n\r\n```\r\nkarthik@MayaMaster:~/galera-deployment$ kubectl exec -it pxc-node2-1007987438-q831l /bin/bash\r\nroot@pxc-node2-1007987438-q831l:/#\r\nroot@pxc-node2-1007987438-q831l:/#\r\nroot@pxc-node2-1007987438-q831l:/# mysql -uroot -p -h pxc-cluster\r\nEnter password:\r\nWelcome to the MySQL monitor.  Commands end with ; or \\g.\r\nYour MySQL connection id is 4\r\nServer version: 5.6.24-72.2-56-log Percona XtraDB Cluster (GPL), Release rel72.2, Revision 43abf03, WSREP version 25.11, wsrep_25.11\r\n\r\nCopyright (c) 2009-2015 Percona LLC and/or its affiliates\r\nCopyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.\r\n\r\nOracle is a registered trademark of Oracle Corporation and/or its\r\naffiliates. Other names may be trademarks of their respective\r\nowners.\r\n\r\nType 'help;' or '\\h' for help. Type '\\c' to clear the current input statement.\r\n\r\nmysql> show databases;\r\n+--------------------+\r\n| Database           |\r\n+--------------------+\r\n| information_schema |\r\n| mysql              |\r\n| performance_schema |\r\n| test               |\r\n| testdb             |\r\n+--------------------+\r\n5 rows in set (0.00 sec)\r\nmysql> use testdb;\r\nDatabase changed\r\nmysql> show tables;\r\n+------------------+\r\n| Tables_in_testdb |\r\n+------------------+\r\n| Hardware         |\r\n+------------------+\r\n1 row in set (0.00 sec)\r\n\r\nmysql> select * from Hardware;\r\n+---------+--------+----------+\r\n| Name    | HWtype | Model    |\r\n+---------+--------+----------+\r\n| TestBox | Server | DellR820 |\r\n+---------+--------+----------+\r\n1 row in set (0.00 sec)\r\n\r\nmysql> exit\r\nBye\r\n```\r\n\r\n4. Now, verify the multi-master capability of the cluster, by writing some additional tables into the db from, say, node3\r\n\r\n```\r\nkarthik@MayaMaster:~/galera-deployment$ kubectl exec -it pxc-node3-82203929-mh5p9 /bin/bash\r\nroot@pxc-node3-82203929-mh5p9:/#\r\nroot@pxc-node3-82203929-mh5p9:/#\r\nroot@pxc-node3-82203929-mh5p9:/# mysql -uroot -p -h pxc-cluster;\r\nEnter password:\r\nWelcome to the MySQL monitor.  Commands end with ; or \\g.\r\nYour MySQL connection id is 6\r\nServer version: 5.6.24-72.2-56-log Percona XtraDB Cluster (GPL), Release rel72.2, Revision 43abf03, WSREP version 25.11, wsrep_25.11\r\n\r\nCopyright (c) 2009-2015 Percona LLC and/or its affiliates\r\nCopyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.\r\n\r\nOracle is a registered trademark of Oracle Corporation and/or its\r\naffiliates. Other names may be trademarks of their respective\r\nowners.\r\n\r\nType 'help;' or '\\h' for help. Type '\\c' to clear the current input statement.\r\n\r\nmysql>\r\n\r\nmysql> show databases;\r\n+--------------------+\r\n| Database           |\r\n+--------------------+\r\n| information_schema |\r\n| mysql              |\r\n| performance_schema |\r\n| test               |\r\n| testdb             |\r\n+--------------------+\r\n5 rows in set (0.00 sec)\r\n\r\nmysql> use testdb;\r\nReading table information for completion of table and column names\r\nYou can turn off this feature to get a quicker startup with -A\r\n\r\nDatabase changed\r\nmysql>\r\n\r\nmysql> INSERT INTO Hardware (Name,HWtype,Model) VALUES ('ProdBox','Server','DellR720');\r\nQuery OK, 1 row affected (0.03 sec)\r\n\r\nmysql> select * from Hardware;\r\n+---------+--------+----------+\r\n| Name    | HWtype | Model    |\r\n+---------+--------+----------+\r\n| TestBox | Server | DellR820 |\r\n| ProdBox | Server | DellR720 |\r\n+---------+--------+----------+\r\n2 rows in set (0.00 sec)\r\n\r\nmysql> exit\r\nBye\r\n```\r\n\r\n\r\n\r\n \r\n"},"comment":{"url":"https://api.github.com/repos/openebs/openebs/issues/comments/341297753","html_url":"https://github.com/openebs/openebs/pull/803#issuecomment-341297753","issue_url":"https://api.github.com/repos/openebs/openebs/issues/803","id":341297753,"user":{"login":"ksatchit","id":21166217,"avatar_url":"https://avatars0.githubusercontent.com/u/21166217?v=4","gravatar_id":"","url":"https://api.github.com/users/ksatchit","html_url":"https://github.com/ksatchit","followers_url":"https://api.github.com/users/ksatchit/followers","following_url":"https://api.github.com/users/ksatchit/following{/other_user}","gists_url":"https://api.github.com/users/ksatchit/gists{/gist_id}","starred_url":"https://api.github.com/users/ksatchit/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ksatchit/subscriptions","organizations_url":"https://api.github.com/users/ksatchit/orgs","repos_url":"https://api.github.com/users/ksatchit/repos","events_url":"https://api.github.com/users/ksatchit/events{/privacy}","received_events_url":"https://api.github.com/users/ksatchit/received_events","type":"User","site_admin":false},"created_at":"2017-11-02T02:14:10Z","updated_at":"2017-11-02T02:14:10Z","author_association":"OWNER","body":"@mahebbar , have reworked the document based on changes suggested"}}
{"actor":{"display_login":"ksatchit"}}