{"action":"created","issue":{"url":"https://api.github.com/repos/pilosa/pilosa/issues/321","repository_url":"https://api.github.com/repos/pilosa/pilosa","labels_url":"https://api.github.com/repos/pilosa/pilosa/issues/321/labels{/name}","comments_url":"https://api.github.com/repos/pilosa/pilosa/issues/321/comments","events_url":"https://api.github.com/repos/pilosa/pilosa/issues/321/events","html_url":"https://github.com/pilosa/pilosa/issues/321","id":208231134,"number":321,"title":"Proposal: Add State and status to Nodes","user":{"login":"raskle","id":585044,"avatar_url":"https://avatars1.githubusercontent.com/u/585044?v=4","gravatar_id":"","url":"https://api.github.com/users/raskle","html_url":"https://github.com/raskle","followers_url":"https://api.github.com/users/raskle/followers","following_url":"https://api.github.com/users/raskle/following{/other_user}","gists_url":"https://api.github.com/users/raskle/gists{/gist_id}","starred_url":"https://api.github.com/users/raskle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/raskle/subscriptions","organizations_url":"https://api.github.com/users/raskle/orgs","repos_url":"https://api.github.com/users/raskle/repos","events_url":"https://api.github.com/users/raskle/events{/privacy}","received_events_url":"https://api.github.com/users/raskle/received_events","type":"User","site_admin":false},"labels":[{"id":632945016,"url":"https://api.github.com/repos/pilosa/pilosa/labels/cluster","name":"cluster","color":"985ee0","default":false},{"id":556208471,"url":"https://api.github.com/repos/pilosa/pilosa/labels/proposal","name":"proposal","color":"e99695","default":false}],"state":"closed","locked":false,"assignee":{"login":"travisturner","id":410544,"avatar_url":"https://avatars3.githubusercontent.com/u/410544?v=4","gravatar_id":"","url":"https://api.github.com/users/travisturner","html_url":"https://github.com/travisturner","followers_url":"https://api.github.com/users/travisturner/followers","following_url":"https://api.github.com/users/travisturner/following{/other_user}","gists_url":"https://api.github.com/users/travisturner/gists{/gist_id}","starred_url":"https://api.github.com/users/travisturner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/travisturner/subscriptions","organizations_url":"https://api.github.com/users/travisturner/orgs","repos_url":"https://api.github.com/users/travisturner/repos","events_url":"https://api.github.com/users/travisturner/events{/privacy}","received_events_url":"https://api.github.com/users/travisturner/received_events","type":"User","site_admin":false},"assignees":[{"login":"travisturner","id":410544,"avatar_url":"https://avatars3.githubusercontent.com/u/410544?v=4","gravatar_id":"","url":"https://api.github.com/users/travisturner","html_url":"https://github.com/travisturner","followers_url":"https://api.github.com/users/travisturner/followers","following_url":"https://api.github.com/users/travisturner/following{/other_user}","gists_url":"https://api.github.com/users/travisturner/gists{/gist_id}","starred_url":"https://api.github.com/users/travisturner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/travisturner/subscriptions","organizations_url":"https://api.github.com/users/travisturner/orgs","repos_url":"https://api.github.com/users/travisturner/repos","events_url":"https://api.github.com/users/travisturner/events{/privacy}","received_events_url":"https://api.github.com/users/travisturner/received_events","type":"User","site_admin":false}],"milestone":null,"comments":1,"created_at":"2017-02-16T20:07:37Z","updated_at":"2017-09-08T14:22:58Z","closed_at":"2017-09-08T14:22:58Z","author_association":"CONTRIBUTOR","body":"@raskle commented on [Thu Feb 16 2017](https://github.com/pilosa/general/issues/14)\n\n# Proposal: Add State and status to Nodes\r\n\r\nAuthor: Michael Baird\r\n\r\nLast updated: 2017/02/16\r\n\r\n## Abstract \r\n  Pilosa nodes are going to need various operational states other than up/down.  These states are necessary are we prepare Pilosa to dynamically grow, shrink, and replace nodes.  \r\n\r\n## Background\r\n  As Pilosa goes into production we will need the ability to make changes to the cluster while it is running.  To facilitate this we will need to define a set of states of behavior for a node.  These sets of behavior will define the life cycle of a node from: birth, growth, reproduction, and death.  For a new node to join an existing cluster it will need different set of behaviors during this phase and will need to communicate this state to the rest of the cluster\r\n\r\n## Proposal\r\n* Each node in a pilosa cluster needs to maintain a state and return its current status\r\n* A pilosa node can be one of the following states\r\n  - Normal: connected to the rest of the cluster and serving all requests\r\n  - Leaving: In the process of leaving the cluster, no longer responding to SetBit requests.  Still responding to Sync endpoints to transfer data\r\n  - Joining: In the process of joining the cluster, not responding to Bitmap queries but can respond the SetBit.\r\n  - Moving: In the process of redistributing Slice Fragments across the cluster, but this node will remain with new hash assignments\r\n\r\nDefine a list of operations allowed by each state\r\n  * Normal: \r\n    -all\r\n  * Leaving: \r\n    - /export\r\n    - /slices/max\r\n    - /fragment/nodes\r\n    - /fragment/data\r\n    - /fragment/blocks\r\n    - /fragment/block/data\r\n    - /nodes\r\n    - /version\r\n    - /status\r\n  * Joining\r\n    - /schema\r\n    - /import\r\n    - /export\r\n    - /slices/max\r\n    - /db\r\n    - /db/time_quantum\r\n    - /db/attr/diff\r\n    - /frame\r\n    - /frame/time_quantum\r\n    - /frame/attr/diff\r\n    - /fragment/nodes\r\n    - /fragment/data\r\n    - /fragment/blocks\r\n    - /fragment/block/data\r\n    - /frame/restore\r\n    - /nodes\r\n    - /version\r\n    - /status\r\n  * Moving\r\n    - I believe this can respond to all requests, but we may need to limited access\r\n\r\n## Compatibility\r\n  Node state maintenance is in preperation for new features to support a dynamic cluster.  \r\n  That being said internode communication is a core feature of Pilosa operating normally\r\n  We will need to ensure that cluster can respond to queries and data ingestion while some nodes are in the Leaving/Joining.  This will require that a minimum replication factor of 2 is in force.\r\n  If not we may want to consider any state other than normal \"under maintenance\" and return a 503 status code on affected endpoints.\r\n\r\n  Node Health is another resource we should cover in separate proposal.\r\n\r\n## Implementation\r\n  We will need to create a new /status API endpoint \r\n    This we be able to return the current node's status or the status of all nodes in the cluster\r\n\r\n  Through periodic Gossip communication across the cluster each node should maintain a status of other nodes it attempts to communicate with\r\n\r\n  We will need to maintain an API permission matrix for each state.\r\n  Proper status codes will be returned for unavailable resources\r\n\n\n"},"comment":{"url":"https://api.github.com/repos/pilosa/pilosa/issues/comments/328116791","html_url":"https://github.com/pilosa/pilosa/issues/321#issuecomment-328116791","issue_url":"https://api.github.com/repos/pilosa/pilosa/issues/321","id":328116791,"user":{"login":"travisturner","id":410544,"avatar_url":"https://avatars3.githubusercontent.com/u/410544?v=4","gravatar_id":"","url":"https://api.github.com/users/travisturner","html_url":"https://github.com/travisturner","followers_url":"https://api.github.com/users/travisturner/followers","following_url":"https://api.github.com/users/travisturner/following{/other_user}","gists_url":"https://api.github.com/users/travisturner/gists{/gist_id}","starred_url":"https://api.github.com/users/travisturner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/travisturner/subscriptions","organizations_url":"https://api.github.com/users/travisturner/orgs","repos_url":"https://api.github.com/users/travisturner/repos","events_url":"https://api.github.com/users/travisturner/events{/privacy}","received_events_url":"https://api.github.com/users/travisturner/received_events","type":"User","site_admin":false},"created_at":"2017-09-08T14:22:58Z","updated_at":"2017-09-08T14:22:58Z","author_association":"CONTRIBUTOR","body":"I'm going to close this as it's part of the work going into #737"}}
{"actor":{"display_login":"travisturner"}}