{"action":"created","issue":{"url":"https://api.github.com/repos/kontena/kontena/issues/2011","repository_url":"https://api.github.com/repos/kontena/kontena","labels_url":"https://api.github.com/repos/kontena/kontena/issues/2011/labels{/name}","comments_url":"https://api.github.com/repos/kontena/kontena/issues/2011/comments","events_url":"https://api.github.com/repos/kontena/kontena/issues/2011/events","html_url":"https://github.com/kontena/kontena/pull/2011","id":215998645,"number":2011,"title":"Refactor agent to use Observable node info","user":{"login":"SpComb","id":3950373,"avatar_url":"https://avatars1.githubusercontent.com/u/3950373?v=3","gravatar_id":"","url":"https://api.github.com/users/SpComb","html_url":"https://github.com/SpComb","followers_url":"https://api.github.com/users/SpComb/followers","following_url":"https://api.github.com/users/SpComb/following{/other_user}","gists_url":"https://api.github.com/users/SpComb/gists{/gist_id}","starred_url":"https://api.github.com/users/SpComb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SpComb/subscriptions","organizations_url":"https://api.github.com/users/SpComb/orgs","repos_url":"https://api.github.com/users/SpComb/repos","events_url":"https://api.github.com/users/SpComb/events{/privacy}","received_events_url":"https://api.github.com/users/SpComb/received_events","type":"User","site_admin":false},"labels":[{"id":213547699,"url":"https://api.github.com/repos/kontena/kontena/labels/agent","name":"agent","color":"0052cc","default":false},{"id":190601902,"url":"https://api.github.com/repos/kontena/kontena/labels/enhancement","name":"enhancement","color":"bfdadc","default":true},{"id":484068465,"url":"https://api.github.com/repos/kontena/kontena/labels/status/design-review","name":"status/design-review","color":"d4c5f9","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2017-03-22T09:23:51Z","updated_at":"2017-03-28T11:41:24Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/kontena/kontena/pulls/2011","html_url":"https://github.com/kontena/kontena/pull/2011","diff_url":"https://github.com/kontena/kontena/pull/2011.diff","patch_url":"https://github.com/kontena/kontena/pull/2011.patch"},"body":"Replace the event-based `Celluloid::Notifications` `subscribe('agent:node_info')` with a new reactive `Observable` -> `Observer` interface that allows actors to restart after a crash and immediately pick up the existing state.\r\n\r\nNeeds #2010 to fix the `Kontena::Logging` helper for the anonymous classes used in the specs.\r\n\r\n## Testing\r\n### `StatsWorker`\r\n\r\n```\r\nkontena-agent | I, [2017-03-22T09:18:14.924600 #1]  INFO -- Kontena::Workers::NodeInfoWorker: initialized\r\nkontena-agent | I, [2017-03-22T09:18:14.948380 #1]  INFO -- Kontena::Workers::StatsWorker: initialized\r\nkontena-agent | I, [2017-03-22T09:18:14.949128 #1]  INFO -- Kontena::Workers::StatsWorker: waiting for cadvisor\r\nkontena-agent | I, [2017-03-22T09:18:14.971066 #1]  INFO -- Kontena::Workers::StatsWorker: cadvisor is running, starting stats loop\r\nkontena-agent | D, [2017-03-22T09:18:14.972339 #1] DEBUG -- Workers::NodeInfoWorker: observe: #<Kontena::Workers::StatsWorker:0x005639615085e8>.node=\r\nkontena-agent | D, [2017-03-22T09:18:16.865729 #1] DEBUG -- RpcServer: rpc notification: Kontena::Rpc::AgentApi#master_info [{\"version\"=>\"1.2.0.dev\"}]\r\nkontena-agent | D, [2017-03-22T09:18:16.865977 #1] DEBUG -- RpcServer: rpc notification: Kontena::Rpc::AgentApi#node_info [{\"id\"=>\"XI4K:NPOL:EQJ4:S4V7:EN3B:DHC5:KZJD:F3U2:PCAN:46EV:IO4A:63S5\", \"created_at\"=>\"2016-10-27 08:15:31 UTC\", \"updated_at\"=>\"2017-03-20 14:40:38 UTC\", \"name\"=>\"core-01\", \"labels\"=>[\"test\"], \"overlay_ip\"=>\"10.81.0.1\", \"peer_ips\"=>[\"192.168.66.102\"], \"node_number\"=>1, \"initial_member\"=>true, \"grid\"=>{\"id\"=>\"development\", \"name\"=>\"development\", \"initial_size\"=>1, \"trusted_subnets\"=>[], \"subnet\"=>\"10.81.0.0/16\", \"supernet\"=>\"10.80.0.0/12\", \"stats\"=>{\"statsd\"=>{\"server\"=>\"127.0.0.2\", \"port\"=>8125}}}}]\r\nkontena-agent | D, [2017-03-22T09:18:16.868890 #1] DEBUG -- Workers::NodeInfoWorker: publishing node information\r\nkontena-agent | D, [2017-03-22T09:18:22.163605 #1] DEBUG -- Workers::NodeInfoWorker: update: #<Node:0x005639618e7d30>\r\nkontena-agent | D, [2017-03-22T09:18:22.164576 #1] DEBUG -- Workers::NodeInfoWorker: notify: #<Kontena::Workers::StatsWorker:0x005639615085e8>.node= #<Node:0x005639618e7d30>\r\nkontena-agent | D, [2017-03-22T09:18:22.164889 #1] DEBUG -- Workers::StatsWorker: configure statds: {\"server\"=>\"127.0.0.2\", \"port\"=>8125}\r\nkontena-agent | I, [2017-03-22T09:18:22.164971 #1]  INFO -- Kontena::Workers::StatsWorker: exporting stats via statsd to udp://127.0.0.2:8125\r\n```\r\n\r\nThe `StatsWorker` now recovers quickly from a crash, without having to wait for a `node.info` notification:\r\n\r\n```\r\nkontena-agent | D, [2017-03-22T09:19:15.076450 #1] DEBUG -- Workers::StatsWorker: starting collection\r\nkontena-agent | E, [2017-03-22T09:19:15.206125 #1] ERROR -- : Actor crashed!\r\nkontena-agent | RuntimeError: crash\r\nkontena-agent | \t/app/lib/kontena/workers/stats_worker.rb:46:in `block in start'\r\nkontena-agent | \t/app/lib/kontena/workers/stats_worker.rb:41:in `loop'\r\nkontena-agent | \t/app/lib/kontena/workers/stats_worker.rb:41:in `start'\r\nkontena-agent | \t/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/calls.rb:28:in `public_send'\r\nkontena-agent | \t/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/calls.rb:28:in `dispatch'\r\nkontena-agent | \t/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/call/async.rb:7:in `dispatch'\r\nkontena-agent | \t/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/cell.rb:50:in `block in dispatch'\r\nkontena-agent | \t/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/cell.rb:76:in `block in task'\r\nkontena-agent | \t/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/actor.rb:339:in `block in task'\r\nkontena-agent | \t/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/task.rb:44:in `block in initialize'\r\nkontena-agent | \t/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/task/fibered.rb:14:in `block in create'\r\nkontena-agent | I, [2017-03-22T09:19:15.221604 #1]  INFO -- Kontena::Workers::StatsWorker: initialized\r\nkontena-agent | I, [2017-03-22T09:19:15.221883 #1]  INFO -- Kontena::Workers::StatsWorker: waiting for cadvisor\r\nkontena-agent | I, [2017-03-22T09:19:15.258843 #1]  INFO -- Kontena::Workers::StatsWorker: cadvisor is running, starting stats loop\r\nkontena-agent | D, [2017-03-22T09:19:20.002071 #1] DEBUG -- Workers::NodeInfoWorker: observe: #<Kontena::Workers::StatsWorker:0x005639623c3f60>.node=\r\nkontena-agent | D, [2017-03-22T09:19:20.002294 #1] DEBUG -- Workers::StatsWorker: configure statds: {\"server\"=>\"127.0.0.2\", \"port\"=>8125}\r\nkontena-agent | I, [2017-03-22T09:19:20.002349 #1]  INFO -- Kontena::Workers::StatsWorker: exporting stats via statsd to udp://127.0.0.2:8125\r\n```\r\n\r\n### `FluentdWorker`\r\n```\r\nkontena-agent | D, [2017-03-22T10:03:58.274701 #1] DEBUG -- Workers::NodeInfoWorker: update: #<Node:0x0055b7fc7f4838>\r\nkontena-agent | D, [2017-03-22T10:03:58.277032 #1] DEBUG -- Workers::NodeInfoWorker: notify: #<Kontena::Workers::FluentdWorker:0x0055b7fd37c818>.node= #<Node:0x0055b7fc7f4838>\r\nkontena-agent | I, [2017-03-22T10:03:58.277313 #1]  INFO -- Kontena::Workers::FluentdWorker: starting fluentd log streaming to 127.0.0.2\r\n```\r\n\r\n`kontena grid update --log-forwarder none development`\r\n```\r\nkontena-agent | D, [2017-03-22T10:04:07.773692 #1] DEBUG -- RpcServer: rpc notification: Kontena::Rpc::AgentApi#node_info [{\"id\"=>\"XI4K:NPOL:EQJ4:S4V7:EN3B:DHC5:KZJD:F3U2:PCAN:46EV:IO4A:63S5\", ..., \"logs\"=>nil}}]\r\nkontena-agent | D, [2017-03-22T10:04:07.775639 #1] DEBUG -- Workers::NodeInfoWorker: update: #<Node:0x0055b7fe67e1a0>\r\nkontena-agent | D, [2017-03-22T10:04:07.784650 #1] DEBUG -- Workers::NodeInfoWorker: notify: #<Kontena::Workers::FluentdWorker:0x0055b7fd37c818>.node= #<Node:0x0055b7fe67e1a0>\r\nkontena-agent | I, [2017-03-22T10:04:07.785990 #1]  INFO -- Kontena::Workers::FluentdWorker: stopping fluentd log streaming\r\n```\r\n\r\n## TODO\r\n\r\n* Link observed actors to deal with crashing Observables\r\n* Chaining Observer as Observable to also replace `network_adapter:start` and `network:ready` notifications"},"comment":{"url":"https://api.github.com/repos/kontena/kontena/issues/comments/289743516","html_url":"https://github.com/kontena/kontena/pull/2011#issuecomment-289743516","issue_url":"https://api.github.com/repos/kontena/kontena/issues/2011","id":289743516,"user":{"login":"jnummelin","id":3205505,"avatar_url":"https://avatars3.githubusercontent.com/u/3205505?v=3","gravatar_id":"","url":"https://api.github.com/users/jnummelin","html_url":"https://github.com/jnummelin","followers_url":"https://api.github.com/users/jnummelin/followers","following_url":"https://api.github.com/users/jnummelin/following{/other_user}","gists_url":"https://api.github.com/users/jnummelin/gists{/gist_id}","starred_url":"https://api.github.com/users/jnummelin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jnummelin/subscriptions","organizations_url":"https://api.github.com/users/jnummelin/orgs","repos_url":"https://api.github.com/users/jnummelin/repos","events_url":"https://api.github.com/users/jnummelin/events{/privacy}","received_events_url":"https://api.github.com/users/jnummelin/received_events","type":"User","site_admin":false},"created_at":"2017-03-28T11:41:24Z","updated_at":"2017-03-28T11:41:24Z","body":"From pure API point of view I think it would be cleaner if `async` is handled within `observe`. This makes the user side of observers a bit cleaner IMO."}}
{"actor":{"display_login":"jnummelin"}}