{"action":"created","comment":{"url":"https://api.github.com/repos/huijunw/heron/pulls/comments/99231386","pull_request_review_id":19902558,"id":99231386,"diff_hunk":"@@ -0,0 +1,234 @@\n+// Copyright 2016 Twitter. All rights reserved.\n+//\n+// Licensed under the Apache License, Version 2.0 (the \"License\");\n+// you may not use this file except in compliance with the License.\n+// You may obtain a copy of the License at\n+//\n+//    http://www.apache.org/licenses/LICENSE-2.0\n+//\n+// Unless required by applicable law or agreed to in writing, software\n+// distributed under the License is distributed on an \"AS IS\" BASIS,\n+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n+// See the License for the specific language governing permissions and\n+// limitations under the License.\n+\n+package com.twitter.heron.metricscachemgr;\n+\n+import java.io.IOException;\n+import java.net.HttpURLConnection;\n+import java.net.InetSocketAddress;\n+import java.util.Arrays;\n+import java.util.concurrent.ExecutionException;\n+import java.util.logging.Level;\n+import java.util.logging.Logger;\n+\n+import com.google.protobuf.GeneratedMessage;\n+import com.google.protobuf.InvalidProtocolBufferException;\n+import com.sun.net.httpserver.HttpExchange;\n+import com.sun.net.httpserver.HttpHandler;\n+import com.sun.net.httpserver.HttpServer;\n+\n+import com.twitter.heron.metricscachemgr.metricscache.MetricsCache;\n+import com.twitter.heron.proto.tmaster.TopologyMaster;\n+import com.twitter.heron.spi.common.Config;\n+import com.twitter.heron.spi.common.Keys;\n+import com.twitter.heron.spi.utils.NetworkUtils;\n+import com.twitter.heron.statemgr.localfs.LocalFileSystemStateManager;\n+\n+\n+/**\n+ * MetricsCacheMgr http server:\n+ * compatible with tmaster and tracker http interface for metrics\n+ * http path:\n+ * \"/stats\" metric query\n+ * \"/exceptions\" exception query\n+ * \"/exceptionsummary\" exception query, with aggregation\n+ * <p>\n+ * Differece from MetricsCacheManagerServer:\n+ * 1. MetricsCacheManagerServer accepts metric publishing message from sinks;\n+ * MetricsCacheManagerHttpServer responds to queries.\n+ * 2. MetricsCacheManagerServer is a HeronServer;\n+ * MetricsCacheManagerHttpServer is a http server\n+ */\n+public class MetricsCacheManagerHttpServer {\n+  // http path, compatible with tmaster stat interface\n+  public static final String PATH_STATS = \"/stats\";","path":"heron/metricscachemgr/src/java/com/twitter/heron/metricscachemgr/MetricsCacheManagerHttpServer.java","position":55,"original_position":55,"commit_id":"dda9d1e4c71f604e9ac24b992bb610c33ec855a4","original_commit_id":"dda9d1e4c71f604e9ac24b992bb610c33ec855a4","user":{"login":"huijunw","id":23125934,"avatar_url":"https://avatars.githubusercontent.com/u/23125934?v=3","gravatar_id":"","url":"https://api.github.com/users/huijunw","html_url":"https://github.com/huijunw","followers_url":"https://api.github.com/users/huijunw/followers","following_url":"https://api.github.com/users/huijunw/following{/other_user}","gists_url":"https://api.github.com/users/huijunw/gists{/gist_id}","starred_url":"https://api.github.com/users/huijunw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/huijunw/subscriptions","organizations_url":"https://api.github.com/users/huijunw/orgs","repos_url":"https://api.github.com/users/huijunw/repos","events_url":"https://api.github.com/users/huijunw/events{/privacy}","received_events_url":"https://api.github.com/users/huijunw/received_events","type":"User","site_admin":false},"body":"done","created_at":"2017-02-02T22:03:30Z","updated_at":"2017-02-02T22:03:30Z","html_url":"https://github.com/huijunw/heron/pull/1#discussion_r99231386","pull_request_url":"https://api.github.com/repos/huijunw/heron/pulls/1","_links":{"self":{"href":"https://api.github.com/repos/huijunw/heron/pulls/comments/99231386"},"html":{"href":"https://github.com/huijunw/heron/pull/1#discussion_r99231386"},"pull_request":{"href":"https://api.github.com/repos/huijunw/heron/pulls/1"}}},"pull_request":{"url":"https://api.github.com/repos/huijunw/heron/pulls/1","id":100974740,"html_url":"https://github.com/huijunw/heron/pull/1","diff_url":"https://github.com/huijunw/heron/pull/1.diff","patch_url":"https://github.com/huijunw/heron/pull/1.patch","issue_url":"https://api.github.com/repos/huijunw/heron/issues/1","number":1,"state":"open","locked":false,"title":"MetricsCacheMgr for SLA","user":{"login":"huijunw","id":23125934,"avatar_url":"https://avatars.githubusercontent.com/u/23125934?v=3","gravatar_id":"","url":"https://api.github.com/users/huijunw","html_url":"https://github.com/huijunw","followers_url":"https://api.github.com/users/huijunw/followers","following_url":"https://api.github.com/users/huijunw/following{/other_user}","gists_url":"https://api.github.com/users/huijunw/gists{/gist_id}","starred_url":"https://api.github.com/users/huijunw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/huijunw/subscriptions","organizations_url":"https://api.github.com/users/huijunw/orgs","repos_url":"https://api.github.com/users/huijunw/repos","events_url":"https://api.github.com/users/huijunw/events{/privacy}","received_events_url":"https://api.github.com/users/huijunw/received_events","type":"User","site_admin":false},"body":"The MetricsCacheMgr is a standalone process in container 0. It is responsible for collecting metrics from MetricsCacheSink and replying queries from both inside and outside process. It will eventually substitute the tmetrics-collector in the Tmaster. It currently may serve as the base process for SLA to be developed on.\r\n\r\nThe MetricsCacheMgr process sets its location in the StateMgr, which is monitored by StMgr and sent to MetricsMgr. MetricsCacheMgr provides three interfaces for metric publishing and querying:\r\n1)publish metrics: MetricsCacheSink sends the metrics to the MetricsCacheMgr process. \r\n2)query from outside process: The protobuf message definition of Tmaster is re-used for future easy migration.\r\n3)query from inside process: GetMetrics(MetricCacheRequest request)\r\n\r\nTo try it out on local Mac laptop, see below step A and step B.\r\n\r\n[A]to run it locally on Mac laptop (same as normal procedures):\r\ngit clone https://github.com/huijunw/heron.git\r\ngit checkout SLAMetricCache\r\ncd [source root directory]\r\n\r\nbazel build --config=darwin heron/...\r\nbazel build --config=darwin scripts/packages:binpkgs\r\nbazel build --config=darwin scripts/packages:tarpkgs\r\n\r\nbazel-bin/scripts/packages/heron-api-install.sh --user \r\nbazel-bin/scripts/packages/heron-client-install.sh --user\r\nbazel-bin/scripts/packages/heron-tools-install.sh --user\r\n\r\n~/bin/heron submit local ~/.heron/examples/heron-examples.jar com.twitter.heron.examples.ExclamationTopology ExclamationTopology --deploy-deactivated --verbose\r\n\r\n[B]to query:\r\nin the [source root directory],\r\nbazel run heron/metricscachemgr/src/java:metricscache-queryclient-unshaded -- ExclamationTopology exclaim1 __emit-count __execute-count __fail-count __ack-count __complete-latency __execute-latency __process-latency __jvm-uptime-secs __jvm-process-cpu-load __jvm-memory-used-mb __jvm-memory-mb-total __jvm-gc-collection-time-ms __server/__time_spent_back_pressure_initiated __time_spent_back_pressure_by_compid\r\n","created_at":"2017-01-10T23:27:14Z","updated_at":"2017-02-02T22:03:30Z","closed_at":null,"merged_at":null,"merge_commit_sha":"3736f6f1b020ca1d98845547dd08a93556206500","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/huijunw/heron/pulls/1/commits","review_comments_url":"https://api.github.com/repos/huijunw/heron/pulls/1/comments","review_comment_url":"https://api.github.com/repos/huijunw/heron/pulls/comments{/number}","comments_url":"https://api.github.com/repos/huijunw/heron/issues/1/comments","statuses_url":"https://api.github.com/repos/huijunw/heron/statuses/dda9d1e4c71f604e9ac24b992bb610c33ec855a4","head":{"label":"huijunw:SLAMetricCache","ref":"SLAMetricCache","sha":"dda9d1e4c71f604e9ac24b992bb610c33ec855a4","user":{"login":"huijunw","id":23125934,"avatar_url":"https://avatars.githubusercontent.com/u/23125934?v=3","gravatar_id":"","url":"https://api.github.com/users/huijunw","html_url":"https://github.com/huijunw","followers_url":"https://api.github.com/users/huijunw/followers","following_url":"https://api.github.com/users/huijunw/following{/other_user}","gists_url":"https://api.github.com/users/huijunw/gists{/gist_id}","starred_url":"https://api.github.com/users/huijunw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/huijunw/subscriptions","organizations_url":"https://api.github.com/users/huijunw/orgs","repos_url":"https://api.github.com/users/huijunw/repos","events_url":"https://api.github.com/users/huijunw/events{/privacy}","received_events_url":"https://api.github.com/users/huijunw/received_events","type":"User","site_admin":false},"repo":{"id":75666065,"name":"heron","full_name":"huijunw/heron","owner":{"login":"huijunw","id":23125934,"avatar_url":"https://avatars.githubusercontent.com/u/23125934?v=3","gravatar_id":"","url":"https://api.github.com/users/huijunw","html_url":"https://github.com/huijunw","followers_url":"https://api.github.com/users/huijunw/followers","following_url":"https://api.github.com/users/huijunw/following{/other_user}","gists_url":"https://api.github.com/users/huijunw/gists{/gist_id}","starred_url":"https://api.github.com/users/huijunw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/huijunw/subscriptions","organizations_url":"https://api.github.com/users/huijunw/orgs","repos_url":"https://api.github.com/users/huijunw/repos","events_url":"https://api.github.com/users/huijunw/events{/privacy}","received_events_url":"https://api.github.com/users/huijunw/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/huijunw/heron","description":"Heron is a realtime, distributed, fault-tolerant stream processing engine from Twitter","fork":true,"url":"https://api.github.com/repos/huijunw/heron","forks_url":"https://api.github.com/repos/huijunw/heron/forks","keys_url":"https://api.github.com/repos/huijunw/heron/keys{/key_id}","collaborators_url":"https://api.github.com/repos/huijunw/heron/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/huijunw/heron/teams","hooks_url":"https://api.github.com/repos/huijunw/heron/hooks","issue_events_url":"https://api.github.com/repos/huijunw/heron/issues/events{/number}","events_url":"https://api.github.com/repos/huijunw/heron/events","assignees_url":"https://api.github.com/repos/huijunw/heron/assignees{/user}","branches_url":"https://api.github.com/repos/huijunw/heron/branches{/branch}","tags_url":"https://api.github.com/repos/huijunw/heron/tags","blobs_url":"https://api.github.com/repos/huijunw/heron/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/huijunw/heron/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/huijunw/heron/git/refs{/sha}","trees_url":"https://api.github.com/repos/huijunw/heron/git/trees{/sha}","statuses_url":"https://api.github.com/repos/huijunw/heron/statuses/{sha}","languages_url":"https://api.github.com/repos/huijunw/heron/languages","stargazers_url":"https://api.github.com/repos/huijunw/heron/stargazers","contributors_url":"https://api.github.com/repos/huijunw/heron/contributors","subscribers_url":"https://api.github.com/repos/huijunw/heron/subscribers","subscription_url":"https://api.github.com/repos/huijunw/heron/subscription","commits_url":"https://api.github.com/repos/huijunw/heron/commits{/sha}","git_commits_url":"https://api.github.com/repos/huijunw/heron/git/commits{/sha}","comments_url":"https://api.github.com/repos/huijunw/heron/comments{/number}","issue_comment_url":"https://api.github.com/repos/huijunw/heron/issues/comments{/number}","contents_url":"https://api.github.com/repos/huijunw/heron/contents/{+path}","compare_url":"https://api.github.com/repos/huijunw/heron/compare/{base}...{head}","merges_url":"https://api.github.com/repos/huijunw/heron/merges","archive_url":"https://api.github.com/repos/huijunw/heron/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/huijunw/heron/downloads","issues_url":"https://api.github.com/repos/huijunw/heron/issues{/number}","pulls_url":"https://api.github.com/repos/huijunw/heron/pulls{/number}","milestones_url":"https://api.github.com/repos/huijunw/heron/milestones{/number}","notifications_url":"https://api.github.com/repos/huijunw/heron/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/huijunw/heron/labels{/name}","releases_url":"https://api.github.com/repos/huijunw/heron/releases{/id}","deployments_url":"https://api.github.com/repos/huijunw/heron/deployments","created_at":"2016-12-05T21:05:15Z","updated_at":"2016-12-05T21:05:21Z","pushed_at":"2017-02-02T19:52:18Z","git_url":"git://github.com/huijunw/heron.git","ssh_url":"git@github.com:huijunw/heron.git","clone_url":"https://github.com/huijunw/heron.git","svn_url":"https://github.com/huijunw/heron","homepage":"http://heronstreaming.io","size":128087,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":1,"forks":0,"open_issues":1,"watchers":0,"default_branch":"master"}},"base":{"label":"huijunw:master","ref":"master","sha":"98c12f772a046e3623d6d48fe17b5baececf9fe9","user":{"login":"huijunw","id":23125934,"avatar_url":"https://avatars.githubusercontent.com/u/23125934?v=3","gravatar_id":"","url":"https://api.github.com/users/huijunw","html_url":"https://github.com/huijunw","followers_url":"https://api.github.com/users/huijunw/followers","following_url":"https://api.github.com/users/huijunw/following{/other_user}","gists_url":"https://api.github.com/users/huijunw/gists{/gist_id}","starred_url":"https://api.github.com/users/huijunw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/huijunw/subscriptions","organizations_url":"https://api.github.com/users/huijunw/orgs","repos_url":"https://api.github.com/users/huijunw/repos","events_url":"https://api.github.com/users/huijunw/events{/privacy}","received_events_url":"https://api.github.com/users/huijunw/received_events","type":"User","site_admin":false},"repo":{"id":75666065,"name":"heron","full_name":"huijunw/heron","owner":{"login":"huijunw","id":23125934,"avatar_url":"https://avatars.githubusercontent.com/u/23125934?v=3","gravatar_id":"","url":"https://api.github.com/users/huijunw","html_url":"https://github.com/huijunw","followers_url":"https://api.github.com/users/huijunw/followers","following_url":"https://api.github.com/users/huijunw/following{/other_user}","gists_url":"https://api.github.com/users/huijunw/gists{/gist_id}","starred_url":"https://api.github.com/users/huijunw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/huijunw/subscriptions","organizations_url":"https://api.github.com/users/huijunw/orgs","repos_url":"https://api.github.com/users/huijunw/repos","events_url":"https://api.github.com/users/huijunw/events{/privacy}","received_events_url":"https://api.github.com/users/huijunw/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/huijunw/heron","description":"Heron is a realtime, distributed, fault-tolerant stream processing engine from Twitter","fork":true,"url":"https://api.github.com/repos/huijunw/heron","forks_url":"https://api.github.com/repos/huijunw/heron/forks","keys_url":"https://api.github.com/repos/huijunw/heron/keys{/key_id}","collaborators_url":"https://api.github.com/repos/huijunw/heron/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/huijunw/heron/teams","hooks_url":"https://api.github.com/repos/huijunw/heron/hooks","issue_events_url":"https://api.github.com/repos/huijunw/heron/issues/events{/number}","events_url":"https://api.github.com/repos/huijunw/heron/events","assignees_url":"https://api.github.com/repos/huijunw/heron/assignees{/user}","branches_url":"https://api.github.com/repos/huijunw/heron/branches{/branch}","tags_url":"https://api.github.com/repos/huijunw/heron/tags","blobs_url":"https://api.github.com/repos/huijunw/heron/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/huijunw/heron/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/huijunw/heron/git/refs{/sha}","trees_url":"https://api.github.com/repos/huijunw/heron/git/trees{/sha}","statuses_url":"https://api.github.com/repos/huijunw/heron/statuses/{sha}","languages_url":"https://api.github.com/repos/huijunw/heron/languages","stargazers_url":"https://api.github.com/repos/huijunw/heron/stargazers","contributors_url":"https://api.github.com/repos/huijunw/heron/contributors","subscribers_url":"https://api.github.com/repos/huijunw/heron/subscribers","subscription_url":"https://api.github.com/repos/huijunw/heron/subscription","commits_url":"https://api.github.com/repos/huijunw/heron/commits{/sha}","git_commits_url":"https://api.github.com/repos/huijunw/heron/git/commits{/sha}","comments_url":"https://api.github.com/repos/huijunw/heron/comments{/number}","issue_comment_url":"https://api.github.com/repos/huijunw/heron/issues/comments{/number}","contents_url":"https://api.github.com/repos/huijunw/heron/contents/{+path}","compare_url":"https://api.github.com/repos/huijunw/heron/compare/{base}...{head}","merges_url":"https://api.github.com/repos/huijunw/heron/merges","archive_url":"https://api.github.com/repos/huijunw/heron/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/huijunw/heron/downloads","issues_url":"https://api.github.com/repos/huijunw/heron/issues{/number}","pulls_url":"https://api.github.com/repos/huijunw/heron/pulls{/number}","milestones_url":"https://api.github.com/repos/huijunw/heron/milestones{/number}","notifications_url":"https://api.github.com/repos/huijunw/heron/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/huijunw/heron/labels{/name}","releases_url":"https://api.github.com/repos/huijunw/heron/releases{/id}","deployments_url":"https://api.github.com/repos/huijunw/heron/deployments","created_at":"2016-12-05T21:05:15Z","updated_at":"2016-12-05T21:05:21Z","pushed_at":"2017-02-02T19:52:18Z","git_url":"git://github.com/huijunw/heron.git","ssh_url":"git@github.com:huijunw/heron.git","clone_url":"https://github.com/huijunw/heron.git","svn_url":"https://github.com/huijunw/heron","homepage":"http://heronstreaming.io","size":128087,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":1,"forks":0,"open_issues":1,"watchers":0,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/huijunw/heron/pulls/1"},"html":{"href":"https://github.com/huijunw/heron/pull/1"},"issue":{"href":"https://api.github.com/repos/huijunw/heron/issues/1"},"comments":{"href":"https://api.github.com/repos/huijunw/heron/issues/1/comments"},"review_comments":{"href":"https://api.github.com/repos/huijunw/heron/pulls/1/comments"},"review_comment":{"href":"https://api.github.com/repos/huijunw/heron/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/huijunw/heron/pulls/1/commits"},"statuses":{"href":"https://api.github.com/repos/huijunw/heron/statuses/dda9d1e4c71f604e9ac24b992bb610c33ec855a4"}}}}
{"actor":{"display_login":"huijunw"}}