{"action":"created","issue":{"url":"https://api.github.com/repos/RevolutionAnalytics/rmr2/issues/179","repository_url":"https://api.github.com/repos/RevolutionAnalytics/rmr2","labels_url":"https://api.github.com/repos/RevolutionAnalytics/rmr2/issues/179/labels{/name}","comments_url":"https://api.github.com/repos/RevolutionAnalytics/rmr2/issues/179/comments","events_url":"https://api.github.com/repos/RevolutionAnalytics/rmr2/issues/179/events","html_url":"https://github.com/RevolutionAnalytics/rmr2/issues/179","id":169798354,"number":179,"title":"rmr2 mapreduce does not produce any output","user":{"login":"byu777","id":1245932,"avatar_url":"https://avatars3.githubusercontent.com/u/1245932?v=4","gravatar_id":"","url":"https://api.github.com/users/byu777","html_url":"https://github.com/byu777","followers_url":"https://api.github.com/users/byu777/followers","following_url":"https://api.github.com/users/byu777/following{/other_user}","gists_url":"https://api.github.com/users/byu777/gists{/gist_id}","starred_url":"https://api.github.com/users/byu777/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/byu777/subscriptions","organizations_url":"https://api.github.com/users/byu777/orgs","repos_url":"https://api.github.com/users/byu777/repos","events_url":"https://api.github.com/users/byu777/events{/privacy}","received_events_url":"https://api.github.com/users/byu777/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-08-07T14:16:36Z","updated_at":"2017-08-20T06:43:48Z","closed_at":null,"body":"I tried the following simple script on rmr2 in Cloudera Quickstart 5.7.0 but mapreduce does not generate any results.  Here is the script:\n\n```\nsmall.ints <- to.dfs(1:10)\nout <- mapreduce(input = small.ints, map = function(k, v) keyval(v, v^2))\nfrom.dfs(out)\n```\n\nHere is the output:\n\n```\n> small.ints <- to.dfs(1:10)\n16/08/07 20:14:42 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library\n16/08/07 20:14:42 INFO compress.CodecPool: Got brand-new compressor [.deflate]\nWarning message:\nS3 methods ‘gorder.default’, ‘gorder.factor’, ‘gorder.data.frame’, ‘gorder.matrix’, ‘gorder.raw’ were declared in NAMESPACE but not found \n> out <- mapreduce(input = small.ints, map = function(k, v) keyval(v, v^2))\n16/08/07 20:14:48 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces\npackageJobJar: [] [/usr/jars/hadoop-streaming-2.6.0-cdh5.7.0.jar] /tmp/streamjob543400947433267521.jar tmpDir=null\n16/08/07 20:14:49 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032\n16/08/07 20:14:49 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032\n16/08/07 20:14:50 INFO mapred.FileInputFormat: Total input paths to process : 1\n16/08/07 20:14:50 INFO mapreduce.JobSubmitter: number of splits:2\n16/08/07 20:14:50 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1470447912721_0016\n16/08/07 20:14:50 INFO impl.YarnClientImpl: Submitted application application_1470447912721_0016\n16/08/07 20:14:50 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1470447912721_0016/\n16/08/07 20:14:50 INFO mapreduce.Job: Running job: job_1470447912721_0016\n16/08/07 20:15:00 INFO mapreduce.Job: Job job_1470447912721_0016 running in uber mode : false\n16/08/07 20:15:00 INFO mapreduce.Job:  map 0% reduce 0%\n16/08/07 20:15:12 INFO mapreduce.Job:  map 50% reduce 0%\n16/08/07 20:15:13 INFO mapreduce.Job:  map 100% reduce 0%\n16/08/07 20:15:13 INFO mapreduce.Job: Job job_1470447912721_0016 completed successfully\n16/08/07 20:15:13 INFO mapreduce.Job: Counters: 30\n    File System Counters\n        FILE: Number of bytes read=0\n        FILE: Number of bytes written=236342\n        FILE: Number of read operations=0\n        FILE: Number of large read operations=0\n        FILE: Number of write operations=0\n        HDFS: Number of bytes read=1001\n        HDFS: Number of bytes written=244\n        HDFS: Number of read operations=14\n        HDFS: Number of large read operations=0\n        HDFS: Number of write operations=4\n    Job Counters \n        Launched map tasks=2\n        Data-local map tasks=2\n        Total time spent by all maps in occupied slots (ms)=19917\n        Total time spent by all reduces in occupied slots (ms)=0\n        Total time spent by all map tasks (ms)=19917\n        Total vcore-seconds taken by all map tasks=19917\n        Total megabyte-seconds taken by all map tasks=9958500\n    Map-Reduce Framework\n        Map input records=3\n        Map output records=0\n        Input split bytes=208\n        Spilled Records=0\n        Failed Shuffles=0\n        Merged Map outputs=0\n        GC time elapsed (ms)=115\n        CPU time spent (ms)=1300\n        Physical memory (bytes) snapshot=239222784\n        Virtual memory (bytes) snapshot=2127200256\n        Total committed heap usage (bytes)=121503744\n    File Input Format Counters \n        Bytes Read=793\n    File Output Format Counters \n        Bytes Written=244\n16/08/07 20:15:13 INFO streaming.StreamJob: Output directory: /tmp/file10106a0b36b6\n> from.dfs(out)\n$key\nNULL\n\n$val\nNULL\n```\n\nto.dfs and from.dfs do work since I tried the following:\n\n```\n> small.ints <- to.dfs(1:10)\n16/08/07 07:15:34 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library\n16/08/07 07:15:34 INFO compress.CodecPool: Got brand-new compressor [.deflate]\n> out <- from.dfs(small.ints)\n16/08/07 07:15:44 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library\n16/08/07 07:15:44 INFO compress.CodecPool: Got brand-new decompressor [.deflate]\n> out\n$key\nNULL\n\n$val\n [1]  1  2  3  4  5  6  7  8  9 10\n```\n"},"comment":{"url":"https://api.github.com/repos/RevolutionAnalytics/rmr2/issues/comments/323567181","html_url":"https://github.com/RevolutionAnalytics/rmr2/issues/179#issuecomment-323567181","issue_url":"https://api.github.com/repos/RevolutionAnalytics/rmr2/issues/179","id":323567181,"user":{"login":"Surender1984","id":31121766,"avatar_url":"https://avatars1.githubusercontent.com/u/31121766?v=4","gravatar_id":"","url":"https://api.github.com/users/Surender1984","html_url":"https://github.com/Surender1984","followers_url":"https://api.github.com/users/Surender1984/followers","following_url":"https://api.github.com/users/Surender1984/following{/other_user}","gists_url":"https://api.github.com/users/Surender1984/gists{/gist_id}","starred_url":"https://api.github.com/users/Surender1984/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Surender1984/subscriptions","organizations_url":"https://api.github.com/users/Surender1984/orgs","repos_url":"https://api.github.com/users/Surender1984/repos","events_url":"https://api.github.com/users/Surender1984/events{/privacy}","received_events_url":"https://api.github.com/users/Surender1984/received_events","type":"User","site_admin":false},"created_at":"2017-08-20T06:43:48Z","updated_at":"2017-08-20T06:43:48Z","body":"Hi byu777/VJ-Vikvy, , any luck to solve this issue. I a, also facing the same issue."}}
{"actor":{"display_login":"Surender1984"}}