{"action":"created","issue":{"url":"https://api.github.com/repos/minio/minio/issues/4176","repository_url":"https://api.github.com/repos/minio/minio","labels_url":"https://api.github.com/repos/minio/minio/issues/4176/labels{/name}","comments_url":"https://api.github.com/repos/minio/minio/issues/4176/comments","events_url":"https://api.github.com/repos/minio/minio/issues/4176/events","html_url":"https://github.com/minio/minio/issues/4176","id":224213868,"number":4176,"title":"[Pachyderm + Minio] Pipeline does not succeed as fails to fetch object","user":{"login":"jonandernovella","id":3052170,"avatar_url":"https://avatars3.githubusercontent.com/u/3052170?v=3","gravatar_id":"","url":"https://api.github.com/users/jonandernovella","html_url":"https://github.com/jonandernovella","followers_url":"https://api.github.com/users/jonandernovella/followers","following_url":"https://api.github.com/users/jonandernovella/following{/other_user}","gists_url":"https://api.github.com/users/jonandernovella/gists{/gist_id}","starred_url":"https://api.github.com/users/jonandernovella/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jonandernovella/subscriptions","organizations_url":"https://api.github.com/users/jonandernovella/orgs","repos_url":"https://api.github.com/users/jonandernovella/repos","events_url":"https://api.github.com/users/jonandernovella/events{/privacy}","received_events_url":"https://api.github.com/users/jonandernovella/received_events","type":"User","site_admin":false},"labels":[{"id":399254082,"url":"https://api.github.com/repos/minio/minio/labels/community","name":"community","color":"006b75","default":false},{"id":574010436,"url":"https://api.github.com/repos/minio/minio/labels/priority:%20high","name":"priority: high","color":"FF530D","default":false}],"state":"open","locked":false,"assignee":{"login":"NitishT","id":5156139,"avatar_url":"https://avatars3.githubusercontent.com/u/5156139?v=3","gravatar_id":"","url":"https://api.github.com/users/NitishT","html_url":"https://github.com/NitishT","followers_url":"https://api.github.com/users/NitishT/followers","following_url":"https://api.github.com/users/NitishT/following{/other_user}","gists_url":"https://api.github.com/users/NitishT/gists{/gist_id}","starred_url":"https://api.github.com/users/NitishT/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NitishT/subscriptions","organizations_url":"https://api.github.com/users/NitishT/orgs","repos_url":"https://api.github.com/users/NitishT/repos","events_url":"https://api.github.com/users/NitishT/events{/privacy}","received_events_url":"https://api.github.com/users/NitishT/received_events","type":"User","site_admin":false},"assignees":[{"login":"NitishT","id":5156139,"avatar_url":"https://avatars3.githubusercontent.com/u/5156139?v=3","gravatar_id":"","url":"https://api.github.com/users/NitishT","html_url":"https://github.com/NitishT","followers_url":"https://api.github.com/users/NitishT/followers","following_url":"https://api.github.com/users/NitishT/following{/other_user}","gists_url":"https://api.github.com/users/NitishT/gists{/gist_id}","starred_url":"https://api.github.com/users/NitishT/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NitishT/subscriptions","organizations_url":"https://api.github.com/users/NitishT/orgs","repos_url":"https://api.github.com/users/NitishT/repos","events_url":"https://api.github.com/users/NitishT/events{/privacy}","received_events_url":"https://api.github.com/users/NitishT/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/minio/minio/milestones/15","html_url":"https://github.com/minio/minio/milestone/15","labels_url":"https://api.github.com/repos/minio/minio/milestones/15/labels","id":2426553,"number":15,"title":"Edge cache","description":"In this milestone:\r\n - caching support when minio runs in the gateway mode.\r\n - support for heal command.\r\n - Misc fixes and improvements. ","creator":{"login":"deekoder","id":1700901,"avatar_url":"https://avatars2.githubusercontent.com/u/1700901?v=3","gravatar_id":"","url":"https://api.github.com/users/deekoder","html_url":"https://github.com/deekoder","followers_url":"https://api.github.com/users/deekoder/followers","following_url":"https://api.github.com/users/deekoder/following{/other_user}","gists_url":"https://api.github.com/users/deekoder/gists{/gist_id}","starred_url":"https://api.github.com/users/deekoder/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/deekoder/subscriptions","organizations_url":"https://api.github.com/users/deekoder/orgs","repos_url":"https://api.github.com/users/deekoder/repos","events_url":"https://api.github.com/users/deekoder/events{/privacy}","received_events_url":"https://api.github.com/users/deekoder/received_events","type":"User","site_admin":false},"open_issues":46,"closed_issues":85,"state":"open","created_at":"2017-04-01T00:36:55Z","updated_at":"2017-04-28T02:03:33Z","due_on":"2017-06-30T07:00:00Z","closed_at":null},"comments":20,"created_at":"2017-04-25T17:28:02Z","updated_at":"2017-04-28T09:28:30Z","closed_at":null,"body":"## Expected Behavior\r\nThe jobs in Pachyderm should output the results/commit after having correctly processed the data in the input repo with any number of workers, and without needing to rerun the pipeline.\r\n\r\n## Current Behavior\r\nWhen the processing of the data finishes successfully, Pachyderm does not seem to make the final commit.\r\n\r\nThe logs from pachd show the following:\r\n\r\n`failed to process datum [file:<commit:<repo:<name:“mrpo” > id:“4c1d47e832cf4b9f8965aabc24008ed8” > path:“/014_CRa_H9M5_M470_Pool_03_alternate_pos_low_mr.mzML” > file_type:FILE size_bytes:98128338 hash: ... `\r\n\r\n`failed to retrieve hashtree after processing for datum [file:<commit:<repo:<name:“mrpo” > id:“4c1d47e832cf4b9f8965aabc24008ed8” > path:“/014_CRa_H9M5_M470_Pool_03_alternate_pos_low_mr.mzML” `\r\n\r\n` rpc error: code = 2 desc = The specified key does not exist., retrying in: 13.707884915s`\r\n\r\n## Steps to Reproduce (for bugs)\r\n1. Deploy Minio in Kubernetes with the minio-server file and make it accessible from your Master node\r\n`sudo vi /etc/resolv.conf (Change default DNS server to your KubeDNS ip-address 10.96.0.10)`\r\n2. Create a S3 bucket to access the Minio server with the Minio client (from your master node)\r\n`wget https://dl.minio.io/client/mc/release/linux-amd64/mc`\r\n`chmod +x mc`\r\n`./mc config host add myalias http://minio.default.svc.cluster.local:9000 <id> <sec>`\r\n`./mc mb myalias/mybucket`\r\n3. Deploy Pachyderm with the given manifest\r\n4. Test the pipeline with the given data below creating 12 workers.\r\n\r\n* `id: AKIAIOSFODNN7TESTPAC`\r\n`sec: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEWAY`\r\n\r\n## My setup and code\r\n* Version used: Pachyderm 1.4.5, Kubernetes 1.5\r\n* Environment: Pachyderm + Minio in OpenStack with local host disk storage back-end\r\n* Pachyderm deployment: https://www.dropbox.com/s/5th8fmz4exq1buh/mymanifestv145.json\r\n* Minio deployment in Kubernetes: https://www.dropbox.com/s/7cyouvk9ipxwar3/minio-server.yaml\r\n* Pipeline: https://www.dropbox.com/s/ldeb3gn6f49rjju/PeakPickerHiRes.json\r\n* Dataset: `wget ftp://ftp.ebi.ac.uk/pub/databases/metabolights/studies/public/MTBLS233/*alternate_pos_low_mr.mzML\r\n`\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/minio/minio/issues/comments/297952667","html_url":"https://github.com/minio/minio/issues/4176#issuecomment-297952667","issue_url":"https://api.github.com/repos/minio/minio/issues/4176","id":297952667,"user":{"login":"harshavardhana","id":622699,"avatar_url":"https://avatars3.githubusercontent.com/u/622699?v=3","gravatar_id":"","url":"https://api.github.com/users/harshavardhana","html_url":"https://github.com/harshavardhana","followers_url":"https://api.github.com/users/harshavardhana/followers","following_url":"https://api.github.com/users/harshavardhana/following{/other_user}","gists_url":"https://api.github.com/users/harshavardhana/gists{/gist_id}","starred_url":"https://api.github.com/users/harshavardhana/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/harshavardhana/subscriptions","organizations_url":"https://api.github.com/users/harshavardhana/orgs","repos_url":"https://api.github.com/users/harshavardhana/repos","events_url":"https://api.github.com/users/harshavardhana/events{/privacy}","received_events_url":"https://api.github.com/users/harshavardhana/received_events","type":"User","site_admin":false},"created_at":"2017-04-28T09:28:30Z","updated_at":"2017-04-28T09:28:30Z","body":"> @NitishT With 16 cores it took me 1'20\". I think that for some reason the latest version of Minio has slowed down a lot my setup.\r\n\r\n@jonandernovella when you say performance has slowed down, how much it used to be do you have any comparison?  with like the last release? "}}
{"actor":{"display_login":"harshavardhana"}}