{"action":"created","issue":{"url":"https://api.github.com/repos/ioos/catalog-ckan/issues/147","repository_url":"https://api.github.com/repos/ioos/catalog-ckan","labels_url":"https://api.github.com/repos/ioos/catalog-ckan/issues/147/labels{/name}","comments_url":"https://api.github.com/repos/ioos/catalog-ckan/issues/147/comments","events_url":"https://api.github.com/repos/ioos/catalog-ckan/issues/147/events","html_url":"https://github.com/ioos/catalog-ckan/issues/147","id":220014520,"number":147,"title":"Dataset Count is low","user":{"login":"mwengren","id":1469921,"avatar_url":"https://avatars3.githubusercontent.com/u/1469921?v=3","gravatar_id":"","url":"https://api.github.com/users/mwengren","html_url":"https://github.com/mwengren","followers_url":"https://api.github.com/users/mwengren/followers","following_url":"https://api.github.com/users/mwengren/following{/other_user}","gists_url":"https://api.github.com/users/mwengren/gists{/gist_id}","starred_url":"https://api.github.com/users/mwengren/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mwengren/subscriptions","organizations_url":"https://api.github.com/users/mwengren/orgs","repos_url":"https://api.github.com/users/mwengren/repos","events_url":"https://api.github.com/users/mwengren/events{/privacy}","received_events_url":"https://api.github.com/users/mwengren/received_events","type":"User","site_admin":false},"labels":[{"id":353615949,"url":"https://api.github.com/repos/ioos/catalog-ckan/labels/bug","name":"bug","color":"ee0701","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2017-04-06T19:58:07Z","updated_at":"2017-04-10T19:04:51Z","closed_at":null,"body":"[Dataset count](https://data.ioos.us/dataset) seems low today, any known issues on the CKAN front that might be the cause?  Currently shows ~4200 vs 7000 in the Registry. \n\nSeems to be across the board (not necessarily one particular provider).  The obvious culprit would be Solr, which seems to have periodic issues.  Any info in log files?    \n\nHave you ever tried looking at the Solr webapp behind the firewall for troubleshooting (data.ioos.us:8080/solr/#/~cores/ckan-schema-2.0, or something like that)?"},"comment":{"url":"https://api.github.com/repos/ioos/catalog-ckan/issues/comments/293047550","html_url":"https://github.com/ioos/catalog-ckan/issues/147#issuecomment-293047550","issue_url":"https://api.github.com/repos/ioos/catalog-ckan/issues/147","id":293047550,"user":{"login":"lukecampbell","id":1155163,"avatar_url":"https://avatars0.githubusercontent.com/u/1155163?v=3","gravatar_id":"","url":"https://api.github.com/users/lukecampbell","html_url":"https://github.com/lukecampbell","followers_url":"https://api.github.com/users/lukecampbell/followers","following_url":"https://api.github.com/users/lukecampbell/following{/other_user}","gists_url":"https://api.github.com/users/lukecampbell/gists{/gist_id}","starred_url":"https://api.github.com/users/lukecampbell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lukecampbell/subscriptions","organizations_url":"https://api.github.com/users/lukecampbell/orgs","repos_url":"https://api.github.com/users/lukecampbell/repos","events_url":"https://api.github.com/users/lukecampbell/events{/privacy}","received_events_url":"https://api.github.com/users/lukecampbell/received_events","type":"User","site_admin":false},"created_at":"2017-04-10T19:04:51Z","updated_at":"2017-04-10T19:04:51Z","body":"I suspect that this issue is related to something I posted earlier:\r\nhttps://github.com/ioos/catalog-harvesting/pull/46\r\n\r\nThe uniqueness per filename is only the first 43 characters. So, if the first 43 characters of a dataset name aren't unique then they'll override the file.\r\n\r\nFor example, in the NANOOS WAF:\r\n\r\nhttp://data.nanoos.org/metadata/ioos/52nsos/\r\n\r\nYou'll spot SEVERAL links that are not uniquely named. Those links won't get downloaded properly. \r\n\r\nIt would be better to show them as errors in the registry to indicate what happened. I'll take a look this afternoon to see if I can find some sort of fix. I was toying around with the idea of naming every single file downloaded a UUID of some sort. "}}
{"actor":{"display_login":"lukecampbell"}}