{"action":"created","issue":{"url":"https://api.github.com/repos/beetbox/beets/issues/2630","repository_url":"https://api.github.com/repos/beetbox/beets","labels_url":"https://api.github.com/repos/beetbox/beets/issues/2630/labels{/name}","comments_url":"https://api.github.com/repos/beetbox/beets/issues/2630/comments","events_url":"https://api.github.com/repos/beetbox/beets/issues/2630/events","html_url":"https://github.com/beetbox/beets/issues/2630","id":243177274,"number":2630,"title":"musixmatch searches failing in beets but work in browser","user":{"login":"anarcat","id":796623,"avatar_url":"https://avatars7.githubusercontent.com/u/796623?v=4","gravatar_id":"","url":"https://api.github.com/users/anarcat","html_url":"https://github.com/anarcat","followers_url":"https://api.github.com/users/anarcat/followers","following_url":"https://api.github.com/users/anarcat/following{/other_user}","gists_url":"https://api.github.com/users/anarcat/gists{/gist_id}","starred_url":"https://api.github.com/users/anarcat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anarcat/subscriptions","organizations_url":"https://api.github.com/users/anarcat/orgs","repos_url":"https://api.github.com/users/anarcat/repos","events_url":"https://api.github.com/users/anarcat/events{/privacy}","received_events_url":"https://api.github.com/users/anarcat/received_events","type":"User","site_admin":false},"labels":[{"id":29900975,"url":"https://api.github.com/repos/beetbox/beets/labels/needinfo","name":"needinfo","color":"e102d8","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2017-07-15T14:07:19Z","updated_at":"2017-07-15T18:06:45Z","closed_at":null,"body":"### Problem\r\n\r\nI'm trying to find the lyrics to a song that I know is on Musixmatch.\r\n\r\nRunning this command in verbose (`-vv`) mode fails:\r\n\r\n```sh\r\n$ beet -vv -l ./library.db lyrics richard desjardins la maison est ouverte\r\nuser configuration: /home/anarcat/.config/beets/config.yaml\r\ndata directory: /home/anarcat/.config/beets\r\nplugin paths: \r\nSending event: pluginload\r\nlyrics: Disabling google source: no API key configured.\r\nlibrary database: /home/anarcat/src/lyrics-ptype/library.db\r\nlibrary directory: /home/anarcat/mp3\r\nSending event: library_opened\r\nlyrics: failed to fetch: http://lyrics.wikia.com/Richard_Desjardins:La_Maison_Est_Ouverte (404)\r\nlyrics: failed to fetch: https://www.musixmatch.com/lyrics/Richard-Desjardins/La-Maison-Est-Ouverte (404)\r\nlyrics: lyrics not found: Richard Desjardins - Boom Boom - La maison est ouverte\r\nSending event: cli_exit\r\n```\r\n\r\nYou can see above that it fails to find the lyrics, yet they are available on MusixMatch at:\r\n\r\nhttps://www.musixmatch.com/fr/paroles/Richard-Desjardins/La-maison-est-ouverte\r\n\r\nNotice the different capitalization. Also note that the result comes up in a search on their website as well:\r\n\r\nhttps://www.musixmatch.com/fr/search/richard%20desjardins%20la%20maison%20est%20ouverte\r\n\r\nI wonder if the latter couldn't be used to get the lyrics in a more reliable way...\r\n\r\nNote that with the Google Search API keys in, I still fail to get the lyrics, even though it *seems* to find a good URL:\r\n\r\n```\r\n$ beet -vvv -l ./library.db lyrics richard desjardins la maison est ouverte\r\nuser configuration: /home/anarcat/.config/beets/config.yaml\r\ndata directory: /home/anarcat/.config/beets\r\nplugin paths: \r\nSending event: pluginload\r\nlibrary database: /home/anarcat/src/lyrics-ptype/library.db\r\nlibrary directory: /home/anarcat/mp3\r\nSending event: library_opened\r\nlyrics: failed to fetch: https://www.musixmatch.com/fr/paroles/Richard-Desjardins/La-maison-est-ouverte (404)\r\nlyrics: failed to fetch: https://www.musixmatch.com/es/letras/Richard-Desjardins-Francois-Gauthier-Mathieu-Desy-Forestare/La-Maison-est-ouverte (404)\r\nlyrics: failed to fetch: http://lyrics.wikia.com/Richard_Desjardins:La_Maison_Est_Ouverte (404)\r\nlyrics: failed to fetch: https://www.musixmatch.com/lyrics/Richard-Desjardins/La-Maison-Est-Ouverte (404)\r\nlyrics: lyrics not found: Richard Desjardins - Boom Boom - La maison est ouverte\r\nSending event: cli_exit\r\n```\r\n\r\nIt's interesting to note that the first URL there, works:\r\n\r\nhttps://www.musixmatch.com/fr/paroles/Richard-Desjardins/La-maison-est-ouverte\r\n\r\n### Setup\r\n\r\n* OS: Debian stretch 9\r\n* Python version 2.7.13\r\n* beets version 1.4.5\r\n* Turning off plugins made problem go away (yes/no): yes, but that's besides the point of course\r\n\r\nMy configuration (output of `beet config`) is:\r\n\r\n```yaml\r\nplugins: fetchart fromfilename duplicates missing copyartifacts lyrics\r\nimport:\r\n    copy: no\r\n    move: no\r\n    write: no\r\nfetchart:\r\n    sources: coverart amazon albumart fanarttv wikipedia filesystem\r\ncopyartifacts:\r\n    print_ignored: yes\r\nlyrics:\r\n    auto: no\r\n```\r\n\r\nThis seems similar to #2546 - maybe there's something fishy going on here that makes musixmatch refuse beets requests? Indeed, if i change the `USER_AGENT` string in `lyrics.py` to `Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36`, things seem to work a little better.\r\n\r\nAny idea how to fix this? A common fix to user agent issues is to shove the usual Mozilla crap up there and append the real user agent in the end... :/ It's hackish, but it's better than downright failures...\r\n\r\nToo bad sites disrespect unusual UAs... :("},"comment":{"url":"https://api.github.com/repos/beetbox/beets/issues/comments/315551795","html_url":"https://github.com/beetbox/beets/issues/2630#issuecomment-315551795","issue_url":"https://api.github.com/repos/beetbox/beets/issues/2630","id":315551795,"user":{"login":"sampsyo","id":188033,"avatar_url":"https://avatars6.githubusercontent.com/u/188033?v=4","gravatar_id":"","url":"https://api.github.com/users/sampsyo","html_url":"https://github.com/sampsyo","followers_url":"https://api.github.com/users/sampsyo/followers","following_url":"https://api.github.com/users/sampsyo/following{/other_user}","gists_url":"https://api.github.com/users/sampsyo/gists{/gist_id}","starred_url":"https://api.github.com/users/sampsyo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sampsyo/subscriptions","organizations_url":"https://api.github.com/users/sampsyo/orgs","repos_url":"https://api.github.com/users/sampsyo/repos","events_url":"https://api.github.com/users/sampsyo/events{/privacy}","received_events_url":"https://api.github.com/users/sampsyo/received_events","type":"User","site_admin":false},"created_at":"2017-07-15T18:06:44Z","updated_at":"2017-07-15T18:06:44Z","body":"We can continue on #2546 (it’s still open). But briefly, yes, the UA string does work for now—but so did the current UA string, briefly, which is why we switched to it from th built-in requests UA. Unless we exactly replicate a popular browser UA, which feels weird, I think we’ll just be continuing the Whac-a-Mole game."}}
{"actor":{"display_login":"sampsyo"}}