{"action":"created","issue":{"url":"https://api.github.com/repos/codelucas/newspaper/issues/128","repository_url":"https://api.github.com/repos/codelucas/newspaper","labels_url":"https://api.github.com/repos/codelucas/newspaper/issues/128/labels{/name}","comments_url":"https://api.github.com/repos/codelucas/newspaper/issues/128/comments","events_url":"https://api.github.com/repos/codelucas/newspaper/issues/128/events","html_url":"https://github.com/codelucas/newspaper/issues/128","id":60496694,"number":128,"title":"Incorrect article extraction","user":{"login":"attodorov","id":300580,"avatar_url":"https://avatars1.githubusercontent.com/u/300580?v=3","gravatar_id":"","url":"https://api.github.com/users/attodorov","html_url":"https://github.com/attodorov","followers_url":"https://api.github.com/users/attodorov/followers","following_url":"https://api.github.com/users/attodorov/following{/other_user}","gists_url":"https://api.github.com/users/attodorov/gists{/gist_id}","starred_url":"https://api.github.com/users/attodorov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/attodorov/subscriptions","organizations_url":"https://api.github.com/users/attodorov/orgs","repos_url":"https://api.github.com/users/attodorov/repos","events_url":"https://api.github.com/users/attodorov/events{/privacy}","received_events_url":"https://api.github.com/users/attodorov/received_events","type":"User","site_admin":false},"labels":[{"id":67580497,"url":"https://api.github.com/repos/codelucas/newspaper/labels/bug","name":"bug","color":"fc2929","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2015-03-10T13:10:38Z","updated_at":"2017-05-30T19:41:56Z","closed_at":null,"body":"Hey,\n\nI really like newspaper - nice job ! It has one major flaw, though - on a lot of sites, like wsj.com, reuters.com, etc. - it doesn't exract the article text precisely, but also includes left/right menus such as \"Top most read articles\" and similar things. If you use a library like the one Instapaper uses, you can extract the actual article text without any extra clutter. Here is one example (written in Java) that works pretty well:\n\nhttps://code.google.com/p/boilerpipe/\n\nThanks\n--Angel\n"},"comment":{"url":"https://api.github.com/repos/codelucas/newspaper/issues/comments/304986270","html_url":"https://github.com/codelucas/newspaper/issues/128#issuecomment-304986270","issue_url":"https://api.github.com/repos/codelucas/newspaper/issues/128","id":304986270,"user":{"login":"agordon","id":523057,"avatar_url":"https://avatars2.githubusercontent.com/u/523057?v=3","gravatar_id":"","url":"https://api.github.com/users/agordon","html_url":"https://github.com/agordon","followers_url":"https://api.github.com/users/agordon/followers","following_url":"https://api.github.com/users/agordon/following{/other_user}","gists_url":"https://api.github.com/users/agordon/gists{/gist_id}","starred_url":"https://api.github.com/users/agordon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/agordon/subscriptions","organizations_url":"https://api.github.com/users/agordon/orgs","repos_url":"https://api.github.com/users/agordon/repos","events_url":"https://api.github.com/users/agordon/events{/privacy}","received_events_url":"https://api.github.com/users/agordon/received_events","type":"User","site_admin":false},"created_at":"2017-05-30T19:41:56Z","updated_at":"2017-05-30T19:41:56Z","body":"Hello,\r\n\r\nFirst paragraph is missed in some Engadget articles (`https://www.engadget.com/2017/05/30/intel-core-i9-extreme/`).\r\nwhile working in others (e.g. `https://www.engadget.com/2017/05/29/whats-on-tv-house-of-cards-fear-the-walking-dead/`).\r\n\r\nFrom a very cursory look, it seems that for articles where the first paragraph(s) are missed,\r\nthe website separates the first paragraphs from the others, perhaps for a \"Read Fulll Article\" kind-of-button, (hinted by the `js-notMobileReferredByFbTw` class in the example below).\r\n\r\nThis is the general structure of the \"intel i9\" article (link above):\r\n```\r\n<div class=\"o-article_block pb-15 pb-5@m- o-subtle_divider mt-n10@m-\">\r\n      <div class=\"grid@tl+\">\r\n        <div class=\"grid@tl+__cell col-8-of-12@tl+\">\r\n          <div class=\"article-text c-gray-1\">\r\n                          <p> [[ CONTENT OF FIRST PARAGRAPH ]] </p>\r\n           </div>\r\n        </div>\r\n      </div>\r\n</div>\r\n\r\n<div class=\"js-notMobileReferredByFbTw\">\r\n      <div class=\"o-article_block pb-15 pb-5@m- mt-n35 mt-n25@m mt-n15@s\">\r\n        <div class=\"grid@tl+\">\r\n          <div class=\"full-width@tp- grid@tl+__cell col-8-of-12@tl+\">\r\n            <div class=\"article-text c-gray-1 no-review\">\r\n\t        <p>  [[ CONTENT OF ALL OTHER PARAGRAPHS ]] </p>\r\n\t\t<p>  </p>\r\n            </div>\r\n        </div>\r\n    </div>\r\n</div>\r\n```\r\n\r\n(BTW, thanks for a great package!)"}}
{"actor":{"display_login":"agordon"}}