{"action":"created","comment":{"url":"https://api.github.com/repos/bigartm/bigartm/pulls/comments/103438178","pull_request_review_id":24220664,"id":103438178,"diff_hunk":"@@ -1,2 +1,71 @@\n-6. Tokens Coocurancy and Coherence Coumputation\n-=======\n+6. Tokens Coocurancy and Coherence Computation\n+==============================================\n+\n+* **Interpretability and Coherence of Topics**\n+\n+The one of main requirements of topic models is interpretability (i.e., do the topics contain tokens that, according to subjective human judgment, are representative of single coherent concept). `Newman at al. <http://www.aclweb.org/anthology/N10-1012>`_ showed the human evaluation of interpretability is well correlated with next automated quality measure called coherence. The coherence formula for topic is defined as\n+\n+:math:`\\mathcal{C}_t = \\cfrac{2}{k(k - 1)} \\sum_{i = 1}^{k - 1} \\sum_{j = 1}^{k} \\mathrm{value}(w_i, w_j)`,\n+\n+where value is some pairwise information about tokens in collection dictionary, which is provided by user according to his goals.\n+\n+* **Tokens Coocurancy Dictionary**\n+\n+BigARTM provides ability of automatic coherence computation. Only one thing you should prepare is file with coocurancies of tokens. Lines of that file should have the next format: ::\n+\n+\tid1 id2 value\n+\n+where ``id1`` and ``id2`` are indexes of tokens in vocabulary file in UCI format (but indexing starts from 0!). Details about UCI format can be found here: :doc:`../datasets`. ``value`` is some pairwise information about tokens with indexes ``id1`` and ``id2``. For instance, ``value`` can be computed by any corpora of texts by formulas:\n+\n+* pointwise PMI: :math:`value(u,v)=\\left[\\log\\cfrac{n(u,v)}{n(u)n(v)}\\right]_{+}`, \n+* smoothed logarithm of conditional probability: :math:`value(u,v)=\\log\\cfrac{n(u,v)+1}{n(v)}`,\n+\n+where `n(u)` and `n(v)` are frequencies of `u` and `v` in corpora, `n(u,v)` is joint frequency of `u` and `v` in corpora in window with some fixed width.\n+\n+* **Usage**\n+\n+Let's assume you have file `cooc.txt` with coocurancies of tokens. Also you should have vocabulary file in UCI format `vocab.txt` corresponding to `cooc.txt` file. \n+\n+To upload coocurancies data into BigARTM you should use ``artm.Dictionary`` object and method ``gather``: ::\n+\n+\tcooc_dict = artm.Dictionary()\n+\tcooc_dict.gather(\n+\t    data_path='batches_folder',\n+\t    cooc_file_path='cooc.txt',\n+\t    vocab_file_path='vocab.txt',\n+\t    symmetric_cooc_values=True)\n+\n+Where \n+\n+* ``data_path`` is path to folder with your collection in internal BigARTM format (batches);\n+* ``cooc_file_path`` is path to file with coocurancies of tokens;\n+* ``vocab_file_path`` is path to vocabulary file corresponding to `cooc.txt`;\n+* ``symmetric_cooc_values`` is Boolean argument. ``False`` means that coocurancies information is not symmetric by order of tokens, i.e. `value(w1, w2)` and `value(w2, w1)` can be different.\n+\n+\n+So, now you can create coherence computation score: ::\n+\n+\tcoherence_score = artm.TopTokensScore(\n+\t                            name='TopTokensCoherenceScore',\n+\t                            class_id='@default_class', \n+\t                            num_tokens=10, \n+\t                            topic_names=[u'topic_0',u'topic_1'],\n+\t                            dictionary=cooc_dict)\n+\n+Arguments:\n+\n+* ``name`` is name of score;\n+* ``class_id`` is name of modality, which contains tokens with coocurancy information;","path":"docs/tutorials/python_userguide/coherence.txt","position":60,"original_position":60,"commit_id":"3f602495d578e1b8d9437b4496b3d22296c6fb2c","original_commit_id":"3f602495d578e1b8d9437b4496b3d22296c6fb2c","user":{"login":"JeanPaulShapo","id":7376288,"avatar_url":"https://avatars.githubusercontent.com/u/7376288?v=3","gravatar_id":"","url":"https://api.github.com/users/JeanPaulShapo","html_url":"https://github.com/JeanPaulShapo","followers_url":"https://api.github.com/users/JeanPaulShapo/followers","following_url":"https://api.github.com/users/JeanPaulShapo/following{/other_user}","gists_url":"https://api.github.com/users/JeanPaulShapo/gists{/gist_id}","starred_url":"https://api.github.com/users/JeanPaulShapo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JeanPaulShapo/subscriptions","organizations_url":"https://api.github.com/users/JeanPaulShapo/orgs","repos_url":"https://api.github.com/users/JeanPaulShapo/repos","events_url":"https://api.github.com/users/JeanPaulShapo/events{/privacy}","received_events_url":"https://api.github.com/users/JeanPaulShapo/received_events","type":"User","site_admin":false},"body":"*cooc**c**ur**re**nce*","created_at":"2017-02-28T11:59:42Z","updated_at":"2017-02-28T12:04:15Z","html_url":"https://github.com/bigartm/bigartm/pull/774#discussion_r103438178","pull_request_url":"https://api.github.com/repos/bigartm/bigartm/pulls/774","_links":{"self":{"href":"https://api.github.com/repos/bigartm/bigartm/pulls/comments/103438178"},"html":{"href":"https://github.com/bigartm/bigartm/pull/774#discussion_r103438178"},"pull_request":{"href":"https://api.github.com/repos/bigartm/bigartm/pulls/774"}}},"pull_request":{"url":"https://api.github.com/repos/bigartm/bigartm/pulls/774","id":108312762,"html_url":"https://github.com/bigartm/bigartm/pull/774","diff_url":"https://github.com/bigartm/bigartm/pull/774.diff","patch_url":"https://github.com/bigartm/bigartm/pull/774.patch","issue_url":"https://api.github.com/repos/bigartm/bigartm/issues/774","number":774,"state":"open","locked":false,"title":"Docs","user":{"login":"Angriff","id":6429935,"avatar_url":"https://avatars.githubusercontent.com/u/6429935?v=3","gravatar_id":"","url":"https://api.github.com/users/Angriff","html_url":"https://github.com/Angriff","followers_url":"https://api.github.com/users/Angriff/followers","following_url":"https://api.github.com/users/Angriff/following{/other_user}","gists_url":"https://api.github.com/users/Angriff/gists{/gist_id}","starred_url":"https://api.github.com/users/Angriff/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Angriff/subscriptions","organizations_url":"https://api.github.com/users/Angriff/orgs","repos_url":"https://api.github.com/users/Angriff/repos","events_url":"https://api.github.com/users/Angriff/events{/privacy}","received_events_url":"https://api.github.com/users/Angriff/received_events","type":"User","site_admin":false},"body":"Work with documentation. Fixing misspelling. Tokens Coocurancy and Coherence Computation documentation page is added ","created_at":"2017-02-28T11:41:13Z","updated_at":"2017-02-28T12:04:15Z","closed_at":null,"merged_at":null,"merge_commit_sha":"a490e79fcec6396573d82637a3893846bf3aed13","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/bigartm/bigartm/pulls/774/commits","review_comments_url":"https://api.github.com/repos/bigartm/bigartm/pulls/774/comments","review_comment_url":"https://api.github.com/repos/bigartm/bigartm/pulls/comments{/number}","comments_url":"https://api.github.com/repos/bigartm/bigartm/issues/774/comments","statuses_url":"https://api.github.com/repos/bigartm/bigartm/statuses/3f602495d578e1b8d9437b4496b3d22296c6fb2c","head":{"label":"Angriff:Docs","ref":"Docs","sha":"3f602495d578e1b8d9437b4496b3d22296c6fb2c","user":{"login":"Angriff","id":6429935,"avatar_url":"https://avatars.githubusercontent.com/u/6429935?v=3","gravatar_id":"","url":"https://api.github.com/users/Angriff","html_url":"https://github.com/Angriff","followers_url":"https://api.github.com/users/Angriff/followers","following_url":"https://api.github.com/users/Angriff/following{/other_user}","gists_url":"https://api.github.com/users/Angriff/gists{/gist_id}","starred_url":"https://api.github.com/users/Angriff/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Angriff/subscriptions","organizations_url":"https://api.github.com/users/Angriff/orgs","repos_url":"https://api.github.com/users/Angriff/repos","events_url":"https://api.github.com/users/Angriff/events{/privacy}","received_events_url":"https://api.github.com/users/Angriff/received_events","type":"User","site_admin":false},"repo":{"id":82933191,"name":"bigartm","full_name":"Angriff/bigartm","owner":{"login":"Angriff","id":6429935,"avatar_url":"https://avatars.githubusercontent.com/u/6429935?v=3","gravatar_id":"","url":"https://api.github.com/users/Angriff","html_url":"https://github.com/Angriff","followers_url":"https://api.github.com/users/Angriff/followers","following_url":"https://api.github.com/users/Angriff/following{/other_user}","gists_url":"https://api.github.com/users/Angriff/gists{/gist_id}","starred_url":"https://api.github.com/users/Angriff/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Angriff/subscriptions","organizations_url":"https://api.github.com/users/Angriff/orgs","repos_url":"https://api.github.com/users/Angriff/repos","events_url":"https://api.github.com/users/Angriff/events{/privacy}","received_events_url":"https://api.github.com/users/Angriff/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/Angriff/bigartm","description":"Fast topic modeling platform","fork":true,"url":"https://api.github.com/repos/Angriff/bigartm","forks_url":"https://api.github.com/repos/Angriff/bigartm/forks","keys_url":"https://api.github.com/repos/Angriff/bigartm/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Angriff/bigartm/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Angriff/bigartm/teams","hooks_url":"https://api.github.com/repos/Angriff/bigartm/hooks","issue_events_url":"https://api.github.com/repos/Angriff/bigartm/issues/events{/number}","events_url":"https://api.github.com/repos/Angriff/bigartm/events","assignees_url":"https://api.github.com/repos/Angriff/bigartm/assignees{/user}","branches_url":"https://api.github.com/repos/Angriff/bigartm/branches{/branch}","tags_url":"https://api.github.com/repos/Angriff/bigartm/tags","blobs_url":"https://api.github.com/repos/Angriff/bigartm/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Angriff/bigartm/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Angriff/bigartm/git/refs{/sha}","trees_url":"https://api.github.com/repos/Angriff/bigartm/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Angriff/bigartm/statuses/{sha}","languages_url":"https://api.github.com/repos/Angriff/bigartm/languages","stargazers_url":"https://api.github.com/repos/Angriff/bigartm/stargazers","contributors_url":"https://api.github.com/repos/Angriff/bigartm/contributors","subscribers_url":"https://api.github.com/repos/Angriff/bigartm/subscribers","subscription_url":"https://api.github.com/repos/Angriff/bigartm/subscription","commits_url":"https://api.github.com/repos/Angriff/bigartm/commits{/sha}","git_commits_url":"https://api.github.com/repos/Angriff/bigartm/git/commits{/sha}","comments_url":"https://api.github.com/repos/Angriff/bigartm/comments{/number}","issue_comment_url":"https://api.github.com/repos/Angriff/bigartm/issues/comments{/number}","contents_url":"https://api.github.com/repos/Angriff/bigartm/contents/{+path}","compare_url":"https://api.github.com/repos/Angriff/bigartm/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Angriff/bigartm/merges","archive_url":"https://api.github.com/repos/Angriff/bigartm/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Angriff/bigartm/downloads","issues_url":"https://api.github.com/repos/Angriff/bigartm/issues{/number}","pulls_url":"https://api.github.com/repos/Angriff/bigartm/pulls{/number}","milestones_url":"https://api.github.com/repos/Angriff/bigartm/milestones{/number}","notifications_url":"https://api.github.com/repos/Angriff/bigartm/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Angriff/bigartm/labels{/name}","releases_url":"https://api.github.com/repos/Angriff/bigartm/releases{/id}","deployments_url":"https://api.github.com/repos/Angriff/bigartm/deployments","created_at":"2017-02-23T14:06:17Z","updated_at":"2017-02-23T14:06:19Z","pushed_at":"2017-02-28T11:38:40Z","git_url":"git://github.com/Angriff/bigartm.git","ssh_url":"git@github.com:Angriff/bigartm.git","clone_url":"https://github.com/Angriff/bigartm.git","svn_url":"https://github.com/Angriff/bigartm","homepage":"http://bigartm.org/","size":16125,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"bigartm:master","ref":"master","sha":"a8a568bca26ddbe7ea559c1d896a40172dd5fbca","user":{"login":"bigartm","id":8783740,"avatar_url":"https://avatars.githubusercontent.com/u/8783740?v=3","gravatar_id":"","url":"https://api.github.com/users/bigartm","html_url":"https://github.com/bigartm","followers_url":"https://api.github.com/users/bigartm/followers","following_url":"https://api.github.com/users/bigartm/following{/other_user}","gists_url":"https://api.github.com/users/bigartm/gists{/gist_id}","starred_url":"https://api.github.com/users/bigartm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bigartm/subscriptions","organizations_url":"https://api.github.com/users/bigartm/orgs","repos_url":"https://api.github.com/users/bigartm/repos","events_url":"https://api.github.com/users/bigartm/events{/privacy}","received_events_url":"https://api.github.com/users/bigartm/received_events","type":"Organization","site_admin":false},"repo":{"id":24072579,"name":"bigartm","full_name":"bigartm/bigartm","owner":{"login":"bigartm","id":8783740,"avatar_url":"https://avatars.githubusercontent.com/u/8783740?v=3","gravatar_id":"","url":"https://api.github.com/users/bigartm","html_url":"https://github.com/bigartm","followers_url":"https://api.github.com/users/bigartm/followers","following_url":"https://api.github.com/users/bigartm/following{/other_user}","gists_url":"https://api.github.com/users/bigartm/gists{/gist_id}","starred_url":"https://api.github.com/users/bigartm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bigartm/subscriptions","organizations_url":"https://api.github.com/users/bigartm/orgs","repos_url":"https://api.github.com/users/bigartm/repos","events_url":"https://api.github.com/users/bigartm/events{/privacy}","received_events_url":"https://api.github.com/users/bigartm/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/bigartm/bigartm","description":"Fast topic modeling platform","fork":false,"url":"https://api.github.com/repos/bigartm/bigartm","forks_url":"https://api.github.com/repos/bigartm/bigartm/forks","keys_url":"https://api.github.com/repos/bigartm/bigartm/keys{/key_id}","collaborators_url":"https://api.github.com/repos/bigartm/bigartm/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/bigartm/bigartm/teams","hooks_url":"https://api.github.com/repos/bigartm/bigartm/hooks","issue_events_url":"https://api.github.com/repos/bigartm/bigartm/issues/events{/number}","events_url":"https://api.github.com/repos/bigartm/bigartm/events","assignees_url":"https://api.github.com/repos/bigartm/bigartm/assignees{/user}","branches_url":"https://api.github.com/repos/bigartm/bigartm/branches{/branch}","tags_url":"https://api.github.com/repos/bigartm/bigartm/tags","blobs_url":"https://api.github.com/repos/bigartm/bigartm/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/bigartm/bigartm/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/bigartm/bigartm/git/refs{/sha}","trees_url":"https://api.github.com/repos/bigartm/bigartm/git/trees{/sha}","statuses_url":"https://api.github.com/repos/bigartm/bigartm/statuses/{sha}","languages_url":"https://api.github.com/repos/bigartm/bigartm/languages","stargazers_url":"https://api.github.com/repos/bigartm/bigartm/stargazers","contributors_url":"https://api.github.com/repos/bigartm/bigartm/contributors","subscribers_url":"https://api.github.com/repos/bigartm/bigartm/subscribers","subscription_url":"https://api.github.com/repos/bigartm/bigartm/subscription","commits_url":"https://api.github.com/repos/bigartm/bigartm/commits{/sha}","git_commits_url":"https://api.github.com/repos/bigartm/bigartm/git/commits{/sha}","comments_url":"https://api.github.com/repos/bigartm/bigartm/comments{/number}","issue_comment_url":"https://api.github.com/repos/bigartm/bigartm/issues/comments{/number}","contents_url":"https://api.github.com/repos/bigartm/bigartm/contents/{+path}","compare_url":"https://api.github.com/repos/bigartm/bigartm/compare/{base}...{head}","merges_url":"https://api.github.com/repos/bigartm/bigartm/merges","archive_url":"https://api.github.com/repos/bigartm/bigartm/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/bigartm/bigartm/downloads","issues_url":"https://api.github.com/repos/bigartm/bigartm/issues{/number}","pulls_url":"https://api.github.com/repos/bigartm/bigartm/pulls{/number}","milestones_url":"https://api.github.com/repos/bigartm/bigartm/milestones{/number}","notifications_url":"https://api.github.com/repos/bigartm/bigartm/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/bigartm/bigartm/labels{/name}","releases_url":"https://api.github.com/repos/bigartm/bigartm/releases{/id}","deployments_url":"https://api.github.com/repos/bigartm/bigartm/deployments","created_at":"2014-09-15T20:26:11Z","updated_at":"2017-02-28T00:40:55Z","pushed_at":"2017-02-28T11:41:13Z","git_url":"git://github.com/bigartm/bigartm.git","ssh_url":"git@github.com:bigartm/bigartm.git","clone_url":"https://github.com/bigartm/bigartm.git","svn_url":"https://github.com/bigartm/bigartm","homepage":"http://bigartm.org/","size":16127,"stargazers_count":144,"watchers_count":144,"language":"C++","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":52,"mirror_url":null,"open_issues_count":83,"forks":52,"open_issues":83,"watchers":144,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/bigartm/bigartm/pulls/774"},"html":{"href":"https://github.com/bigartm/bigartm/pull/774"},"issue":{"href":"https://api.github.com/repos/bigartm/bigartm/issues/774"},"comments":{"href":"https://api.github.com/repos/bigartm/bigartm/issues/774/comments"},"review_comments":{"href":"https://api.github.com/repos/bigartm/bigartm/pulls/774/comments"},"review_comment":{"href":"https://api.github.com/repos/bigartm/bigartm/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/bigartm/bigartm/pulls/774/commits"},"statuses":{"href":"https://api.github.com/repos/bigartm/bigartm/statuses/3f602495d578e1b8d9437b4496b3d22296c6fb2c"}}}}
{"actor":{"display_login":"JeanPaulShapo"}}