{"action":"created","issue":{"url":"https://api.github.com/repos/nvaccess/nvda/issues/2990","repository_url":"https://api.github.com/repos/nvaccess/nvda","labels_url":"https://api.github.com/repos/nvaccess/nvda/issues/2990/labels{/name}","comments_url":"https://api.github.com/repos/nvaccess/nvda/issues/2990/comments","events_url":"https://api.github.com/repos/nvaccess/nvda/issues/2990/events","html_url":"https://github.com/nvaccess/nvda/issues/2990","id":116033811,"number":2990,"title":"Automatic language detection based on unicode ranges","user":{"login":"nvaccessAuto","id":14253915,"avatar_url":"https://avatars2.githubusercontent.com/u/14253915?v=4","gravatar_id":"","url":"https://api.github.com/users/nvaccessAuto","html_url":"https://github.com/nvaccessAuto","followers_url":"https://api.github.com/users/nvaccessAuto/followers","following_url":"https://api.github.com/users/nvaccessAuto/following{/other_user}","gists_url":"https://api.github.com/users/nvaccessAuto/gists{/gist_id}","starred_url":"https://api.github.com/users/nvaccessAuto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nvaccessAuto/subscriptions","organizations_url":"https://api.github.com/users/nvaccessAuto/orgs","repos_url":"https://api.github.com/users/nvaccessAuto/repos","events_url":"https://api.github.com/users/nvaccessAuto/events{/privacy}","received_events_url":"https://api.github.com/users/nvaccessAuto/received_events","type":"User","site_admin":false},"labels":[{"id":284590815,"url":"https://api.github.com/repos/nvaccess/nvda/labels/enhancement","name":"enhancement","color":"84b6eb","default":true},{"id":284597243,"url":"https://api.github.com/repos/nvaccess/nvda/labels/needsCodeReview","name":"needsCodeReview","color":"ffffff","default":false},{"id":401238340,"url":"https://api.github.com/repos/nvaccess/nvda/labels/p2","name":"p2","color":"5319e7","default":false},{"id":284594694,"url":"https://api.github.com/repos/nvaccess/nvda/labels/Speech","name":"Speech","color":"ffffff","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":55,"created_at":"2013-02-13T12:26:42Z","updated_at":"2017-08-16T05:49:47Z","closed_at":null,"body":"**Reported by ragb on 2013-02-13 12:26**\n\nThis is kind of a spin-of of #279.\n\nAs settled some time ago, this proposal aims to implement automatic text “language” detection for NVDA. The main goal of this feature is for users to read text in different languages (or better said, language families) using proper synthesizer voices. By using unicode character ranges, one can understand at least the language family of a bunch of text: Latine-based (english, german, portuguese, spanish, french,…),, cyrilic (russian, ukrainian,…), kanji (japanese, maybe korean, - I that already written but it is too much for my memory), greek, arabic (arabic, farsy), and others more. \n\nIn broad terms, the implementation of this feature in NVDA requires the addition of a detection module in the speech sub system, that intercepts speech commands and adds “fake” language commands for the synth to change language, based on changes on text characters. It is also needed an interface for the user to tell NVDA what particular language to choose for some language family, that is, what to assume for latin-based, what to assume for arabic-based characters, etc.\n\nI’ve implemented a prototype of this feature in a custome vocalizer driver, with no interface to choose the “proper” language. Prliminary testing with arabic users, using arabi and english vocalizer voices, has shown good results, that is, people like the idea. Detection language code was adapted from the [Guess_language](https://github.com/dsc/guess-language) module, removing some of the detection code which was not applicable (tri-gram detection for differentiating latin languages, for instance).\n\nI’ll explain the decision to use, for now, only unicode based language detection. Language detection could also be done using trigrams (see [here for instance](http://code.activestate.com/recipes/326576-language-detection-using-character-trigrams/)), dictionaries, or other heuristics of that kind. However, the text that is passed each time for the synthesizer is very very small (a line of text, a menu name, etc), which makes these processes, which are probabilistic by nature, very very error-prone. From my testing, applying trigram detect for latin languages in NVDA showed completely unusable, further from adding a noticeable delay when speaking. For bigger text content (books, articles, etc.) it seems to work well, however I don’t know if this can by applied somehow in the future, say by analyzing virtuel buffers, or anything.\n\nRegarding punctuation, digits, and other general characters, I’m defaulting to the current language (and voice) of the synth.\n\nI’ll create a branch with my detection module integrated within NVDA, with no interface.\n\nRegarding the interface for selecting what language to assume for each given language group (when applicable, greek, for instance, is only itself), I see a dialog with various combo boxes, each one for each language family, to choose the language to be used. I think restricting the available language choices from the available languages of the current synth may improve usability. I don’t know where to put that dialog, or what to call it (“language detection options”?).\n\nAny questions please ask.\n\nRegards,\n\nRui Batista \n**Blocked by #5427, #5438**\n"},"comment":{"url":"https://api.github.com/repos/nvaccess/nvda/issues/comments/322672647","html_url":"https://github.com/nvaccess/nvda/issues/2990#issuecomment-322672647","issue_url":"https://api.github.com/repos/nvaccess/nvda/issues/2990","id":322672647,"user":{"login":"dineshkaushal","id":5061823,"avatar_url":"https://avatars1.githubusercontent.com/u/5061823?v=4","gravatar_id":"","url":"https://api.github.com/users/dineshkaushal","html_url":"https://github.com/dineshkaushal","followers_url":"https://api.github.com/users/dineshkaushal/followers","following_url":"https://api.github.com/users/dineshkaushal/following{/other_user}","gists_url":"https://api.github.com/users/dineshkaushal/gists{/gist_id}","starred_url":"https://api.github.com/users/dineshkaushal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dineshkaushal/subscriptions","organizations_url":"https://api.github.com/users/dineshkaushal/orgs","repos_url":"https://api.github.com/users/dineshkaushal/repos","events_url":"https://api.github.com/users/dineshkaushal/events{/privacy}","received_events_url":"https://api.github.com/users/dineshkaushal/received_events","type":"User","site_admin":false},"created_at":"2017-08-16T05:49:46Z","updated_at":"2017-08-16T05:49:46Z","body":"Thanks @nishimotz for the fix.\n\n \n\nBut I thought this scenario should be covered by the common Unicode category? My understanding is that the algorithm does not work for your example of 1個 as number comes before the character 個. Can you verify if number coming after the Japanese character works fine.\n\n \n\nIn that case instead of adding “Number” as a separate category, we could change the processing so that language code will apply for previous common category string if there is no earlier language code. This should solve the above scenario. the current implementation should take the default language code for the common category so this problem should not occur.\n\n \n\nCould you also give me log so that I could check what default language is showing for the above example with a Japanese synthesizer?\n\n \n\nThanks for other improvements as well, the code is looking better.\n\n \n\n \n\nFrom: James Teh [mailto:notifications@github.com] \nSent: Wednesday, August 16, 2017 2:06 AM\nTo: nvaccess/nvda <nvda@noreply.github.com>\nCc: dineshkaushal <dineshkaushal@gmail.com>; Mention <mention@noreply.github.com>\nSubject: Re: [nvaccess/nvda] Automatic language detection based on unicode ranges (#2990)\n\n \n\nNo; you can't choose the specific voice. This choice is made by the synth. Supporting this will be possible using the same technique we will use to suppoort synth switching.\n\n\n\n—\nYou are receiving this because you were mentioned.\nReply to this email directly, view it on GitHub <https://github.com/nvaccess/nvda/issues/2990#issuecomment-322581214> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AE08v_LGb-klkJ-QfXu4bRh-JaA09BAoks5sYgEVgaJpZM4LHyH_> .  <https://github.com/notifications/beacon/AE08v0j6ZBkXkARiV5D_WHoFamvW8fNaks5sYgEVgaJpZM4LHyH_.gif> \n\n"}}
{"actor":{"display_login":"dineshkaushal"}}