{"action":"opened","issue":{"url":"https://api.github.com/repos/unicode-cookbook/cookbook/issues/18","repository_url":"https://api.github.com/repos/unicode-cookbook/cookbook","labels_url":"https://api.github.com/repos/unicode-cookbook/cookbook/issues/18/labels{/name}","comments_url":"https://api.github.com/repos/unicode-cookbook/cookbook/issues/18/comments","events_url":"https://api.github.com/repos/unicode-cookbook/cookbook/issues/18/events","html_url":"https://github.com/unicode-cookbook/cookbook/issues/18","id":268308103,"number":18,"title":"Chapter 2 teminology","user":{"login":"bambooforest","id":1073582,"avatar_url":"https://avatars3.githubusercontent.com/u/1073582?v=4","gravatar_id":"","url":"https://api.github.com/users/bambooforest","html_url":"https://github.com/bambooforest","followers_url":"https://api.github.com/users/bambooforest/followers","following_url":"https://api.github.com/users/bambooforest/following{/other_user}","gists_url":"https://api.github.com/users/bambooforest/gists{/gist_id}","starred_url":"https://api.github.com/users/bambooforest/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bambooforest/subscriptions","organizations_url":"https://api.github.com/users/bambooforest/orgs","repos_url":"https://api.github.com/users/bambooforest/repos","events_url":"https://api.github.com/users/bambooforest/events{/privacy}","received_events_url":"https://api.github.com/users/bambooforest/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/unicode-cookbook/cookbook/milestones/1","html_url":"https://github.com/unicode-cookbook/cookbook/milestone/1","labels_url":"https://api.github.com/repos/unicode-cookbook/cookbook/milestones/1/labels","id":2863057,"number":1,"title":"Address comments by reviewers","description":"Make the cookbook publishable with [http://langsci-press.org/](http://langsci-press.org/}.","creator":{"login":"bambooforest","id":1073582,"avatar_url":"https://avatars3.githubusercontent.com/u/1073582?v=4","gravatar_id":"","url":"https://api.github.com/users/bambooforest","html_url":"https://github.com/bambooforest","followers_url":"https://api.github.com/users/bambooforest/followers","following_url":"https://api.github.com/users/bambooforest/following{/other_user}","gists_url":"https://api.github.com/users/bambooforest/gists{/gist_id}","starred_url":"https://api.github.com/users/bambooforest/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bambooforest/subscriptions","organizations_url":"https://api.github.com/users/bambooforest/orgs","repos_url":"https://api.github.com/users/bambooforest/repos","events_url":"https://api.github.com/users/bambooforest/events{/privacy}","received_events_url":"https://api.github.com/users/bambooforest/received_events","type":"User","site_admin":false},"open_issues":11,"closed_issues":0,"state":"open","created_at":"2017-10-24T19:12:51Z","updated_at":"2017-10-25T08:25:20Z","due_on":null,"closed_at":null},"comments":0,"created_at":"2017-10-25T08:25:19Z","updated_at":"2017-10-25T08:25:19Z","closed_at":null,"author_association":"OWNER","body":"Chapter 2, \"Unicode Pitfalls\", is arranged as a list of common problems when\r\nworking with Unicode. Even just looking at the some section names, \r\n\"Characters are not glyphs\", \"Characters are not graphs\", \"Blocks\", \"Names\",\r\nhints at the main problem of using Unicode being one of terminology.\r\nIn particular for programmers or researchers using a computing environment like\r\npython or R, this is certainly the case, because many of these environments -\r\nbeing mainly used and developed in the English-speaking world - get away with\r\nnot distinguishing two things, namely \"sequences of code points\" and \"text\r\nencoded as bit patterns\". Now looking up \"Character encoding\" on Wikipedia [1]\r\nwe learn that the term \"encoding\" is used for all levels - code points or\r\nbit patterns. But in programming, the standard terminology seems to have\r\nsettled on reserving \"encoding\" for the bit-pattern level, while using terms\r\nlike \"string\" or \"unicode text\" for sequences of Unicode code points [2].\r\n\r\nSo while it may be technically correct to say\r\n\r\n    The Unicode Standard is a CHARACTER ENCODING SYSTEM (page 13)\r\n\r\nthis sounds a bit odd to the Unicode-aware programmer, who would rather say\r\n\"UTF-8 is a Unicode Encoding system\". So the solution of the Unicode Standard\r\nfor the problem of many mutually conflicting encoding systems was to introduce\r\na new layer of indirection between character and bit pattern - the code point.\r\nUnfortunately, the Unicode Consortium itself seems to shy away from clearer\r\n*new* terminology, when it compares Unicode to encoding systems (not saying\r\nUnicode is one, though), but does not coin a new term for what Unicode actually\r\nis, but rather defining it purely functional as \"Unicode provides a unique number \r\nfor every character\" [3].\r\nIt's hard to tell whether attempting to coin such a term could have succeeded\r\nwith this book, but it is a bit sad, that the blurry terminology used everywhere\r\nis perpetuated here. Still, the authors should try not to distract or confuse\r\nthe reader as with footnote 10 on page 13, where the short excursion into\r\nnumeral systems seems to suggest, that Unicode code point U+0070 is always\r\n\"encoded\" as bit pattern 1110 0000.\r\n\r\nSection \"Pitfall: File formats\" may profit most from a more transparent\r\nterminology, distinguishing levels of encoding, rather than talking about how\r\ntexts \"appear inside some kind of computer file\". This section should also\r\nmention that line breaks are actually part of the text, and as such also\r\ncovered by the Unicode standard. Again regarding line breaks, the \r\nrecommendation \"that everybody use this encoding [only LF] whenever possible\"\r\n(page 31) seems less well motivated (and specific) than most other \r\nrecommendations in the book. Some common formats, e.g. CSV (as specified by \r\nRFC 4180 [4]), specify \"CRLF\" as line break. Thus, the perceived \"strong \r\ntendency\" towards \"LF\" may be just that.\r\n\r\nOne Unicode pitfall which may be worth adding is the problem with incomplete\r\nimplementations of the Unicode Standard. This is a problem most big-enough\r\nstandards suffer from, e.g. SQL, XML, XSLT. So although the Unicode Standard\r\nmandates that comparison of Unicode text should always be done using normalized\r\ntext, this is not how the equality operator \"==\" is implemented for Unicode\r\ntext in the Python programming language.\r\n\r\n[1] https://en.wikipedia.org/wiki/Character_encoding\r\n[2] https://docs.python.org/3/howto/unicode.html#encodings\r\n[3] http://www.unicode.org/standard/WhatIsUnicode.html\r\n[4] https://www.ietf.org/rfc/rfc4180.txt"}}
{"actor":{"display_login":"bambooforest"}}