{"action":"created","issue":{"url":"https://api.github.com/repos/lampepfl/dotty/issues/2491","repository_url":"https://api.github.com/repos/lampepfl/dotty","labels_url":"https://api.github.com/repos/lampepfl/dotty/issues/2491/labels{/name}","comments_url":"https://api.github.com/repos/lampepfl/dotty/issues/2491/comments","events_url":"https://api.github.com/repos/lampepfl/dotty/issues/2491/events","html_url":"https://github.com/lampepfl/dotty/issues/2491","id":230180922,"number":2491,"title":"Consider syntax with significant indentation","user":{"login":"odersky","id":795990,"avatar_url":"https://avatars1.githubusercontent.com/u/795990?v=3","gravatar_id":"","url":"https://api.github.com/users/odersky","html_url":"https://github.com/odersky","followers_url":"https://api.github.com/users/odersky/followers","following_url":"https://api.github.com/users/odersky/following{/other_user}","gists_url":"https://api.github.com/users/odersky/gists{/gist_id}","starred_url":"https://api.github.com/users/odersky/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/odersky/subscriptions","organizations_url":"https://api.github.com/users/odersky/orgs","repos_url":"https://api.github.com/users/odersky/repos","events_url":"https://api.github.com/users/odersky/events{/privacy}","received_events_url":"https://api.github.com/users/odersky/received_events","type":"User","site_admin":false},"labels":[{"id":203754560,"url":"https://api.github.com/repos/lampepfl/dotty/labels/itype:language%20enhancement","name":"itype:language enhancement","color":"cc317c","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":16,"created_at":"2017-05-20T21:45:20Z","updated_at":"2017-05-21T16:06:26Z","closed_at":null,"body":"I was playing for a while now with ways to make Scala's syntax indentation-based. I always admired the neatness of Python syntax and also found that F# has benefited greatly from its optional indentation-based syntax, so much so that nobody seems to use the original syntax anymore. I had some good conversations with @lihaoyi at Scala Exchange in 2015 about this. At the time, there were some issues with which I was not happy yet, notably how to elide braces of arguments to user-defined functions. I now have a proposal that addresses these issues.\r\n\r\n# Proposal in a Nutshell\r\n\r\n - If certain keywords are followed by an end-of-line and an indented code block, assume\r\n    block structure as if braces were inserted around the indented block. Example:\r\n\r\n    ```scala\r\n    def f(x: Int) =\r\n      val y = x * x\r\n      y + 1\r\n    ```\r\n    is treated as equivalent to\r\n    ```scala\r\n    def f(x: Int) = {\r\n      val y = x * x\r\n      y + 1\r\n    }\r\n    ```\r\n    Or, for match expressions:\r\n    ```scala\r\n    xs match\r\n      case x :: xs1 => ...\r\n      case Nil => ...\r\n    ```\r\n\r\n   Or, using the new syntax for if-then-else:\r\n    ```scala\r\n    if condition then\r\n      println(\"taken\")\r\n      x\r\n    else\r\n      println(\"not taken\")\r\n      y\r\n    ```\r\n    Or for for-expressions:\r\n\r\n    ```scala\r\n    for\r\n      x <- xs\r\n      y <- ys\r\n    yield\r\n      f(x, y)\r\n    ```\r\n\r\n - Use `with` + indentation as an alternative way to delimit statement sequences in objects and classes. Example:\r\n\r\n    ```scala\r\n    object Obj with\r\n      class C(x: Int) with\r\n        def f = x + 3\r\n      def apply(x: Int) = new C(x)\r\n    ```\r\n\r\n - Also use `with` + indentation as an alternative way to pass arguments that were formerly in braces to functions. Examples:\r\n\r\n    ```scala\r\n    xs.map with x =>\r\n      x + 2\r\n\r\n    xs.collect with\r\n      case P1 => E1\r\n      case P2 => E2\r\n    ```\r\n\r\n# Motivation\r\n\r\nWhy use indentation-based syntax?\r\n\r\n - _Cleaner typography_: We are all used to write\r\n\r\n    ```scala\r\n    def f() = {\r\n      ...\r\n    }\r\n    ```\r\n\r\n     But if we look at it with fresh eyes it's really quite weird how the braces embrace nothing but empty space. Geometrically the braces point _away_ from the enclosed space `...`. One could argue that other brace schemes are better. But there are good reasons why the scheme above is the most popular and in a sense arguments how to make braces look less awkward are themselves an indication that braces are fundamentally problematic. It's much simpler and cleaner to get rid of them:\r\n\r\n    ```scala\r\n    def f() =\r\n      ...\r\n    ```\r\n\r\n - _Regain control of vertical white space_. Most of us are very particular how to organize horizontal whitespace, with strict rules for indentation and alignment. We are much less demanding on vertical whitespace. With braces, we cannot be, because vertical whitespace is fully determined by the number of closing braces. So two definitions might be separated by a single blank line, or by many (almost) blank lines if there are closing braces. One could avoid this by putting several closing braces on one line, but this looks weird and therefore has not caught on.\r\n\r\n - _Ease of learning_. There are [some powerful arguments](http://okasaki.blogspot.ch/2008/02/in-praise-of-mandatory-indentation-for.html) why indentation based syntax is easier to learn.\r\n\r\n - _Less prone to errors_. Braces are a weak signal, much weaker than indentation. So when\r\n    brace structure and indentation differ, we misunderstand what was written. The code below\r\n    exhibits a common problem:\r\n\r\n    ```scala\r\n    if (condition)\r\n      println(\"something\")\r\n      action()\r\n    ```\r\n\r\n    Indentation fools us to believe that `action` is executed only if `condition` is true. But of course that's not the case, because we forgot to add braces.\r\n\r\n - _Easier to change_. A situation like the one above happens particularly often when one adds the first `println` statement after the fact. To protect against modification problems like this, some people suggest to always write braces even if they only enclose a single statement or expression. But that sort of boilerplatey defensive programming is generally not considered good practice in Scala.\r\n\r\n - _Better vertical alignment_. In the most commonly used brace scheme, an if-then-else reads like this:\r\n\r\n    ```scala\r\n    if (cond1) {\r\n      ...\r\n    } else if (cond2) {\r\n      ...\r\n    } else {\r\n      ...\r\n    }\r\n    ```\r\n\r\n    Instead of nicely aligned keywords, we find weird looking closing braces `}` at the beginning of the most important lines in this code. We are all used to this, but that does not make it good. I have recently changed my preferred style to:\r\n\r\n    ```scala\r\n    if (cond1) {\r\n      ...\r\n    }\r\n    else if (cond2) {\r\n      ...\r\n    }\r\n    else {\r\n      ...\r\n    }\r\n    ```\r\n\r\nThis solves the alignment issue: The `if` and the `else`s are now vertically aligned. But it gives up even more control over vertical whitespace.\r\n\r\n# Impediments\r\n\r\nWhat are the reasons for preferring braces over indentations?\r\n\r\n - _Provide visual clues where constructs end_. With pure indentation based syntax it is sometimes\r\n   hard to tell how many levels of indentation are terminated at some point. In the code below, it is still easy to see that `j` is on the same nesting level as `g`. But if there were many more lines between the definitions, it might not be.\r\n\r\n    ```scala\r\n    def f =\r\n      def g =\r\n        def h =\r\n          def i = 1\r\n          i\r\n      def j = 2\r\n    ```\r\n   The proposal includes with _end comments_ a way to mitigate this issue.\r\n\r\n - _Susceptibility to off-by-one indentation_. It's easy to make a mistake so that indentation is off by a space to the left or the right. Without proper guards, indentation based syntax would interprete misalignment as nesting which can lead to errors that are very hard to diagnose.\r\n\r\n     In the proposal, an indented block is always preceded by a keyword that _must_ be followed by a nested expression or statement sequence. So it is impossible to accidentally introduce nesting by veering off to the right. I tried to experiment with the following additional rule, which would\r\nmake it unlikely to accidentally terminate nesting by veering off to the left:\r\n\r\n   - When terminating an indented block by a new statement that starts further to the left than the block, it is checked that the new statement aligns exactly with previous statements at the same indentation level. \r\n\r\n   That rule proves to be quite constraining (for instance it would outlaw the chained `filter` and `map` operations in the example below), so it is currently not implemented.\r\n\r\n - _Editor support_. Editors typically have ways to highlight matching pairs of braces. Without that support, it becomes harder to understand the nesting structure of a program. On the other hand,\r\nit's also straightforward to provide navigation help for indentation-based syntax, for instance by providing a command to go to the start of the previous or next definition at the indentation level of the cursor. According to @lihaoyi's comment, major editors do something like this already.\r\n\r\nBut neither of these points are the strongest argument against indentation. The strongest argument is clearly\r\n\r\n -  _Cost of change_. It would be expensive in many dimensions to change to indentation based syntax. To be sure, the present proposal for indentation based syntax still allows braces, so existing programs would still compile. But there are other costs as well. If the new indentation syntax is not universally adopted, we incur the cost that there will be two visually distinct ways to structure Scala code. People used to one way will be less comfortable reading the other. If the new indentation syntax _does_ take over as a universal standard (which I would expect), we have rendered outmoded all blogs, books, StackOverflow answers and other technical information that used the old syntax. It will be a long time to change all that, and the transition will be awkward.\r\n\r\n# Proposal in Detail\r\n\r\n## Expanded use of `with`\r\n\r\nWhile we are about to phase out `with` as a connective for types, we propose to add it in two new roles for definitions and terms. For definitions, we allow `with` as an optional prefix of (so far brace-enclosed) statement sequences in templates, packages, and enums. For terms, we allow `with` as another way to express function application. `f with { e }` is the same as `f{e}`. This second rule looks redundant at first, but will become important once significant indentation is added. The proposed syntax changes are as described in [this diff](https://github.com/lampepfl/dotty/pull/2488/commits/a3e51e6083653f386d9c9bbdb57993e39d86fd37#diff-4aac18b6866f1e8800b42f4255a82780).\r\n\r\n## Significant Indentation\r\n\r\nIn code outside of braces, parentheses or brackets we maintain a stack of _indentation levels_. At the start of the program, the stack consists of the indentation level zero.\r\n\r\nIf a line ends in one of the keywords `=`, `if`, `then`, `else`, `match`, `for`, `yield`, `while`, `do`, `try`, `catch`, `finally` or `with`, and the next token starts in a column greater than the topmost indentation level of the stack, an open brace `{` is implicitly inserted and the starting column of the token is pushed as new top entry on the stack.\r\n\r\nIf a line starts in a column smaller than the current topmost indentation level, it is checked that there is an entry in the stack whose indentation level precisely matches the start column. The stack is popped until that entry is at the top and for each popped entry a closing brace `}` is implicitly inserted. If there is no entry in the stack whose indentation level precisely matches the start column an error is issued.\r\n\r\nNone of these steps is taken in code that is enclosed in braces, parentheses or brackets.\r\n\r\n## Lambdas with `with`\r\n\r\nA special convention allows the common layout of lambda arguments without braces, as in:\r\n\r\n```scala\r\nxs.map with x =>\r\n  ...\r\n```\r\n\r\nThe rule is as follows: If a line contains an occurrence of the `with` keyword, and that same line ends in a `=>` and is followed by an indented block, and neither the `with` nor the `=>` is enclosed by braces, parentheses or brackets, an open brace `{` is assumed directly following the `with` and a matching closing brace is assumed at the end of the indented block.\r\n\r\nIf there are several occurrences of `with` on the same line that match the condition above, the last one is chosen as the start of the indented block.\r\n\r\n## Interpreted End-Comments\r\n\r\nIf a statement follows a long indented code block, it is sometimes difficult as a writer to ensure that the statement is correctly indented, or as a reader to find out to what indentation level the new statement belongs. Braces help because they show that something ends here, even though they do not say by themselves what. We can improve code understanding by adding comments when a long definition ends, as in the following code:\r\n\r\n```scala\r\n    def f =\r\n       def g =\r\n          ...\r\n          (long code sequence)\r\n          ...\r\n    // end f\r\n\r\n    def h\r\n```\r\n\r\nThe proposal is to make comments like this one more useful by checking that the indentation of the `// end` comment matches the indentation of the structure it refers to. In case of discrepancy, the compiler should issue a warning like:\r\n\r\n    // end f\r\n    ~~~~~~\r\n    misaligned // end, corresponds to nothing\r\n\r\nMore precisely, let an \"end-comment\" be a line comment of the form\r\n\r\n    // end <id>\r\n\r\nwhere `<id>` is a consecutive sequence of identifier and/or operator characters and `<id>` either ends the comment or is followed by a punctuation character `.`, `;`, or `,`. If `<id>` is one of the strings `def`, `val`, `type`, `class`, `object`, `enum`, `package`, `if`, `match`, `try`, `while`, `do`, or `for`, the compiler checks that the comment is immediately preceded by a syntactic construct described by a keyword matching `<id>` and starting in the same column as the end comment. If `<id>` is an identifier or operator name, the compiler checks that the comment is immediately preceded by a definition of that identifier or operator that starts in the same column as the end comment. If a check fails, a warning is issued.\r\n\r\n# Implementation\r\n\r\nThe proposal has been implemented in #2488. The implementation is quite similar to the way optional semicolons are supported. The bulk of the implementation can be done in the lexical analyzer, looking only at the current token and line indentation. The rule for \"lambdas with `with`\" requires some lookahead in the lexical analyzer to check the status at the end of the current line. The parser needs to be modified in a straightforward way to support the new syntax with the\r\ngeneralized use of `with`.\r\n\r\n## Example\r\n\r\nHere's some example code, which has been compiled with the implementation in #2488.\r\n\r\n```scala\r\nobject Test with\r\n\r\n  val xs = List(1, 2, 3)\r\n\r\n// Plain indentation\r\n\r\n  xs.map with\r\n       x => x + 2\r\n    .filter with\r\n       x => x % 2 == 0\r\n    .foldLeft(0) with\r\n       _ + _\r\n\r\n// Using lambdas with `with`\r\n\r\n  xs.map with x =>\r\n      x + 2\r\n    .filter with x =>\r\n      x % 2 == 0\r\n    .foldLeft(0) with\r\n      _ + _\r\n\r\n// for expressions\r\n\r\n  for\r\n    x <- List(1, 2, 3)\r\n    y <- List(x + 1)\r\n  yield\r\n    x + y\r\n\r\n  for\r\n    x <- List(1, 2, 3)\r\n    y <- List(x + 1)\r\n  do\r\n    println(x + y)\r\n\r\n\r\n// Try expressions\r\n\r\n  try\r\n    val x = 3\r\n    1.0 / x\r\n  catch\r\n    case ex: Exception =>\r\n      0\r\n  finally\r\n    println(\"done\")\r\n\r\n// Match expressions\r\n\r\n  xs match\r\n    case Nil =>\r\n      println()\r\n      0\r\n    case x :: Nil =>\r\n      1\r\n    case _ => 2\r\n\r\n// While and Do\r\n\r\n  do\r\n    println(\"x\")\r\n    println(\"y\")\r\n  while\r\n    println(\"z\")\r\n    true\r\n\r\n  while\r\n    println(\"z\")\r\n    true\r\n  do\r\n    println(\"x\")\r\n    println(\"y\")\r\n\r\n  // end while\r\n\r\n// end Test\r\n\r\npackage p with\r\n\r\n  object o with\r\n\r\n    class C extends Object\r\n               with Serializable with\r\n\r\n      val x = new C with\r\n          def y = 3\r\n\r\n      val result =\r\n        if x == x then\r\n          println(\"yes\")\r\n          true\r\n        else\r\n          println(\"no\")\r\n          false\r\n\r\n    // end C\r\n  // end o\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/lampepfl/dotty/issues/comments/302945954","html_url":"https://github.com/lampepfl/dotty/issues/2491#issuecomment-302945954","issue_url":"https://api.github.com/repos/lampepfl/dotty/issues/2491","id":302945954,"user":{"login":"polkovnikov-ph","id":2983969,"avatar_url":"https://avatars3.githubusercontent.com/u/2983969?v=3","gravatar_id":"","url":"https://api.github.com/users/polkovnikov-ph","html_url":"https://github.com/polkovnikov-ph","followers_url":"https://api.github.com/users/polkovnikov-ph/followers","following_url":"https://api.github.com/users/polkovnikov-ph/following{/other_user}","gists_url":"https://api.github.com/users/polkovnikov-ph/gists{/gist_id}","starred_url":"https://api.github.com/users/polkovnikov-ph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/polkovnikov-ph/subscriptions","organizations_url":"https://api.github.com/users/polkovnikov-ph/orgs","repos_url":"https://api.github.com/users/polkovnikov-ph/repos","events_url":"https://api.github.com/users/polkovnikov-ph/events{/privacy}","received_events_url":"https://api.github.com/users/polkovnikov-ph/received_events","type":"User","site_admin":false},"created_at":"2017-05-21T16:06:26Z","updated_at":"2017-05-21T16:06:26Z","body":"Great news.\r\n\r\n> If certain keywords are followed\r\n\r\nCould it be simplified to a much simpler rule: enclose every extra level of tabulation with a pair of curly braces? I've implemented significant whitespace this way in my own similar language, and had no issues with context at all.\r\n\r\n> Susceptibility to off-by-one indentation\r\n\r\nThat's easily solved by also enforcing tabs at the beginning of the line. As a side effect, there would be no more need to have an exact count of spaces per tab in style guides."}}
{"actor":{"display_login":"polkovnikov-ph"}}