Terminating Tokens #72
Comments
munificent
added
the
feature
|
Example: { v
-- x}Since { v
-- }would have the same termination behavior as the first example, at least if you only look one token ahead. |
If I've understood this correctly it sounds like there'll be a mix of code with semicolons and code without semicolons? If so, it sounds like it could be pretty confusing. |
Not really. Look at Kotlin. The language allows having optional semicolons but to be honest, I have never seen anyone actually mixing both unless the code has been automatically converted to Kotlin from Java and user didn't have the diligence to clean it up. |
|
@MarkOSullivan94 Sounds like a case for dartfmt :-) |
|
Moving this to icebox for now; for more details, see https://github.com/dart-lang/language/blob/master/resources/optional-semicolons-prototype.md |
|
@mit-mit this is a feature I was really looking forward to, and can add a lot of "Coolest kid on the block" factor if done properly. People I've spoken to all thought Dart felt dated, it sounds dumb and trivial but making semicolons optional adds a lot of flash to a language. |
|
@PhilParisot Forgive me for saying so, but to me, it sounds incredibly arbitrary to say a language feels dated just because it uses semicolons. There are plenty of other features in Dart that make it feel like quite a fresh language (and there are a lot of reasons that Javascript, for example, feels like a dated language despite having semicolons be optional). |
|
@andrewackerman for sure and I completely agree I was just speaking about my personal experience and the people I spoke with which is obviously quite a limited perspective. But if anything can give Dart (and by consequence Flutter) more appeal, even if just a little, we have to explore it further. Flutter is gaining great momentum and revived Dart as a language but before Flutter, Dart was almost lost beyond the point of no return. I now spent a lot of resources on Flutter and I don't want to see it fail, and for it not to it needs to become widely adopted. I just see semicolons as a way to make it more appealing to some, maybe just a few, but a number we probably can't ignore. Flutter's ecosystem depends on it. Flutter's ecosystem is arguably its biggest drawback right now. Just my two cents. |
|
@PhilParisot The ecosystem is absolutely one of Flutter's biggest drawbacks. But that goes well beyond such a minor concern like optional semicolons. There needs to be more comprehensive tools/tooling, robust third party libraries, in-depth and easy-to-use learning resources, streamline some things that need streamlining... the list goes on and on for things that Dart/Flutter could improve. The make-or-break point on whether Dart and Flutter catches wind and takes off or eats dirt is not going to be some small feature that is both complicated to implement and entirely cosmetic with subjective-at-best benefits. We should not be trying to make Dart into a language that everyone will love, because that's not possible to achieve (and many languages in the past have fallen on their face for trying). Dart should focus on carving its niche and filling it well, and it's making great progress in doing that. Now the biggest things it needs require maturing the engine, fleshing out the more important and comprehensive features, and allowing the community to grow and develop their own contributions to the ecosystem. (And in my honest opinion, people who decide not to use Dart/Flutter for such a petty reason as it having mandatory semicolons aren't people we should be taking seriously anyway. Being a programmer means learning many different programming languages, and that means getting used to a multitude of different styles and syntaxes. People who aren't willing to do that are, in my eyes, aren't that serious about being programmers.) |
|
I think we should take all potential users seriously and users do have aesthetic preferences and those preferences do change in aggregate ways over time. If you opened a restaurant thirty years ago, formica tables and disco pumping through the stereo would be great. Today, those might impact the restaurant's success, even though the food is equally good either way. When we design a language's syntax, we have to be thoughtful about the expectations and psychology of our choices. Users have varying complex, often emotional relationships to the products they use. Of course, no one aesthetic choice will appeal to everyone because people are different. So what we do is aim for broad trends that appeal to a lot of people. |
|
Why don't you guys ask for Flutter's users opinion on the next "Flutter User Survey Q3"? |
The Flutter Survey has asked about semicolons, so we have some of that data now. I'm not sure what the rules are around sharing it publicly, and we don't have a lot of detailed data, but there is some. Of course, this data is biased towards existing Flutter users, which is good and bad. Good because it focuses on the priority of real users. But it also unweights language changes that might improve adoption since non-users aren't counted. |
You have a interesting point. |
Those platforms also use Objective-C and Java - languages that DO require semicolons. And while there are probably more people who primarily use Swift for iOS development than Objective-C (though this gets less true the more years of experience an iOS developer has), there are a TON of Android developers who use Java, not least of which because it's an extremely popular first language in universities and online courses for both mobile and general programming. And let's not also forget the people coming over from React(-Native) and the Javascript world, who still can't make up their collective mind on whether or not semicolons are the root of all evil and likely never will. I'd argue against making semicolons optional if only to prevent that hundred-year war from spilling into Dart as well.
I can agree that going from a semicolon language to a non-semicolon language or vice versa is quite jarring the first time. That being said, I still consider it quite the leap in logic to go from saying a language that mandates semicolons is quite a different experience than a language that omits them to saying the approach of mandating semicolons as a whole is "outdated".
I can definitely sympathize with the idea that semicolons are a bit of an inconvenience, but really it just takes experience and eventually, you get to the point where you don't even have to think about it. And once you get to that point, it's a lot easier to remember not to use them than the other way around. |
|
If "semi-colons" just can't find a proper end, why not give the users the choice to use it or not? Maybe just a flag on the ".yaml" could be enough. |
Because it's not that easy. Making a language omit semicolons is already a somewhat complicated choice since you need to treat some whitespace as a statement terminator but also be able to tell when it's not terminating the statement. Making a language use semicolons as an option makes things even more complicated since you now need to be able to support when statements are explicitly terminated. And all this complication is assuming that you are designing a language with those rules from scratch; adding the rules to an already developed and in production language means you have to change the language rules to incorporate all these considerations without making any breaking changes (or at the very least as few as possible). Making semicolons optional at this stage of Dart's life is going to be a very involved task, and it's going to have long-reaching consequences. As such, it's going to require an immense amount of effort from the Dart team, effort that could be spent on other more immediate, more universally beneficial, and/or less controversial concerns. And it's not like they can just work on this quietly while they work on other things since every language feature going forward would also need to be designed with optional semicolons in mind. For these and a hundred other more specific reasons, this is not a decision that can be made so lightly. And since this is all over a language feature that is purely cosmetic and entirely subjective whether it's at all beneficial, my vote is to please just don't bother. Focus instead on making Dart more powerful, more flexible, and more extensible than ever before, and don't essentially press pause on Dart's development for months while the Dart team figure out how to approach this. |
|
My best guess is that Dart, with its current syntax, cannot be modified to make semicolons unnecessary without a massive change to how code is written (and a migration of all existing code). Bob has already tried to deduce the rule system that would be needed to account for the way we already break statements, so that we won't have to modify too much code. The rule system was too complicated to be practically useful; users will need to remember the rules, and nobody would be able to do that. The most likely incremental approach would be to make the syntax indentation dependent. That has other issues (it's nice enough when code is written, but how do you make a formatter for something where parsing depends on formatting, what happens if you cut-n-paste code with different indentation, how will an editor work with incomplete or unaligned code). It's not impossible, but it has its own issues. So, maybe we should just create a completely new syntax, |
|
Creating a new language syntax from scratch would be just as much work if not more than to incorporate optional semicolons into existing Dart. Now, once Dart has better tooling support, someone (not in the Dart team) could independently create a DartX or whatever that will either transpile into Dart or compile into the same bytecode, but as it stands, it makes even less sense to justify essentially creating an entirely new official language that would need to be supported alongside Dart just to make semicolons optional. |
Any new syntax would have a lot in common with existing Dart and other languages. We wouldn't say, reboot the entire expression syntax from scratch. Large portions would remain the same. |
|
@navicstein Not unless there is some obscure VS Code extension for it. There is no tangible plan to make semicolons optional in Dart, though there is an intent to revisit language-wide syntax and ergonomics once nullable types hit production. |
|
If there's a will there's a way.... |
|
As a general engineering principle, less is more. Any syntax that doesn't need to exist, shouldn't. It only creates room for frequent mistakes (hence the plethora of "missing semicolon" jokes) and potentially inhibits readability. Maybe I'm the odd one out or maybe I'm just one of a small number of uninvested potential users willing to bother commenting but I landed here because I got excited about Flutter and started digging into Dart, only to then discover that it surprisingly requires semicolons. Having spent the last decade largely working with JavaScript, Python, Kotlin, Ruby and shell scripting, it just seemed so dated and unnecessary. I don't miss C or Java and I'm definitely not looking to go back to the overly verbose syntax I had to write 20 years ago. I'm not sure yet if Flutter is worth the pain and nuisance. Languages should get out of the way, not in it. Verbosity and rigidity is what drove me away from Go. Kotlin is an absolutely brilliant example of how to design a language that's exciting to use. Also, FWIW, Python has seen huge adoption and the syntactic whitespace hasn't inhibited that - plus, whitespace is optional when brackets and semicolons are used. Syntactic whitespace may not be awesome from a grammar standpoint but it can make the user experience awesome. With all that said, I know Google doesn't have a reputation of listening to users so I was already hesitant about trying to adopt another Google technology. Just thought I'd try again to share some feedback. |
|
Having also spent the last decade working in languages like Java, Dart, and C#, but also languages like Javascript, Swift, Kotlin, and Python, the presence of semicolons in a language (or lack thereof) has never been an issue - a language has it or it doesn't, and it doesn't take much of a mental effort to "switch modes" between the two. Likewise, having or not having semicolons has never impacted readability in the slightest in real-world code that I have written or seen, much like the inclusion of a period at the end of a single sentence doesn't suddenly make me hyper-aware of its existence. The only programmers I have ever seen who have extensive issues with semicolons being included/excluded are people who operate exclusively in one idiom or the other, and they seem to have somehow generated their own subjective arguments why their side is objectively better. Programmers such as myself who switch frequently between them don't generally care as long as the language is consistent with its choice. It takes a little bit of effort to get to that point, sure, but it's not that big of a deal and, frankly, is part of the job description. Sure, I may have my personal opinions on the matter (I greatly dislike Python's use of indentation for blocks as I think it makes longer code files much harder to visually parse), I don't take to the streets with my torch and pitchforks because it's just that - a subjective opinion on an inherently subjective matter, and I certainly won't let those opinions limit my growth as a developer. I would like to dispel one notion though. There is nothing "outdated" about a language utilizing semicolons as a terminator. FORTRAN is one of the oldest programming languages in existence (predating C by 15 years) and it doesn't use semicolons, and it was shortly followed by ALGOL which did, and then Lisp which didn't. For as long as there have been high-level programming languages, there have been languages that used semicolons and there have been languages that didn't use semicolons. The removal of semicolons isn't a recent development or modern fad - it's a cosmetic choice that has co-existed with the opposite idiom for both of their relative histories. At any rate, what you say is true - less is more. However, that logic can also be applied to Dart's current situation: don't exert unnecessary effort. For better or for worse, Dart already uses semicolons, so to remove them now would be a monumental undertaking. As I have said in this very thread more than once, as passionate as people get about this topic, at the end of the day it is a cosmetic change with little practical benefit. I would much rather the Dart team work on improving the language in other more practical ways (such as ADTs, data enums, and the upcoming NNBD) than spending time on this. I will close with this. If you think Dart (and, by extension, Flutter) are going to be a headache and a pain simply because they use semicolons, then you have sheltered yourself as a developer. You have lost the ability to be flexible and reactive by restricting yourself to a particular "family" of languages, and you have established a protective little bubble that you refuse to operate outside of. Not that there's anything wrong with picking a preferred specialty, of course, but that does tend to make you somewhat biased when weighing in on matters that fall outside of that specialty. Having such a stance as a developer means you are going to miss out on a lot of emerging frameworks and technologies simply because you can't be bothered to train yourself to be flexible in what tools you work with. (And I still don't know how your complaints regarding the verbosity of C, Java, and Go have anything to do with semicolons.) |
|
That's quite the leap of assumptions but ok. Yes, semicolons have come and gone and still exist in plenty of languages. Newer and more popular languages today have made them optional. It's a trend and that makes them feel dated. I have worked with an enormous variety of tech and I prefer things that don't impose more decisions or limitations than necessary on me as a developer and that seek to make life easy. To me this is bigger than semicolons. My concern is the language design philosophy and decision making approach. More than having something that is technically cool, I need something that is practically cool. In my experience, that tends to go hand-in-hand with the willingness of a technology creator's willingness to listen to users. Creators define the community and the community largely defines the experience. ✌️ |
Newer programming languages like what? Julia doesn't use them, Elm does. Swift doesn't use them, Rust does. Kotlin doesn't use them, Dart does. It seems pretty even across the aisle of "newer" languages in terms of which ones use them and which ones don't. And as far as the "popular" languages, the following is the top 10 programming languages in popular use according to the KIOBE index at the time of this writing (* = semicolons required, ** = semicolons optional):
Of those 10, 5 of them require semicolons, 2 require no semicolons, and 2 have optional semicolon use (though 1 of which still highly recommends their use for security purposes). Don't confuse "trendy" with "popular" or "modern". Only time tells which languages actually stick around, and so far time has told that having or not having semicolons plays no role whatsoever in that distinction.
There is no objective practical benefit toward requiring or omitting semicolon use. It is a purely subjective preference.
And my concern is the "don't waste effort on cosmetic improvements when there are more practical and useful things to work on" approach. This would've been a good discussion to have years ago when the Dart language spec was first being deliberated. Now it's largely pointless, as making semicolons optional at this point would be FAR more trouble than it's worth, which is especially egregious considering the mile-long list of other things they could be working on instead.
And which members of the community should the creators listen to? From what I can tell, there are two equally-sized vocal minorities that either demand mandatory semicolons or demand their removal, and everyone else doesn't really seem to ultimately care. |
|
You can listen to the users that agree with your personal preferences or collect data about what makes devs happy and act on it. Just depends on the goals of the language and who it's trying to attract: people like you or people in general. Anyway, I don't care enough to argue a about this but it's worth pointing out that the TIOBE index is not useful for understanding what languages are actually most popular for new projects or what languages developers actually want to use. Github stats and the annual Stack Overflow surveys are better ways to gauge that. |
Well, there's your problem. The Dart team made their decision a long time ago on which direction they want to go where semicolons are concerned so that ship was sailed. And as far as attracting "people in general", that's not possible. Semicolon terminators are a polarizing issue so you will never please everyone, and making it optional is even worse in many ways. When you try to please everyone, you end up pleasing no one. Might as well just stick with what you have.
StackOverflow's 2020 survey listed Rust as the most loved language, and it won by A LOT. However, it's also worth mentioning that it is a list made by random programmers visiting StackOverflow and that the vast majority of respondents had less than 5 years of professional experience. I have similar reservations with Github stats as literally anyone can throw code up into a repository and Github is by far the most popular repository platform out there. For example, Github lists Javascript as its most popular language and Python as a close second, which sounds impressive until you realize that a growing majority of the aging pip and node ecosystems consist of low quality or abandoned packages and that every single one of them has a repository somewhere. A surface glance also ignores two very crucial facts about those respective languages: Javascript is the de facto language of the web, and Python is the de facto language of data science and machine learning (not to mention a very capable scripting language), so they are naturally going to have massive numbers. What's far more interesting is that Java, despite being an ancient language by modern standards with many glaring flaws, is in third place, and Go, being a far younger language than any of those three, is not far behind. StackOverflow surveys only show what is trendy and Github stats fail to show the whole picture. TIOBE may not measure popularity for new projects, but so what? New projects don't measure popularity or longevity, just what people are experimenting with. Instead, TIOBE measures what has been proven useful and marketable by the industry at large, not by whatever individual programmers happen to be tinkering with at a given time. |
|
Hi there, Michael from the language team. Thanks for all the passionate comments. I'd like to clarify the current status. This feature has been "put on ice", not because we feel there is a lack of demand, but because we after multiple investigations concluded that it wasn't practically possible to do this within the current Dart syntax. Bob (@munificent) wrote up a detailed explanation. For example, in Dart variable declarations don't start with a keyword, but rather just the type. That makes it hard to disambiguate with our statement grammar. Further, we have very few reserved keywords, so even a basic line like Right now we're in the middle of adding sound null safety. This is a very, very large change. Layering a change to our core language syntax op top of that would add a tremendous amount of risk to an already risky feature, which we found unacceptable. We may revisit this feature again in the future, but would then likely have to do so in the context of making some fairly fundamental changes to the Dart syntax. |
|
I think the fundamental difference between dart and java, rust and many other languages is that most people learn dart just for flutter, and this part of people, including me, is very important to the programming user experience. Also, thanks to munificent's article, I realized that my two concerns (type declaration and not having to write semicolons) turned out to be relevant in DSL parsing let a:string = 3 Or we can expect a big break change in dart 3 to bring these changes someday
Looking forward to this moment |
Now that null-safe-by-default has landed, yes, please, follow, for example, Scala's lead and put types on the right. If optional semi-colons come about naturally from that change, fantastic, kill N birds with one stone. And thanks for being open to pushing Dart into the brave new world of data classes, pattern matching, immutable collections (one can dream), meta programming, etc. |
|
@godenji That's a massive breaking change to the entire language which will affect every code repository, package, Flutter plugin, app, and server backend that has ever been written in Dart. It's not worth it for such a subjective cosmetic feature. |
Actually, not. Just making it optional like Kotlin, will break anything. |
|
@shinayser Please read the Dart team's reason for putting this feature request on ice (not to mention the comment that I was responding to). TL;DR - Making semicolons optional introduces too many compiler ambiguities to be practically viable with the language syntax as it is right now with type information coming before identifiers (e.g. |
At the time of release of dart-types-on-the-right, exactly zero existing code bases will break. And depending on the implementation, code bases that elect to upgrade to this hypothetical dart version, won't break either (i.e. if the language maintainers keep deprecated types-on-the-left around for a few releases). Look, types-on-the-right isn't just about actually enabling optional semi-colons; it's about improving the language. Scala, Swift, Kotlin, etc. put the type definition on the right for improved type inference, and, as a rather nice side effect, ambiguities like those mentioned in the why-we-can't-implement-optional-semi-colons document, simply don't exist -- shouldn't be a surprise then that semi-colons are optional in all modern types-on-the-right languages. That Flutter/Dart developers have brought up pain points such as:
indicates that all is not well in the house of Dart, this despite the absolutely compelling product that is Flutter. Developers coming from other modern languages will natually be taken aback by the current state of affairs. Fortunately at least some of the above items will be addressed, hopefully much sooner than later. |
|
@godenji Of all the things you listed, the first item is the only item that requires type-on-the-right to be implemented. Everything else might be helped with such a change, but it is far from necessary (although granted, neither of us are in a position to concretely make that claim one way or the other, so we would have to wait for a Dart team member to weigh in on that). And several of the things you listed are features of functional programming, and seeing that Dart is not an FP language, there's no real need for it to support let alone prioritize. Also, how would it break zero code-bases? It's a syntactical 180. Sure, they can release scripts or compiler flags or whatever to support/convert code written for previous versions, but that would require package developers to actually use them, which means old packages that aren't actively maintained but still provide a useful utility are doomed to fall by the wayside. The Dart team had a hard enough time supporting this transition with null-safety, and that was a change that involved little in terms of breaking syntax and most of what it did break in existing code bases were the product of bad coding practices that needed refactoring anyway.
I really wish people would stop using the lack of semicolons as evidence of a semicolon being modern. Semicolon-less languages have been around just as long as languages using a semicolon or some other delimiter. In fact, some of the first text-based programming languages written simply assumed one statement per line, thus predating the need for delimiters entirely. A design choice that predates semicolons by definition cannot be considered a "modern feature". It's a cosmetic choice with no objective effect on development within the language. |
This really hits the nail on the head. The only people I see complaining about a language having semicolons are the ones who have pigeon-holed themselves into only using the same handful of mainstream languages that don't use them (Kotlin, Scala, Swift, Javascript/Typescript) using the same tired arguments that are subjective at best and betrays their inexperience at worst (it looks better, it's cleaner/decluttered, it's "modern", etc.). Instead of insisting that every language behaves like what you've convinced yourself is the "best", recognize that different languages are made by and for different mindsets and different practical uses and make yourself more flexible as a developer (which you should be doing anyway). This is like complaining that Japanese is a dated, cluttered, or otherwise inferior language because "all modern languages use gendered nouns" (which is a feature of many European languages). |
|
When you're constantly switching between plugins and dart code, it becomes hard to write semicolons in dart, while omitting them in Kotlin/Swift. |
|
@SashaFarkas As someone who frequently switches between Dart, Kotlin, Javascript (with a semicolons-on style requirement), Python, Swift, and the occasional Rust and Scala for a living, no it doesn't. It's a learned skill called fluency, which you won't learn by just complaining about how hard it is and wishing every language was one way or the other. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
I am sad to see this closed. Is there other way to not have to type semicolons after every statement? It may not be a big issue / only cosmetic change for someone, but it is a big issue for me. It is a subjective preference. I'm not saying to remove it (because it will cause big backward compatibility issues), but just make it optional or have compiler infer it. |
|
@hlung this issue is not closed. |
It's not officially closed, but it has been put on ice due to it requiring increasingly complex rule changes to the language syntax and thus requiring far more bandwidth on the team's effort estimate than they are currently willing or able to allocate in favor of other priorities. It may be revisited in the future, but it has been tabled for now. |
Making it optional is even more of an ask than removing it entirely, and both are bigger asks than you realize. Read the link in my other reply for details. |
|
Hi, I marked a couple of comments as "off-topic". They aren't totally off topic because they are somewhat talking about other languages that do or do not have optional semicolons. But they are quickly veering into language war territory and absolutely nothing good has ever come from a discussion like that. The current status of optional semicolons for Dart is:
It's not particularly helpful to argue with each other on this thread to try to sway others to your side. My experience is that most programmers are only persuadable on semicolons by getting first-hand experience with a language that takes one or the other approach. Words don't seem to carry much weight because programmers have an almost tactile relationship with language syntax. It's like trying to talk someone out of their favorite ice cream flavor. |
This is a proposed solution #69, optional semicolons.
I'm calling it "terminating tokens" to differentiate it from other semicolon insertion rules we might investigate. In this proposal, the rules are roughly:
A "terminating token" is a token with a newline between it and the previous token.
In places where a semicolon is currently allowed, if the next token is a terminating token, no semicolon is required.
This leads to ambiguity since the existing grammar rules ignore whether or not a token is terminating. To resolve this, the grammar is changed so that a newline cannot be ignored in each place where that leads to a known ambiguity.
The "known ambiguity" part means that newlines are only used to act as implicit semicolons or to resolve ambiguities. A newline won't cause a syntax error on the next line. This lets us ignore most newlines, which minimizes the amount of migration and breakage.
On the other hand, the rules are fairly complex and subtle. We may not catch all of the ambiguities, and it may make it harder to introduce syntax in the future because it might cause a new ambiguity.
The text was updated successfully, but these errors were encountered: