proposal: Go 2: indentable multiline strings

[VinGarcia]

First a disclaimer, I don't think this proposal is appropriate for Go1.0 it would only make sense if we were planning on rehashing the syntax a little bit, but regardless I think it was important to share because it's something that can have practical benefits.

The Problem

When working with multiline strings we often have a dilemma:

1. Identing it to match the surrounding indentation
2. Not indenting it so we reduce the size of the string by having fewer whitespaces in it.

And both solutions are not ideal:

1. Identing to match the identation does not use gofmt which means that the confusion between tabs or spaces comes back
   causing problems on merges (which is the problem that actually convinced me to share this idea).
2. Not indenting breaks the abstraction of a function block and causes the programmers to feel that it's wrong, so this option is not often used even though it would be the less problematic option.

Please note that always using solution 2 doesn't solve the problem since it's hard to enforce it on other programmers, especially if I have just moved to a new company and the code was already written using the first method.

Proposed Syntactical Solution

With that problem in mind, I thought about a syntactical solution that would fix this problem and both reduce the number of whitespaces on the string and allow indentation to take place normally.

Note: In the example below note that I am not recommending specifically the use of the " character for this, we can use any character or combination of characters that makes sense.

func example() {
    exampleText := "Starting a multiline identable string
        "continuing the string
        "this is the final line of the string"

    // This should print "true":
    fmt.Println(exampleText == "Starting a multiline identable string\ncontinuing the string\nthis is the final line of the string")
}

In the example above the idea is that if the programmer doesn't terminate the string in the same line as it started it the compiler would interpret the end of the line as an actual \n character that was part of the string. Then it would ignore the beginning of the next line until it found another " character and then continue to parse the string until it found another line break or a quote " for ending the string.

Other slightly different forms of using this syntax would be:

func example() {
    exampleText1 := "start here
        "
        "continue here, after an empty line
        "" // end here just because I wanted the string to end in a \n

    exampleText2 := "
        "started on the previous line because I like
        "how the indentation looks like when I do it like this"
}

I would also expect the following code to cause a compilation error:

func example() {
    badExampleText := "start here and skip one line without any quotes...
    
        "end the string here" // Syntax error because of an unterminated multiline string
}

Expected Benefits

This idea should give us 3 benefits:

1. Have fewer merge conflicts regarding whitespaces on multiline strings (very common on big queries)
2. Have a better-looking code without growing the size of it with whitespaces
3. Normalizing how Gophers should write multiline strings so that everyone's code looks the same.

[seankhliao]

Please fill out https://github.com/golang/proposal/blob/master/go2-language-changes.md when proposing language changes

[VinGarcia]

Answering the requested questions, please note that I omitted the ones asking for examples of the proposed change since its already described above.

Also @seankhliao I was following this website for instructions on how to write a Go Proposal:

• https://github.com/golang/proposal

Maybe updating it to include a link to https://github.com/golang/proposal/blob/master/go2-language-changes.md in the section https://github.com/golang/proposal#the-proposal-process could be beneficial.

Template Questions:

Would you consider yourself a novice, intermediate, or experienced Go programmer?
Experienced

What other languages do you have experience with?
C/C++, JavaScript, Python, Bash, SQL, Ruby

Would this change make Go easier or harder to learn, and why?
If this was the only option it would be easier. Since it's currently not slightly harder.

Has this idea, or one like it, been proposed before?

In #36863 it proposes a solution to this same problem that is either not backward compatible or more invasive than this one.

This ticket mentioned above was flagged as a duplicate of #32590 which doesn't really address the indentation issue.

If so, how does this proposal differ?

So this proposal differs from both alternatives since it is backward compatible, less invasive and actually concerns the indentation issue instead of the issue of escaping backticks as in #32590

This proposal also doesn't require any new special characters inside strings unlike #36863

Who does this proposal help, and why?

Programmers trying to make the code more maintainable and normalized for all programmers.

Is this change backward compatible?

Yes

Show example code before and after the change.

func currentSyntax() error {
    query := `SELECT
      users.username,
      users.age,
      users.id,
      posts.id,
      posts.title,
      -- ... several other values ...
    FROM
      users
      JOIN posts ON users.id = posts.user_id
      -- ... possibly more joins
    WHERE users.id = $1
      AND posts.category in ( ... )
    `

    // ...

    err := db.QueryRow(query, params...).Scan(args...)
    if err != nil {
      return err
    }
}

func updatedSyntax() error {
    // Indented by gofmt:
    query := "SELECT
        "  users.username,
        "  users.age,
        "  users.id,
        "  posts.id,
        "  posts.title,
        "  -- ... several other values ...
        "FROM
        "  users
        "  JOIN posts ON users.id = posts.user_id
        "  -- ... possibly more joins
        "WHERE users.id = $1
        "  AND posts.category in ( ... )
        ""

    // ...

    err := db.QueryRow(query, params...).Scan(args...)
    if err != nil {
      return err
    }
}

What is the cost of this proposal? (Every language change has a cost).

One extra way of writing multiline strings that will have to be taught and learned.

How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?

• gofmt
• gopls
• go vet

As far as I can tell.

What is the compile time cost?

It should be negligible since each character would still be read only once.

What is the run time cost?

Very small, but with smaller strings, there is some performance improvement when parsing it.

Can you describe a possible implementation?

I can't.

Orthogonality: how does this change interact or overlap with existing features?

It occupies the same niche as the backtick strings.

Is the goal of this change a performance improvement?

No

If so, what quantifiable improvement should we expect?

Fewer merge conflicts regarding whitespace in multiline strings and more likelihood that people are going to write code in a same way, although this second statement would make more sense in Go2 if we could remove the backtick string, which would also have more consequences.

How would we measure it?

If necessary the only way I can think of is with a github crawler or similar that would find golang merge commits with multiline strings and check how many conflicts were due to the differences in spaces vs tabs.

Does this affect error handling?

No

Is this about generics?

No

[fzipp]

The nice property of the current raw string literal is that you simply place the cursor between ``, paste a multiline text from the clipboard, and you're done. You can also do the reverse, select the string contents, say an SQL statement as in your example, copy it to another tool, say a database query editor, and execute the query.

With your proposed syntax one would have to prefix each line with a " (and remove the prefixes again if you want to copy the string to another tool). That's only slightly less annoying than concatenating each line as a separate string literal with +.

I do not feel that the different indentation compared to the surrounding code is a big deal. With syntax highlighting it's even more obvious that something is a multiline string (one of the few instances in which syntax highlighting is actually helpful).

[VinGarcia]

You have a point @fzipp, but your use-case is mostly for data, my current use-case (which I have no idea if it is a common use-case) is for code, i.e. things that you might actually want to merge such as SQL queries.

So having a point in favor of backtick multiline strings does not breaks the argument in favor of indentable strings, since we would still get fewer conflicts on merge requests and if the user is not pasting text like you suggested having an auto-indented string with fewer whitespace characters is an advantage.

[VinGarcia]

Also regarding:

I do not feel that the different indentation compared to the surrounding code is a big deal. With syntax highlighting it's even more obvious that something is a multiline string (one of the few instances in which syntax highlighting is actually helpful).

You are assuming everyone is going to unindent these strings, but what I see most programmers doing is indenting it even if it is arguably not the best option.

So if I start unindenting the backtick strings in this codebase I would be breaking the local idiom, which is actually what gofmt tries to fix: It formats everything so we don't need to argue over which way is better to indent and etc.

[earthboundkid]

For SQL, put the code into a .sql file and import it with //go:embed.

[VinGarcia]

@carlmjohnson I think your suggestion would apply to any data we might want to put in a multiline string. I could always save it in a file and use the embed package (which is an awesome addition to the Go language btw).

But arguing against myself here, defending this as an idiom for the company (of embedding all SQL code) would probably be easier than trying to argue with them about how to indent multiline strings.

But what about small queries?

Reading SELECT foo, bar FROM foobar WHERE id = $1 might be more informative than database.Queries.ReadFile("select_user_by_id.sql"), but I must admit that not by a much, also small queries are less likely to have merge conflicts, so this would be a minor issue in this situation.

Yeah I think I rest my case, I still think this idea of indentable multiline strings is interesting, but I can work using embed as a default solution and it is easy to refactor existing code to use this concept, e.g.:

• database/
  • queries.go
  • select_user_by_id.sql
  • select_user_by_username.sql
  • etc

queries.go:

import "embed"

//go:embed *
var Queries embed.FS

So I will close this issue.