Skip to content

Commit

Permalink
triple quoted strings: don't dedent interpolates
Browse files Browse the repository at this point in the history
Before:

  julia> s = """
             $("\n    ")
             """
  "\n\n"

After:

  julia> s = """
             $("\n    ")
             """
  "\n    \n"

Plus some minor cleanup of 333bb87.
  • Loading branch information
nolta committed Jun 26, 2015
1 parent 0e27875 commit 4f4938e
Show file tree
Hide file tree
Showing 2 changed files with 31 additions and 16 deletions.
44 changes: 28 additions & 16 deletions src/julia-parser.scm
Original file line number Diff line number Diff line change
Expand Up @@ -1632,33 +1632,45 @@
(define (take-char p)
(begin (read-char p) p))

; map the first element of lst
(define (map-first f lst)
(if (null? lst) ()
(cons (f (car lst)) (cdr lst))))

; map the elements of lst where (pred index) is true
; e.g., (map-at odd? (lambda (x) 0) '(a b c d)) -> '(a 0 c 0)
(define (map-at pred f lst)
(define (map-at- pred f lst i r)
(if (null? lst) (reverse r)
(let* ((x (car lst))
(y (if (pred i) (f x) x)))
(map-at- pred f (cdr lst) (+ i 1) (cons y r)))))
(map-at- pred f lst 0 ()))

(define (parse-string-literal s custom)
(let ((p (ts:port s)))
(if (eqv? (peek-char p) #\")
(if (eqv? (peek-char (take-char p)) #\")
(strip-first-newline
(map-first strip-leading-newline
(dedent-triplequoted-string
(parse-string-literal- 2 (take-char p) s custom)))
(list ""))
(parse-string-literal- 0 p s custom))))

(define (strip-first-newline lst)
(let* ((f (car lst))
(n (sizeof f)))
(if (and (> n 0) (eqv? (string.char f 0) #\newline))
(cons (string.sub f 1 n) (cdr lst))
lst)))
(define (strip-leading-newline s)
(if (and (> (sizeof s) 0) (eqv? (string.char s 0) #\newline))
(string.tail s 1)
s))

(define (dedent-triplequoted-string lst)
(let ((prefix (triplequoted-string-indentation lst)))
(if (length> prefix 0)
(map (lambda (s)
(if (string? s)
(string-swap s
(list->string (cons #\newline prefix))
#\newline)
s))
lst)
(map-at even?
(lambda (s)
(string-replace s
(list->string (cons #\newline prefix))
#\newline))
lst)
lst)))

(define (triplequoted-string-indentation lst)
Expand Down Expand Up @@ -1716,8 +1728,8 @@
(string-split- s sep (+ i (sizeof sep)) (cons (string.sub s start i) splits))
(reverse (cons (string.sub s start (sizeof s)) splits)))))

; swap all occurrences of a in s with b
(define (string-swap s a b)
; replace all occurrences of a in s with b
(define (string-replace s a b)
(string.join (string-split s a) b))

(define (parse-interpolate s)
Expand Down
3 changes: 3 additions & 0 deletions test/strings.jl
Original file line number Diff line number Diff line change
Expand Up @@ -937,6 +937,9 @@ s = " p"
@test """
foo
bar\t""" == "foo$(nl)bar\t"
@test """
$("\n ")
""" == "\n \n"

# bytes2hex and hex2bytes
hex_str = "d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592"
Expand Down

9 comments on commit 4f4938e

@StefanKarpinski
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@ScottPJones
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Is there no PR for this? 2) Isn't this what I asked about in my review?

@StefanKarpinski
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no PR. I don't recall you asking about this, but sure, that's possible.

@ScottPJones
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go read #11815

How does this treat totally blank lines? The old code ignored them when calculating the leading indentation, I had to treat them specially, so that if they were followed by interpolation, they were still counted when calculating the common prefix, but ignored when not, so things like:

"""
a
$var

b"""

@nolta
Copy link
Member Author

@nolta nolta commented on 4f4938e Jun 27, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

9cd6ad4:

julia> s = """
           a
           $var

           b"""
"a\n0\n\nb"

333bb87:

julia> s = """
           a
           $var

           b"""
"a\n0\n\nb"

4f4938e:

julia> s = """
           a
           $var

           b"""
"a\n0\n\nb"

@ScottPJones
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you are trying to show there. 4f4938e fixed one of the things I was trying to bring up in #11815. I thought a problem still remained with totally blank lines, but I can't reproduce now.

@StefanKarpinski
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nolta is showing that the example that you provided in that comment works identically in all three commits, so it seems irrelevant to this change.

@ScottPJones
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I lost the indenting in cut and paste, sorry, now I see the confusion. Also depends on what var is set to, if var is a set to a blank line, or something with indentation.

@nolta
Copy link
Member Author

@nolta nolta commented on 4f4938e Jun 29, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't understand what you're trying to say, Scott. It doesn't matter what var is set too:

julia> v = "
           "
"\n    "

julia> s = """
           a
           $v

           b"""
"a\n\n    \n\nb"

(this is true on 9cd6ad4, 333bb87, and 4f4938e)

4f4938e fixes a very obscure corner case, only triggered when interpolating raw string literals.

Please sign in to comment.