Skip to content

Commit

Permalink
Write files line-wise
Browse files Browse the repository at this point in the history
Instead of joining the list of lines into a single string and writing
that at once using `write-string`. This is much more efficient for large
files as otherwise the majority of runtime is spent joining the list
to a string.

This was discovered while editing large files in Guix (e.g. the package
definitions from gnu/packages/python-xyz.scm) using the edward text
editor.
  • Loading branch information
nmeum committed Jul 31, 2024
1 parent 7703c0e commit 0eea82e
Show file tree
Hide file tree
Showing 3 changed files with 19 additions and 9 deletions.
16 changes: 13 additions & 3 deletions lib/ed/cmd.scm
Original file line number Diff line number Diff line change
Expand Up @@ -503,15 +503,25 @@
(values #t (string-copy fn 1))
(values #f fn))))

;;> Write given data to given filename. If filename starts with `!` (i.e.
;;> Write a list of `lines` (represented as a string without a terminating
;;> newline) to a given `filename`. If `filename` starts with `!` (i.e.
;;> is a command according to [filename-cmd?](#filename-cmd?)), write data
;;> to standard input of given command string.
;;>
;;> Returns amount of bytes written to the `filename` on success and false
;;> if an error occurred.

(define (write-to filename data)
(define (write-lines filename lines)
(let-values (((fn-cmd? fn) (filename-unwrap filename)))
(with-io-error-handler fn
(lambda ()
(let ((proc (lambda (port) (write-string data port))))
(let ((proc (lambda (port)
(fold (lambda (line num)
(let* ((line (string-append line "\n"))
(bytes (string->utf8 line)))
(write-bytevector bytes port)
(+ num (bytevector-length bytes))))
0 lines))))
(if fn-cmd?
(call-with-output-pipe fn proc)
(call-with-output-file fn proc)))))))
Expand Down
2 changes: 1 addition & 1 deletion lib/ed/cmd.sld
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
;; Utility procedures to define custom editor commands.
(export define-edit-cmd define-input-cmd define-print-cmd define-file-cmd
parse-cmd-char parse-cmd parse-re parse-re-pair unwrap-command-list
exec-command-list parse-filename parse-file-cmd write-to
exec-command-list parse-filename parse-file-cmd write-lines
read-from call-when-confirmed subst-nomatch-handler
filename-cmd? register-command exec-command-list-interactive)

Expand Down
10 changes: 5 additions & 5 deletions lib/ed/posix.scm
Original file line number Diff line number Diff line change
Expand Up @@ -356,12 +356,12 @@
;;;;

(define (exec-write editor lines filename)
(let ((fn (editor-filename editor filename))
(data (lines->string (editor-get-lines editor lines))))
(unless (write-to fn data)
(let* ((fn (editor-filename editor filename))
(lines (editor-get-lines editor lines))
(written (write-lines fn lines)))
(unless written
(editor-raise "cannot open output file"))
;; Assuming write-to *always* writes all bytes.
(editor-verbose editor (count-bytes data))
(editor-verbose editor written)

(unless (filename-cmd? filename)
(if (empty-string? (text-editor-filename editor))
Expand Down

0 comments on commit 0eea82e

Please sign in to comment.