alias ll and issues with -q or $QUOTING_STYLE

[HaleTom]

Given a directory listing (quoted so you can see what's going on):

% ls -l --quoting-style=shell-escape
total 0
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:40 '$dollar'
-rw-r--r-- 1 ravi ravi 0 Sep  1 12:04 'a & b'
-rw-r--r-- 1 ravi ravi 0 Aug 31 21:52 'a b'
-rw-r--r-- 1 ravi ravi 0 Sep  2 16:40 'back'$'\b''space'
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:41 'double"'
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:42  normal
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:41 "single'"
%

Note the ^H or backspace between back and space.

ll shows it diffrently:

% ll --quoting-style=shell-escape
-rw-r--r-- 1 ravi ravi 0 Aug 31 21:52 [1]  'a b'
-rw-r--r-- 1 ravi ravi 0 Sep  1 12:04 [2]  'a & b'
-rw-r--r-- 1 ravi ravi 0 Sep  2 16:40 [3]  'back'$''space'
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:41 [4]  'double"'
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:42  [5]  normal
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:41 [6]  "single'"
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:40 [7]  '$dollar'

Note:

1. With the file normal, (the only file without quotes) it's shortcut [5] doesn't align.
2. The file 'back'$'\b''space' is no longer shown correctly by scm_breeze (the \b is removed) - I can't copy / paste this filename, eg if I want to do rm -- 'back'$'\b''space'.

Trying to print the empty file with cat:

% ge cat 3
/usr/bin/cat: '/home/ravi/code/scm_breeze/tmp/'\''back'\''$'\'''$'\b'\'''\''space'\''': No such file or directory
%

Things only seem to work properly with --quoting-style=literal:

% ll --quoting-style=literal
-rw-r--r-- 1 ravi ravi 0 Aug 31 21:52 [1]  a b
-rw-r--r-- 1 ravi ravi 0 Sep  1 12:04 [2]  a & b
-rw-r--r-- 1 ravi ravi 0 Sep  2 16:40 [3]  bacspace
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:41 [4]  double"
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:42 [5]  normal
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:41 [6]  single'
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:40 [7]  $dollar
% ge cat 3
%

Note the above successful shortcut - no error message from cat (empty file).

Any other quoting style will make for a non-existent shortcut, as the quoted string picked up from the ls output doesn't exist as a filename:

% ll -q
-rw-r--r-- 1 ravi ravi 0 Aug 31 21:52 [1]  a b
-rw-r--r-- 1 ravi ravi 0 Sep  1 12:04 [2]  a & b
-rw-r--r-- 1 ravi ravi 0 Sep  2 16:40 [3]  back?space
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:41 [4]  double"
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:42 [5]  normal
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:41 [6]  single'
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:40 [7]  $dollar

The shortcut still doesn't work:

% ge cat 3
/usr/bin/cat: '/home/ravi/code/scm_breeze/tmp/back?space': No such file or directory

[ghthor]

I've experienced this issue before. Any idea how difficult it is going to be to solve this?

[HaleTom]

@ghthor I believe that fixing this issue without race conditions is not trivial.

Others, please see #204 for some background on this issue before reading further.

This page lists all the possible quoting styles for GNU ls.

Note that --quoting-style=clocale and --quoting-style=locale will both expand into a myriad of quoting styles.

The best thing we can do (while still parsing ls output) to get accurate filenames is to do ls --quoting-style=literal and hope that no file has a \n in its name. (--show-control-chars not needed as output is not to the terminal, but rather captured in a variable)

We would still need to honour the user's quoting style:

• ls could be run again with the users quoting style
  • Race condition: deleting the first listed file and adding a new one between runs would invalidate all $e# variables. (BAD)
• We could print the filename obtained via --quoting-style=literal for human readability using printf '%q' "$filename", ignoring the user's quoting style
  • We could possibly allow a user to name a function to print filenames in their preferred quoting style.

Non solutions:

• Use find to produce \0 terminated filenames, generating $e# variables based on this listing - user's ls sort options may produce a different order to that generated by find. Also, running find on multiple targets may be ugly.
• Use the -i flag to print inodes and then match based on that - won't work in the hardlink case (ln without -s)
• Force shell-escape quoting style - this is only available in GNU ls, and only in quite recent versions.

[HaleTom]

For now I propose to prepend QUOTING_STYLE=literal to the ls call.

This will cause the shortcuts to work except in the case that a user calls ll with a --quoting-style=...:

% QUOTING_STYLE=literal ll
-rw-r--r-- 1 ravi ravi 0 Aug 31 21:52 [1]  a b
-rw-r--r-- 1 ravi ravi 0 Sep  1 12:04 [2]  a & b
-rw-r--r-- 1 ravi ravi 0 Sep  2 16:40 [3]  bacspace
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:41 [4]  double"
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:42 [5]  normal
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:41 [6]  single'
-rw-r--r-- 1 ravi ravi 0 Sep  1 13:40 [7]  $dollar

Note the letter k is removed in the presentation, but the variable is set correctly:

% printf '%q\n' "$e3"
/home/ravi/code/scm_breeze/tmp/back$'\b'space
%

@ghthor @ndbroadbent are you comfortable with this initial approach?

[ghthor]

This seems alright to me. Do we have any test cases for this committed? Don't feel that you have to make them in addition to these upgrades. I sort of need to explore this entire space of QUOTING_STYLE and what it means.

I'm somewhat concerned about this now that you've brought it all forward and I'd like to use this as a platform to teach and push people into more and more modern shell versions(Looking at you Apple).