Clink settings and history file changed

[i-s-o]

Purpose of the issue

• Bug report (encountered problems/errors)

Version Information

1.3.17

Description of the issue

The new version of Clink appears to have revamped its settings system:

1. Instead of %CMDER_USER_CONFIG%\settings, it is now %CMDER_USER_CONFIG%\clink_settings (i.e. the default settings or existing settings are not picked up).
2. The actual names of settings keys are now different (e.g. history.max_lines instead of history_file_lines).
3. The history is now saved to %CMDER_USER_CONFIG%\clink_history instead of %CMDER_USER_CONFIG%\.history.
   a. the .history file is ignored for existing installations
   b. history command (i.e. alias) is not working properly

See: https://chrisant996.github.io/clink/clink.html#configuring-clink

Old

C:\tools\cmder
λ clink set
Available options:

ctrld_exits ............... 1      Pressing Ctrl-D exits session
esc_clears_line ........... 1      Toggle if pressing Esc clears line
match_colour .............. -1     Match display colour
exec_match_style .......... 2      Executable match style
space_prefix_match_files .. 1      Whitespace prefix matches files
prompt_colour ............. -1     Colour of the prompt
terminate_autoanswer ...... 0      Auto-answer terminate prompt
history_file_lines ........ 10000  Lines of history saved to disk
history_ignore_space ...... 0      Skip adding lines prefixed with whitespace
history_dupe_mode ......... 0      Controls how duplicate entries are handled
history_io ................ 1      Read/write history file each line edited
history_expand_mode ....... 3      Sets how command history expansion is applied
use_altgr_substitute ...... 1      Support Windows' Ctrl-Alt substitute for AltGr
strip_crlf_on_paste ....... 2      Strips CR and LF chars on paste
ansi_code_support ......... 1      Enables basic ANSI escape code support

Settings path: C:\tools\cmder\config/settings

New

C:\tools\cmder
λ clink set
clink.paste_crlf                 space
clink.path
clink.promptfilter               True
cmd.auto_answer                  off
cmd.ctrld_exits                  True
color.cmd                        bold
color.doskey                     bright cyan
color.filtered                   bold
color.hidden
color.input
color.interact                   bold
color.modmark
color.readonly
doskey.enhanced                  True
exec.cwd                         True
exec.dirs                        True
exec.enable                      True
exec.path                        True
exec.space_prefix                True
files.hidden                     True
files.system                     False
files.unc_paths                  False
history.dont_add_to_history_cmd  exit history
history.dupe_mode                erase_prev
history.expand_mode              not_quoted
history.ignore_space             True
history.max_lines                5000
history.save                     True
history.shared                   False
lua.break_on_error               False
lua.break_on_traceback           False
lua.debug                        False
lua.path
lua.traceback_on_error           False
match.ignore_case                relaxed
match.sort_dirs                  with
match.wild                       True
readline.hide_stderr             False
terminal.emulation               auto
terminal.modify_other_keys       True

[daxgames]

It never fails. Will take a look.

[chrisant996]

I assume Clink v1.x pre-release builds are being included because v0.4.9 has too many issues, e.g. the hooking crash?

While it's intentional that Clink behaves differently now, maybe there's a way to do a migration and/or a change in Cmder that could make the transition more automatic.

Anyway, I'll take a look this evening.

[daxgames]

Thats what I was thinking as well.

[daxgames]

Just FYI. I updated the release with a note regarding this issue.

@chrisant996

I was starting to look into how to have Cmder's init.bat migrate Clink settings and history when I realized I would need a lot of help from you translating Cmders default Clink vendor/settings.default file to a new vendor/clink_settings.default THEN I noticed you opened an enhancement request on Clink to possibly migrate settings and history.

Thank you for your support.

[chrisant996]

Clink v1.1.14 includes automatic migration.

History should be fully migrated.
Settings should be migrated as much as possible.

The documentation now has a section on upgrading, and has various compatibility notes called out.

[chrisant996]

@i-s-o can you clarify this point:

3. b. history command (i.e. alias) is not working properly

Can you describe specifically what about the history command isn't working properly?
Or did you just mean that since history wasn't migrated, naturally there's nothing for the history command to report?

Thank you for reporting the issue! Hopefully v1.1.14 addresses the migration issues (see previous comment).

Since you already have clink_history and clink_settings files now, to make migration happen you'll have to delete those two files manually. Sorry for the inconvenience!

[daxgames]

Cmder defaults provide a doskey alias for the history command like:

history=cat -n "%CMDER_ROOT%\config\.history"

In my testing of 1.1.10 it seems the new clink_history filename appears to be variable? I currently have a clink_history, clink_history_[nnnn], and a clink_history_[nnn]~.

Testing 1.1.14 now.

[daxgames]

Changing the Cmder history doskey alias for history to history=clink history results in:

dgames@DTG-SP4 ~\cmder - dev (master -> origin) λ alias history=clink history

dgames@DTG-SP4 ~\cmder - dev (master -> origin) λ history
'clink' is not recognized as an internal or external command,
operable program or batch file.

It seems Clink history is now kept in two files clink_history(previous session history), and clink_history_[nnnn](current_session history). Typing clink history combines these two into a session specific history and outputs the indexed list to the screen.

I am doing the following to resolve the history command issue.

From the command line:

alias history="^%cmder_root^%\vendor\clink\clink.bat" history

which results in an alias like:

dgames@DTG-SP4 ~\cmder - dev (master -> origin) λ alias history
history="%cmder_root%\vendor\clink\clink.bat" history

I will also make init.bat migrate this alias to the new 1.x.x clink alias and change it in the Cmder default aliases file.

All available in a future release.

[daxgames]

Testing 1.1.14:

1. I deleted my %cmder_root%\config\clink_history and %cmder_root%\config\clink_settings files.
2. Launching a new session Clink migrated my %cmder_root%\config\.history file to %cmder_root%\config\clink_history and left the %cmder_root%\config\.history file.
3. Did nothing to my %cmder_root%\config\settings file so I do not have a %cmder_root%\config\clink_settings.

The %cmder_root%\config\settings file derives from the %cmder_root%\vendor\clink_settings.default file. I assume since settings is a best effort migration there is nothing to migrate?

[daxgames]

Try artifacts from here: https://ci.appveyor.com/project/MartiUK/cmder/builds/36991693/artifacts

[chrisant996]

Oh, I see.

1. Clink itself defines a history alias with various flags and commands. Cmder can simply do nothing, and then the history command should work.

2. Clink has two history modes, described in the docs. In shared mode all instances share a single master history file, and an instance sees new commands from the other instances as soon as it starts a new prompt. In non-shared mode each instance has a separate session history file, and session files aren't appended to the master file until their session ends. This ensures each session's own history always appears at the tail of the history in that session.

3. Migration for settings is "lazy" or "on read". In other words, reading always migrates old settings -- they aren't "locked in" until you run clink set and change a setting. This choice was intended to avoid race conditions if you start multiple Clink instances at once -- so that they don't contend with each other all trying to update the settings concurrently. It's possible to add locking and force writing a new clink_setting file, that just gets more complicated is all.

[daxgames]

@chrisant996 If I type clink set history.shared true Clink writes the below to the %cmder_root%\config\clink_settings file.

# name: Shell command completions
# type: color
color.cmd = bold

# name: Doskey completions
# type: color
color.doskey = bright cyan

# name: Filtered completion color
# type: color
color.filtered = bold

# name: For user-interaction prompts
# type: color
color.interact = bold

# name: Message area color
# type: color
color.message = default

# name: Sets how command history expansion is applied
# type: enum
# options: off,on,not_squoted,not_dquoted,not_quoted
history.expand_mode = not_dquoted

# name: Skip adding lines prefixed with whitespace
# type: boolean
history.ignore_space = False

# name: The number of history lines to save
# type: integer
history.max_lines = 10000

# name: Share history between instances
# type: boolean
history.shared = True

The above file contents contain the setting I manually set of history.shared = true as well as settings migrated from my original %cmder_root%\config\settings file like history.max_lines = 10000

I assume this is my new %cmder_root%\vendor\clink_settings.default file based on the lazy migration you described above?

[daxgames]

I have changed the fix PR #2452 again. New binaries to try: https://ci.appveyor.com/project/MartiUK/cmder/builds/36993661/artifacts

[chrisant996]

@chrisant996 If I type clink set history.shared true Clink writes the below to the %cmder_root%\config\clink_settings file.

# name: Shell command completions
# type: color
color.cmd = bold

# name: Doskey completions
# type: color
color.doskey = bright cyan

# name: Filtered completion color
# type: color
color.filtered = bold

# name: For user-interaction prompts
# type: color
color.interact = bold

# name: Message area color
# type: color
color.message = default

# name: Sets how command history expansion is applied
# type: enum
# options: off,on,not_squoted,not_dquoted,not_quoted
history.expand_mode = not_dquoted

# name: Skip adding lines prefixed with whitespace
# type: boolean
history.ignore_space = False

# name: The number of history lines to save
# type: integer
history.max_lines = 10000

# name: Share history between instances
# type: boolean
history.shared = True

The above file contents contain the setting I manually set of history.shared = true as well as settings migrated from my original %cmder_root%\config\settings file like history.max_lines = 10000

I assume this is my new %cmder_root%\vendor\clink_settings.default file based on the lazy migration you described above?

Three thoughts regarding the default settings file:

1. Remove the color.* settings. Those are all the default values, and it looks like there's a bug that's accidentally writing out default values for colors (default values are meant to be omitted from the settings file).
2. What is the reason for history.expand_mode to be "not_dquoted" instead of the default "not_quoted"? I ask because depending on the reason, it could potentially be desirable to change either Clink's default or Cmder's default (or maybe both are fine as-is).
3. Maybe Important: Under what circumstances will Cmder create a clink_settings file? It should only be created if neither settings nor clink_settings exist. If a settings file already exists and Cmder were to copy a default clink_settings file into place, then that would defeat the migration code and accidentally discard the user's current settings.

[daxgames]

1. Ok
2. Dunno it was generated by the migration based on existing settings. The setting predates me, I think.
3. Yes only if neither exist. If both exist then the old is removed for both settings and .history

[chrisant996]

Cool, sounds great.

[daxgames]

@chrisant996 Testing 1.1.19 breaks Cmder Prompt

Default Cmder Prompt:

My Custom Prompt:

[daxgames]

My Custom Prompt lua loaded from clink profile folder: %cmder_root%/config:

prompt.lua

-- Much of the below was 'borrowed' from https://github.com/AmrEldib/cmder-powerline-prompt
-- Symbol displayed for the home dir in the prompt.
if not prompt_homeSymbol then
        prompt_homeSymbol = "~"
end

-- Symbol displayed in the new line below the prompt.
if not prompt_lambSymbol then
        prompt_lambSymbol = "λ"
end

-- Extracts only the folder name from the input Path
-- Ex: Input C:\Windows\System32 returns System32
---
local function get_folder_name(path)
        local reversePath = string.reverse(path)
        local slashIndex = string.find(reversePath, "\\")
        return string.sub(path, string.len(path) - slashIndex + 2)
end

function prompt_filter()
    cwd = clink.get_cwd() .. ' '
    if prompt_useHomeSymbol and string.find(cwd, clink.get_env("HOME")) then
        cwd = string.gsub(cwd, clink.get_env("HOME"), prompt_homeSymbol)
    end

    uah = ''
    if prompt_useUserAtHost then
        uah = clink.get_env("USERNAME") .. "@" .. clink.get_env("COMPUTERNAME") .. ' '
    end

    cr = "\n"
    if prompt_singleLine then
      cr = ' '
    end

    prompt = uah_color .. "{uah}" .. cwd_color .. "{cwd}{git}{hg}{svn}" .. lamb_color .. cr .. "{lamb} \x1b[0m"
    uah_value = string.gsub(prompt, "{uah}", uah)
    new_value = string.gsub(uah_value, "{cwd}", cwd)
    clink.prompt.value = string.gsub(new_value, "{lamb}", prompt_lambSymbol)
end

clink.prompt.register_filter(prompt_filter, 1)

prompt_config.lua

-- All of the below was 'borrowed' from https://github.com/AmrEldib/cmder-powerline-prompt

--- REQUIRED. config_prompt_type is whether the displayed prompt is the full path or only the folder name
 -- Use:
 -- "full" for full path like C:\Windows\System32
 -- "folder" for folder name only like System32
 -- "smart" to switch in git repo to folder name instead of full path
 -- default is full
prompt_type = "smart"
--- REQUIRED. config_prompt_useHomeSymbol is whether to show ~ instead of the full path to the user's home folder
 -- Use true or false
 -- default is true
prompt_useHomeSymbol = true

-- Symbols
-- REQUIRED. Prompt displayed instead of user's home folder e.g. C:\Users\username
prompt_homeSymbol = "~"

-- REQUIRED. Symbol displayed in the new line below the prompt.
prompt_lambSymbol = "λ"

-- REQUIRED. Adds [user]@[host] to the beginning of the prompt like bash
prompt_useUserAtHost = true

-- Prompt Attributes
--
-- Colors
-- Green:      "\x1b[1;33;40m"
-- Yellow:     "\x1b[1;32;40m"
-- Light Grey: "\x1b[1;30;40m"

-- Prompt Element Colors
cwd_color = "\x1b[1;33;40m" -- Green
uah_color = "\x1b[1;32;40m" -- Yellow
lamb_color = "\x1b[1;30;40m" -- Light Grey

prompt_singleLine = true

[chrisant996]

Can you point out specifically what's broken? I'm having trouble noticing the specific issue.

[daxgames]

@chrisant996 I can send you a screenshot of each from the current release of Cmder when I get home.

[chrisant996]

The report so far has screenshots and says it's broken. I'm not sure if more screenshots are needed, but I do need a sentence describing the nature of the breakage.

[daxgames]

Additional screenshots will show you what it should look like better than I could describe it.

[chrisant996]

Is it that {git}{hg}{svn} shows up?
If so, it seems that the custom prompt code you cited is relying on something later on replacing those tags.
And apparently that other thing isn't running or isn't isn't replacing them.

I wonder if maybe there is a script error message being reported, and this is a case of the issue where the combination of Cmder and ConEmu intermittently causes some of Clink's output to be suppressed.

I'll see if I can reproduce the issue.

Please let me know if this is what you're reporting, or if there's something else.

Thanks!

[daxgames]

My custom prompt does not seem to be replacing things as you said. The defailt prompt is not working at all and is the standard cmd.exe prompt.

[daxgames]

Default Prompt Running Clink 1.14:

Custom Prompt Running Clink 1.1.14:

Text in () is source code branch.

[chrisant996]

Oh, I see. It's because Cmder replaces Clink's core clink.lua script.

Clink 0.4.9 actually ignores any scripts named clink.lua except for "the main one".
Clink 1.x no longer uses a clink.lua script at all.

To fix a compatibility bug, I made Clink 1.x ignore any scripts named clink.lua, exactly like 0.4.9 does.
But since Clink 1.x doesn't use a clink.lua at all, it doesn't load "the main one" because there's no such thing anymore.

And since that's how Cmder was taking over Clink, it means Cmder fails to take over Clink anymore.

On the one hand, this is excellent! Clink is properly defending itself from being taken over!
On the other hand, it was a benevolent takeover, and there needs to be some way for Cmder to work without completely taking over Clink's entire built in lua system.

UPDATE:

I'll make it so Clink loads clink.lua from the main scripts directory first, if the file exists. That should be a good compromise that covers all the bases.

Yep, that's gotten it working again.

I'll publish an update sometime this weekend, probably. I've almost got lua key bindings working... Now you can bind keys in the inputrc file to lua functions. 😎

[daxgames]

Did this behavior change after clink 1.1.14?

[chrisant996]

Did this behavior change after clink 1.1.14?

Yes. I already verified the fix.

[chrisant996]

v1.1.20 is published and should resolve the Cmder integration issues.

[wg485733]

I'm on cmder 1.3.17

While waiting for the merge of PR and a new release, I was wondering if there's any temporary solution for this (like manually upgrade clink exes in the vender folder?)

[chrisant996]

I'm on cmder 1.3.17

While waiting for the merge of PR and a new release, I was wondering if there's any temporary solution for this (like manually upgrade clink exes in the vender folder?)

Yes, manually copying Clink files into the vendor\clink directory should resolve all known issues to date.

[chrisant996]

This can be closed, yes?