v2.8

This release comes with a number of experimental but stable features that might eventually become standard practice. The features are there to use if you with but adjacent from the older code so there is zero risk in updating to this version.

Breaking Changes:

• None

Features:

• New data-type jsonc (Concatenated JSON). This is an experimental parser for reading multiple JSON documents in a single file or stream. The expectation is that it will eventually replace jsonl (jsonlines) and possibly even the default json data-type

• Autocomplete can now split on =. However this is an experimental feature

• New autocomplete directive: Goto. This is an experimental way to add a little more structure to autocomplete definitions without resorting to using dynamic code everywhere

• autocomplete directive FlagValues now supports two values: "*" and "". These define default properties regardless of the flag used

Minor Changes:

• Unmatched records in index ([) when parsing a table now produces the same configurable behavior (erroring by default but which can be disabled) as when using index against an object

• Autocompletion suggestions for yarn

• Data-type field in function is now optional

• Lots of improvements to the docs

Bug Fixes:

• $MyVar[[/path/to/element]] now behaves the same when used infixed as it did when used as a function

v2.7

Breaking Changes:

• zero length arrays returned from subshells (eg echo @{g this-file-does-not-exist}) should fail by default, like unset variables. This is enabled by default but can be disabled via config: set proc strict-arrays false

• autoglob should fail if it doesn't match any results. eg @g echo this-file-does-not-exist.*

• Subshells should fail parent command when used inside try and trypipe blocks. eg try { echo ${false} }

Features:

• function now supports defining parameters

• Added support fro && eg do-something && do-something-else for chaining successful commands

• Added support for || eg do-something || do-something-else for chaining unsuccessful commands

• Added support for writing to the terminal emulator's titlebar via config: set shell titlebar-func { block }

• titlebar-func can also be written to your tmux window title via config: set shell tmux-echo true.

• New reserved variable: $HOSTNAME

• New reserved variables: $1 (and upwards) which correlates to the scope's parameter index. $1 is the functions first parameter. $2 is the second. $13 is the thirteenth.

• New reserved variable: $0 which contains the function name

• New event: onCommandCompletion (this is experimental and subject to change in the next release)

• Macro variables. eg echo Hello ^$name will prompt the user to enter a name. Macro variables are only support in the REPL

• read now supports flags (eg default option, etc) to allow for a better experience in shell scripting

Minor Changes:

• You can now overwrite onKeyPress events. This no longer produces an error forcing you to remove the old event before adding the new one

• Autocompletion suggestions shouldn't be sorted is results include paths (improves the ordering of autocompletion suggestions)

• Autocompletion suggestions for openagent builtin

• Autocompletion suggestions for hashtags

• Test counts re-added to website

• Windows should show file extensions by default in autocompletion results

Bug Fixes:

• Fix inverted logic on forceTTY: config: get proc force-tty false set by default, which then proxies STDERR and prints them in red

• ctrl+c (^c) now currectly sends SIGTERM rather than just ending the child process

• Better handling of SIGTERM

• SIGTSTP isn't working. Switched to SIGSTOP when (^z) doesn't

• Fix panic in event onFilesystemChange where fileRef is not getting passed correctly

• Fix panic in event onFilesystemChange where path is zero length string

• Some improvements to variable previews in the REPL

• count should check if it is a method

• AST cache now checked more regukarly. This is to reduce the risk of memory leaks during fuzz or load testing

• murex-docs still referred to len builtin. That should be changed to count

• Lots of fuzzing added -- a few edge case bugs discovered

v2.6

Breaking Change:

• strict-vars is enabled by default. This means if a variable doesn't exist / hasn't been instantiated then your command will fail. You can disable this by putting config: set proc strict-vars false inside any affected functions and/or in your ~/.murex_profile

Features:

• select builtin now added to the default build. This built in will stream tabulated data into an in memory sqlite3 database and take SQL arguments to query that data. Thus allowing you to inline SQL queries as part of your native command line. This addition will add complications for anyone building from source on Windows. If you are a Windows user you are recommended to use WSL or, if you desperately need to use murex natively on Windows, download the appropriate per-compiled binary from the downloads page
• When used in WSL, any Windows mounts included in $PATH will now be subject to the same inclusion rules as Windows file systems when murex is used in Windows. This means only executables (ie files with .exe, amongst other, extensions are present). This has dramatically cleaned up autocomplete suggestions when running murex in WSL.
• New autocomplete directive: FileRegexp. Use this if you want to return only files where the file name matches a defined pattern (eg only .txt files: \.txt$)
• HTTP client (as used by get, getfile, post, open, etc) supports additional logic with dynamically guessing the correct data type to assign to STDOUT where the Content-Type hasn't been mapped. Any Content-Type with a +json suffix will be assumed to be json data type
• getfile output has been revamped. If STDOUT is a TTY then you'll now see a progress bar
• .tfstate (Terraform state files) have been added to supported JSON extensions
• ~/.ssh/config hosts have been added to SSH (et al)'s autocomplete suggestions
• murex-package: reload now only reloads packages, not .murex_preload nor .murex_profile`
• website tweaks: updated INSTALL doc, CSS fixes
• debug message added to warn when fork FIDs are being killed

Non-user facing changes (internal changes to the murex code base):

• Additional tests
• Older function definitions replaced with the newer (correct) way to define functions and methods. This allows for greater discovery in the interactive terminal
• Root unit tests finish quicker

Bug fixes:

• HTTP client (as used by get, getfile, post, open, etc) wasn't breaking down the Content-Type header correctly. This only affected unknown MIME types where murex had to guess the correct data type to assign to STDOUT. This is an edge case bug.
• while, when used without a separate parameter for the conditional logic, was buffering the STDOUT for each loop and outputting it after that iteration had ended. while now uses a "tee" stdio writer which means it can stream STDOUT while the iteration is in progress

v2.5

Features:

• alter now supports --sum where structures are merged and numeric values are added together

• New builtin count. This has deprecated len however len will stick around as an alias for backwards compatibility

• New operators added to let: +=, -=, /=, *=

• New builtin addheading for adding headings to lists

• Compiled profile will now always execute even if murex flags set to ignore the modules/user profile. This is so that aliases like len get set and thus murex can still move forward with changes to builtins but without breaking backwards compatibility

• autocomplete now passes ISMETHOD variable to dynamic completions so those dynamic completions are aware if a command requesting auto-completion suggestions is being invoked as a method (mid-pipeline) or function (start of a pipeline)

• Index, [, now supports inlining element, [[, lookups. eg [ a b [/map/c] ]

• Dynamic auto-completions that include @IncFiles or @IncDirs will now automatically append files and/or directories to their auto-completion suggestions

• New autocomplete directives: IncExeAll (like IncExePath but includes builtins, functions, aliases), IncManPage (include results from the man page parser -- usually suppressed when autocomplete config is set)

• Disabled 'Loading default profile' message -- this was always pretty redundant but now that the compiled profile is being loaded all the time (eg murex -c 'command' or when called in a shebang), it's also now ugly too

• select now supports passing a file in the FROM syntax. eg select * FROM ./example.csv. The caveat here is this breaks currently auto-complete on column names

• select now supports multiple tables using either named pipes (eg select * FROM <table1>, <table2>) or variables (eg select * FROM \$table1, \$table2) passed in the FROM syntax. Variables should be escaped and you cannot mix and match between named pipes, file names nor variables. You can use any number of tables from 1 to 2^63-1 (64bit systems) or 1 to 2^31-1 (32bit systems). Which should be more than enough ;)

• config option for select to define default output data type where multiple tables are imported

• Lots of new and updated documentation!

Non-user facing changes (internal changes to the murex code base):

• open functions can now be called by other functions to take advantage of auto-typing and auto gunzip etc.

• tmp.Close() should return err. This isn't a bug but it might catch future bugs

• LazyLogging created to speed up writing tests against data structures

• utils/List package created to handle list / array / map functions. Also makes testing more complex routines easier

Bug fixes:

• Regression bug fixed where prepend was invoking append

• streams.ReadCloser not setting context

• parameters.StringArray() should copy values instead of a pointer to ensure the underlying parameters are immutable

v2.4

There are a number of new features in this release

Breaking Changes:

• mkarray (a et al) no longer returns an error if the start and end ranges
  are the same. Instead it returns an array of 1 value.

User Facing Changes:

• Strict variables now supported (like set -u in Bash). This will mean any
  variables passed that haven't been initialized beforehand will cause that
  pipeline to fail. Currently this is opt in, a future release of murex will
  flip that and make it opt out. So take this opportunity to enable it in your
  ~/.murex_profile and test your scripts. Enable this via config:

  config: set proc strict-vars true
  

  This feature was requested in issue #380.

• New builtin: datetime. This builtin allows you to convert date and/or time
  strings of one format into strings of another format. datetime is a
  supercharged alternative to the UNIX command date aimed at making scripting
  easier.

• mkarray (a et al) now supports dates. eg [01-Jan-20..05-May-21]. If no
  start nor end date appears then mkarray assumes range starts or ends at
  current date.

• openagent profile for image data types has been improved. murex now
  better supports tmux and iTerm2.

• runtime --config now displays FileRef for every set as well as define,
  named FileRefSet and FileRefDefine respectively. So you can now easily
  trace where global config is being set and defined.

• Better error messages in the interactive terminal.

• Prompt now defaults to only displaying current directory rather than the full
  path. You can revert this change by adding your own prompt in config. eg:

  config: set shell prompt {
      out "{RESET}{YELLOW}${pwd_short} {RESET}» "
  }
  
  config: set shell prompt-multiline {
      let len = ${pwd_short -> wc -c} - 1
      printf "%${$len}s » " $linenum
  }
  

• Parser updated to better support multiline pipelines where the newline is
  escaped and a comment exists after issue #379.
  This only applies to shell scripts, the interactive terminal hasn't yet been
  updated to reflect this change.

• Fixed regression bugs with autocomplete parameters that affected some dynamic
  blocks.

• readline now caches syntax highlighting and hint text to improve the
  responsiveness of the interactive terminal. This is set to a hard limit of
  200 cached items however that will be a configurable metric in a future
  release. Also planned for the future is caching autocompletion suggestions.

• Loading message added for the default profile, ie the one that is compiled
  into and thus shipped with murex.

• Fixed bug with fid-list and jobs where they were outputting the p.Name
  struct rather than p.Name.String(). This lead to the process name appearing
  garbled under some circumstances.

• {BG-BLUE} emitted the wrong ANSI escape code, this has been corrected.

• Several readline bug fixes.

Non-User Facing / Maintenance Changes

• Thread safe copying of parameters upon fork. The previous code never actually
  generated any race conditions and I don't think ever could. However it was
  ambiguous. This new code makes the copy more explicit and appears to have
  also brought some minor performance improvements in benchmarks too.

• Behavioural test framework has been refactored to make it easier to add new
  behavioural tests.

• Lots of new tests added.

• Updated documentation.

v2.3

This release brings some major improvements to the discoverability of commands
and their parameters in the interactive terminal.

User Facing Changes:

• New builtin: method. This is used to help turn some commands into a
  "method", similar in concept to object orientated development where the tab-
  completer will return a list of other commands that are compatible with the
  formers output data types. It's purely an autocompletion tool and has no
  effect in shell scripts.

• Autocompletion suggestions will differ based on what pipes are used. The idea
  behind this is to improve the discoverability of commands. All tokens can
  still be used to pipeline to any command type (like before). This rule only
  affects autocomplete suggestions in the interactive prompt.

  • | will show all available aliases, functions, builtins and external
    commands, as usual

  • -> will default to showing only compatible commands defined in method
    (see above), however if no methods are found then -> will fall back to
    suggesting all available aliases, functions, builtins and external commands,
    like |.

  The idea here is for -> to surface commands that are commonly
  expected to be pipelined together but without forcing the user to pause and
  remember which token exposes what commands. While | is still available for
  familiarity

• Autocompletion de-duplication now places hyphen-prefixed flags at the end of
  the results when sorting. This helps bring more relevant flags forward, with
  hyphen-prefixed flags still visible when users press -

• External commands return a error message if the exit status is non-zero
  (issue #299)

  » exec: false
  Error in `exec` (1,1): exit status 1
  

• Index() ([) now returns a different data type from the root object if the
  element is a string, integer, float64, boolean or nil (issue #316)

• runtime was missing a few flags for reporting supported data types against
  the newer APIs

• Generic types (* / generic) to include two spaces between columns when
  marshalled

• Syntax highlighter colour tweaks -- no longer using background highlighting
  because that looks weird and unreadable on some terminal colour schemes

• Bug fixes to the fast parser

• Bug fixes to readline merged from v2.2.1200 (issue #362)

Non-User Facing / Maintenance Changes

• hcl and bson types now use ReadArray templates so their behavior should
  mimic JSON and YAML

• Fast parser is now aware of which pipe token (eg |, ->, =>, ?) is used

• Unit test concurrency fix

• New tests written for the fast parser

• Updated documentation

v2.2

What's new in murex v2.2

This release sees new optional features plus major bug fixes to the existing
code base:

Breaking Changes:

One config shell key has changed it's name:

• spellcheck-block -> spellcheck-func

This is to bring spellcheck inline with other keys that contain executable
code.

This change might break some of your existing profile config!

User Facing Changes:

• Windows coreutils no longer returns exit code of 13 for aliases

• Fixed regression bug in fg (added stronger testing here to prevent future
  regressions)

• Generic data type, *, now uses tabwriter to align columns

• Spellchecker error handling tweaked to be more user friendly

• Environmental variables can been created per command:

» env  | grep foo
foo=bar

• stdstream.SetDataType() only works on first call -- ignored thereafter.
  This will impact pipes that are written to by many processes but results
  in more predictable code due possible timing errors where different
  processes are overwriting the stream data type

• stdstream.SetDataType() only works on first call -- ignored thereafter.
  This will impact pipes that are written to by many processes but results
  in more predictable code due possible timing errors where different
  processes are overwriting the stream data type

Non-User Facing / Maintenance Changes

• All dependencies have been updated, pinned and the vendors directory
  rebuilt

• Fixed some issues flagged up in goreportcard.com

• Some internal API changes that have no UI/UX ramifications but makes the
  code more maintainable

• Lots more unit tests added

• Lots of code rewritten to avoid potential data races