Project Goals

High level Goals

Design a modern Unix shell that can do everything bash/zsh/etc. can do, and more:

• System Administration -- Building Linux distributions. Startup scripts, configure and build scripts. Reproducible Builds.
• Distributed Computing -- Building containers, specifying remote jobs. Feedback and Monitoring.
  • Performance measurement
  • Security testing
• Scientific Computing -- heterogeneous "big data" and small data pipelines. The language should scale down as well as scale up, i.e. low startup latency for small jobs. Incorporate features of "workflow languages" and systems in the MapReduce family. Reproducible Science.
• Interactive Computing -- a general purpose REPL. May hook into Jupyter at first.

Oil Language Design Goals

• Easy upgrade path from bash, the most popular shell in the world. To do this, I've written a very compatible bash parser, which will allow automatic conversion of bash (osh) to oil. So the language has a different syntax and a superset of bash semantics.

• Consistent syntax. POSIX sh and bash have evolved many quirks.

• Semantic fixes, such as fixing arrays.

• Treat the shell as a real programming language. Example: Completion functions in bash are fairly difficult have a bad API involving globals. It should feel more like writing completion in Python or JavaScript.

  • Fill in obvious gaps, like abspath, etc.
  • selected influences: Python, R, Ruby, Perl 6, Lua (API), ML, C and C++. Power Shell.

• Reduce language cacophony in shell programming. Example: combine shell, awk, and make.

  • Also combine tools like find (which has its own expression parser and starts processes), and xargs/GNU parallel, which both start processes. GNU parallel is actually mentioned in the bash manual.

• Security
  • in emitting strings: escaping
  • in reading strings: error checking should be easy, better control over "read" delimiters, etc.
  • fix issues with globs and flags, i.e. untrusted file system and untrusted variables

Implementation Goals

• Proper error messages like Clang/Swift. Static Parsing.
• Provide end-to-end tracing and profiling tools (e.g. for pipelines that run for hours)

Longer Term Goals

• Expose our toolkit for little languages -- lexing, parsing, AST representation, etc. So that other languages can be built in the same way.
• Metaprogramming with ASTs as first class data structures.