Thou shalt not make unto thee any graven image, or any likeness of any thing that is in heaven above, or that is in the earth beneath, or that is in the water under the earth.
Graven Image is a Common Lisp portability library (a less fancier name
might’ve been trivial-debugging) for better interaction and
debugging of a running Lisp image. One can inspect and debug all the
things under heaven and above it—all in their own REPL-resident Lisp
image!
Graven Image reuses compiler internals to improve/redefine the existing standard functions. This “improvement” often comes at a cost of changing the API of a function (for better customizability) or making it slightly less reliable (due to unstable, compiler-internal, or otherwise hacky implementation.)
NOTE: Graven Image is currently being refactored into more focused libraries. Like trivial-time and trivial-inspect.
The library is purposefully limited in scope:
- Improving the standard functions has a priority over introducing new
ones.
- repl-utilities as a contrasting approach: it defines lots of “DWIM” functions, significantly altering the REPL interaction.
- There are helpers like
function-lambda-list*in Graven Image, but these are merely aliases/one-liners over standard/improved functions.- I get carried away sometimes, though.
benchmark*is one of such nerd snipes. But still, it’s quite close to the underlyingwith-time*😉
- I get carried away sometimes, though.
- The interaction paradigm of the improved functions should stay
standard (i.e. use
*query-io*where standard requires*query-io*.)- This is to ensure that libraries like Ndebug can rely on standard facilities safely when used with Graven Image.
- Function arglist can be modified for convenience, but it should preferably stay as close to standard/implementation-specific arglist as possible.
- Graven Image strives to not modify the REPL/image/shell in any way,
relying on implementation defaults instead.
- CIEL is taking a different direction: redefining the REPL for increased utility.
- flight-recorder uses a separate shell script to add a new functionality.
- Portably reusing implementation-specific functionality is better
than re-implementing it. But if some functionality is e.g. unique to
SBCL, it might be dropped as non-portable.
- tracer, supertrace, cl-framegraph, and other non-portable SBCL-specific improvements.
Clone the Git repository:
git clone --recursive https://github.com/aartaka/graven-image ~/common-lisp/And load :graven-image in the REPL:
(asdf:load-system :graven-image)
;; or, if you use Quicklisp
(ql:quickload :graven-image)You can also install Graven Image via Guix, using the bundled
guix.scm file:
guix package -f guix.scmSomeday (after at least minor version bump) I’ll send a patch to Guix
so that you can also install it with guix install. Until then—stay
tuned (or send the patch yourself—I don’t mind.)
If you want your REPL to start with Graven Image constructs accessible without package prefix, then simply use the package directly.
(asdf:load-system :graven-image)
;; Imports external symbols of Graven Image into the current package
;; (CL-USER?) Safe, because Graven Image shadows no CL symbols.
(use-package :graven-image)Or, if you’re more orderly and disciplined about packages than I am,
you can always use trivial-package-local-nicknames and define a
shorter Graven Image nickname:
(trivial-package-local-nicknames:add-package-local-nickname
:g :graven-image :cl-user)If you want to replace the standard utilities with Graven Image ones, you can safely do so with the familiar:
;; For functions
(fmakunbound 'apropos)
(setf (fdefinition 'apropos) (fdefinition 'gimage:apropos*))
;; For macros
(setf (macro-function 'time) (macro-function 'gimage:time*))Enhanced functions (mostly from ANSI CL Debugging Utilities chapter)
The functions that Graven Image exposes are safely :use-able
star-appended functions/macros (i.e. describe* instead of
describe). Currently improved ones are:
y-or-n-p*/yes-or-no-p*apropos*/apropos-list*function-lambda-expression*time*describe*/inspect*dribble*documentation*
All of the functions exposed by Graven Image are generics, so one can
easily define :around and other qualified methods for them.
Signature:
y-or-n-p* &optional control &rest arguments => generalized-boolean
yes-or-no-p* &optional control &rest arguments => generalized-booleanImprovements are:
- Both functions accept options from
graven-image:*yes-or-no-options*, thus allowing for “nope” or “ay” to be valid responses too. - Both functions mean the same now, because it makes no sense in
differentiating them (and because most Emacs users use a magical
(fset 'yes-or-no-p 'y-or-n-p)in their config, setting the precedent for shorter yes/no queries). - No beeps (just define a
yes-or-no-p* :beforemethod to add beeps if you like ‘em; see the “Customization” section below).
Signature:
apropos-list* string &optional (package nil) exported-only docs-too => list of symbols
apropos* string &optional (package nil) exported-only docs-too => no valuesapropos-list* now allows listing exported symbols only (with
exported-only), which was a non-portable privilege of SBCL/Allegro
until now. Search over docs (more intuitive for apropos(-list)* than
mere name search) is possible with docs-too.
Based on this foundation, apropos* lists symbols with their types,
values, and documentation, so that implementation-specific formats are
gone for a better and more unified listing:
(apropos* :max)
;; MAX [FUNCTION (NUMBER &REST
;; MORE-NUMBERS) : Return the greatest of its arguments; among EQUALP greatest, return...]
;; :MAX [SELF-EVALUATING]
;; CFFI::MAX-ALIGN
;; SB-ASSEM::MAX-ALIGNMENT [CONSTANT = 5]
;; ...
;; SB-C::MAXES
;; ALEXANDRIA:MAXF [MACRO (#:PLACE &REST NUMBERS) : Modify-macro for MAX. Sets place designated by the first argument to the...]
;; SB-KERNEL::MAXIMAL-BITMAP
;; ...
;; SB-LOOP::LOOP-ACCUMULATE-MINIMAX-VALUE [MACRO (LM OPERATION FORM)]
;; SB-LOOP::LOOP-MAXMIN-COLLECTION [FUNCTION (SPECIFICALLY)]
;; SB-LOOP::LOOP-MINIMAX [CLASS (STRUCTURE-OBJECT)]
;; ...Signature:
function-lambda-expression* function/macro/method/symbol &optional force => list, list, symbol, list
;; Alias:
lambda-expression* function/macro/method/symbol &optional force => list, list, symbol, listThis function tries to read source files, process the definitions of
functions, and build at least a barebones lambda from the arglist and
documentation of the function. So that CL function-lambda-expression
returns:
(function-lambda-expression #'identity)
;; => NIL, T, IDENTITY
(function-lambda-expression #'print-object)
;; => NIL, T, PRINT-OBJECTWhile the new Graven Image function-lambda-expression now returns:
(function-lambda-expression* #'idenitity)
;; => (LAMBDA (THING) "This function simply returns what was passed to it." THING),
;; NIL, IDENTITY, (FUNCTION (T) (VALUES T &OPTIONAL))
(function-lambda-expression* #'print-object t) ; Notice the T for FORCE, to build a dummy lambda.
;; => (LAMBDA (SB-PCL::OBJECT STREAM)), NIL, PRINT-OBJECT, (FUNCTION (T T) *)Which means:
identityis actually not a closure, and has a reliable source!print-objectis a generic and thus is not really inspectable, so we build a dummy lambda for it whenforceargument is provided.- This might be a questionable choice, but it at least allows us to
get function arglists from
function-lambda-expressionin a portable-ish way. The standard doesn’t provide us with much ways to know an arglist of a function beside this.
- This might be a questionable choice, but it at least allows us to
get function arglists from
Things that function-lambda-expression* now returns are:
- Lambda expression.
- For lambda functions, their source.
- For regular functions, their
defunturned into alambda. - For anything else, a constructed empty
(lambda (arglist...) documentation nil)(only whenforceis T). - Or, in case all the rest fails, NIL.
- Whether the thing is a closure
- If it is, might return an alist of the actual closed-over values, whenever accessible (not for all implementations).
- If closed-over values are not accessible, returns T.
- If it’s not a closure, returns NIL.
- Function name. Mostly falls back to the standard
function-lambda-expression, but also inspects implementation-specific function objects if necessary. - Function type, whenever accessible (SBCL and ECL).
Based on these new features of function-lambda-expression*, here are
some Graven Image-specific helpers:
function-lambda-list*- Get the lambda list of a function.
function-arglist*- Alias.
lambda-list*- Alias for
function-lambda-list*. arglist*- Alias.
function-name*- Get the name of a function.
function-type*- Get its ftype.
function-lambda-list* function => list
function-arglist* function => list
lambda-list* function => list
arglist* function => list
function-name* function => symbol
function-type* function => listSignature:
time* &rest forms => return-valuesThe improved time* from Graven Image reuses as much
implementation-specific APIs as possible, with the predictable output
format.
And it also allows providing several forms, yay!
Signature:
benchmark* (&optional (repeat 1000)) &body forms => return-valuesWhile time* is the standard benchmarking/profiling solution, it’s
almost always too simple for proper benchmarking. Most systems getting
complex enough end up with some form of custom
benchmarking. Shinmera’s trivial-benchmark is one such example. Graven
Image benchmark* is heavily inspired by trivial-benchmark, but has
a more portable foundation in the form of with-time*.
As many other benchmarking macros, benchmark* repeats its body a
certain number of times, collecting timing stats for every run, and
then prints aggregate statistics for the total runs.
(gimage::benchmark* (20) ;; Repeat count.
(loop for i below 1000 collect (make-list i) finally (return 1)))
;; Benchmark for 20 runs of
;; (LOOP FOR I BELOW 1000
;; COLLECT (MAKE-LIST I)
;; FINALLY (RETURN 1))
;; - MINIMUM AVERAGE MAXIMUM TOTAL
;; REAL-TIME 0.0 0.00175 0.019 0.035
;; USER-RUN-TIME 0.000668 0.0016634 0.016315 0.033268
;; SYSTEM-RUN-TIME 0.0 0.00021195 0.003794 0.004239
;; GC-RUN-TIME 0.0 0.00085 0.017 0.017
;; BYTES-ALLOCATED 7997952.0 8008154.5 8030464.0 160163090.0Signature:
with-time* (&rest time-keywords) (&rest multiple-value-args) form &body bodyAs the implementation detail of time* and benchmark*, with-time*
allows to get the timing data for interactive
querying. time-keywords allow &key-matching the timing data (like
:gc time or bytes :allocated) for processing in the body. While
multiple-value-args allow matching against the return values of the
form. So we get best of the both worlds: timing data and return
values. This flexibility enables time*, with its requirements of
printing the data and returning the original values at the same time.
For example, here’s how one would track the allocated bytes and garbage collection times when running a cons-heavy code:
(gimage:with-time* (&key aborted gc-count gc allocated)
(lists lists-p)
(loop for i below 1000
collect (make-list i :initial-element :hello)
into lists
finally (return (values lists t)))
(unless aborted
(format t "Bytes allocated: ~a, GC ran ~d times for ~a seconds"
allocated gc-count gc)))
;; Bytes allocated: 7997952, GC ran NIL times for 0 secondsSignature:
describe* object &optional (stream t) respect-methodsDescribes the object to the stream, but this time with portable
format of description (determined by graven-image:description* and
specified for many standard classes) and with predictable set of
properties (graven-image:fields*). In Graven Image, both
describe and inspect have the same format and the same set of
fields.
As a note of respect to the original describe, Graven Image one
allows to reuse the describe-object methods defined for user
classes. To enable this, pass T to respect-methods.
Signature:
fields* object &key strip-null &allow-other-keysReturns an undotted alist of properties for the object. Custom
fields provided by Graven Image are named with keywords, while the
implementation-specific ones use whatever the implementation
uses. Arrays and hash-tables are inlined into fields to allow
indexing these right from the inspector.
See fields* documentation for more details.
Signature:
description* object &optional streamConcise and informative description of object to the
stream. Useful information from most of the implementations
tested—united into one description header.
Signature:
inspect* object &optional strip-nullNew’n’shiny inspect* has:
- Most commands found in other implementation, with familiar names.
- Abbreviations like
H -> HELP(inspired by SBCL). - Ability to set object field values with
(:set key value)command (inspired by CCL). - Built-in pagination with ways to scroll it (
:next-page,:previous-page,:home) and change it (:length). - Property indexing by both integer indices and property names (with abbreviations for them too!).
- Ability to ignore
nilproperties withstrip-nullargument (inspired by SBCL). On by default! - And the ability to evaluate arbitrary expressions (with
:evaluatecommand or simply by inputting something that doesn’t match any command).
And here’s a help menu of the new inspect* (in this case, inspecting
*readtable*), just to get you teased:
This is an interactive interface for 5
Available commands are:
:? Show the instructions for using this interface.
:HELP Show the instructions for using this interface.
:QUIT Exit the interface.
:EXIT Exit the interface.
(:LENGTH NEW) Change the page size.
(:WIDTH NEW) Change the page size.
(:WIDEN NEW) Change the page size.
:NEXT Show the next page of fields (if any).
:PREVIOUS Show the previous page of fields (if any).
:PRINT Print the current page of fields.
:PAGE Print the current page of fields.
:HOME Scroll back to the first page of fields.
:RESET Scroll back to the first page of fields.
:TOP Scroll back to the first page of fields.
:THIS Show the currently inspected object.
:SELF Show the currently inspected object.
:REDISPLAY Show the currently inspected object.
:SHOW Show the currently inspected object.
:CURRENT Show the currently inspected object.
:AGAIN Show the currently inspected object.
(:EVAL EXPRESSION) Evaluate the EXPRESSION.
:UP Go up to the previous level of the interface.
:POP Go up to the previous level of the interface.
:BACK Go up to the previous level of the interface.
(:SET KEY VALUE) Set the KEY-ed field to VALUE.
(:MODIFY KEY VALUE) Set the KEY-ed field to VALUE.
(:ISTEP KEY) Inspect the object under KEY.
(:INSPECT KEY) Inspect the object under KEY.
:STANDARD Print the inspected object readably.
:AESTHETIC Print the inspected object aesthetically.
Possible inputs are:
- Mere symbols: run one of the commands above, matching the symbol.
- If there's no matching command, then match against fields.
- If nothing matches, evaluate the symbol.
- Integer: act on the field indexed by this integer.
- If there are none, evaluate the integer.
- Any other atom: find the field with this atom as a key.
- Evaluate it otherwise.
- S-expression: match the list head against commands and fields,
as above.
- If the list head does not match anything, evaluate the
s-expression.
- Inside this s-expression, you can use the `$' function to fetch
the list of values under provided keys.
Signature:
dribble* &optional pathname (if-exists :append)Dribble the REPL session to pathname. Unlike the
implementation-specific dribble, this one formats all of the session
as load-able Lisp file fully reproducing the session. So all the
input forms are printed verbatim, and all the outputs are commented
out.
Beware: using any interactive function (like inspect etc.) breaks
the dribble REPL. But then, it’s unlikely one’d want to record
interactive session into a dribble file.
Signature:
documentation* object &optional (doc-type t)
doc* object &optional (doc-type t)Improved version of documentation. Two main improvements are:
doc-type is now optional, and doc* alias is available for
convenience.
documentation.lisp also defines more documentation methods (and
respective setf method) to simplify documentation fetching and
setting. In particular, method on (symbol (eql t)) to simplify
symbol documentation search; and (t (eql 'package)) with a new
doc-type for package documentation convenience.
Signature:
break* &rest argumentsA more useful wrapper for break, listing the function it’s called
from and the provided symbol values. See examples in the docstring.
Graven Image is made to be extensible. That’s why most of the improved
functions are generic: one can define special methods for their data
and patch the behavior with :before, :after, and :around
methods. Most of Graven Image functions mention the variables/things
influencing them in the docstring. Here’s a set of useful
customizations:
Restoring the standard-ish (beeping with bell (ASCII 7) character) behavior:
(defmethod gimage:yes-or-no-p* :before (&optional control &rest arguments)
(declare (ignore control arguments))
(write-char (code-char 7) *query-io*)
(finish-output *query-io*));; Make it strict yes/no as per standard.
(defmethod gimage:yes-or-no-p* :around (&optional control &rest arguments)
(declare (ignore control arguments))
(let ((gimage:*yes-or-no-options*
'(("yes" . t)
("no" . nil))))
(call-next-method)))
;; Add more yes/no options (Russian, for example).
(defmethod gimage:y-or-n-p* :around (&optional control &rest arguments)
(declare (ignore control arguments))
(let ((gimage:*yes-or-no-options*
(append
gimage:*yes-or-no-options*
'(("да" . t)
("ага" . t)
("нет" . nil)
("не" . nil)
("неа" . nil)))))
(call-next-method)))Implementations are not good at sorting things, and their results are
not often useful. Sorting things the way one needs is a useful
extension. Here’s a simple yet effective :around method that sorts
things by string occurence:
(defmethod gimage:apropos-list* :around (string &optional packages external-only docs-too)
"Sort symbols by the relation of subSTRING count to the length of symbol."
(declare (ignorable packages external-only docs-too))
(let ((result (call-next-method)))
(sort
(remove-duplicates result)
;; For more comprehensive matching, see
;; a1b4ebd649e0268b1566e80709e7cea41363d006 and other commits
;; before c090d6dc14e05c561cf5c39cf5f6cc02e8cd04c5.
#'> :key (lambda (sym)
(let ((match-count 0))
(uiop:frob-substrings
(string sym) (list (string string))
(lambda (sub frob)
(incf match-count)
(funcall frob sub)))
(/ match-count (length (string sym))))))))Graven Image inspect* function uses *interface-lines* for the
number of properties to list. If your screen is more than 20 lines
high, you might want to add more lines:
(defmethod gimage:inspect* :around (object)
(declare (ignore object))
(let ((gimage:*interface-lines* 45))
(call-next-method)))Most of Graven Image functions also rely on implementation/REPL-specific printer variables, which might be un-intuitive, overly verbose, or too short. Binding printer variables around Graven Image functions helps that too:
(defmethod gimage:apropos* :around (string &optional package external-only docs-too)
(declare (ignore string package external-only docs-too))
;; Note that you can also use
;; `sb-ext:*compiler-print-variable-alist*' and
;; `sb-ext:*debug-print-variable-alist*' on SBCL.
(let ((*print-case* :downcase)
(*print-level* 2)
(*print-lines* 2)
(*print-length* 10))
(call-next-method)))A noisy apropos function listing like
X86::*X86-OPERAND-TYPE-NAMES* [VARIABLE = ((:REG8 . 1) (:REG16 . 2) (:REG32 . 4) (:REG64 . 8) (:IMM8 . 16) (:IMM8S . 32) (:IMM16 . 64) (:IMM32 . 128) (:IMM32S . 256) (:IMM64 . 512) (:IMM1 . 1024) (:BASEINDEX . 2048) (:DISP8 . 4096) (:DISP16 . 8192) (:DISP32 . 16384) (:DISP32S . 32768) (:DISP64 . 65536) (:INOUTPORTREG . 131072) (:SHIFTCOUNT . 262144) (:CONTROL . 524288) (:DEBUG . 1048576) (:TEST . 2097152) (:FLOATREG . 4194304) (:FLOATACC . 8388608) (:SREG2 . 16777216) (:SREG3 . 33554432) (:ACC . 67108864) (:JUMPABSOLUTE . 134217728) (:REGMMX . 268435456) (:REGXMM . 536870912) (:ESSEG . 1073741824) (:INVMEM . 2147483648) (:REG . 15) (:WORDREG . 14) (:IMPLICITREGISTER . 75890688) (:IMM . 1008) (:ENCIMM . 464) (:DISP . 126976) (:ANYMEM . 2147547136) (:LLONGMEM . 2147547136) (:LONGMEM . 2147547136) (:SHORTMEM . 2147547136) (:WORDMEM . 2147547136) (:BYTEMEM . 2147547136) (:LABEL . 4294967296) (:SELF . 8589934592))]turns into a much more readable
x86::*x86-operand-type-names* [variable = ((:reg8 . 1) (:reg16 . 2) (:reg32 . 4) (:reg64 . 8) (:imm8 . 16) (:imm8s . 32) (:imm16 . 64) (:imm32 . 128) (:imm32s . 256) (:imm64 . 512) ...)]Several implementations throw errors when trying to get documentation for non-existent method combinations, classes, etc. It’s convenient to suppress these:
(defmethod gimage:documentation* :around (object &optional doc-type)
(ignore-errors (call-next-method)))Actually, one can try to write an :around method for regular
documentation, but this modification is not guaranteed to work on
all implementations.
You can help with any of the open issues most are well-described and split into bite-sized tasks. See .github/CONTIBUTING.md for the contributing guidelines.