bydi

bydi extends ert with some simple mocking macros.

These macros allow you to mock (or spy on) any function and make assertions on what was (or wasn’t) called and with what arguments. You can also watch variables and make assertions on what was (or wasn’t) set and to what value.

It grew out of my configuration and therefore is currently still heavily geared towards my usage.

Integration

You can add it as a dependency in your Cask file:

(development
 ;; ... your other dependencies
 (depends-on "bydi" :git "https://github.com/Walheimat/bydi" :branch "trunk"))

Usage in Tests

Let’s look at some simple examples. Say you want to test a function that calls some other external functions but you’re actually just interested in verifying your composition.

(defun my-project-name ()
  "Inform about project name."
  (when-let* ((proj (project-current nil))
              (name (project-name proj)))

    (message "My project is named %s" name)
    name))

(ert-deftest mock-always ()
  (bydi ((:mock project-current :with always) ; Make sure `project-current' returns t.
         (:mock project-name :return "Test Project")) ; Make `project-name' return constant value.

    (should (string= (my-project-name) "Test Project"))

    ;; Verify `project-current' was called with nil.
    (bydi-was-called-with project-current nil)
    ;; or
    (bydi-was-called-with project-current (list nil))

    ;; Verify that `project-name' was called.
    (bydi-was-called project-name)))

The example introduces the core macro bydi. bydi accepts a list of mocking instructions and a body to execute while these instructions are active.

There are a few short-hands for useful mocks using ignore and always, and keyword var to create a binding. Using the example above, we could have written instead:

(ert-deftest mock-always ()
  (bydi ((:always project-current)
         (:mock project-name :var name :initial "Test Project"))
    (should (string= (my-project-name) "Test Project"))
    (bydi-was-called-with project-current nil)
    (bydi-was-called project-name)

    (setq name "Mock Project")
    (should (string= (my-project-name) "Mock Project"))))

So what if your function branches? There’s another short-hand for that.

(defun my-branching-project-name ()
  "Inform about project name."
  (if-let* ((proj (project-current nil))
            (name (project-name proj)))
      (progn
        (message "My project is named %s" name)
        name)
    (message "You're not in a project")
    nil))

(ert-deftest mock-sometimes ()
  (bydi ((:sometimes project-current) ; Make `project-current' only
                                      ; sometimes return t. The
                                      ; opposite is `:othertimes'.

         (:mock project-name :return "Test Project")) ; Make `project-name' return constant value.

    (should (string= (my-branching-project-name) "Test Project"))
    (bydi-was-called project-name)

    (bydi-toggle-sometimes) ;; Toggles all `:sometimes' and
                            ;; `:othertimes' mocks, this will
                            ;; automatically clear mocks for them.

    ;; OR.
    (bydi-toggle-volatile 'project-current) ;; Toggle `project-current' only.

    (should-not (my-branching-project-name))
    (bydi-was-not-called project-name)))

What if a mocked function is called repeatedly and we care about the amount and arguments? There’s more macros.

(require 'subr-x)

(defun my-org-file-counter (directory)
  "Count the number of org files in DIRECTORY."
  (let* ((files (directory-files directory))
         (real-files (nthcdr 2 files))
         (fun (apply-partially #'string= "org")))

    (thread-last real-files
      (mapcar #'file-name-extension)
      (seq-count fun))))

(ert-deftest count-one ()
  (bydi ((:mock directory-files :return '("." ".." "one.txt" "two.org" "three.org" "four.cpp"))
         (:mock file-name-extension :return "org")
         file-name-base)

    (should (eq 4 (my-org-file-counter "/some/dir")))
    (bydi-was-not-called file-name-base)
    (bydi-was-called file-name-extension)

     ;; Verify the number of calls.
    (bydi-was-called-n-times file-name-extension 4)
    (bydi-was-called-n-times directory-files 1)

    ;; Verify the passed arguments for a specific invocation.
    (bydi-was-called-nth-with file-name-extension "three.org" 2)))

Sometimes you want a specific function reliably fail. You can use :fail for this.

(defun one-word-forward-two-chars-back ()
  "Go forward one word and two chars back."
  (forward-word)
  (backward-char)
  (backward-char))

(ert-deftest words-fail-me ()
  (bydi ((:spy forward-word)
         (:fail backward-char))

    (should-error (one-word-forward-two-chars-back) :type 'error)

    (bydi-was-called forward-word)))

You can also use :with to pass the name of another error (for example user-error) and you can use :args to pass arguments.

Sometimes you don’t want to replace a function but still know if and how it was called. You can use :spy for this.

;; Using `my-org-file-counter' defined above.

(ert-deftest count-two ()
  (bydi ((:mock directory-files :return '("." ".." "one.txt" "two.org" "three.org" "four.cpp"))
         (:spy file-name-extension))

    ;; Actual implementation means we match correctly.
    (should (eq 2 (my-org-file-counter "/some/dir")))

    (bydi-was-called file-name-extension)

    (bydi-was-called-n-times file-name-extension 4)
    (bydi-was-called-n-times directory-files 1)

    ;; This is 0-indexed
    (bydi-was-called-nth-with file-name-extension "three.org" 2)

    (bydi-was-called-nth-with file-name-extension "four.cpp" 3)
    ;; or
    (bydi-was-called-last-with file-name-extension "four.cpp")))

Spying has another advantage, you can selectively mock using macro bydi-when. This does not work for mocks.

It allows you to stipulate when a function should return a certain value. The condition is the arguments with which the function is called.

(defun keep-adding-one-more (&rest numbers)
  "Keep adding NUMBERS, but always one more."
  (1+ (apply '+ numbers)))

(ert-deftest sometimes-add-two-more ()
  (bydi ((:spy keep-adding-one-more))

    (should (eq 7 (keep-adding-one-more 1 2 3)))

    ;; Return 1 when `keep-adding-one-more' is called with '(1 2 3)
    ;; but only do that once.
    (bydi-when keep-adding-one-more :called-with '(1 2 3) :then-return 1 :once t)

    (should (eq 7 (keep-adding-one-more 3 2 1)))
    (should (eq 1 (keep-adding-one-more 1 2 3)))
    (should (eq 7 (keep-adding-one-more 1 2 3)))))

Similarly, you can watch variables.

(defvar my-variable nil)

(defun friendly-function (new-val)
  "Update with NEW-VAL."
  (setq my-variable new-val))

(defun unfriendly-function ()
  "Does nothing but `let'-bind."
  (let ((my-variable 'evil-eye))

    (ignore)))

(ert-deftest friendly-setting ()
  (bydi ((:watch my-variable))

    (friendly-function 'test)

    (bydi-was-set-to my-variable 'test)

    (unfriendly-function)

    (bydi-was-set-to my-variable 'evil-eye)

    ;; OR
    (bydi-was-set-to-nth my-variable 'test 0)
    (bydi-was-set-to-last my-variable 'evil-eye)))

As you can see, this works both for setq and let bindings. In fact, watchers work mostly like mocks and offer the same verification macros just using {called=>set} and {with=>to}.

Back to mocking functions. You can also provide an alternate implementation using :with for more fine-grained control.

;; Also using `my-org-file-counter' defined above.

(ert-deftest count-three ()
  (let ((files (list (list "." ".." "one.txt" "two.org")
                     (list "." ".." "three.org" "four.org"))))

    (bydi ((:mock directory-files :with (lambda (&rest _) (pop files))))

      (should (eq 1 (my-org-file-counter "/some/dir")))
      (should (eq 2 (my-org-file-counter "/some/dir"))))))

If you want to verify a function call’s arguments but are only interested in some of them matching, you can use elision by providing a list containing the bydi-elision variable ('... by default).

(defun many-args (a b c d e)
  "Return a list of A, B, C, D and E."
  (list a b c d e))

(ert-deftest many-args ()
  (bydi (many-args)
    (many-args 1 2 3 4 5)

    (bydi-was-called-with many-args '(... 4 5))
    ;; or
    (bydi-was-called-with many-args '(1 ... 3))))

Macros bydi-was-{called,set}{-with,-to} accept an optional argument to clear the the history for that function or variable (might allow for some easier chaining in some scenarios).

(defvar useful-var nil)

(defun useful (person)
  "Message about a useful PERSON."
  (when (string= person "Claire")
    (setq useful-var 'claire)
    (message "Claire is useful")))

(ert-deftest useful ()
  (bydi (message
         (:watch useful-var))
    (useful "Claire")

    ;; Check but then clear history for `message'.
    (bydi-was-called-with message "Claire is useful" :clear t)
    (bydi-was-set-to useful-var 'claire :clear t)

    (useful "Jack")

    ;; Can verify it wasn't called this time now.
    (bydi-was-not-called message)
    (bydi-was-not-set useful-var)))

Verifying macro expansion can be done with bydi-match-expansion.

(defmacro my-useful-macro (name &rest body)
  "Notify about expansion of BODY named NAME."
  (declare (indent defun))

  `(progn
     (message ,(format "Expanding your %s" name))
     ,@body))

(my-useful-macro hello
  (message "yes"))

(ert-deftest match-expansion ()
  (bydi-match-expansion
   (my-useful-macro macro
     (setq some-variable 'some-value))
   '(progn
     (message "Expanding your macro")
     (setq some-variable 'some-value))))

Limitations

Inline functions (those using defsubst) can’t be mocked. If you’re using cl-defstruct you could pass option :noinline to keep slot accessors mockable. This will make functions run slower, so you might be better off creating a helper setup macro for your structs.

(cl-defstruct (horse (:noinline t))
  "A horse with a name."
  name)

(bydi ((:mock horse-name :return "no-name"))
  (should (string= "no-name" (horse-name 'not-a-horse))))

;; Or better.

(cl-defstruct (horse)
  "A horse with a name."
  name)

(defun horse-name-redirect (horse)
  "Return the horse's name."
  (horse-name horse))

(bydi ((:mock horse-name-redirect :return "no-name"))
  (should (string= "no-name" (horse-name 'not-a-horse))))

Also confer variable bydi-mock--risky for a (incomplete) list of functions that shouldn’t be mocked because it will likely lead to execution errors.

You can silence warnings emitted when mocking these.

(defun indirect-string= (a b)
  "Indirect version of `string='.

Compares A and B."
  (when (fboundp 'string=)
    (string= a b)))

(let ((bound nil))

  (bydi ((:risky-mock fboundp :return bound))

    (should-not (indirect-string= "test" "test"))

    (setq bound t)

    (should (indirect-string= "test" "test"))))

Experimental

You may stack bydi forms and share history between them using keyword :history like so:

;; Function `bydi--make-table' is used internally to create histories.
(let ((shared (bydi--make-table)))

  (bydi ((:always forward-char)) :history shared

    (forward-char)

    (bydi ((:always backward-char)) :history shared
      (backward-char))

    (bydi-was-called forward-char)
    (bydi-was-called backward-char)))

You also may do this for volatile functions using keyword :volatile like so:

(let ((shared (bydi--make-table)))

  (bydi ((:sometimes buffer-live-p)) :volatile shared

    (should (buffer-live-p))

    (bydi ((:othertimes dired)) :volatile shared

      (should-not (dired))
      (should (buffer-live-p))

      (bydi-toggle-sometimes)

      (should (dired))
      (should-not (buffer-live-p)))))