path: root/README.org

A [[https://en.wiktionary.org/wiki/testiere][testiere]] is armor for the head of a horse and ~testiere~ is armor
for the your lisp forms.

* Testiere

With ~testiere~, you embed test expressions directly into your
code. When you compile, those tests are run. If any tests fail, you
are dropped into the debugger where you can decide what to do.

This approach has several beneifts:

1. **Does Not Add Dependencies** You do not need to add ~testiere~ as
   a dependency to your project. It is enough to load ~testiere~ into
   your Lisp image and evoke ~(testiere:on)~. 
2. **TDD** Common Lisp is a language well suited to interactive
   development. Why should testing be any different? With ~testiere~
   you can test functions as you =C-c C-c= them in SLIME, or whenever
   you load or compile a file.
3. **Self Documentation** Because tests are in the source (but do not
   end up compiled into executable code unless ~testiere~ is "on"),
   you get purposeful documentation of your code for free.  Why read a
   comment when there's a test!?
4. **Automatic Test Suite Definition** Tests are automatically added
   to test suites. There is one suite per package in which testiere
   tests appear. These tests can be run using ~testiere:run-suites~.

Out of the box, ~testiere~ supports testing of the following:

- ~defun~
- ~defmethod~
- ~deftype~
- ~defclass~
- ~defstruct~

** A Basic Example

#+begin_src lisp

(defun add3 (x y z)
  "Adds three numbers"
  #+testiere
  (:tests
   (= 6 (add3 1 2 3))
   (:fails (add3 "hey"))
   (:fails (add3 1 2)))
  (+ x y z))
  
#+end_src

This compiles as normal. If you wish to run the tests in the
~(:tests ...)~ form, however, you need to turn testiere on.

#+begin_src lisp

(testiere:on)

#+end_src

Now if you try recompiling =add3= those tests will be run.

This approach lets you add tests to functions without actually
including the testiere source in your distributed code. You need only
have testiere loaded and turned on during development.

You can, of course, turn testiere off too:

#+begin_src lisp

(testiere:off)

#+end_src

** Tests Expressions

Within the body of a ~(:tests ...)~ form are test expressions.

| Expression                                         | Description                                          |
|----------------------------------------------------+------------------------------------------------------|
| ~(:is form)~                                       | The test fails if ~form~ evaluates to NIL.           |
|----------------------------------------------------+------------------------------------------------------|
| ~(pred form1 form2)~                               | E.g  ~(= (foo) 10)~  Provides more informative       |
|                                                    | error messages than ~:is~                            |
|----------------------------------------------------+------------------------------------------------------|
| ~(:funcall function arg1 ...)~                     | Calls a function with some arguments. If this        |
|                                                    | function signals an error, then the test fails.      |
|                                                    | Useful when running  several complext tests.         |
|----------------------------------------------------+------------------------------------------------------|
| ~(:fails form)~                                    | Evaluates ~form~ and expects it to singal an error.  |
|                                                    | If it does not signal an error, then the test fails. |
|----------------------------------------------------+------------------------------------------------------|
| ~(:signals condition form)~                        | Evaluates ~form~ and expects it to signal a          |
|                                                    | condition of type ~condition~. If it does not, then  |
|                                                    | the test fails.                                      |
|----------------------------------------------------+------------------------------------------------------|
| ~(:let bindings test1 ...)~                        | Runs test expressions in the context of some bound   |
|                                                    | variables.                                           |
|----------------------------------------------------+------------------------------------------------------|
| ~(:with-defuns ((name args body) ...) tests ... )~ | Mimics ~labels~ syntax. Used for stubbing / mocking  |
|                                                    | functions will which have temporary definitions for  |
|                                                    | the duration of the ~:with-defuns~ form.             |
|----------------------------------------------------+------------------------------------------------------|
| ~(:with-generic name methods tests ... )~          | Temporarily redefine the an entire generic           |
|                                                    | function for the duration of the enclosed            |
|                                                    | ~tests~. ~methods~ is a list of forms, each of       |
|                                                    | is essentially anything  that normally follows       |
|                                                    | ~(defmethod name ...)~.                              |
|                                                    | E.g. ~((x string) (string-upcase x))~ or             |
|                                                    | ~(:after (x string) (print "after"))~                |
|----------------------------------------------------+------------------------------------------------------|
| ~(:do form)~                                       | Evaluate ~form~ for its side effects. Useful         |
|                                                    | within a ~:let~, ~:with-defuns~, or ~:with-generic~  |
|                                                    |                                                      |
** Examples

#+begin_src lisp
(defpackage :testiere.examples
  (:use #:cl #:testiere))

(defpackage :dummy
  (:use #:cl))

(in-package :testiere.examples)

;;; Turn Testiere On.
(testiere:on)

;;; BASIC TESTS

(defun add3 (x y z)
  "Adds three numbers"
  #+testiere
  (:tests
   (= 6 (add3 1 2 3))
   (:is (evenp (add3 2 2 2)))
   (:fails (add3))
   (:fails (add3 1 2 "oh no")))
  (+ x y z))

;;; Using external tests

(defun dummy::test-add10 (n)
  "Tests add10 in the same way N times. Obviously useless. We define
this in a separate package to give you an idea that you can embed
tests that aren't part of the package you're testing."
  (loop :repeat n :do 
    (assert (= 13 (add10 3)))))

(defun add10 (x)
  "Adds 10 to X"
  #+testiere
  (:tests
   (:funcall 'dummy::test-add10 1))
  (+ x 10))

;;; Adding some context to tests with :LET

(defvar *count*)

(defun increment-count (&optional (amount 1))
  "Increments *COUNT* by AMOUNT"
  #+testiere
  (:tests
   (:let ((*count* 5))
     (:do (increment-count))
     (= *count* 6)
     (:do (increment-count 4))
     (= *count* 10))
   (:let ((*count* -10))
     (= (increment-count) -9)))
  (incf *count* amount))

;;; Stubbing functions with :WITH-DEFUNS

(defun dummy::make-drakma-request (url)
  "Assume this actually makes an HTTP request using drakma"
  )

(defun test-count-words-in-response ()
  (assert (= 3 (count-words-in-response "blah"))))

(defun count-words-in-response (url)
  "Fetches a url and counts the words in the response."
  #+testiere
  (:tests
   (:with-defuns
       ((dummy::make-drakma-request (url)
                                    "Hello     there    dudes"))
     (= 3 (count-words-in-response "dummy-url"))
     (:funcall 'test-count-words-in-response)))
  (loop
    :with resp string := (dummy::make-drakma-request url)
    :with in-word? := nil
    :for char :across resp
    :when (and in-word? (not (alphanumericp char)))
      :count 1 :into wc
      :and :do (setf in-word? nil)
    :when (alphanumericp char)
      :do (setf in-word? t)
    :finally (return
               (if (alphanumericp char) (1+ wc) wc))))

;;; Testing Classes

(defclass point ()
  ((x
    :accessor px
    :initform 0
    :initarg :x)
   (y
    :accessor py
    :initform 0
    :initarg :y))
  #+testiere
  (:tests
   (:let ((pt (make-instance 'point :x 10 :y 20)))
     (= 20 (py pt))
     (= 10 (px pt))
     (:is (< (px pt) (py pt))))))

;;; Testing Structs

(defstruct pt
  x y
  #+testiere
  (:tests
   (:let ((pt (make-pt :x 10 :y 20)))
     (= 20 (pt-y pt))
     (:is (< (pt-x pt) (pt-y pt))))))

;;; Testing Types

(deftype optional-int ()
  #+testiere
  (:tests
   (:is (typep nil 'optional-int))
   (:is (typep 10 'optional-int))
   (:is (not (typep "foo" 'optional-int))))
  '(or integer null))
#+end_src
** Running Test Suites

The above also defines a test suite for the forms defined in the
~:testiere.examples~ package.

The ~RUN-SUITES~ function lets you run test suites associated with
packages.  The ~:AUTOMATIC-CONTINUE~ argument avoids dropping into the
debugger, instead printing a test failure.

If the ~:PACKAGES~ argument is empty, then all test suites known to
Testiere are run.

#+begin_src lisp

(run-suites :packages '(:testiere.examples)
            :automatic-continue t)

#+end_src

#+begin_example 

Running tests for package "TESTIERE.EXAMPLES"
  Testing DEFUN ADD3                                             [pass]
  Testing DEFUN ADD10                                            [pass]
  Testing DEFUN INCREMENT-COUNT                                  [pass]
  Testing DEFUN COUNT-WORDS-IN-RESPONSE                          [pass]
  Testing DEFCLASS POINT                                         [pass]
  Testing DEFTYPE OPTIONAL-INT                                   [pass]

#+end_example



** How does it work?

Under the hood, ~testiere~ defines a custom ~*macroexpand-hook*~ that
consults a registry of hooks.  If a macro is found in the registery,
tests are extracted and run whenever they appear. Otherwise the hook
expands code normally.

** Extending

Users can register ~testiere~ hooks by calling
~testiere:register-hook~ on three arguments:

1. A symbol naming a macro
2. A function designator for a function that extracts tests from a
   macro call (from the ~&whole~ of a macro call), returning the
   modified form and a list of the extracted test expressions. All of
   the built-ins hooks use the ~testiere::standard-extractor~.
3. An optional function accepting the same ~&whole~ of the macro call,
   and returning a list of restart handlers that are inserted as-is
   into the body of a ~restart-case~.  See =src/standard-hooks.lisp=
   for examples.

Any macro that has been so registered will be available for testing at
compile time.