colin/testiere.git - TDD-like dev for Common Lisp

‎

1. Testiere

A testiere is armor for the head of a horse and testiere is armor for the your lisp forms.

1 Testiere

With testiere, you embed test expressions directly into your code. When you compile, those tests are run. If any tests fail, you are dropped into the debugger where you can decide what to do.

This approach has several beneifts:

Does Not Add Dependencies You do not need to add testiere as a dependency to your project. It is enough to load testiere into your Lisp image and evoke (testiere:on).
TDD Common Lisp is a language well suited to interactive development. Why should testing be any different? With testiere you can test functions as you C-c C-c them in SLIME, or whenever you load or compile a file.
Self Documentation Because tests are in the source (but do not end up compiled into executable code unless testiere is "on"), you get purposeful documentation of your code for free. Why read a comment when there's a test!?
Automatic Test Suite Definition Tests are automatically added to test suites. There is one suite per package in which testiere tests appear. These tests can be run using testiere:run-suites.

Out of the box, testiere supports testing of the following:

defun
defmethod
deftype
defclass
defstruct

1.1 A Basic Example

(defun add3 (x y z)
  "Adds three numbers"
  #+testiere
  (:tests
   (= 6 (add3 1 2 3))
   (:fails (add3 "hey"))
   (:fails (add3 1 2)))
  (+ x y z))

This compiles as normal. If you wish to run the tests in the (:tests ...) form, however, you need to turn testiere on.

(testiere:on)

Now if you try recompiling add3 those tests will be run.

This approach lets you add tests to functions without actually including the testiere source in your distributed code. You need only have testiere loaded and turned on during development.

You can, of course, turn testiere off too:

(testiere:off)

1.2 Tests Expressions

Within the body of a (:tests ...) form are test expressions.

Expression	Description
`(:is form)`	The test fails if `form` evaluates to NIL.
`(pred form1 form2)`	E.g `(= (foo) 10)` Provides more informative
	error messages than `:is`
`(:funcall function arg1 ...)`	Calls a function with some arguments. If this
	function signals an error, then the test fails.
	Useful when running several complext tests.
`(:fails form)`	Evaluates `form` and expects it to singal an error.
	If it does not signal an error, then the test fails.
`(:signals condition form)`	Evaluates `form` and expects it to signal a
	condition of type `condition`. If it does not, then
	the test fails.
`(:let bindings test1 ...)`	Runs test expressions in the context of some bound
	variables.
`(:with-defuns ((name args body) ...) tests ... )`	Mimics `labels` syntax. Used for stubbing / mocking
	functions will which have temporary definitions for
	the duration of the `:with-defuns` form.
`(:with-generic name methods tests ... )`	Temporarily redefine the an entire generic
	function for the duration of the enclosed
	`tests`. `methods` is a list of forms, each of
	is essentially anything that normally follows
	`(defmethod name ...)`.
	E.g. `((x string) (string-upcase x))` or
	`(:after (x string) (print "after"))`
`(:do form)`	Evaluate `form` for its side effects. Useful
	within a `:let`, `:with-defuns`, or `:with-generic`

1.3 Examples

(defpackage :testiere.examples
  (:use #:cl #:testiere))

(defpackage :dummy
  (:use #:cl))

(in-package :testiere.examples)

;;; Turn Testiere On.
(testiere:on)

;;; BASIC TESTS

(defun add3 (x y z)
  "Adds three numbers"
  #+testiere
  (:tests
   (= 6 (add3 1 2 3))
   (:is (evenp (add3 2 2 2)))
   (:fails (add3))
   (:fails (add3 1 2 "oh no")))
  (+ x y z))

;;; Using external tests

(defun dummy::test-add10 (n)
  "Tests add10 in the same way N times. Obviously useless. We define
this in a separate package to give you an idea that you can embed
tests that aren't part of the package you're testing."
  (loop :repeat n :do 
    (assert (= 13 (add10 3)))))

(defun add10 (x)
  "Adds 10 to X"
  #+testiere
  (:tests
   (:funcall 'dummy::test-add10 1))
  (+ x 10))

;;; Adding some context to tests with :LET

(defvar *count*)

(defun increment-count (&optional (amount 1))
  "Increments *COUNT* by AMOUNT"
  #+testiere
  (:tests
   (:let ((*count* 5))
     (:do (increment-count))
     (= *count* 6)
     (:do (increment-count 4))
     (= *count* 10))
   (:let ((*count* -10))
     (= (increment-count) -9)))
  (incf *count* amount))

;;; Stubbing functions with :WITH-DEFUNS

(defun dummy::make-drakma-request (url)
  "Assume this actually makes an HTTP request using drakma"
  )

(defun test-count-words-in-response ()
  (assert (= 3 (count-words-in-response "blah"))))

(defun count-words-in-response (url)
  "Fetches a url and counts the words in the response."
  #+testiere
  (:tests
   (:with-defuns
       ((dummy::make-drakma-request (url)
                                    "Hello     there    dudes"))
     (= 3 (count-words-in-response "dummy-url"))
     (:funcall 'test-count-words-in-response)))
  (loop
    :with resp string := (dummy::make-drakma-request url)
    :with in-word? := nil
    :for char :across resp
    :when (and in-word? (not (alphanumericp char)))
      :count 1 :into wc
      :and :do (setf in-word? nil)
    :when (alphanumericp char)
      :do (setf in-word? t)
    :finally (return
               (if (alphanumericp char) (1+ wc) wc))))

;;; Testing Classes

(defclass point ()
  ((x
    :accessor px
    :initform 0
    :initarg :x)
   (y
    :accessor py
    :initform 0
    :initarg :y))
  #+testiere
  (:tests
   (:let ((pt (make-instance 'point :x 10 :y 20)))
     (= 20 (py pt))
     (= 10 (px pt))
     (:is (< (px pt) (py pt))))))

;;; Testing Structs

(defstruct pt
  x y
  #+testiere
  (:tests
   (:let ((pt (make-pt :x 10 :y 20)))
     (= 20 (pt-y pt))
     (:is (< (pt-x pt) (pt-y pt))))))

;;; Testing Types

(deftype optional-int ()
  #+testiere
  (:tests
   (:is (typep nil 'optional-int))
   (:is (typep 10 'optional-int))
   (:is (not (typep "foo" 'optional-int))))
  '(or integer null))

1.4 Running Test Suites

The above also defines a test suite for the forms defined in the :testiere.examples package.

The RUN-SUITES function lets you run test suites associated with packages. The :AUTOMATIC-CONTINUE argument avoids dropping into the debugger, instead printing a test failure.

If the :PACKAGES argument is empty, then all test suites known to Testiere are run.

(run-suites :packages '(:testiere.examples)
            :automatic-continue t)


Running tests for package "TESTIERE.EXAMPLES"
  Testing DEFUN ADD3                                             [pass]
  Testing DEFUN ADD10                                            [pass]
  Testing DEFUN INCREMENT-COUNT                                  [pass]
  Testing DEFUN COUNT-WORDS-IN-RESPONSE                          [pass]
  Testing DEFCLASS POINT                                         [pass]
  Testing DEFTYPE OPTIONAL-INT                                   [pass]

1.5 How does it work?

Under the hood, testiere defines a custom *macroexpand-hook* that consults a registry of hooks. If a macro is found in the registery, tests are extracted and run whenever they appear. Otherwise the hook expands code normally.

1.6 Extending

Users can register testiere hooks by calling testiere:register-hook on three arguments:

A symbol naming a macro
A function designator for a function that extracts tests from a macro call (from the &whole of a macro call), returning the modified form and a list of the extracted test expressions. All of the built-ins hooks use the testiere::standard-extractor.
An optional function accepting the same &whole of the macro call, and returning a list of restart handlers that are inserted as-is into the body of a restart-case. See src/standard-hooks.lisp for examples.

Any macro that has been so registered will be available for testing at compile time.

Created: 2024-04-19 Fri 09:21

Validate

Table of Contents