aboutsummaryrefslogtreecommitdiff
path: root/README.org
diff options
context:
space:
mode:
Diffstat (limited to 'README.org')
-rw-r--r--README.org357
1 files changed, 223 insertions, 134 deletions
diff --git a/README.org b/README.org
index 755a885..0e4ca0c 100644
--- a/README.org
+++ b/README.org
@@ -1,175 +1,264 @@
A [[https://en.wiktionary.org/wiki/testiere][testiere]] is armor for the head of a horse and ~testiere~ is armor
-for the head of your ~defun~ forms.
+for the your lisp forms.
* Testiere
-With ~testiere~ you can program in an interactive TDD-like
-fashion. Tests are included at the top of a ~defun/t~ form. When you
-recompile your functions interactively, the tests are run. If any
-fail, you are dropped into a debugger where you can decide to revert
-the definition to the last known working version, or you can choose to
-unbind it altogether.
+With ~testiere~, you embed test expressions directly into your
+code. When you compile, those tests are run. If any tests fail, you
+are dropped into the debugger where you can decide what to do.
-The system supports mocking and stubbing in your tests, so that you
-can, e.g. test the system in different dynamic contexts or by mocking
-network request functions.
+This approach has several beneifts:
-Here is an example:
+1. **Does Not Add Dependencies** You do not need to add ~testiere~ as
+ a dependency to your project. It is enough to load ~testiere~ into
+ your Lisp image and evoke ~(testiere:on)~.
+2. **TDD** Common Lisp is a language well suited to interactive
+ development. Why should testing be any different? With ~testiere~
+ you can test functions as you =C-c C-c= them in SLIME, or whenever
+ you load or compile a file.
+3. **Self Documentation** Because tests are in the source (but do not
+ end up compiled into executable code unless ~testiere~ is "on"),
+ you get purposeful documentation of your code for free. Why read a
+ comment when there's a test!?
-#+begin_src lisp
-
-(defun/t sum-3 (x y &key (z 10))
- "Sums three numbers, Z has a default value of 10"
- :tests
- (:program some-test-function)
- (= (1 2) 13) ; (sum-3 1 2) == 13
- (= (1 2 :z 3) 6) ; (sum-3 1 2 :z 3) == 6
- (:outputp (0 0) ; tests that (sum-3 0 0) passes the predicate
- (lambda (result) (= 10 result)))
- (:fails ; ensures that (sum-3 "strings" "ain't" :z "numbers") fails
- ("strings" "ain't" :z "numbers"))
- :end
- (+ x y z))
-
-#+end_src
-
-In the above, a function ~sum-3~ is defined with five embedded
-tests. The test specification syntax is detailed below. If any of the
-tests fail, the function will not be redefined and you will drop into
-the debugger, which asks you how you'd like to proceed.
+Out of the box, ~testiere~ supports testing of the following:
-The approach to TDD-like development taking by ~testiere~ may not be
-appropriate to all circumstances, but it is good for interactive
-development of interactive applications (😉) whose "main loop"
-involves a good sized collection of unit-testable functions.
+- ~defun~
+- ~defmethod~
+- ~deftype~
+- ~defclass~
+- ~defstruct~
-** Test Specification
-
-There are a few kinds of tests available.
-
-*** For the Impatient, Just Use =:program= Tests
-
-Most users will probably benefit from the ~:program~ style test. Here
-is a quick example:
+** A Basic Example
#+begin_src lisp
-(defun test-fibble ()
- (assert (= 13 (fibble 1 2))))
-
-(defun/t fibble (x y &key (z 10))
- "Adds three numbers, one of which defaults to 10."
- :tests
- (:program test-fibble)
- :end
+(defun add3 (x y z)
+ "Adds three numbers"
+ #+testiere
+ (:tests
+ (= 6 (add3 1 2 3))
+ (:fails (add3 "hey"))
+ (:fails (add3 1 2)))
(+ x y z))
-
+
#+end_src
-In the above test, we insist that the ~test-fibble~ function not
-signal an error condition in order for ~fibble~ to be successfully
-(re)compiled.
-
-*** Basic Test Specifications
+This compiles as normal. If you wish to run the tests in the
+~(:tests ...)~ form, however, you need to turn testiere on.
-A test suite is a list of forms that appear between ~:tests~ and
-~:end~ in the body of a ~defun/t~ form. The test suite must appear
-after any optional docstring and before the function body actually
-begins.
-
-A catalog of test form specifications follows.
+#+begin_src lisp
-**** Comparator Test Specifications
+(testiere:testiere-on)
-: (comparator (&rest args...) value)
+#+end_src
-The ~comparator~ should be the name of a binary predicate (like ~<~ or
-~eql~). These tests proceed by calling ~(comparator (apply my-fun args) value)~
-If the comparison fails, an error condition is signaled.
+Now if you try recompiling =add3= those tests will be run.
-Amending the above example, we include a comparator test:
+This approach lets you add tests to functions without actually
+including the testiere source in your distributed code. You need only
+have testiere loaded and turned on during development.
+You can, of course, turn testiere off too:
#+begin_src lisp
-(defun/t fibble (x y &key (z 10))
- "Adds three numbers, one of which defaults to 10."
- :tests
- (:program test-fibble)
- (= (0 0 :z 30) 30) ; (assert (= (fibble 0 0 :z 30) 30))
- :end
- (+ x y z))
+
+(testiere:testiere-off)
#+end_src
-**** Other Test Specifications
+** Tests Expressions
+
+Within the body of a ~(:tests ...)~ form are test expressions.
+
+| Expression | Description |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:is form)~ | The test fails if ~form~ evaluates |
+| | to NIL. |
+|----------------------------------------------------+------------------------------------------------|
+| ~(pred form1 form2)~ | E.g ~(= (foo) 10)~ Provides more |
+| | informative error messages than ~:is~ |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:funcall function arg1 ...)~ | Calls a function with some arguments. |
+| | If this function signals an error, |
+| | then the test fails. Useful when |
+| | running many or complex tests. |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:fails form)~ | Evaluates ~form~ and expects it to |
+| | signal an error. If it does not |
+| | signal an error, the test fails. |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:signals condition form)~ | Evaluates ~form~ and expects it to |
+| | signal a condition of type |
+| | ~condition~. If it does not, then |
+| | the test fails. |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:let bindings test1 ...)~ | Runs test expressions in the context |
+| | of some bound variables. |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:with-defuns ((name args body) ...) tests ... )~ | Mimics ~labels~ syntax. Used for |
+| | stubbing / mocking functions will which |
+| | have temporary definitions for the |
+| | duration of the ~:with-defuns~ form. |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:with-generic name methods tests ... )~ | Temporarily redefine the an entire generic |
+| | function for the duration of the enclosed |
+| | ~tests~. ~methods~ is a list of forms, each of |
+| | is essentially anything that normally follows |
+| | ~(defmethod name ...)~. |
+| | E.g. ~((x string) (string-upcase x))~ or |
+| | ~(:after (x string) (print "after"))~ |
+
+** Examples
-Every other form appearing in a test suite is a list that starts with
-a keyword.
+#+begin_src lisp
+(defpackage :testiere.examples
+ (:use #:cl #:testiere))
-- ~(:program FUNCTION-NAME ARGS...)~ runs a function named
- FUNCTION-NAME with arguments ARGS. This function is meant to act as
- a test suite for the function being defined with defun/t. It may
- call that function and ASSERT things about it.
-- ~(:outputp (..ARGS...) PREDICATE)~ asserts that the output passes
- the one-argument predicate.
-- ~(:afterp (...ARGS...) THUNK)~ asserts that the thunk should return
- non-nil after the function has run. Good for testing values of
- dynamic variables that the function might interact with.
-- ~(:fails (...ARGS...))~ asserts that the function will produce an
- error with the given arguments.
-- ~(:signals (...ARGS...) CONDITION)~ where ~CONDITION~ is the name of
- a condition. Asserts that the function will signal a condition of
- the supplied type when called with the provided arguments.
+(defpackage :dummy
+ (:use #:cl))
+(in-package :testiere.examples)
-*** Mocking and Stubbing
+;;; Turn Testiere On.
+(testiere-on)
-The following test forms allow for the running of tests inside a
-context in which certain functions or global values are bound:
+;;; BASIC TESTS
-Binding variables looks like
+(defun add3 (x y z)
+ "Adds three numbers"
+ #+testiere
+ (:tests
+ (= 6 (add3 1 2 3))
+ (:is (evenp (add3 2 2 2)))
+ (:fails (add3))
+ (:fails (add3 1 2 "oh no")))
+ (+ x y z))
-: (:let LET-BINDINGS TESTS)
-
-and are useful for binding dynamic variables for use during a set of
-tests.
+;;; Using external tests
+
+(defun dummy::test-add10 (n)
+ "Tests add10 in the same way N times. Obviously useless. We define
+this in a separate package to give you an idea that you can embed
+tests that aren't part of the package you're testing."
+ (loop :repeat n :do
+ (assert (= 13 (add10 3)))))
+
+(defun add10 (x)
+ "Adds 10 to X"
+ #+testiere
+ (:tests
+ (:funcall 'dummy::test-add10 1))
+ (+ x 10))
+
+;;; Adding some context to tests with :LET
+
+(defvar *count*)
+
+(defun increment-count (&optional (amount 1))
+ "Increments *COUNT* by AMOUNT"
+ #+testiere
+ (:tests
+ (:let ((*count* 5))
+ (:funcall #'increment-count)
+ (= *count* 6)
+ (:funcall #'increment-count 4)
+ (= *count* 10))
+ (:let ((*count* -10))
+ (= (increment-count) -9)))
+ (incf *count* amount))
+
+;;; Stubbing functions with :WITH-DEFUNS
+
+(defun dummy::make-drakma-request (url)
+ "Assume this actually makes an HTTP request using drakma"
+ )
+
+(defun test-count-words-in-response ()
+ (assert (= 3 (count-words-in-response "blah"))))
+
+(defun count-words-in-response (url)
+ "Fetches a url and counts the words in the response."
+ #+testiere
+ (:tests
+ (:with-defuns
+ ((dummy::make-drakma-request (url)
+ (declare (values (simple-array character)))
+ "Hello there dudes"))
+ (= 3 (count-words-in-response "dummy-url"))
+ (:funcall 'test-count-words-in-response)))
+ (loop
+ :with resp string := (dummy::make-drakma-request url)
+ :with in-word? := nil
+ :for char :across resp
+ :when (and in-word? (not (alphanumericp char)))
+ :count 1 :into wc
+ :and :do (setf in-word? nil)
+ :when (alphanumericp char)
+ :do (setf in-word? t)
+ :finally (return
+ (if (alphanumericp char) (1+ wc) wc))))
+
+;;; Testing Classes
+
+(defclass point ()
+ ((x
+ :accessor px
+ :initform 0
+ :initarg :x)
+ (y
+ :accessor py
+ :initform 0
+ :initarg :y))
+ #+testiere
+ (:tests
+ (:let ((pt (make-instance 'point :x 10 :y 20)))
+ (= 20 (py pt))
+ (= 10 (px pt))
+ (:is (< (px pt) (py pt))))))
+
+;;; Testing Structs
+
+(defstruct pt
+ x y
+ #+testiere
+ (:tests
+ (:let ((pt (make-pt :x 10 :y 20)))
+ (= 20 (pt-y pt))
+ (:is (< (pt-x pt) (pt-y pt))))))
+
+;;; Testing Types
+
+(deftype optional-int ()
+ #+testiere
+ (:tests
+ (:is (typep nil 'optional-int))
+ (:is (typep 10 'optional-int))
+ (:is (not (typep "foo" 'optional-int))))
+ '(or integer null))
-For example
+#+end_src
-#+begin_src lisp
- (defvar *count*)
-
- (defun/t increment-count ()
- "Increments the *count* variable."
- :tests
- (:let ((*count* 4))
- (:afterp () (lambda () (= *count* 5))) ; 5 after the first call
- (= () 6) ; 6 after the second
- (:outputp () (lambda (x) (= x 7)))) ; and 7 after the third
- :end
- (incf *count*))
-#+end_src
-
-The ~:with-stubs~ form is similar, except that it binds temporary
-values to functions that might be called by the form in
-questions. Useful for mocking.
+** How does it work?
-#+begin_src lisp
+Under the hood, ~testiere~ defines a custom ~*macroexpand-hook*~ that
+consults a registry of hooks. If a macro is found in the registery,
+tests are extracted and run whenever they appear. Otherwise the hook
+expands code normally.
+** Extending
- (defun just-a-function ()
- (print "Just a function."))
-
- (defun/t call-just-a-function ()
- "Calls JUST-A-FUNCTION."
- :tests
- (:with-stubs ((just-a-function () (print "TEMP JUST-A-FUNCTION.")))
- (equal () "TEMP JUST-A-FUNCTION."))
- :end
- (just-a-function))
+Users can register ~testiere~ hooks by calling
+~testiere:register-hook~ on three arguments:
-#+end_src
-
-In the above, the temporary redefinition of ~JUST-A-FUNCTION~ is used.
+1. A symbol naming a macro
+2. A function designator for a function that extracts tests from a
+ macro call (from the ~&whole~ of a macro call), returning the
+ modified form and a list of the extracted test expressions. All of
+ the built-ins hooks use the ~testiere::standard-extractor~.
+3. An optional function accepting the same ~&whole~ of the macro call,
+ and returning a list of restart handlers that are inserted as-is
+ into the body of a ~restart-case~. See =src/standard-hooks.lisp=
+ for examples.