Testiere2

Add examples

changed some internal names; improved some error messages

Added more examples

renaming exports

Added New Readme

1 files changed, 223 insertions, 134 deletions

diff --git a/README.org b/README.org
index 755a885..0e4ca0c 100644
--- a/README.org
+++ b/README.org
@@ -1,175 +1,264 @@
 A [[https://en.wiktionary.org/wiki/testiere][testiere]] is armor for the head of a horse and ~testiere~ is armor
-for the head of your ~defun~ forms.  
+for the your lisp forms.
 
 * Testiere
 
-With ~testiere~ you can program in an interactive TDD-like
-fashion. Tests are included at the top of a ~defun/t~ form. When you
-recompile your functions interactively, the tests are run. If any
-fail, you are dropped into a debugger where you can decide to revert
-the definition to the last known working version, or you can choose to
-unbind it altogether.
+With ~testiere~, you embed test expressions directly into your
+code. When you compile, those tests are run. If any tests fail, you
+are dropped into the debugger where you can decide what to do.
 
-The system supports mocking and stubbing in your tests, so that you
-can, e.g. test the system in different dynamic contexts or by mocking
-network request functions.
+This approach has several beneifts:
 
-Here is an example:
+1. **Does Not Add Dependencies** You do not need to add ~testiere~ as
+   a dependency to your project. It is enough to load ~testiere~ into
+   your Lisp image and evoke ~(testiere:on)~. 
+2. **TDD** Common Lisp is a language well suited to interactive
+   development. Why should testing be any different? With ~testiere~
+   you can test functions as you =C-c C-c= them in SLIME, or whenever
+   you load or compile a file.
+3. **Self Documentation** Because tests are in the source (but do not
+   end up compiled into executable code unless ~testiere~ is "on"),
+   you get purposeful documentation of your code for free.  Why read a
+   comment when there's a test!?
 
-#+begin_src lisp
-
-(defun/t sum-3 (x y &key (z 10))
-  "Sums three numbers, Z has a default value of 10"
-  :tests
-  (:program some-test-function)
-  (= (1 2) 13)        ; (sum-3 1 2) == 13
-  (= (1 2 :z 3) 6)    ; (sum-3 1 2 :z 3) == 6
-  (:outputp (0 0)      ; tests that (sum-3 0 0) passes the predicate
-           (lambda (result) (= 10 result)))
-  (:fails             ; ensures that (sum-3 "strings" "ain't" :z "numbers") fails
-   ("strings" "ain't" :z "numbers"))
-  :end
-  (+ x y z))
-
-#+end_src
-
-In the above, a function ~sum-3~ is defined with five embedded
-tests. The test specification syntax is detailed below. If any of the
-tests fail, the function will not be redefined and you will drop into
-the debugger, which asks you how you'd like to proceed.
+Out of the box, ~testiere~ supports testing of the following:
 
-The approach to TDD-like development taking by ~testiere~ may not be
-appropriate to all circumstances, but it is good for interactive
-development of interactive applications (😉) whose "main loop"
-involves a good sized collection of unit-testable functions.
+- ~defun~
+- ~defmethod~
+- ~deftype~
+- ~defclass~
+- ~defstruct~
 
-** Test Specification 
-
-There are a few kinds of tests available. 
-
-*** For the Impatient, Just Use =:program= Tests
-
-Most users will probably benefit from the ~:program~ style test.  Here
-is a quick example:
+** A Basic Example
 
 #+begin_src lisp
 
-(defun test-fibble ()
-  (assert (= 13 (fibble 1 2))))
-
-(defun/t fibble (x y &key (z 10))
-  "Adds three numbers, one of which defaults to 10."
-  :tests
-  (:program test-fibble)
-  :end
+(defun add3 (x y z)
+  "Adds three numbers"
+  #+testiere
+  (:tests
+   (= 6 (add3 1 2 3))
+   (:fails (add3 "hey"))
+   (:fails (add3 1 2)))
   (+ x y z))
-
+  
 #+end_src
 
-In the above test, we insist that the ~test-fibble~ function not
-signal an error condition in order for ~fibble~ to be successfully
-(re)compiled.
-
-*** Basic Test Specifications
+This compiles as normal. If you wish to run the tests in the
+~(:tests ...)~ form, however, you need to turn testiere on.
 
-A test suite is a list of forms that appear between ~:tests~ and
-~:end~ in the body of a ~defun/t~ form.  The test suite must appear
-after any optional docstring and before the function body actually
-begins.
-
-A catalog of test form specifications follows.
+#+begin_src lisp
 
-**** Comparator Test Specifications
+(testiere:testiere-on)
 
-: (comparator (&rest args...) value)
+#+end_src
 
-The ~comparator~ should be the name of a binary predicate (like ~<~ or
-~eql~).  These tests proceed by calling ~(comparator (apply my-fun args) value)~
-If the comparison fails, an error condition is signaled. 
+Now if you try recompiling =add3= those tests will be run.
 
-Amending the above example, we include a comparator test:
+This approach lets you add tests to functions without actually
+including the testiere source in your distributed code. You need only
+have testiere loaded and turned on during development.
 
+You can, of course, turn testiere off too:
 
 #+begin_src lisp
-(defun/t fibble (x y &key (z 10))
-  "Adds three numbers, one of which defaults to 10."
-  :tests
-  (:program test-fibble)
-  (= (0 0 :z 30) 30)     ; (assert (= (fibble 0 0 :z 30) 30))
-  :end
-  (+ x y z))
+
+(testiere:testiere-off)
 
 #+end_src
 
-**** Other Test Specifications
+** Tests Expressions
+
+Within the body of a ~(:tests ...)~ form are test expressions.
+
+| Expression                                         | Description                                    |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:is form)~                                       | The test fails if ~form~ evaluates             |
+|                                                    | to NIL.                                        |
+|----------------------------------------------------+------------------------------------------------|
+| ~(pred form1 form2)~                               | E.g  ~(= (foo) 10)~  Provides more             |
+|                                                    | informative error messages than ~:is~          |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:funcall function arg1 ...)~                     | Calls a function with some arguments.          |
+|                                                    | If this function signals an error,             |
+|                                                    | then the test fails. Useful when               |
+|                                                    | running many or complex tests.                 |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:fails form)~                                    | Evaluates ~form~ and expects it to             |
+|                                                    | signal an error. If it does not                |
+|                                                    | signal an error, the test fails.               |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:signals condition form)~                        | Evaluates ~form~ and expects it to             |
+|                                                    | signal a condition of type                     |
+|                                                    | ~condition~. If it does not, then              |
+|                                                    | the test fails.                                |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:let bindings test1 ...)~                        | Runs test expressions in the context           |
+|                                                    | of some bound variables.                       |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:with-defuns ((name args body) ...) tests ... )~ | Mimics ~labels~ syntax. Used for               |
+|                                                    | stubbing / mocking functions will which        |
+|                                                    | have temporary definitions for the             |
+|                                                    | duration of the ~:with-defuns~ form.           |
+|----------------------------------------------------+------------------------------------------------|
+| ~(:with-generic name methods tests ... )~          | Temporarily redefine the an entire generic     |
+|                                                    | function for the duration of the enclosed      |
+|                                                    | ~tests~. ~methods~ is a list of forms, each of |
+|                                                    | is essentially anything  that normally follows |
+|                                                    | ~(defmethod name ...)~.                        |
+|                                                    | E.g. ~((x string) (string-upcase x))~ or       |
+|                                                    | ~(:after (x string) (print "after"))~          |
+
+** Examples
 
-Every other form appearing in a test suite is a list that starts with
-a keyword. 
+#+begin_src lisp
+(defpackage :testiere.examples
+  (:use #:cl #:testiere))
 
-- ~(:program FUNCTION-NAME ARGS...)~ runs a function named
-  FUNCTION-NAME with arguments ARGS. This function is meant to act as
-  a test suite for the function being defined with defun/t. It may
-  call that function and ASSERT things about it.
-- ~(:outputp (..ARGS...) PREDICATE)~ asserts that the output passes
-  the one-argument predicate.
-- ~(:afterp (...ARGS...) THUNK)~ asserts that the thunk should return
-  non-nil after the function has run. Good for testing values of
-  dynamic variables that the function might interact with.
-- ~(:fails (...ARGS...))~ asserts that the function will produce an
-  error with the given arguments.
-- ~(:signals (...ARGS...) CONDITION)~ where ~CONDITION~ is the name of
-  a condition. Asserts that the function will signal a condition of
-  the supplied type when called with the provided arguments.
+(defpackage :dummy
+  (:use #:cl))
 
+(in-package :testiere.examples)
 
-*** Mocking and Stubbing 
+;;; Turn Testiere On.
+(testiere-on)
 
-The following test forms allow for the running of tests inside a
-context in which certain functions or global values are bound:
+;;; BASIC TESTS
 
-Binding variables looks like
+(defun add3 (x y z)
+  "Adds three numbers"
+  #+testiere
+  (:tests
+   (= 6 (add3 1 2 3))
+   (:is (evenp (add3 2 2 2)))
+   (:fails (add3))
+   (:fails (add3 1 2 "oh no")))
+  (+ x y z))
 
-: (:let LET-BINDINGS TESTS) 
-    
-and are useful for binding dynamic variables for use during a set of
-tests.
+;;; Using external tests
+
+(defun dummy::test-add10 (n)
+  "Tests add10 in the same way N times. Obviously useless. We define
+this in a separate package to give you an idea that you can embed
+tests that aren't part of the package you're testing."
+  (loop :repeat n :do 
+    (assert (= 13 (add10 3)))))
+
+(defun add10 (x)
+  "Adds 10 to X"
+  #+testiere
+  (:tests
+   (:funcall 'dummy::test-add10 1))
+  (+ x 10))
+
+;;; Adding some context to tests with :LET
+
+(defvar *count*)
+
+(defun increment-count (&optional (amount 1))
+  "Increments *COUNT* by AMOUNT"
+  #+testiere
+  (:tests
+   (:let ((*count* 5))
+     (:funcall #'increment-count)
+     (= *count* 6)
+     (:funcall #'increment-count 4)
+     (= *count* 10))
+   (:let ((*count* -10))
+     (= (increment-count) -9)))
+  (incf *count* amount))
+
+;;; Stubbing functions with :WITH-DEFUNS
+
+(defun dummy::make-drakma-request (url)
+  "Assume this actually makes an HTTP request using drakma"
+  )
+
+(defun test-count-words-in-response ()
+  (assert (= 3 (count-words-in-response "blah"))))
+
+(defun count-words-in-response (url)
+  "Fetches a url and counts the words in the response."
+  #+testiere
+  (:tests
+   (:with-defuns
+       ((dummy::make-drakma-request (url)
+                                    (declare (values (simple-array character)))
+                                    "Hello     there    dudes"))
+     (= 3 (count-words-in-response "dummy-url"))
+     (:funcall 'test-count-words-in-response)))
+  (loop
+    :with resp string := (dummy::make-drakma-request url)
+    :with in-word? := nil
+    :for char :across resp
+    :when (and in-word? (not (alphanumericp char)))
+      :count 1 :into wc
+      :and :do (setf in-word? nil)
+    :when (alphanumericp char)
+      :do (setf in-word? t)
+    :finally (return
+               (if (alphanumericp char) (1+ wc) wc))))
+
+;;; Testing Classes
+
+(defclass point ()
+  ((x
+    :accessor px
+    :initform 0
+    :initarg :x)
+   (y
+    :accessor py
+    :initform 0
+    :initarg :y))
+  #+testiere
+  (:tests
+   (:let ((pt (make-instance 'point :x 10 :y 20)))
+     (= 20 (py pt))
+     (= 10 (px pt))
+     (:is (< (px pt) (py pt))))))
+
+;;; Testing Structs
+
+(defstruct pt
+  x y
+  #+testiere
+  (:tests
+   (:let ((pt (make-pt :x 10 :y 20)))
+     (= 20 (pt-y pt))
+     (:is (< (pt-x pt) (pt-y pt))))))
+
+;;; Testing Types
+
+(deftype optional-int ()
+  #+testiere
+  (:tests
+   (:is (typep nil 'optional-int))
+   (:is (typep 10 'optional-int))
+   (:is (not (typep "foo" 'optional-int))))
+  '(or integer null))
 
-For example 
+#+end_src
 
-#+begin_src lisp
 
-    (defvar *count*)
-    
-    (defun/t increment-count ()
-      "Increments the *count* variable."
-      :tests
-      (:let ((*count* 4))
-        (:afterp () (lambda () (= *count* 5))) ; 5 after the first call
-        (= () 6)                               ; 6 after the second
-        (:outputp () (lambda (x) (= x 7))))    ; and 7 after the third
-      :end
-      (incf *count*))
-#+end_src
-    
-The ~:with-stubs~ form is similar, except that it binds temporary
-values to functions that might be called by the form in
-questions. Useful for mocking.
+** How does it work?
 
-#+begin_src lisp
+Under the hood, ~testiere~ defines a custom ~*macroexpand-hook*~ that
+consults a registry of hooks.  If a macro is found in the registery,
+tests are extracted and run whenever they appear. Otherwise the hook
+expands code normally.
 
+** Extending
 
-    (defun just-a-function ()
-      (print "Just a function."))
-    
-    (defun/t call-just-a-function ()
-      "Calls JUST-A-FUNCTION."
-      :tests
-      (:with-stubs ((just-a-function () (print "TEMP JUST-A-FUNCTION.")))
-        (equal () "TEMP JUST-A-FUNCTION."))
-      :end
-      (just-a-function))
+Users can register ~testiere~ hooks by calling
+~testiere:register-hook~ on three arguments:
 
-#+end_src
-      
-In the above, the temporary redefinition of ~JUST-A-FUNCTION~ is used.
+1. A symbol naming a macro
+2. A function designator for a function that extracts tests from a
+   macro call (from the ~&whole~ of a macro call), returning the
+   modified form and a list of the extracted test expressions. All of
+   the built-ins hooks use the ~testiere::standard-extractor~.
+3. An optional function accepting the same ~&whole~ of the macro call,
+   and returning a list of restart handlers that are inserted as-is
+   into the body of a ~restart-case~.  See =src/standard-hooks.lisp=
+   for examples.