diff options
Diffstat (limited to 'README.org')
| -rw-r--r-- | README.org | 357 |
1 files changed, 223 insertions, 134 deletions
@@ -1,175 +1,264 @@ A [[https://en.wiktionary.org/wiki/testiere][testiere]] is armor for the head of a horse and ~testiere~ is armor -for the head of your ~defun~ forms. +for the your lisp forms. * Testiere -With ~testiere~ you can program in an interactive TDD-like -fashion. Tests are included at the top of a ~defun/t~ form. When you -recompile your functions interactively, the tests are run. If any -fail, you are dropped into a debugger where you can decide to revert -the definition to the last known working version, or you can choose to -unbind it altogether. +With ~testiere~, you embed test expressions directly into your +code. When you compile, those tests are run. If any tests fail, you +are dropped into the debugger where you can decide what to do. -The system supports mocking and stubbing in your tests, so that you -can, e.g. test the system in different dynamic contexts or by mocking -network request functions. +This approach has several beneifts: -Here is an example: +1. **Does Not Add Dependencies** You do not need to add ~testiere~ as + a dependency to your project. It is enough to load ~testiere~ into + your Lisp image and evoke ~(testiere:on)~. +2. **TDD** Common Lisp is a language well suited to interactive + development. Why should testing be any different? With ~testiere~ + you can test functions as you =C-c C-c= them in SLIME, or whenever + you load or compile a file. +3. **Self Documentation** Because tests are in the source (but do not + end up compiled into executable code unless ~testiere~ is "on"), + you get purposeful documentation of your code for free. Why read a + comment when there's a test!? -#+begin_src lisp - -(defun/t sum-3 (x y &key (z 10)) - "Sums three numbers, Z has a default value of 10" - :tests - (:program some-test-function) - (= (1 2) 13) ; (sum-3 1 2) == 13 - (= (1 2 :z 3) 6) ; (sum-3 1 2 :z 3) == 6 - (:outputp (0 0) ; tests that (sum-3 0 0) passes the predicate - (lambda (result) (= 10 result))) - (:fails ; ensures that (sum-3 "strings" "ain't" :z "numbers") fails - ("strings" "ain't" :z "numbers")) - :end - (+ x y z)) - -#+end_src - -In the above, a function ~sum-3~ is defined with five embedded -tests. The test specification syntax is detailed below. If any of the -tests fail, the function will not be redefined and you will drop into -the debugger, which asks you how you'd like to proceed. +Out of the box, ~testiere~ supports testing of the following: -The approach to TDD-like development taking by ~testiere~ may not be -appropriate to all circumstances, but it is good for interactive -development of interactive applications (😉) whose "main loop" -involves a good sized collection of unit-testable functions. +- ~defun~ +- ~defmethod~ +- ~deftype~ +- ~defclass~ +- ~defstruct~ -** Test Specification - -There are a few kinds of tests available. - -*** For the Impatient, Just Use =:program= Tests - -Most users will probably benefit from the ~:program~ style test. Here -is a quick example: +** A Basic Example #+begin_src lisp -(defun test-fibble () - (assert (= 13 (fibble 1 2)))) - -(defun/t fibble (x y &key (z 10)) - "Adds three numbers, one of which defaults to 10." - :tests - (:program test-fibble) - :end +(defun add3 (x y z) + "Adds three numbers" + #+testiere + (:tests + (= 6 (add3 1 2 3)) + (:fails (add3 "hey")) + (:fails (add3 1 2))) (+ x y z)) - + #+end_src -In the above test, we insist that the ~test-fibble~ function not -signal an error condition in order for ~fibble~ to be successfully -(re)compiled. - -*** Basic Test Specifications +This compiles as normal. If you wish to run the tests in the +~(:tests ...)~ form, however, you need to turn testiere on. -A test suite is a list of forms that appear between ~:tests~ and -~:end~ in the body of a ~defun/t~ form. The test suite must appear -after any optional docstring and before the function body actually -begins. - -A catalog of test form specifications follows. +#+begin_src lisp -**** Comparator Test Specifications +(testiere:testiere-on) -: (comparator (&rest args...) value) +#+end_src -The ~comparator~ should be the name of a binary predicate (like ~<~ or -~eql~). These tests proceed by calling ~(comparator (apply my-fun args) value)~ -If the comparison fails, an error condition is signaled. +Now if you try recompiling =add3= those tests will be run. -Amending the above example, we include a comparator test: +This approach lets you add tests to functions without actually +including the testiere source in your distributed code. You need only +have testiere loaded and turned on during development. +You can, of course, turn testiere off too: #+begin_src lisp -(defun/t fibble (x y &key (z 10)) - "Adds three numbers, one of which defaults to 10." - :tests - (:program test-fibble) - (= (0 0 :z 30) 30) ; (assert (= (fibble 0 0 :z 30) 30)) - :end - (+ x y z)) + +(testiere:testiere-off) #+end_src -**** Other Test Specifications +** Tests Expressions + +Within the body of a ~(:tests ...)~ form are test expressions. + +| Expression | Description | +|----------------------------------------------------+------------------------------------------------| +| ~(:is form)~ | The test fails if ~form~ evaluates | +| | to NIL. | +|----------------------------------------------------+------------------------------------------------| +| ~(pred form1 form2)~ | E.g ~(= (foo) 10)~ Provides more | +| | informative error messages than ~:is~ | +|----------------------------------------------------+------------------------------------------------| +| ~(:funcall function arg1 ...)~ | Calls a function with some arguments. | +| | If this function signals an error, | +| | then the test fails. Useful when | +| | running many or complex tests. | +|----------------------------------------------------+------------------------------------------------| +| ~(:fails form)~ | Evaluates ~form~ and expects it to | +| | signal an error. If it does not | +| | signal an error, the test fails. | +|----------------------------------------------------+------------------------------------------------| +| ~(:signals condition form)~ | Evaluates ~form~ and expects it to | +| | signal a condition of type | +| | ~condition~. If it does not, then | +| | the test fails. | +|----------------------------------------------------+------------------------------------------------| +| ~(:let bindings test1 ...)~ | Runs test expressions in the context | +| | of some bound variables. | +|----------------------------------------------------+------------------------------------------------| +| ~(:with-defuns ((name args body) ...) tests ... )~ | Mimics ~labels~ syntax. Used for | +| | stubbing / mocking functions will which | +| | have temporary definitions for the | +| | duration of the ~:with-defuns~ form. | +|----------------------------------------------------+------------------------------------------------| +| ~(:with-generic name methods tests ... )~ | Temporarily redefine the an entire generic | +| | function for the duration of the enclosed | +| | ~tests~. ~methods~ is a list of forms, each of | +| | is essentially anything that normally follows | +| | ~(defmethod name ...)~. | +| | E.g. ~((x string) (string-upcase x))~ or | +| | ~(:after (x string) (print "after"))~ | + +** Examples -Every other form appearing in a test suite is a list that starts with -a keyword. +#+begin_src lisp +(defpackage :testiere.examples + (:use #:cl #:testiere)) -- ~(:program FUNCTION-NAME ARGS...)~ runs a function named - FUNCTION-NAME with arguments ARGS. This function is meant to act as - a test suite for the function being defined with defun/t. It may - call that function and ASSERT things about it. -- ~(:outputp (..ARGS...) PREDICATE)~ asserts that the output passes - the one-argument predicate. -- ~(:afterp (...ARGS...) THUNK)~ asserts that the thunk should return - non-nil after the function has run. Good for testing values of - dynamic variables that the function might interact with. -- ~(:fails (...ARGS...))~ asserts that the function will produce an - error with the given arguments. -- ~(:signals (...ARGS...) CONDITION)~ where ~CONDITION~ is the name of - a condition. Asserts that the function will signal a condition of - the supplied type when called with the provided arguments. +(defpackage :dummy + (:use #:cl)) +(in-package :testiere.examples) -*** Mocking and Stubbing +;;; Turn Testiere On. +(testiere-on) -The following test forms allow for the running of tests inside a -context in which certain functions or global values are bound: +;;; BASIC TESTS -Binding variables looks like +(defun add3 (x y z) + "Adds three numbers" + #+testiere + (:tests + (= 6 (add3 1 2 3)) + (:is (evenp (add3 2 2 2))) + (:fails (add3)) + (:fails (add3 1 2 "oh no"))) + (+ x y z)) -: (:let LET-BINDINGS TESTS) - -and are useful for binding dynamic variables for use during a set of -tests. +;;; Using external tests + +(defun dummy::test-add10 (n) + "Tests add10 in the same way N times. Obviously useless. We define +this in a separate package to give you an idea that you can embed +tests that aren't part of the package you're testing." + (loop :repeat n :do + (assert (= 13 (add10 3))))) + +(defun add10 (x) + "Adds 10 to X" + #+testiere + (:tests + (:funcall 'dummy::test-add10 1)) + (+ x 10)) + +;;; Adding some context to tests with :LET + +(defvar *count*) + +(defun increment-count (&optional (amount 1)) + "Increments *COUNT* by AMOUNT" + #+testiere + (:tests + (:let ((*count* 5)) + (:funcall #'increment-count) + (= *count* 6) + (:funcall #'increment-count 4) + (= *count* 10)) + (:let ((*count* -10)) + (= (increment-count) -9))) + (incf *count* amount)) + +;;; Stubbing functions with :WITH-DEFUNS + +(defun dummy::make-drakma-request (url) + "Assume this actually makes an HTTP request using drakma" + ) + +(defun test-count-words-in-response () + (assert (= 3 (count-words-in-response "blah")))) + +(defun count-words-in-response (url) + "Fetches a url and counts the words in the response." + #+testiere + (:tests + (:with-defuns + ((dummy::make-drakma-request (url) + (declare (values (simple-array character))) + "Hello there dudes")) + (= 3 (count-words-in-response "dummy-url")) + (:funcall 'test-count-words-in-response))) + (loop + :with resp string := (dummy::make-drakma-request url) + :with in-word? := nil + :for char :across resp + :when (and in-word? (not (alphanumericp char))) + :count 1 :into wc + :and :do (setf in-word? nil) + :when (alphanumericp char) + :do (setf in-word? t) + :finally (return + (if (alphanumericp char) (1+ wc) wc)))) + +;;; Testing Classes + +(defclass point () + ((x + :accessor px + :initform 0 + :initarg :x) + (y + :accessor py + :initform 0 + :initarg :y)) + #+testiere + (:tests + (:let ((pt (make-instance 'point :x 10 :y 20))) + (= 20 (py pt)) + (= 10 (px pt)) + (:is (< (px pt) (py pt)))))) + +;;; Testing Structs + +(defstruct pt + x y + #+testiere + (:tests + (:let ((pt (make-pt :x 10 :y 20))) + (= 20 (pt-y pt)) + (:is (< (pt-x pt) (pt-y pt)))))) + +;;; Testing Types + +(deftype optional-int () + #+testiere + (:tests + (:is (typep nil 'optional-int)) + (:is (typep 10 'optional-int)) + (:is (not (typep "foo" 'optional-int)))) + '(or integer null)) -For example +#+end_src -#+begin_src lisp - (defvar *count*) - - (defun/t increment-count () - "Increments the *count* variable." - :tests - (:let ((*count* 4)) - (:afterp () (lambda () (= *count* 5))) ; 5 after the first call - (= () 6) ; 6 after the second - (:outputp () (lambda (x) (= x 7)))) ; and 7 after the third - :end - (incf *count*)) -#+end_src - -The ~:with-stubs~ form is similar, except that it binds temporary -values to functions that might be called by the form in -questions. Useful for mocking. +** How does it work? -#+begin_src lisp +Under the hood, ~testiere~ defines a custom ~*macroexpand-hook*~ that +consults a registry of hooks. If a macro is found in the registery, +tests are extracted and run whenever they appear. Otherwise the hook +expands code normally. +** Extending - (defun just-a-function () - (print "Just a function.")) - - (defun/t call-just-a-function () - "Calls JUST-A-FUNCTION." - :tests - (:with-stubs ((just-a-function () (print "TEMP JUST-A-FUNCTION."))) - (equal () "TEMP JUST-A-FUNCTION.")) - :end - (just-a-function)) +Users can register ~testiere~ hooks by calling +~testiere:register-hook~ on three arguments: -#+end_src - -In the above, the temporary redefinition of ~JUST-A-FUNCTION~ is used. +1. A symbol naming a macro +2. A function designator for a function that extracts tests from a + macro call (from the ~&whole~ of a macro call), returning the + modified form and a list of the extracted test expressions. All of + the built-ins hooks use the ~testiere::standard-extractor~. +3. An optional function accepting the same ~&whole~ of the macro call, + and returning a list of restart handlers that are inserted as-is + into the body of a ~restart-case~. See =src/standard-hooks.lisp= + for examples. |