Common Lisp programming is often presented as “interactive”. In most languages, modifications to your program are applied by recompiling it and restarting it. In contrast, Common Lisp lets you incrementally modify your program while it is running.
While this approach is convenient, especially for exploratory programming, it also means that the state of your program during execution does not always reflect the source code. You do not just define new constructs: you look them up, inspect them, modify them or delete them. I had to learn a lot of subtleties the hard way. This article is a compendium of information related to the interactive nature of Common Lisp.
Variables
In Common Lisp variables are identified by symbols. Evaluating (SETQ A 42)
creates or updates a variable with the integer 42 as value, and associates
it to the A symbol. After the call to SETQ, (BOUNDP 'A) will return T
and (SYMBOL-VALUE 'A) will return 42.
You do not delete a variable: instead, you remove the association between the
symbol and the variable. You do so with MAKUNBOUND. Following the previous
example, (MAKUNBOUND 'A) will remove the association between the A symbol
and the variable. And (BOUNDP 'A) returns NIL as expected. As for
(SYMBOL-VALUE 'A), it now signals an UNBOUND-VARIABLE error as mandated by
the standard.
What about DEFVAR and DEFPARAMETER? They are also used to declare
variables (globally defined ones), associating them with symbols. Both define
“special” variables (i.e. variables for which all bindings are dynamic; see
CLtL2 9.2). The difference is that the initial value passed to DEFVAR is not
evaluated if it already has a value. MAKUNBOUND will work on variables
declared with DEFVAR or DEFPARAMETER as expected.
DEFCONSTANT is a bit more complicated. CLtL21 5.3.2 states that “once a
name has been declared by defconstant to be constant, any further assignment
to or binding of that special variable is an error”, but does not clearly
define whether MAKUNBOUND should or should not be able to be used on
constants. However, CLtL2 5.3.2 also states that “defconstant […] does assert
that the value of the variable name is fixed and does license the compiler to
build assumptions about the value into programs being compiled”. If the
compiler is allowed to rely on the value associated with the variable name, it
would make sense not to allow the deletion of the binding. Thus it is
recommended to only use constants for values that are guaranteed to never
change, e.g. mathematical constants. Most of the time you want DEFPARAMETER.
Note that MAKUNBOUND does not apply to lexical variables.
Functions
Common Lisp is a Lisp-2, meaning that variables and functions are part of two separate namespaces. Despite this clear separation, functions behave similarly to variables.
Using DEFUN will either create or update the global function associated with
a symbol. SYMBOL-FUNCTION returns the globally defined function associated
with a symbol, and FMAKUNBOUND deletes this association.
Let us point out a common mistake when referencing functions: (QUOTE F)
(abbreviated as 'F) yields a symbol while (FUNCTION F) (abbreviated as
#'F) yields a function. The function argument of FUNCALL and APPLY can
be either a symbol or a function (see CLtL2 7.3) It has two consequences:
First, one can write a function referencing F as (QUOTE F) with the
expectation that F will later be bound to a function. The following function
definition is perfectly valid even though F has not been defined yet:
(defun foo (a b)
(funcall 'f a b))
Second, redefining the F function will update its association (or binding)
to the F symbol, but the previous function will still be available if it has
been referenced somewhere before the update. For example:
(setf (symbol-function 'foo) #'1+)
(let ((old-foo #'foo))
(setf (symbol-function 'foo) #'1-)
(funcall old-foo 42))
What about macros? Since macros are a specific kind of functions (CLtL2 5.1.4
“a macro is essentially a function from forms to forms”), it is not surprising
that they share the same namespace and can be manipulated in the same way as
functions with FBOUNDP, SYMBOL-FUNCTION and FMAKUNBOUND.
Symbols and packages
While functions and variables are familiar concepts to developers, Common Lisp symbols and packages are a bit more peculiar.
A symbol is interned when it is part of a package. The most explicit way to
create an interned symbol is to use INTERN, e.g. (INTERN "FOO"). INTERN
interns the symbol in the current package by default, but one can pass a
package as second argument. After that, (FIND-SYMBOL "FOO") will return our
interned symbol as expected.
More surprising, the reader automatically interns symbols. You can test it by
evaluating (READ-FROM-STRING "BAR"). After evaluation, BAR is a symbol
interned in the current package. This also means that it is very easy to pollute
a package with symbols in ways you did not necessarily expect. To clean up,
simply use UNINTERN. Remember to refer to the right symbol: to remove the
symbol BAR from the package FOO, use (UNINTERN 'FOO::BAR "BAR").
A symbol is either internal or external. EXPORT will make a symbol external
to its package while UNEXPORT will make it internal. As for UNINTERN,
confusion usually arises around which symbol is affected. (UNEXPORT 'FOO:BAR "FOO") correctly refers to the external symbol in the FOO package and makes
it internal again. (UNEXPORT 'BAR "FOO") will signal an error since the
BAR symbol is not part of the FOO package (unless of course the current
package happens to be FOO).
Packages themselves can be created with MAKE-PACKAGE and destroyed with
DELETE-PACKAGE. Developers are usually more familiar with DEFPACKAGE, a
macro allowing the creation of a package and its configuration (package use
list, imported and exported symbols, etc.) in a declarative way. A surprising
and frustrating behavior is that evaluating a DEFPACKAGE form for a package
that already exists will result in undefined behavior if the new declaration
“is not consistent” (CLtL2 11.7) with the current state of the package. As an
example, adding symbols to the export list is perfectly fine. Removing one
will result in undefined behavior (usually an error) due to the inconsistency
of the export list. Fortunately, Common Lisp offers all the necessary
functions to manipulate packages and their symbols: use them!
Classes
The Common Lisp standard includes CLOS, the Common Lisp Object System. Unsurprisingly it provides multiple ways to interact with classes and objects dynamically.
As variables or functions, classes are identified by symbols and FIND-CLASS
returns the class associated with a symbol. Class names are part of a separate
namespace shared with structures and types.
The DEFCLASS macro is the only way to define or redefine a class. Redefining
a class means that instances created afterward with MAKE-INSTANCE will use
the new definition. Existing instances are updated: newly added slots are
added (either unbound or using the value associated with :INITFORM) and
slots that are not defined anymore are deleted.
UPDATE-INSTANCE-FOR-REDEFINED-CLASS is particularly interesting: developers
can define methods for this generic function in order to control how instances
are updated when their class is redefined.
Defining classes may imply implicitly defining methods: the :ACCESSOR,
:READER and :WRITER slot keyword arguments will lead to the creation of
generic functions. When a class is redefined, methods associated with slots
that have been removed will live on.
A limitation of CLOS is that classes cannot be deleted. FIND-CLASS can be
used as a place, and (SETF (FIND-CLASS 'FOO) NIL) will remove the
association between the FOO symbol and the class, but the class itself and
its instances will not disappear. While this limitation may seem strange, ask
yourself how an implementation should handle instances of a class that has
been deleted.
The class of an instance can be changed with CHANGE-CLASS: slots that exist
in the new class will be conserved while those that do not are deleted. New
slots are either unbound or set to the value associated with :INITFORM in
the new class. In a way similar to UPDATE-INSTANCE-FOR-REDEFINED-CLASS,
UPDATE-INSTANCE-FOR-DIFFERENT-CLASS lets developers control precisely the
process.
Generics and methods
Generics are functions which can be specialized based on the class (and not type as one could expect) of their arguments and which can have a method combination type.
Generics can be created explicitly with DEFGENERIC or implicitly when
DEFMETHOD is called and the list of parameter specializers and method
combination does not match any existing generic function. Since generics are
functions, FBOUNDP, SYMBOL-FUNCTION and FMAKUNBOUND will work as
expected.
Methods themselves are either defined as part of the DEFGENERIC call or
separately with DEFMETHOD. Discovering the different methods associated with
a generic function is a bit more complicated. There is no standard way to list
the methods associated with a generic, but it is at least possible to look up
a method with FIND-METHOD. Do remember to pass a function (and not a symbol)
as the generic, and to pass classes (and not symbols naming classes) in the
list of specializers.
Redefinition is not as obvious as for non-generic functions. When redefining a
generic with DEFGENERIC all methods defined as part of the previous
DEFGENERIC form are removed and methods defined in the redefinition are
added. However, methods defined separately with DEFMETHOD are not affected.
For example, in the following code, the second call to DEFGENERIC will
replace the two methods specialized on INTEGER and FLOAT respectively by a
single one specialized on a STREAM, but the method specialized on STRING
will remain unaffected.
(defgeneric foo (a)
(:method ((a integer))
(format nil "~A is an integer" a))
(:method ((a float))
(format nil "~A is a float" a)))
(defmethod foo ((a string))
(format nil "~S is a string" a))
(defgeneric foo (a)
(:method ((a stream))
(format nil "~A is a stream" a)))
Note that trying to redefine a generic with a different parameter lambda list will cause the removal of all previously defined methods since none of them can match the new parameters.
Removing a method will require you to find it first using FIND-METHOD and
then use REMOVE-METHOD. With the previous example, removing the method
specialized on a STRING argument is done with:
(remove-method #'foo (find-method #'foo nil (list (find-class 'string)) nil))
Working with methods is not always easy, and two errors are very common.
First, remember that changing the combinator in a DEFMETHOD will define a
new method. If you realize that your :AFTER method should use :AROUND and
reevaluate the DEFMETHOD form, remember to delete the method with the
:AFTER combinator or you will end up with two methods being called.
Second, when defining a method for a generic from another package, remember to
correctly refer to the generic. If you want to define a method on the BAR
generic from package FOO, use (DEFMETHOD FOO:BAR (…) …) and not
(DEFMETHOD BAR (…) …). In the latter case, you will define a new BAR
generic in the current package.
Meta Object Protocol
While CLOS is already quite powerful, various interactions are impossible. One cannot create classes or methods programmatically, introspect classes or instances for example to list their slots or obtain all their superclasses, or list all the methods associated with a generic function.
In addition of an example of a CLOS implementation, The Art of the Metaobject Protocol2 defines multiple extensions to CLOS including metaclasses, metaobjects, dynamic class and generic creation, class introspection and much more.
Most Common Lisp implementations implement at least part of these extensions, usually abbreviated as “MOP”, for “MetaObject Protocol”. The well-known closer-mop system can be used as a compatibility layer for multiple implementations.
Structures
Structures are record constructs defined with DEFSTRUCT. At a glance they
may seem very similar to classes, but they have a fundamental limitation:
the results of redefining a structure are undefined (CLtL2 19.2).
While this property allows implementations to handle structures in a more efficient way than classes, it makes structures unsuitable for incremental development. As such, they should only be used as a last resort, when a regular class has been proved to be a performance bottleneck.
Conditions
While conditions look very similar to classes the Common Lisp standard does not define them as classes. This is one of the few differences between the standard and CLtL2 which clearly states in 29.3.4 that “Common Lisp condition types are in fact CLOS classes, and condition objects are ordinary CLOS objects”.
This is why one uses DEFINE-CONDITION instead of DEFCLASS and
MAKE-CONDITION instead of MAKE-INSTANCE. This also means that one should
not use slot-related functions (including the very useful WITH-SLOTS macro)
with conditions.
In practice, most modern implementations follow CLtL2 and the
CLOS-CONDITIONS:INTEGRATE X3J13 Cleanup
Issue
and implement conditions as CLOS classes, meaning that conditions can be
manipulated and redefined as any other classes. And the same way as any other
classes, they cannot be deleted.
Types
Types are identified by symbols and are part of the same namespace as classes (which should not be surprising since defining a class automatically defines a type with the same name).
Types are defined with DEFTYPE, but documentation is surprisingly silent on
the effects of type redefinition. This can lead to interesting situations. On
some implementations (e.g. SBCL and CCL), if a class slot is defined as having
the type FOO, redefining FOO will not be taken into account and the type
checking operation (which is not mandated by the standard) will use the
previous definition of the type. Infortunatly Common Lisp does not mandate
any specific behavior on slot type mismatches (CLtL2 28.1.3.2).
Thus developers should not expect any useful effect from redefining types. Restarting the implementation after substantial type changes is probably best.
In the same vein interactions with types are very limited. You cannot find a
type by its symbol or even check whether a type exists or not. Calling
TYPE-OF on a value will return a type this value satisfies, but the nature
of the type is implementation-dependent (CLtL2 4.9): it could be any
supertype. In other words, TYPE-OF could absolutly return T for all values
but NIL. At least SUBTYPE-P lets you check whether a type is a subtype of
another type.
Going further
Common Lisp is a complex language with a lot of subtleties, way more than what can be covered in a blog post. The curious reader will probably skip the standard (not because you have to buy it, but because it is a low quality scan of a printed document and jump directly to CLtL2 or the Common Lisp HyperSpec. The Art of the Metaobject Protocol is of course the normative reference for the CLOS extensions usually referred to as “MOP”.