Threads and Synchronisation

Beneath the job system, cl-threads has threads, locks and semaphores. Their interface follows Bordeaux-threads where that is what people expect. Use them for long-lived threads of your own, such as a render thread or an audio thread, and to coordinate threads with each other. For splitting work across processors, use a job pool instead of making threads.

Threads

(make-thread function &key name bindings on-error) → thread
(join-thread thread) → the function's values
(current-thread) → thread
(thread-name thread) → string
(thread-finished-p thread) → boolean
(interrupt-thread thread function) → boolean

make-thread starts a thread that calls function with no arguments, and returns it at once:

(defvar *thread* (th:make-thread (lambda () (load-everything)) :name "loader"))
*thread*                        ; => #<THREAD "loader">
(th:thread-name *thread*)       ; => "loader"
(th:join-thread *thread*)       ; => what LOAD-EVERYTHING returned
(th:thread-finished-p *thread*) ; => T
*thread*                        ; => #<THREAD "loader" (finished)>

The name is a string, shown by debuggers and in the thread's printed form. Without :name, a thread is called "cl-threads thread"; name the threads you will want to find in a debugger.

Special variables in a new thread

A new thread sees the global values of special variables, not the bindings of the thread that made it. :bindings gives it some of its own, as a list of (symbol value), which it keeps for its whole life, as a job pool gives its workers:

(th:make-thread #'game-loop :name "game"
                            :bindings `((*random-state* ,(make-random-state t))))

(th:join-thread (th:make-thread (lambda () *print-base*) :bindings '((*print-base* 16))))
;; => 16

The values are taken when make-thread is called. A mistake in :bindings signals threads-error in the calling thread, before any thread starts:

(*PRINT-BASE* 16) are not bindings.  Give a list of (SYMBOL VALUE), such as ((*print-base* 16)).
PI is a constant, which a thread cannot bind.  Bind special variables only.

Joining

join-thread waits until the thread has ended, its function having returned or been unwound, and returns the values the function returned:

(th:join-thread (th:make-thread (lambda () (+ 1 2))))           ; => 3
(th:join-thread (th:make-thread (lambda () (values :a :b))))    ; => :A, :B

• Any number of threads may join a thread, any number of times, and each gets the values. Several threads may wait for the same thread at once. Joining a finished thread returns at once.

• A thread whose function was unwound by a non-local exit, such as (abort), returned nothing, so joining it returns no values. An error that ended it does what :on-error says; see Errors in a thread.

• A thread has finished, for thread-finished-p, once it has ended: a moment after its function has returned or been unwound, and always by the time a join returns.

• Joining the calling thread itself signals threads-error instead of waiting forever:

  #<THREAD "self-join"> cannot join itself: it would wait forever.  Join it from another thread.
  

• Only threads made by make-thread can be joined; see below.

Joining makes the thread's writes visible to the joiner.

Errors in a thread

An error a thread's function does not handle goes to the debugger in that thread, as any unhandled error does: in an interactive Lisp you get the thread's stack to look at, and a thread nobody joins, such as a render thread, loses none of its errors. That is :on-error :debug, the default.

A thread whose maker waits for its result can hand its errors to whoever joins it instead. With :on-error :signal, an error the function does not handle unwinds it, and join-thread signals a thread-error that carries it, in every thread that joins, each time:

(let ((thread (th:make-thread (lambda () (error "Boom.")) :on-error :signal)))
  (handler-case (th:join-thread thread)
    (th:thread-error (c)
      (th:thread-error-condition c))))   ; => the "Boom." error

The function of #<THREAD "cl-threads thread" (finished)> ended in an error.  It was made with :ON-ERROR :SIGNAL, which hands its errors to the threads that join it.
  The thread's error: Boom.

Errors the function handles itself, and its own non-local exits, stay its own. :signal suits a test's helper thread: in a Lisp whose debugger is disabled, as a batch run's is, an error in a thread made with :debug ends the whole Lisp. A job pool's :on-error makes the same choice for its jobs; see Errors in Jobs.

Knowing yourself

current-thread returns the calling thread, in any thread.

• In a thread made by make-thread, it is the object make-thread returned, from the thread's first instruction. A thread can therefore identify itself, for example to register itself somewhere, before make-thread has even returned it to the caller.
• In any other thread, such as the initial thread or a REPL thread, it is a thread that cl-threads makes the first time that thread asks, and returns ever after. It has the implementation's name for the thread, and thread-finished-p tells whether the thread still runs. It cannot be joined: cl-threads did not call its function, so it has no values of it to return, and join-thread signals threads-error instead of waiting.

(th:current-thread)                    ; => #<THREAD "main thread">
(eq (th:current-thread) (th:current-thread)) ; => T

So comparing threads always works, for example to check that code runs on the render thread:

(defun render-thread-p ()
  (eq (th:current-thread) *render-thread*))

After its first call in a thread, current-thread allocates nothing. Job pool workers are made by make-thread, so current-thread works in jobs too.

Interrupting a thread

interrupt-thread makes a thread call a function, with no arguments, in the middle of whatever it is doing, and returns at once. When the function returns, the thread goes on where it was. It is for looking at a thread, or for entering the debugger in it:

(th:interrupt-thread render-thread #'break)   ; the debugger shows the render thread's stack

• It returns t if the interrupt was sent, which the thread runs unless it finishes first, and nil if the thread had already finished.
• Any thread may be interrupted: one made by make-thread, one that current-thread gave, and the calling thread itself, which runs the function at once.
• Interrupts are enabled while the function runs, so it may be interrupted in turn, as a debugger entered from it must be.

Never leave the function by a non-local exit. It runs at once even when the thread is inside a foreign call. A throw, return-from or restart out of it would unwind the thread through the foreign code and leave that code unfinished, holding whatever it held, such as a lock that every other thread then waits for forever. In the debugger, a person chooses a restart, seeing the stack, and can tell whether that is safe.

Locks

(make-lock &optional name) → lock
(with-lock (lock) body ...)

A lock gives mutual exclusion: one thread at a time holds it.

(defvar *count* 0)
(defvar *count-lock* (th:make-lock "count"))

(th:parallel-for *pool* 0 10000
                 (lambda (start end)
                   (th:with-lock (*count-lock*)
                     (incf *count* (- end start)))))

• make-lock takes an optional name, a string, which debuggers show. Without one, the lock is called "cl-threads lock".
• with-lock evaluates the body holding the lock, and releases it however the body is left, by a normal return, a non-local exit or an error. It returns the body's values and allocates nothing.
• Locks are not recursive: a thread that holds a lock must not take it again, even through a function it calls.
• Keep the body short. Every other thread that needs the lock waits for it.

Semaphores

(make-semaphore &key (count 0)) → semaphore
(signal-semaphore semaphore &optional (count 1))
(wait-on-semaphore semaphore &key timeout) → boolean

A semaphore holds a count. signal-semaphore raises it, waking threads that wait. wait-on-semaphore waits until the count is above 0 and lowers it by one.

(defvar *ready* (th:make-semaphore))

(th:make-thread (lambda ()
                  (initialise-audio)
                  (th:signal-semaphore *ready*))
                :name "audio")

(th:wait-on-semaphore *ready*)   ; returns T once audio is initialised

Each unit of count lets one wait through. Signals are counted, so a signal before anyone waits is not lost. A count of 0 does nothing, so a computed count needs no check.

Counts. A semaphore made with :count starts with that many waits allowed through, which suits a pool of N interchangeable resources. (signal-semaphore semaphore n) raises the count by n at once, waking up to n waiting threads:

(defvar *slots* (th:make-semaphore :count 4))  ; four uploads at a time

(defun upload (data)
  (th:wait-on-semaphore *slots*)
  (unwind-protect (send-to-gpu data)
    (th:signal-semaphore *slots*)))

Timeouts

With :timeout, a number of seconds, wait-on-semaphore gives up after that long:

Call	Returns
(wait-on-semaphore s)	t, once it has lowered the count. Waits as long as it takes.
(wait-on-semaphore s :timeout 0.1)	t if it lowered the count within a tenth of a second, nil if it timed out.
(wait-on-semaphore s :timeout 0)	t if the count was above 0, lowered at once; nil without waiting otherwise.

The timeout is any non-negative real: 0.1, 1/10 and 2 all work, and so does 0, although SBCL's own wait does not take one.

Choosing between them

You want to	Use
Spread work over processors	parallel-for, or a job group
Run something in the background and check on it	a job group
Run a long-lived loop of your own, such as a render thread	make-thread
Compute something on a thread of its own and get the result	make-thread, then join-thread
See what a running thread is doing, or debug it	interrupt-thread with #'break
Protect data that several threads change	a lock
Let one thread wait for another's signal	a semaphore
Limit how many threads use something at once	a semaphore with a :count
Hand items from one thread to another	a lock and a semaphore together; see Recipes