Job Pools

A job pool is a set of worker threads that run jobs: the chunks of parallel-for and the jobs of job groups. Each group queues its own jobs, and the groups with jobs to run take turns: an idle worker runs a job of one group, then of the next. Idle workers sleep; they use no processor time until there is work.

A worker keeps nothing a job referred to alive once the job counts as finished, so what a job used can be collected as soon as wait-for-jobs returns: a font whose glyphs a job drew goes when you drop it, not when the workers next run something. The worker lets go of a job before it counts it finished, and since SBCL takes any word on a stack that looks like a pointer for a reference, it also clears the part of its stack the job used.

Making and destroying a pool

(make-job-pool &key workers name on-error error-handler bindings) → job-pool
(destroy-job-pool pool)
(with-job-pool (pool &rest options) body ...)

(defvar *pool* (th:make-job-pool :workers 4 :name "example worker"))
*pool*                           ; => #<JOB-POOL 4 workers>
(th:job-pool-worker-count *pool*) ; => 4
(th:destroy-job-pool *pool*)
*pool*                           ; => #<JOB-POOL 4 workers (destroyed)>

Option	Meaning	Default
`:workers`	How many worker threads, at least 1.	`(default-worker-count)`
`:name`	The workers' thread name; each is called `name` and its index, such as `"example worker 0"`.	`"cl-threads worker"`
`:on-error`	What a job's error does: `:signal` or `:debug`.	`:signal`
`:error-handler`	A function of the condition, the default handler of the groups this pool makes.	`nil`
`:bindings`	`((symbol value) ...)`: special variables to bind in every worker.	`()`

:on-error and :error-handler are explained in Errors in Jobs.

with-job-pool binds a pool made with the options, evaluates the body, and destroys the pool however the body is left:

(th:with-job-pool (pool :workers 2 :name "loader")
  ...)

destroy-job-pool finishes the pool's jobs, then stops the workers and waits for them:

from the moment it is called, the pool takes no more jobs from other threads;
it runs every job already queued or running, and the jobs those queue, since the pool's own workers may still submit while it closes;
once no job is queued or running, it stops the workers and waits for them, then returns;
calling it again, while the pool closes or after, does nothing;
calling it from one of the pool's own workers signals threads-error, since it would wait for itself.

So nothing that was submitted is dropped, and nobody waiting for a group is left waiting forever. A job that never finishes, such as one blocked on a semaphore nobody signals, keeps destroy-job-pool waiting too.

Once destroying has started, parallel-for, submit-job and submit-parallel-for from other threads signal threads-error:

#<JOB-POOL 1 worker (destroyed)> has been destroyed: it runs no more jobs.

How many workers

(default-worker-count) is one worker for each processor beyond two, which a game engine keeps for its main and render threads, and at least one. On a 32-processor machine it is 30. Where cl-threads cannot ask how many processors there are, it is 1.

Platform	How the processor count is found
Linux	`sysconf(_SC_NPROCESSORS_ONLN)`
Windows	`GetActiveProcessorCount(ALL_PROCESSOR_GROUPS)`: the active processors of every processor group
Elsewhere	unknown, so one worker

Outside Linux and Windows, pass :workers explicitly.

Workers that block, on a file, a socket or a lock, hold a worker without using a processor. For pools that mostly wait, such as one that reads files, more workers than processors can make sense.

Which worker is running

(worker-index) returns the calling worker's index in its pool, from 0 below job-pool-worker-count, or nil in a thread that is not a worker. It allocates nothing.

(th:worker-index) ; => NIL, at the REPL

Workers are threads made by make-thread, so (th:current-thread) and th:thread-name work in them too.

What workers see

Special variables. Workers see the global values of special variables, not the bindings of the thread that made the pool. Pass :bindings for the ones they need:

(defvar *level* :global)

(let ((*level* :local))
  (th:with-job-pool (pool :workers 1)
    ...))                                ; jobs see *level* as :GLOBAL

(let ((*level* :local))
  (th:with-job-pool (pool :workers 1 :bindings (list (list '*level* *level*)))
    ...))                                ; jobs see *level* as :LOCAL

The values are taken when the pool is made. A job sees the binding, not later changes made by the maker's let.

Output. *standard-output*, *error-output* and *trace-output* in the workers go where the maker's went, so a job's format t or trace output appears at your REPL. Several workers writing at once interleave their output; give each its own string, or write under a lock, when the order matters.

To send the workers' output somewhere else, bind those streams in :bindings. A stream given there takes the place of the maker's:

(th:make-job-pool :bindings (list (list '*standard-output* *engine-log*)
                                  (list '*error-output* *engine-log*)))

Any thread may give a pool work and wait for it. One pool for the whole program is the usual design: several pools compete for the same processors.

cl-threads

Reference

Job Pools

Making and destroying a pool

How many workers

Which worker is running

What workers see

Sharing a pool