The Scanner

cl-wl/scanner turns Wayland protocol XML into cl-wl bindings. It plays the role of wayland-scanner but emits Lisp. It is a build-time tool: the generated files are committed next to their XML in protocols/, so loading cl-wl never loads the scanner or an XML parser.

(asdf:load-system :cl-wl/scanner)

It depends on xmls.

Two stages

1. The reader (scanner/reader.lisp) parses XML into a protocol model of plain, read-only structs. It checks the file against the protocol format and reports where any problem is. It is the only part that knows about XML, so replacing xmls means rewriting that one file. The reader tests in tests/reader-test.lisp define what a replacement must produce.
2. The generator (scanner/generator.lisp) reads only the model, and writes define-enum, define-interface and define-request forms. cl-wl's macros expand them into classes, functions and the tables libwayland needs.

Reading a protocol

(cl-wl/scanner:read-protocol #p"protocols/viewporter.xml")   ; from a file, as UTF-8
(cl-wl/scanner:parse-protocol "<protocol name=…>…</protocol>") ; from a string

Both return a protocol. The model keeps the protocol's own names ("wp_viewport", "set_destination") unchanged:

(let ((protocol (cl-wl/scanner:read-protocol "protocols/viewporter.xml")))
  (mapcar (lambda (interface)
            (list (cl-wl/scanner:interface-name interface)
                  (cl-wl/scanner:interface-version interface)
                  (mapcar #'cl-wl/scanner:message-name
                          (cl-wl/scanner:interface-requests interface))))
          (cl-wl/scanner:protocol-interfaces protocol)))
;; => (("wp_viewporter" 1 ("destroy" "get_viewport"))
;;     ("wp_viewport" 1 ("destroy" "set_source" "set_destination")))

The model is a description of the XML, not of the loaded bindings. Its interface-version, for instance, is the version attribute of the file. For the loaded bindings, use cl-wl's own introspection: cl-wl:supported-version, cl-wl:interface-requests and so on.

The model

All structs are read-only and may be shared between threads.

Struct	Slots (readers are struct-slot)
protocol	name, copyright (string or NIL), description, interfaces
interface	name, version, frozen-p, description, requests, events, enums
message	name, kind (:request/:event), opcode, since, deprecated-since, destructor-p, description, args
arg	name, type, interface, allow-null-p, enum ("error" or qualified "wl_shm.format"), summary, description
enum	name, since, bitfield-p, description, entries
entry	name, value, since, deprecated-since, summary, description
description	summary, text (common indentation and blank edge lines removed)

A message's destructor-p is true for type="destructor": a request that destroys its object, or an event after which the object is gone, such as wl_callback.done.

Types: wire-type (:int :uint :fixed :string :object :new-id :array :fd), message-kind, version ((integer 1)) and opcode ((unsigned-byte 16)). Each struct also has a make- constructor and a -p predicate, so you can build a model by hand.

protocol-error

The reader and generator signal protocol-error for XML that is not well formed or does not follow the format. Examples are a missing attribute, an unknown element or type, allow-null on an int, and a reference to an unknown interface or enum. The report gives the path to the problem:

protocol x, interface a, request r, arg s: unknown argument type "strin"

Readers: protocol-error-source (the pathname, or NIL for a string), protocol-error-context (the path as a list of strings, outermost first) and protocol-error-message. An error from read-protocol names the file first:

/home/me/protocols/x.xml: protocol x, interface a, request r, arg s: unknown argument type "strin"

The path is added to the error as it passes out of each element the scanner was reading or writing, and the file as it passes out of read-protocol; nothing global holds them. A handler-bind around a scanner call therefore sees the whole path on the condition that was signalled, before anything unwinds.

It is also signalled for a model built by hand without a part the format requires, such as (make-interface :name "x"), which has no version, and by bindings-text for a name that is not one of the protocols cl-wl ships:

MAKE-INTERFACE needs :VERSION: every interface of a protocol has one.
"viewportr" is not one of the protocols cl-wl ships bindings for, which are wayland, xdg-shell, …

Generating bindings

(cl-wl/scanner:generate-bindings protocol stream
                                 :package "cl-wl/viewporter"
                                 :source "viewporter.xml"
                                 :dependencies (list (cons core-protocol "cl-wl/wayland"))
                                 :array-elements (cl-wl/scanner:shipped-array-elements))

• :package is the package the bindings define.
• :source is named in the header comment.
• :dependencies lists, as (protocol . package), the protocols whose interfaces this one refers to. References into them are written with their package prefix, such as cl-wl/wayland:wl-surface.
• :array-elements gives the element types of arrays (see below).

The output is laid out by hand rather than by the pretty printer, whose layout can change between SBCL releases, so every release produces byte-identical files.

The protocols cl-wl ships

(shipped-protocols) returns them in dependency order as (name package dependencies):

(("wayland" "cl-wl/wayland" ())
 ("xdg-shell" "cl-wl/xdg-shell" ("wayland"))
 ("xdg-decoration" "cl-wl/xdg-decoration" ("wayland" "xdg-shell"))
 ("viewporter" "cl-wl/viewporter" ("wayland"))
 ("fractional-scale" "cl-wl/fractional-scale" ("wayland"))
 ("tablet" "cl-wl/tablet" ("wayland"))
 ("cursor-shape" "cl-wl/cursor-shape" ("wayland" "tablet"))
 ("text-input" "cl-wl/text-input" ("wayland"))
 ("pointer-constraints" "cl-wl/pointer-constraints" ("wayland"))
 ("relative-pointer" "cl-wl/relative-pointer" ("wayland"))
 ("xdg-dialog" "cl-wl/xdg-dialog" ("wayland" "xdg-shell")))

• (bindings-text "viewporter") returns the text generated from protocols/viewporter.xml. It reads the protocols listed before the one asked for too, since it may refer to them.
• (regenerate-bindings) rewrites every protocols/NAME.lisp and returns the pathnames written.

The test committed-bindings checks that every committed file matches what the scanner generates. If you change the generator or an XML file, regenerate, review the diff, and commit the result together with the change.

Array element types

The protocol format gives an array no element type. The prose describes the contents. (shipped-array-elements) returns the transcription of those descriptions, so that the bindings can decode the arrays:

((("wl_keyboard" "enter" "keys") :uint)
 (("xdg_toplevel" "configure" "states") :uint "state")
 (("xdg_toplevel" "wm_capabilities" "capabilities") :uint "wm_capabilities"))

Each entry is ((interface message arg) element &optional enum). element is :int or :uint. enum names the enum the values come from, in the same interface or qualified. An array with an enum is delivered as a list of keywords, one with only an element type as a (vector (unsigned-byte 32)) or (signed-byte 32) vector, and any other array as octets. An entry naming an argument that is not an array signals protocol-error.

Adding a protocol

To add, for example, primary-selection-unstable-v1, as pointer-constraints and the other protocols after the first five were added:

1. Vendor the XML. Copy it from a tagged wayland-protocols release into protocols/primary-selection.xml. Note the release in the README's "Protocol files" section.
2. List it in shipped-protocols (scanner/bindings.lisp) after the protocols it refers to:

   ("primary-selection" "cl-wl/primary-selection" ("wayland"))
   

3. Describe its arrays, if any of its array arguments have contents the prose specifies, in shipped-array-elements.
4. Generate:

   (asdf:load-system :cl-wl/scanner)
   (cl-wl/scanner:regenerate-bindings)
   

5. Add a system to cl-wl.asd:

   (defsystem "cl-wl/primary-selection"
     :description "cl-wl bindings for the primary-selection-unstable-v1 protocol: …"
     :author "dxtr <dxtr@fakehacker.cc>"
     :license "MIT"
     :depends-on ("cl-wl")
     ;; Generated by cl-wl/scanner from protocols/primary-selection.xml.
     :components ((:file "protocols/primary-selection")))
   

   and add it to cl-wl/test's :depends-on, so that committed-bindings covers it.
6. Test the parts with behaviour of their own: decoded arrays, enum arguments, objects created by events, and destructor events. Use the test compositor.
7. Run the suite with stift test.

The generated forms

A generated file is a package definition followed by three sections: enums, then interfaces, then requests. That order guarantees that every type a request's ftype names is already defined.

(define-enum wl-seat-capability ("wl_seat" "capability" :bitfield t)
  "seat capability bitmask …"
  (:pointer 1)
  (:keyboard 2)
  (:touch 4))

(define-interface wl-pointer ("wl_pointer" 11)
  "pointer input device … Events, each passed to its EVENT-HANDLER …"
  (:enter 0 "enter"
   (serial :uint)
   (surface :object :interface wl-surface)
   (surface-x :fixed)
   (surface-y :fixed))
  …)

(define-interface wl-callback ("wl_callback" 1)
  "callback object … (:done CALLBACK-DATA) -- done event (destroys the proxy after its handler) …"
  (:done 0 "done" :destructor t
   (callback-data :uint)))

(define-request wl-registry-bind (wl-registry "bind" 0)
  "bind an object to the display …"
  (name :uint)
  (id :new-id))

An event's options come between its wire name and its arguments; the one option is :destructor t. Request options are :since (the version that introduced it) and :destructor. Argument options are :interface, :allow-null, :enum and :element. The macros are documented in the API Reference, and what they expand into is described in Internals.