Vendoring

stift vendor copies the build list into the project, so that a clone of it builds without the network and without the repositories it requires: should one of them disappear, or a force-push leave a pinned commit on no branch, the project still has every file it needs.

stift vendor
git add vendor
git commit -m "Vendor the build list"

What stift vendor does

Each project of the build list goes into vendor/HOST/PATH/, such as vendor/gitlab.common-lisp.net/alexandria/alexandria/: the files of its commit, checked out by git as stift checks out its cache's commits, with their bytes and their modes. A repository on the local machine, named by a file:// URL, goes into vendor/file/PATH/. A project that a replacement (URL :local DIR) takes from a directory stays where it is.

vendor/projects records the copies: each project's URL, commit, the ID of the commit's tree, its directory, and its ref, test mark and replacement as the build list has them. It also holds a hash of the stiftfile's requirements and replacements, by which stift knows whether the copies still match the stiftfile:

;;; -*- mode: lisp-data -*-
;;; Written by stift vendor: the copies of the build list in vendor/.
;;; Do not edit.  stift vendor writes it again, and stift verify checks
;;; the copies against their commits.
(:stiftfile "9b25b5a63cb60b73d69089650a2cc0414b7b3dab"
 :projects (("gitlab.common-lisp.net/alexandria/alexandria"
             :url "https://gitlab.common-lisp.net/alexandria/alexandria.git"
             :commit "f283e25efbe9f4323419a138b22ad2fc027f6641"
             :tree "2752b9557d3d00235ecbd3ef158775b4626ccc24"
             :directory "vendor/gitlab.common-lisp.net/alexandria/alexandria/"
             :branch "master")))

vendor/.gitattributes and vendor/.gitignore, which stift vendor writes too, keep the copies whole in a commit of the project. The first holds * -text, so that git changes no line ending in a copy, whatever core.autocrlf says: Git for Windows would otherwise check text out with CR LF. The second holds !*, so that the project's own .gitignore leaves no file of a copy out, such as one under a bin/ it ignores. A plain git add vendor then commits the copies as made.

stift vendor takes the build list from the stiftfile's pins, whatever workspace the project is in, as tidy does, and fetches what the cache lacks. Run it again whenever the stiftfile changes: it copies each project of the new build list afresh, and removes the copies of projects that left it, with the directories they leave empty. Two projects whose copies would lie one inside the other, such as example.org/a and example.org/a/b, cannot be vendored together.

Building from the copies

While vendor/projects exists, and no workspace is in use, stift builds from the copies. exec, test, build, check, setup, list and download need neither the network nor the cache's repositories, so a fresh clone builds with an empty cache and --offline. stift list marks each copy vendored:

example.org/me/hello                          main project
gitlab.common-lisp.net/alexandria/alexandria  f283e25efbe9  branch master  vendored

why and graph still read the stiftfiles of the projects required from the cache, and get, update, remove and tidy work on the stiftfile as ever.

Before it builds, stift checks that the copies were made for the stiftfile as it is. When its requirements or replacements have changed since stift vendor ran, stift stops with a vendor-mismatch, which asks for stift vendor; comments and layout do not count. A copy that is missing stops it too.

In a workspace, the members' checkouts and the cache count, as without vendoring, and vendor/ does not; --no-workspace builds from the copies.

What vendor/ holds

The copies belong to stift. A project's own code, such as a library patched by hand, goes elsewhere in its tree and is named by a replacement (URL :local "third-party/foo/"), or lives in a repository of its own, named with :with (see Choosing Commits).

The vendor/ of a project with a stiftfile is stift's: its .asd files are not the project's systems, in the main project or in a project required. A project without a stiftfile keeps its vendor/ as its own.

verify

stift verify checks that each copy holds its commit's files and nothing else. It reads the tree recorded for the copy into an index of its own, adds the copy's files over it, ignored ones included, and compares the tree git writes with the one recorded. It prints what differs, project by project, and exits with 1 when anything does:

$ stift verify
The copy of example.org/me/lib, /home/me/hello/vendor/example.org/me/lib/, is not the files of commit 1a2b3c4d5e6f:
  modified  lib.lisp
  added     notes.txt
  deleted   src/more.lisp
The copies in vendor/ belong to stift: run stift vendor to copy the commit again, and put changes of your own in a replacement (URL :local DIR) instead.

or Every copy holds the files of its commit. with the exit status 0.

The recorded tree comes from the cache's repository of the project, or, once vendor/ is committed as stift made it, from the project's own repository, so that a clone verifies with an empty cache and offline. When neither holds it, stift fetches the commit; offline, it says it could not check the copy, and exits with 1.

File modes count: a script that lost its executable bit is a change. On Windows, where files have no executable bit, a copy keeps the modes of its commit, as git keeps them in a checkout there.

Without vendor/projects, or in a workspace, stift verify checks the cache's checkouts of the build list instead, which stift never changes once made. One that differs from its commit was changed by hand; delete it, and stift checks the commit out again.

With --remote, stift verify also checks over the network that each recorded tag still points to its commit, and each recorded branch still contains its commit: a moved tag or a force-push shows before stift update stops on it. --remote cannot run with --offline.

From Lisp

vendor-projects does what stift vendor does, and returns the build list from the copies. verify-projects does what stift verify does, with :remote, and returns a discrepancy for each project found wrong: its kind, :edited, :unchecked or :ref; the project; the directory checked; the files that differ, as (CHANGE PATH) lists; and the message the command line prints.

(stift:vendor-projects :directory "hello/")
(mapcar #'stift:discrepancy-files (stift:verify-projects :directory "hello/"))
;; => (((:modified "lib.lisp") (:added "notes.txt")))

Functions that write a project's vendor/, or its stiftfile, must not run at the same time for that project.