6 Netfarm server
The Netfarm server is a decentralise2 system, which retrieves a partition of all the objects available to it, runs scripts for them, and makes the objects and computed values available to its connections.
In the introduction to decentralise2, we argued that the terms “client” and “server” do not have enough meaning to be useful when discussing Netfarm software; but we ended up calling this module a “server”.
6.1 Database protocol
6.1.1 Implementations of the database protocol
A diagram of Netfarm server classes, including system and database mixin classes. Shaded classes are instantiable, and not shaded classes are not instantiable. Dashed classes are decentralise2 classes included for completion.
memory-system <: netfarm-systemClass
A system that stores everything in memory. Compare this to decentralise2’s memory-database-mixin.
simple-memory-system <: netfarm-systemClass
A system that stores everything in memory, using the simplest feasible implementations of the database protocol.
6.1.1.1 Filesystem database
A mixin that stores objects in a filesystem.
A .object file that contains the object, and computed values appended to the end of the object,
A .affected-by file that contains a list of object names that this object has been affected by, and
A .affects file that contains a list of object names that this object affects.
A dependency-graph file that stores the dependency graph as an association list,
A interesting-set file that stores the other interesting set,
A unpresentable-set file that stores the inverse of the presentable set (as it would be smaller than the presentable set),
TODO: A fulfilled-dependencies file that stores object names that have had all dependencies fulfilled, but have not been further processed, and
TODO: A script-machines file that stores information about the script machines that need to be run.
The directory that objects should be stored in.
6.1.1.2 Caching
A mixin that caches some objects in memory, but defers to another database implementation when it does not have an object stored.
The maximum rendered size of objects that can be cached at once, in bytes. This defaults to 100 megabytes (10^8 bytes).
:fallback-vague-object-size Initarg
The size that vague objects should be assumed to take when rendered, should the real database not provide a size.
6.1.2 Vague object storage
6.1.2.1 Presentable set storage
6.1.2.2 Side effect storage
objects-affected-by system nameGeneric Function
Return a list of the object hashes affected by a given object.
objects-affecting system nameGeneric Function
Return a list of the object hashes which affect a given object.
objects-affecting-hash system name &key testGeneric Function
Compute a hash value for the objects-affecting list, by
\prod_{a \in \text{affecting}} a \pmod{2^{256}}
If a test function is provided, only object names which satisfy the function will be hashed.
When a test is not provided, hashing can be implemented incrementally, but the default method computes it from the entire objects-affecting list. (Recall multiplication and modular multiplication are commutative - a database could store the last hash, and then compute h := h \times a \pmod{2^{256}} when adding a.)
(We were initially going to reduce logxor over the set, but bitwise exclusive-or, among all additive groups, can be attacked by (Wagner 2002). It is unclear what such an attack could achieve, but we are going to avoid it by using a multiplicative group instead.)
6.1.3 Other interesting set
add-other-interesting-block system nameGeneric Function
6.1.4 Dependency graph
A Netfarm server maintains a dependency graph, consisting of a set of dependency edges, which each store the name of a dependent (a string), the name of a dependency (another string), and a type (a keyword).
For the avoidance of doubt (because we frequently got confused by the names): when a relevant edge exists, storing a dependent object is prevented until a dependency object is retrieved.
All functions that take edge types except for add-dependency-edge also accept the type :all to retrieve all edges, regardless of type.
map-dependents function system type dependencyGeneric Function
map-dependencies function system type dependenciesGeneric Function
A mixin that implements a dependency graph by storing a list of its edges. This mixin may be slower than dependency-table-mixin, but it is easier to read about, as it does not do any tricky indexing.