ALPHABET-ABC(1)ALPHABET-ABC(1)NAME
abc - alphabet declarations
SYNOPSIS
load alphabet
typeset /abc
type /abc
DESCRIPTION
Grid is a typeset for alphabet (see sh-alphabet(1)) which enables
allows direct interconnection of remote and local processing compo‐
nents. It defines one new type, endpoint, which represents a place in
the network to which two parties can connect and exchange data.
In the following descriptions, if a type is not endpoint or a type
defined in the root typeset (see alphabet-main(1)), it is assumed to be
of type /string.
Modules currently provided within the /grid typeset include:
farm [-lnkavA] endpoint addr tasktype [arg...] -> endpoint
Farm connects to a grid labour exchange (see scheduler(4)) at
addr, starts a new job of type workflow, and passes all the
data read from endpoint to be processed by the currently
available labour. The data is split into records, each one
of which will be processed on a worker node by tasktype, with
its associated arguments. Other than -A, which specifies
unauthenticated access to the scheduler, the various options
are all passed verbatim to workflow: -l causes it to split
its input on newline-separated records; -n specifies that no
record separation is necessary on output; -k specifies that
intermediate data for failed tasks should be kept around; -a
specifies that intermediate data for all tasks should be kept
around, and -v specifies that workflow should produce a wordy
description of what it is doing.
local endpoint -> /fd
Local reads everything from endpoint, and writes it to fd.
remote [-a addr] fd -> endpoint
Remote is the inverse of local: it reads data from fd and
writes it to a newly created endpoint. If -a is given, addr
specifies the network address of an endpoint server on which
to create the new endpoint.
rexec [-A] endpoint addr cmd -> endpoint
Rexec connects to a remote execution server at addr (unau‐
thenticated if -A is specified), and arranges to execute the
alphabet expression cmd there. The expression should be com‐
patible with usage fd -> fd. Data from the argument endpoint
will be piped through this expression, and made available as
the resulting endpoint endpoint.
EXAMPLES
The examples below that a local endpoint is available, and the follow‐
ing alphabet declarations:
load alphabet
typeset /grid
type /string /endpoint /fd
import /grid/local /grid/remote
autoconvert fd endpoint remote
Set up a rendering pipeline:
-{/read /tmp/somedata |
remote |
rexec tcp!node1!rexec "{(/fd); /filter $1 "{os render_stage1}} |
rexec tcp!node2!rexec "{(/fd); /filter $1 "{os render_stage2}} |
/create /tmp/somedata.result
}
SOURCE
/appl/alphabet/grid.b, /appl/alphabet/gridtypes.b
/appl/cmd/grid/*.b
SEE ALSOsh-alphabet(1), alphabet-main(1), sh(1)ALPHABET-ABC(1)