SYS-TOKENIZE(2)SYS-TOKENIZE(2)NAME
tokenize - split string into words
SYNOPSIS
include "sys.m";
sys := load Sys Sys->PATH;
tokenize: fn(s, delim: string): (int, list of string);
DESCRIPTION
Tokenize breaks s into words separated by characters in delim. The
returned tuple holds the number of words and an ordered list of those
words (whose hd gives the leftmost word from s).
Words are delimited by the maximal sequences of any character from the
delim string. Tokenize skips delimiter characters at the beginning and
end of s, so each element in the returned list has non-zero length.
If s is nil or contains no words, tokenize returns a count of zero and
a nil list.
Delim may be nil or the empty string, specifying no delimiter characā
ters. The resulting word list will be nil (if s is nil or the empty
string) or a single-item list with a copy of s.
SEE ALSOsys-intro(2)SYS-TOKENIZE(2)