ovdb(5) InterNetNews Documentation ovdb(5)NAMEovdb - Overview storage method for INN
DESCRIPTION
Ovdb is a storage method that uses the BerkeleyDB library
to store overview data. It requires version 2.6.x or
later of the BerkeleyDB library; it is known to work well
with version 2.7.7. There is also a 3.0.55, but it is not
very stable (though ovdb will compile correctly with it).
Check the web page http://www.avalon.net/~hakehoe/inn/ for
the latest on BerkeleyDB compatibility.
Ovdb makes use of the full transaction/logging/locking
functionality of the BerkeleyDB environment. BerkeleyDB
may be downloaded from http://www.sleepycat.com .
The BerkeleyDB distribution is (by default) installed into
/usr/local/BerkeleyDB, with subdirectories lib/, include/,
and bin/ for the library, headers, and support tools,
respectively. Version 3.x uses /usr/local/BerkeleyDB.3.x
instead. Ovdb uses some of the support tools in the bin/
subdirectory; namely db_archive, db_checkpoint, and
db_deadlock.
INSTALLATION
To build ovdb support into INN, specify the option
`--with-berkeleydb' when running the configure script. By
default, configure will search for a BerkeleyDB tree in
several likely locations, and choose the highest version
(based on the name of the directory, e.g., BerkeleyDB.3.0)
that it finds. There will be a message in the configure
output indicating the chosen pathname.
You can override this pathname by adding a path to the
option, e.g., `--with-berkeleydb=/usr/BerkeleyDB.3.1'.
This directory is expected to have subdirectories include,
lib, and bin containing db.h, the library itself, and the
db_* utilities, respectively.
The ovdb database will take up more disk space for a given
spool than the other overview methods. Plan on needing at
least 1.1 KB for every article in your spool (not counting
crossposts). So, if you have 5 million articles, you'll
need at least 5.5 GB of disk space for ovdb. With Berke
leyDB 2.x, the db files are 'grow only'; the library will
not shrink them, even if data is removed. So, reserving
extra space above the estimate is a good idea. Plus,
you'll need additional space for transaction logs: at
least 40 MB, preferably 100 MB or more. (The logs may be
on a different filesystem -- see the DB_CONFIG section.)
CONFIGURATION
To enable ovdb, set the ovmethod parameter in inn.conf to
`ovdb'. The ovdb database is stored in the directory
specified by the pathoverview paramter in inn.conf. This
is the "DB_HOME" directory. To start out, this directory
should be empty (other than an optional DB_CONFIG file,
see the DB_CONFIG section for details) and innd (or make
history) will create the files as necessary in that direc
tory.
Other parameters for configuring ovdb are in the optional
ovdb.conf configuration file. See also the sample
ovdb.conf.
numdbfiles
Overview data is split between this many files. Cur
rently, innd will keep all of the files open, so don't
set this too high or innd may run out of file descrip
tors. The nnrpds only open one at a time, regardless.
May be set to one, or just a few, but only do that if
your OS supports large (>2G) files. Warning: do NOT
change this on an already established database.
Default is 32.
txn_nosync
If txn_nosync is set to false, BerkeleyDB flushes the
log after every transaction. This minimizes the num
ber of transactions that may be lost in the event of a
crash, but results in significantly degraded perfor
mance. Default is true.
cachesize
Size of the memory pool cache, in Kilobytes. The
cache will have a backing store file in the DB direc
tory which will be at least as big. In general, the
bigger the cache, the better. Use `db_stat -m' to see
cache hit percentages. If they're less than 80%, try
increasing the cache size. To make a change of this
parameter take effect with an existing database,
remove the __db* files while there are no database
processes running. Default is 8000 K, which may be
adequate for small to medium-sized servers. Full-feed
servers will probably need at least 14000.
pagesize
Sets the page size for the DB files (in bytes). Must
be a power of 2. Best choices are 4096, 8192, or
16384. The default is 8192. You can not change the
pagesize of an existing DB, it must be recreated.
minkey
Sets the minimum number of keys per page. See the
BerkeleyDB documentation for more info. Default is
based on page size:
default_minkey = pagesize / 2048
You can not change the minkey of an existing DB, it
must be recreated.
DB_CONFIG
A file called DB_CONFIG may be placed in the database
directory to customize where the various database files
and transaction logs are written. By default, all of the
files are written in the "DB_HOME" directory. One way to
improve performance is to put the transaction logs on a
different disk. To do this, put:
DB_LOG_DIR /path/to/logs
in the DB_CONFIG file. If the pathname you give starts
with a /, it is treated as an absolute path; otherwise, it
is relative to the "DB_HOME" directory. Make sure that
any directories you specify exist and have proper owner
ship/mode before starting INN, because they won't be cre
ated automatically. Also, don't change the DB_CONFIG file
while anything that uses ovdb is running.
The DB_CONFIG functionality is part of BerkeleyDB itself.
RUNNING
You don't need to do anything special when starting or
stopping INN when using ovdb, as long as you use rc.news.
When starting, rc.news:
Runs `ovdb_recover'. This will perform any needed
recovery on the database; e.g., if there was a crash
that may have left the database in an inconsistent
state. This is done before innd is started. If the
innd.pid file is present but innd is not running
(i.e., unclean shutdown), rc.news will supply the `-f'
argument to `ovdb_recover'
Runs `dbprocs start'. This starts the DB housekeeping
processes.
And when stopping INN, rc.news calls `dbprocs stop' after
the other INN processes have been shut down.
DIAGNOSTICS
Problems relating to ovdb are logged to news.err with
"OVDB" in the error message.
If a program accessing the database crashes, or otherwise
exits uncleanly, it might leave a stale lock in the
database. This lock could cause other processes to dead
lock on that stale lock. To fix this, shut down all news
processes (using `kill -9' if necessary) then run
`ovdb_recover -f' as the news user, e.g.: `su news -c
"/PATHBIN/ovdb_recover -f"' This will remove all locks
plus repair any damage caused by killing deadlocked pro
cesses.
FILES
inn.conf
The ovmethod and pathoverview parameters are relevant
to ovdb.
ovdb.conf
Optional configuration file for tuning. See CONFIGU
RATION section above.
pathoverview
Directory where the database goes. BerkeleyDB calls
it the 'DB_HOME' directory.
pathoverview/DB_CONFIG
Optional file to configure the layout of the database
files.
TO DO
I've modified innshellvars and innshellvars.pl to add the
BerkeleyDB bin dir to PATH and set DB_HOME to
innconf->pathoverview. Since I don't know TCL, someone
needs to modify innshellvars.tcl in the same way.
Implement a way to limit how many databases can be open at
once (to reduce file descriptor usage); maybe using some
thing similar to the cache code in ov3.c
HISTORY
Written by Heath Kehoe <hakehoe@avalon.net> for
InterNetNews
SEE ALSOrc.news(8), inn.conf(5), dbprocs(8), ovdb_recover(8),
ovdb_upgrade(8)
BerkeleyDB documentation, in the docs directory of the
BerkeleyDB source distribution, or on the Sleepycat web
page: http://www.sleepycat.com/
2000-06-14 INN 2.3 ovdb(5)
