The Karoo Project Documentation

The Karoo Project basics

When you create something with The Karoo Project, that something is called a "service".

In a service, there may be many programs, which are each a "rock".

In a rock, there may be many threads, each implemented as a "queue", and a task runnable in a queue is a "pebble".

There are C++ class structures for each of these things (except for the service, more about this below).

The rock class has convenience methods for getting command line parameters. It also contains all the support methods for sending messages and for setting up handlers for incoming messages.

The queues provided are: "exeque" and "exepool". Their names are self descriptive - a single queue for executing pebbles in, and a pool of queues for executing pebbles in. In the first case, everything is executed sequentially. In the second case, things can execute concurrently, although there may be some latency.

A pebble is just a convenient class that you can inherit from for anything that must be run in a queue. The pebble maintains a pointer to the exeque that it is currently running in. So you can add it back in again, if the pebble must loop. There are methods for adding pebbles immediately, or after a time delay.

To reduce resource usage, mechanisms like sleep() and wait() are strongly discouraged. Basically, every component in a pebble should execute quickly. If it needs to wait for something, then it can be added to a queue with a delayed start.

Services

A service is a collection of rocks. As these are each distinct programs, they need to be started, and something needs to keep a watch over them to see if any exit or die, and restart them.

There is a "manager" program, which constantly checks an XML file for changes, and keeps the collection of rocks running as specified. E.g. if you edit this file and change one of the command line parameters for a rock, then it will stop the one rock (cleanly, by first sending it a "finish " message, and waiting a (specified) number of seconds for it to exit on its own. If not, it then sends it a KILL signal). Then it will restart the rock with the correct parameters.

This XML file is called a "fabric" file. ("Fabric" as in the way it fits together.) It defines the programs that are the rocks and their command line parameters. It also defines the hierachy of rocks.

The manager only manages rocks on one host. Each host must run its own manager. Its easy for a rock to communicate with its parent or any of its children, but it will need to know the specific address (IP and port) of a rock if its outside its hierachy. The rocks that are running on another host can be specified in the fab file. These won't be managed by the localhost manager, but they will form part of the hierachy. Ideally, there will be one fab file for an entire service, and it will be sourced by all the managers on all the hosts.

User Interface

Though the use of streams is discouraged (heavy on resources), stream support is an itegral part of The Karoo Project (though messaging is done by UDP), and there is a "cloud computing" style of user interface included as part of The Karoo Project.

This is simply a mechanism for sending and receiving messages via an http proxy (a rock called "surf"). In the browser end there is a library of Javascript objects and functions that is used for sending messages via http, and retrieving replies (by polling the surf).

The surf is a web server as well as proxy, so it can provide the html files to the client browser. There is no need to involve Apache.

Zwartberg Reseach & Development is a registered trading name of Open Source Software Consulting CC.
Phone:+27235411462, fax:+27235411379, mobile:+27796977082,
brian@zwartberg.com, P.O. Box 2, Prince Albert, 6930, South Africa.