Karoo project runtime

Runtime Architecture

Introduction to the runtime architecture

There are probably two main reasons why people don't create applications from more than one OS process task:

• it's difficult to manage all the makefiles
• it's difficult to keep all the processes running and communicating

This section of documentation deals with how to keep all the processes running and communicating.

Mechanism used to keep things running

The Karoo Project uses a manager program to start the individual rocks. (A rock is an executable program ... which when run becomes an OS process task in the Karoo Project.)

It does not use a script to manage them, though this would be tempting, along the lines of the rc scripts.

Rather, it uses a manager program that start the rocks using execv().

It would be possible to use fork to achieve much the same thing, more easily (no need for separate programs and all those makefiles), but this would not work across hosts. It would be possible to use a combination: separated programs for the other hosts, and fork()ing inside of each host, but that limits scalability.

The general architecture is that the rocks are started by the manager (or by a human for that matter). When they start, they start communicating with the manager to keep it aware of their status. If the status datagram is not received after a specific elapsed time, then the manager will assume the rock is dead, and will start another rock of the same type. Then after a further elapsed time, will be sure it has either died or locked up, and will send a SIG_KILL to be sure. The manager is also able to instruct a rock to stop tidily. Thus it is able to manage the total number of rocks of each type according to load.

Communication in the runtime architecture

In a cluster, all the managers communicate with each other, band the rocks communicate with each other and their manager.

As all the rocks communicate directly with each other, then each of them must have a different port number. The rock library contains simple mechanisms for scheduling a message to be sent. it will automatically handle making sure the datagram is delivered.

Each rock has a textual name and a textual type. The type is the same as the class name of the class that inherits from rock. The name has a one_or_many-to-one relationship to the type. This can be described in XML, e.g. if a rock's name is "kopje" and the rock's class is "myrock", then we could have:

 <karoo:myrock name="kopje"/>
 <karoo:myrock name="hill"/>
 <karoo:myrock name="myproject.section.subsection.something"/>

Note that this XML representation is just a way of referring to the rocks. In the above example, there are three instances of the myrock class. Three executables. The name is assigned from the command line during startup. E.g.

 myrock --name kopje