"Supercomputers" etc...

Why the Karoo Project is needed

Great services such as google and Gmail and many research institutions use concepts such as clustering, beowulf, grid computing, server farms, etc. for true scalability.

These techniques are not simple to begin to use, though there are a number of tools already existing for creating a supercomputer, such as:

• Linux Cluster HOWTO
• PVM (Parallel Virtual Machine)
• MPI - Message Passing Interface
• BCCD - Bootable Cluster CD
• OSCAR Clustering Software - methods for building, programming, and using clusters.
• ROCKS Clustering Software
• Scyld Beowulf Cluster Operating System

... So if all this is already available, why another project?

Whether I use these tools or write my own, I still run into issues with: multiprocessing, multithreading, memory coherency, cache coherency, synchronisation, application checkpointing. When creating great services, we will need to use threads, semaphores (or mutexes and conditions), OS process tasks, and inter process communication. Somehow, we must avoid deadlocks, race conditions, and bottlenecks.

A messaging library is not enough, and a parallel virtual machine simply makes the computer bigger... you still have all the problems mentioned above.

I wanted a messaging system, a professional queue system, a multithreading library, and a distribution architecture and management system.

But... I'm not thinking so much of scientific applications, or huge calculations... I'm thinking of an application framework for large scalable database applications.

See The Karoo Project.

Top 500 super computer sites
Supercomputer news.
Vector and Scalar Parallel Supercomputers