Karoo project Surf setup

Web service setup

The Surf application is a web server. It can be set up as a plain vanilla web server just serving up web pages, or serving up dynamically generated content and as the middle-ware server component in a client/server system using the Karoo system messaging.

Plain Vanilla Web Server setup

To do this, create an XML file (using the "fab" syntax) which describes the location of the HTML documents, e.g.

 <?xml version="1.0" encoding="utf-8"?>
 <surf xmlns="http://www.zwartberg.com/fab/1.0">
   <host name="www.yourhost.com">
     <mime-type ext='html' type='text/html'/>
     <mime-type ext='gif' type='image/gif'/>
     <mime-type ext='png' type='image/png'/>
     <mime-type ext='jpg' type='image/jpeg'/>
     <mime-type ext='js' type='text/javascript'/>
     <mime-type ext='css' type='text/css'/>
     <mime-type ext='xsl' type='text/xsl'/>
     <document-root path="/your/directory/for/html/documents"/>
     <default-document path="/index.html"/>
     <rock name="my-cave"/>
     <rock type="surf"/>
   </host>
 </surf>

The example above first describes the mime types, and then the location of the root of your web site, and also the default file that should be served up if there is no file name in the URL.

Note that the rocks must also be specified. These are the rocks that its legal for any incoming http request to this host. The rock can be specified by name and/or type, depending on how its accessed via the http requests. Obviously the surf rock type must be able to be accessed.

See the dtd file http://www.zwartberg.com/fab/1.0/surf.dtd for a completed description of the syntax.

Then to start the web server, you would start up the surf application and tell it the URL of this XML file. E.g.

 /usr/local/bin/surf --url file:///usr/local/etc/surf.xml

Javascript Client Interface

If you are here, there is a chance you are interested in the Javascript interface:

Javascript interface

Dynamic Content

The surf application can be used to generate content from normal HTML. It has a "conduit" system where you can set up any HTML tag to be cloned multiple times with dynamic content, each clone corresponding to a database retrieval. E.g.

 <p id="one">
   <!-- conduit rock-name:my-cave service-id:get-comments; -->
   <!-- column comment >comment goes here<!-- end -->
   &nbsp;
   <!-- column date >2009-11-01
 </p>

If you view this HTML in a web browser it will show this:

 comment goes here   2009-11-01

(Nothing in the above example breaks the HTML standard.)

... but if you serve it up from surf, it will replace this paragraph with zero to many paragraphs, each dynamically generated from data in the database. The SQL query will be defined in a service called "get-comments" in a cave rock named "my-cave". The SQL query will return at least these two columns: "comment" and "date", which will be used in the dynamically generated HTML. Following is an example of the configuration of the "my-cave" rock:

 <?xml version="1.0" encoding="utf-8"?>
 <cave xmlns="http://www.zwartberg.com/fab/1.0" poll-delay="15">
   <database name="KarooDB" username="bmodra" password="" connect_timeout="1" connection-pool-size="3"/>
   <service name="comments" id="get-comments">
     <sql>
       select comment,date from comments where name=
         <parameter name="username"/>;
     </sql>
   </service>
 </cave>

This also requires that the "username" parameter is passed to the surf application.

Passing of parameters is achieved by sending them in the URL. The format of the URL is:

 http://host.name.something/fab/myhtmlfile/by_username/bmodra.html

... where

the "/fab/" is required,
"myhtmlfile" is the basename of the file you wrote (containing the above example HTML) e.g. if the file name is "example.html", then you'd use "example" in this part of the URL.
"/by_" is required
"username" is the name of the parameter
"bmodra" is the value being passed as the username
".html" is the file extension.

You can pass more parameter names and values. Just delimit them with underscores. e.g.

 http://host.name.something/fab/myhtmlfile/by_username_fromdate/bmodra_2009-11-01.html

You may ask why I did not just use the URL encoded parameters (using ?& etc). Good question, and these will be implemented. The reason (so far) they are not implemented is that I wanted to get the hard part finished first, so that plain looking URLs work (better for search engines.)

Conduit Specification

The conduit is the mechanism used to get data from the cave into the surf. By coincidence this sounds like something pirates will be doing: getting their treasures from the cave where they stashed them, via a conduit cut into the rock, and out into the surf where their ship is waiting.

The cave application is the database interface. It exports an API which is used by the surf object to implement the conduit. The conduit configuration specifies the service, and the surf application will send a request via the conduit to the service, which will return rows of data organised into columns.

These rows are then used to replace parts of the web page and replicate it.

The conduit is specified via a HTML comment. This comment must be the first child element in the XML of the element that is to be replicated. E.g. if it's a table row that will be replicated (a TR element), then the XML will look something like this:

 <tr id="property-row">
   <!-- conduit rock-name:ke-cave; service-id:list-properties; -->
   ...

The ID attribute in the TR element is required. It must be specified. The conduit becomes attached to that element, using the element's ID as a key in an index in the implementation code. The element (in this case a TR) can be any element. It can't be a comment, text, or IP.

The components of the conduit (HTML comment) are:

the text "conduit" - which must be the first word, followed by a space
the specification of the cave, "rock-name;", followed by (no spaces) the name of the cave rock, followed by (no space) a semicolon, then a space.
the specification of the service, "service-id:", followed by (no space), the ID of the service. followed by (no space), a semicolon.

Note again, that the conduit (HTML comment) must be the first child XML element of the element that will be replicated. In the example above, the TR element with ID "property-row", including all its child elements, will be cloned and replicated and modified, for as many rows as are returned from the service.

Once you have specified the conduit, you can add the column and condition specifications to children of the element that is associated with the conduit.

The column and condition specifications are also HTML comments.

Column Specification

The column specification is made up of:

the text "column" - which must be the first word, followed by a space
the column name, followed by a space
optionally, the name of the attribute to replace, a colon (no spaces), followed by the replace string.

Note that the column specification has two modes: with the (optional) attribute and replace string, and without it. If the attribute and replace string is not specified, then the surf will replace all the text following the comment with exactly the column data returned via the conduit. It will replace up until the next element or comment. E.g.

 <!-- column price -->100.00<!-- end -->&nbsp;USD

This will replace the "100.00" with whatever the price column contained. Similarly:

 <!-- column price -->100.00<span>&nbsp;USD</span>

... will also replace just the "100.00" with the contents of the price column.

On the other hand, if the field and replace string are specified, then it will not replace text, but will rather replace that attribute. E.g.

 <!-- column id src:/images/%%.jpg --><img src="/images/2.jpg" alt="photo"/>

This will replace the SRC attribute of the IMG element with an image named by the ID column retrieved from the database. There is nothing complicated about the replacement string. The contents of the column will be inserted in place of the string "%%". Its also possible to put a "%" into the replacement string, by using 3 percent characters: "%%%". If the replacement string contains an environment variable name between two percent characters, e.g. "%cave-address%" then it will replace that with the value of the environment variable. (Environment variables can be specified in the surf configuration XML file, e.g.

 <?xml version="1.0" encoding="utf-8"?>
 <surf xmlns="http://www.zwartberg.com/fab/1.0">
   <mime-type ext='html' type='text/html'/>
   etc...
   <env name="cave-address" value="192.168.1.233:8080"/>
 </surf>

(This particular environment variable comes in handy to specify the base address of the web server that the cave rock can provide... surf is a web server, and cave also can be a - quite limited - web server. This is useful when storing images in a database. Its more convenient just to stream them directly via HTTP, because they are typically quite large. More about this in the cave documentation: Karoo project Cave setup )

Condition Specification

The condition specification is made up of:

the text "condition" - which must be the first word, followed by a space
the column name

Then the element that immediately follows it (and all its children) will be conditionally written, or removed, based on the value of the column. If the value of the column is true, or non zero, then it will output, otherwise not.

E.g.

 <!-- condition bedrooms -->
   <div>
     <img src="/images/icon_bedroom.png" alt="bedroom"/>
     <p>bedrooms: <!-- column bedrooms -->3</p>
   </div>

In the above example, the DIV and its children (the image and the paragraph) will be conditionally output only if the number of bedrooms in the database is non zero.

Surf Document Root

The Surf application implements two document roots: one that is accessed via a URL starting with "/" and another starting with "/surf/". The configuration file for the surf app. can be set up to specify both these roots, e.g.

 <?xml version="1.0" encoding="utf-8"?>
 <surf xmlns="http://www.zwartberg.com/fab/1.0">
   ...etc
   <document-root path="/usr/local/share/httpdocs"/>
   <surf-root path="/usr/local/share/surf-httpdocs/"/>
   <default-document path="/index.html"/>
   ... etc
 </surf>

In the above example, it specifies the surf root as: "/usr/local/share/surf-httpdocs/", and the main document root as: "/usr/local/share/httpdocs". The surf root must contain the files from the karoo/trunc/src/surf-httpdocs/ directory, which by default are installed into /usr/local/share/surf-httpdocs/.

This directory contains the cave.js and client.js files, as well as a number of other generally useful files, such as the karoo.css file.