Berkeley DB

Berkeley DB is a Java based non-relational database that is scalable and offers multiple thread and process support.  This is ideal for use in the Solar Decathlon home management system, now renamed to iHale.  The main idea is shown in the diagram below:

Client-server-database communication

On the client side, Wicket provides the front end user interface via a webpage.  Through this webpage, a user may request to perform a GET, PUT, or DELETE operation.  The client communicates to the server via HTTP over the Restlet framework, where the request is interpreted and executed by the server.  Information in the database is persisted through a file saved on the disk.

To get familiar with Berkeley DB, I worked on two code katas that dealt with the storage and retrieval of contacts.  These were modified versions of the katas I completed on Restlet in the last two blog posts.  More details can be seen here.

Kata 1: Timestamp as a secondary index

The primary index for a contact is their unique ID.  This means that entries can be added or searched in the database via a unique ID.  The task was to add a timestamp as a secondary index.  The timestamp is appended to a contact when it is created.  The most time-consuming part of Kata 1 was writing unit tests and learning Berkeley DB's API to query the database.  It was fairly straightforward and took about 3 hours to complete.

Command line usage of all the available operators: put, get, get-timestamp, get-range.

Kata 2: Wicket, REST, and Berkeley DB: A match made in heaven

This kata is a combination of all the technologies I have learned so far which includes Wicket, REST, and Berkeley DB.  It combines the use of Wicket in Restlet Part I, concepts of persistency from Restlet Part II, and Berkeley DB from Kata 1 above.  The difference is that contact information is now persisted in a file on the disk via Berkeley DB instead of in-memory.

Webpage for adding entries and querying the database.

The overall layout of Wicket, REST, and Berkeley DB is readily understood, but getting it all working properly together is a whole different story.  So far, this kata has been a debug-fest.  It took about 5 hours to code the core Wicket, resource server, database, and ant build files, but debugging has taken over 7 hours.

Some of the problems I encountered were trivial, such as forgetting to attach the timestamp to a contact XML document.  These resulted in exceptions when trying to get or put a contact into the database.  Other problems were due to communication errors between Wicket and the resource server, which resulted in odd exceptions such as Internal resource error 500.  I was able to resolve these by desk checking my code several times.

The lessons that I learned from this exercise is that one must be extra careful when copying and pasting code from working modules in past projects.  This method is extremely error-prone.  Also, it is ideal to import small portions of code at a time and perform frequent testing instead of importing huge blocks of code all at once and testing everything afterwards.

The final distribution of my katas can be downloaded here.