[Oz] Using Oz for large databases

[Peter Van Roy]

> 
> Is it a good idea to use Oz to build systems that manage large amounts
> of fairly passive data?  An example from retailing would be
> transactional data on sales; each time the cash register rings there is
> a record that someone (possibly known through a credit card number)
> bought an item which has a variety of attributes including its price.
> Over time this adds up to hundreds of megabytes or possibly many
> gigabytes depending on the size of the store chain.
> 
> The example is not artificial.  I am seeking a better way to manage such
> data than by storing it in yet another relational database.  Assuming
> one is willing to invest significantly in hardware, for example opening
> up the possibility of gigabytes of RAM and multiple processors, does
> this class of problem become addressable with Oz/ Mozart?
> 
> Please be aware that this question is coming from someone new, not only
> to Oz, but to functional languages in general.  (Apologies to the group
> if I am mislabeling Oz with this sentence :)  In fact, I have to confess
> that I am writing this note concurrent with digging through the Oz
> documentation to see how file access or persistent data is handled at
> all.
> 
> Thanks
> Bill Hunter
> 

The current system has the following ways of handling large amounts of data:

1. In RAM.  The current system has 32-bit addressing, and due to the
   tagging and GC schemes, a program can have a maximum of 512MB of active
   data at any instant.  Within this limit, data structures can be of almost
   arbitrary size.  To verify this, I created just now a dictionary of 1000000
   elements and did some operations with it.  (An Alpha port that does full
   64-bit addressing is in progress.)

2. The Dictionary module manages memory efficiently (e.g., it recovers
   memory when elements are deleted, just like it should) while maintaining
   constant-time access through a Literal key.  I verified this by removing
   all elements from the above dictionary.

3. The Pickle module lets to save and load arbitrary stateless data
   in a file, possibly addressed by a URL.  No encoding or decoding is
   necessary by the program.  Stateless data includes functors, classes,
   and procedures as well as records and numbers.

4. File access is similar to standard Unix, see the Open module.  

5. There is an interface to the GDBM database; see the Contributed Libraries
   manual.

Hope this helps!

Peter Van Roy
-
Please send submissions to users at mozart-oz.org
and administriva mail to users-request at mozart-oz.org.
The Mozart Oz web site is at http://www.mozart-oz.org/.