The end of SQL and relational databases? (part 2 of 3)

In "The end of SQL and relational databases? (part 1 of 3)" I covered some background on the SQL language and relational databases, the current and future for relational databases, the rise of frameworks that hide some of the complexities of database programming and the rise of the NoSQL databases. In this second installment I will take a walk through (some of) the currently available open and closed source NoSQL databases. Then, in part 3, I will point you to NoSQL Internet resources, past/upcoming events and offer some guidance for developers.

It's amazing to see how many open and closed source alternative data stores have appeared. At the same time more are appearing every day. If I have left off one or more of your favorite NoSQL databases post a comment. Below you will find many different types of NoSQL databases: Document-Oriented, Collection-Oriented, Column-Oriented, Object-Oriented, Graph-Oriented, Set-Oriented, Row-Oriented and more.


Company/Org:Franz Inc.
Type: Graph
Description: Modern, high performance, persistent graph database.
Storage: Disk based, meta-data and data triples.
API(s): SPARQL, Prolog


Type: Key/Value
Description: C language embeddable library for enterprise-grade, concurrent, transactional storage services. Thread safe to avoid data corruption or loss
Storage: B-tree, hash table, persistent queue
API(s): C, C++ and Java
Notes: Use BerkleyDB XML layer on top of BerkleyDB for XML based applications. Comparison of BerkleyDB and relational databases


Type: Sparse, distributed, persistent multidimensional sorted map.
Description: Distributed storage system for structured data. Data model provides dynamic control over data layout and format. Data can live in memory or on disk.
Storage: Data is stored as an uninterpreted array of bytes. Client applications can create structured and semi-structured data inside the byte arrays.
API(s): Python, GQL, Sawzall API, REST, various.
Notes: Overview: Bigtable: A Distributed Storage System for Structured Data (PDF format)


Type: Dimensional hash table
Description: Highly scalable distributed database. Combines Dynamo's distributed design and Bigtable's column family data model.
Storage: Clusters of multiple keyspaces. The keyspace is a name space for column families. Columns are comprised of a name, value and timestamp.
API(s): Java, Ruby, perl, Python, C#, Thrift framework.
Notes: Open sourced by Facebook in 2008. Wiki, FAQ, Examples


Type: Document
Description: Distributed database with incremental replication, bi-directional conflict detection and management.
Storage: Ad-hoc and schema-free with a flat address space.
API(s): RESTful JSON API. JavaScript query language.
Notes: CouchDB Introduction, Technical Overview


Type: Object
Description: Java and .NET dual license (commercial and open source) object database.
Storage: Data objects are stored in the way they are defined in the application.
API(s): Java, .NET languages.
Notes: db4o db4o database runtime engine, about db4o


Company/Org:Millstone Creative Works
Type: JSON-based
Description: Schemaless database similar to Amazon's SimpleDB. Open source, standalone Java application server.
Storage: JSON data format, "bags" (similar to tables).
API(s): HTTP and Javascript APIs
Notes: Dovetaildb JavaScript API reference manual


Company/Org:Cliff Moon
Type: Key/Value
Description: Open source Amazon Dynamo clone written in Erlang.
Storage: Distributed key/valve store, Pluggable storage engines.
API(s): Thrift API
Notes: Dynomite Wiki

eXtreme Scale

Type: In-memory grid/cache
Description: Distributed cache processes, partitions, replicates and manages data across servers.
Storage: Data and database cache, "near cache" for local subset of data. Java persistent cache. Map reduce support.
API(s): Java APIs, REST data service
Notes: eXtreme Scale Document library web site


Type: Hierarchical, multi-dimensional sparse arrays, content associative memory
Description: Small footprint, multi-dimensional array with fill support for ACID transactions, optimistic concurrency and software transactional memory.
Storage: Unstructured array of bytes. Can be Key/Value, document oriented, schema-less, dictionary or any other data model.
API(s): Mumps, C/C++, SQL
Notes: GT.M FAQ


Company/Org:Christoph Rupp
Type: Embedded storage library
Description: Lightweight embedded database engine. Supports on disk and in memory databases.
Storage: B+tree with variable length keys.
API(s): C++, Python, .NET and Java
Notes: hamsterdb FAQ, examples, tutorial


Type: Sparse, distributed, persistent multidimensional sorted map.
Description: Open source, distributed, column-oriented, "Bigtable like" store
Storage: Data row has a sortable row key and an arbitrary number of columns, each containing arrays of bytes.
API(s): Java API, Thrift API, RESTful API
Notes: Part of Apache Hadoop project. HBase Wiki, FAQ


Company/Org:Zvents Inc.
Type: Sparse, distributed, persistent multidimensional sorted map.
Description: High performance distributed data storage system designed to run on distributed filesystems (but can run on local filesystems). Modeled after Google Bigtable.
Storage: Row key (primary key), column family, column qualifier, time stamp.
API(s): C++, Thrift API, HQL
Notes: Hypertable Architectural overview, FAQ


Company/Org:JBoss Community
Type: Grid/Cache
Description: Scalable, highly available, peer to peer, data grid platform.
Storage: Key/Value pair with optional expiration lifespan.
API(s): Java, PHP, Python, Ruby, C
Notes: Infinispan FAQ, Wiki


Type: Graph
Description: Internet graph database made up on nodes and edges. Supports in-memory and persistent storage alternatives including RDBMS, file system, file grid, and custom storage.
Storage: Nodes (meshobjects) and edges (relationships). Meshobjects can have entity types, properties and participage in relationships. MeshObjects raise events.
API(s): RESTful web services.
Notes: InfoGrid Overview, FAQ


Type: Key/Value
Description: Distributed (master/slave) key-value data store delivering strong consistency, fault-tolerance and high availability.
Storage: Uses BErkeleyDB library for For local storage. Key/Value pairs and their state are replicated to multiple servers.
API(s): C/C++, Python, PHP, HTTP
Notes: Keyspace Overview, FAQ


Type: Key/Value
Description: High performance, high realiability persistent storage engine for key/value object storage.
Storage: Uses BerkeleyDB as storage library/backend.
API(s): Memcache protocol, C, Python, Java, perl
Notes: MemcacheDB complete guide (PDF format)


Type: Key/Value
Description: Multiuser distributed database including support for replication and dynamic reconfiguration.
Storage: Organized as a set of tables made up of Erlang records. Tables also have properties including type location, persistence, etc.
API(s): Erlang
Notes: Mnesia Reference manual


Type: Document
Description: Scalable, high-performance, open source, schema-free, document-oriented database
Storage: JSON-like data schemas, Dynamic queries, Indexing, replication, MapReduc
API(s): C,C++, Java, JavaScript, perl, PHP, Python, Ruby, C#, Erlang, Go, Groovy, Haskell, Scala, F#
Notes: MongoDB Documentation Index


Company/Org:Neo Technology
Type: Graph
Description: Embedded, small footprint, disk based, transactional graph database written in Java. Dual license - free and commercial.
Storage: Graph-oriented data model with nodes, relationships and properties.
API(s): Java, Python, Ruby, Scala, Groovy, PHP, RESTful API.
Notes: Neo4J Wiki, API, FAQ


Type: Key/Value
Description: Key/Value store with the dataset kept in memory and saved to disk asynchronously. "not just another key-value DB"
Storage: Values can be strings, lists sets and sorted sets.
API(s): Python, Ruby, PHP, Erlang, Lua, C, C#, Java, Scala, perl
Notes: Redis Wiki


Type: Item/Attribute/Value
Description: Scalable Web Service providing data storage, query and indexing in Amazon's cloud.
Storage: Items (like rows of data), Attributes (like column headers), and Values (can be multiple values)
Notes: SimpleDB FAQ, Getting Started Guide, Developer Guide, API

Tokyo Cabinet

Company/Org:Mikio Hirabayashi
Type: Key/Value
Description: Library (written in C) of functions for managing files of key/value pairs. Multi-thread support.
Storage: Keys and Values can have variable byte length. Binary data and strings can be used as a key and a value.
API(s): C, perl, Ruby, Java, Lua.
Notes: Tokyo Cabinet Specifications, presentation (PDF format). Also available: Tokyo Tyrant (remote service), Tokyo Distopia (full text search), Tokyo Promenade (content management).


Type: Hash Table
Description: "It is basically just a big, distributed, persistent, fault-tolerant hash table." High performance and availability.
Storage: Each key is unique to a store. Each key can have at most one value. Supported types: JSON, string, identity, protobuf, java-serialization.
API(s): Java, C++, custom clients
Notes: Project Voldemort Wiki, Client how-to

It's one thing to have lots of choices for non-relational databases. Building up a NoSQL knowledge and experience base will definitely help managers, architects and developers compare and contrast what they already know about relational databases. Relational databases and the SQL language are still the architecture and lingua franca for the design, development and management of database applications. While we are still at the beginning of the use of databases in cloud infrastructures, we can move forward faster because of all of the work and collaboration that is taking place. Depending on the user and business requirements, we can choose between existing relational database technologies or the NoSQL alternatives.

Stay tuned to the conclusion (part three) of this blog series where I'll give you links to additional NoSQL Internet resources, past/upcoming events, offer some guidance for developers and highlight some of the comments and answer some of the questions from the first two parts.

Are you using a non-relational database? Have you given up on SQL and relational databases? Are you moving your data to a public or private cloud? Post a comment.

The end of SQL and relational databases?

Part 1 || Part 2 || Part 3

Programming is Life!

Recent news for developers:

David Intersimone (David I) is the Vice President of Developer Relations and Chief Evangelist for Embarcadero Technologies. My company blog is at

Copyright © 2010 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon