The end of SQL and relational databases? (part 3 of 3)

In "The end of SQL databases - part 1," I covered some background of the SQL language and relational databases, the current status and future for relational databases, the rise of frameworks that hide some of the complexities of database programming, and the rise of the NoSQL category of databases.  "The end of SQL databases - part 2" contained an overview of some of the currently available open and closed source NoSQL databases.  In part 3, I will point you to NoSQL Internet resources, past and upcoming events, and highlight some of the comments posted about the first two parts.

If you are looking for more information about NoSQL and non-relational databases, check out the following sites, blog posts, and articles:

There are several upcoming and recent NoSQL conferences and meet ups with great information and solutions for architects and developers.  Here are just a few:

It has been great to see the comments and suggestions that have been posted on my Computerworld blog and also on Digg.  Thank you to everyone for joining the discussion about relational and non-relational database development.  Here are excerpts from the many comments.

  • Emil Eifrem (Neo4j) commented: "You talk about scaling to size and handling Facebook's 100M user profiles. That's an important use case and one that for example a key-value store handles brilliantly. But it turns out most companies aren't Facebook. You can categorize the four emerging categories of NOSQL databases (key-value stores, column family stores, document dbs and graph databases) along the axes of scaling to size and scaling to complexity. For more information about that, see this blog post. Graph databases (like e.g. Neo4j, which I'm involved with, or Sones) excels at representing complex and rapidly evolving domain models and then traversing them with high performance."
  • Mongo-DB Developer commented: "We have seen the most common use case to date being use of nosql solutions as operational data store of web infrastructure projects. By operational, I mean, problems with real time writes and reads (contrast with data warehousing with bulk occasional loading). For these sort of problems these solutions work well and also fit well with agile development methods where the somewhat 'schemaless' (or more accurately, columnless) nature of some of the solutions, and the dynamically typed nature of the storage, really helps."
  • Peter R commented: "I have already seen, in the domain I work in, the movement away from straight up SQL databases. XML databases are one technology that will be stealing a lot of SQL's thunder (if they haven't already). Do I think SQL will ever die? No. But the key is that there will be/are more options that need to be thought about when designing a system now."
  • Anonymous commented: "I agree object databases have a purpose. They are great for large datasets that need to be replicated and called by a key. However SQL provides a very important capability and that it is to be able to query data across a number of datasets very efficiently, this will be very hard to duplicate in a simple key value database."
  • Johannes Ernst commented: "One of the difficulties for "normal" developers with many of the NoSQL technologies that you've described so far has been the learning curve and the additional work required: e.g. it's easy and everybody knows how to put "every customer can place one or more orders" into a relational database, but what if the only thing you have is keys and opaque values? Compared to many other NoSQL alternatives, graph databases provide a high level of abstraction, freeing developers to concentrate on their application, while still bringing many of the same NoSQL benefits. For example, in InfoGrid (http://infogrid.org/), a project I'm involved in, you can define "Customer" and "Order" and their relationship, and the InfoGrid graph database takes care of storing and retrieving data and enforcing the relationship. In our experience, that makes graph databases much more approachable to developers than many other NoSQL technologies."
  • Database-ed commented: "The problem is that when folks think about storing information that they need to retrieve, they are so ingrained to SQL that they fail to think of other means. The Facebook example is a case in point. Who is ever going to ask for an accurate report of every user in Facebook? If you miss something the first time you go looking, you can always present it later. The end user doesn't know you lost it, they assume it didn't exist at the time and now it does. Yet you still need to store the data for easy retrieval. One problem with SQL is that it ties you into the relationships. Facebook is about letting people build the relationships based on the fields they want to build them on, not the ones you might think of. I know, it can be done within the confines of SQL, but it is a lot harder to do when the size gets large."
  • Raptor007 commented: "Some tasks that are poorly serviced by SQL may get switched over to a new method, but other implementations that are perfectly suited to SQL will continue using it. As they quoted Eric Evans in the article, "the whole point of seeking alternatives is that you need to solve a problem that relational databases are a bad fit for."
  • Miracle Blue commented: "While I highly doubt there's going to be any significant migration away from SQL and the like any time soon, I think more web developers will start experimenting with data stores and other data solutions as we move further into the cloud."
  • TheUnGod commented: "And as companies turn to ask their SQL DBAs what they think of this, they'll say "lets stick with SQL." Honestly, there are so many people that support SQL right now that will not switch any time soon this article is just bogus. You can't make a switch like that until people can support it properly."
  • SteelChicken commented: "Document centric is pretty dumb if you plan on doing any sort of analytics and data mining. Great for workflow and such."
  • Angusm commented: "The significance of the NoSQL movement is that it adds new tools that offer better solutions to specific problems. The future probably belongs to NoSQL in the sense of 'not-only SQL', rather than 'no SQL'. Don't imagine that NoSQL solutions offer a free lunch though. I had an educational experience when I changed a view definition in a CouchDB data store and my first trivial query took an hour to come back. CouchDB can be pleasingly fast when all its indexes are built, but if you have to rebuild those indexes from scratch ... well, let's just say that's not something you want to do on a live client-facing site."
  • Afex2win commented: "digg is one of the bigger proponents of Cassandra, a distributed data store in the vein of which the article is talking about. http://about.digg.com/blog/looking-future-cassandra"
  • Drmangrum commented: "SQL will be around for awhile. It's good at doing what it was designed to do. However, there are many times when people use SQL simply because there is nothing better out there. As data complexity rises, a new method for accessing and persisting that data will have to be investigated. Part of the problem with many of the alternate solutions is that few people know how to use them."

For years to come, most of us will still use relational databases and SQL.  At the same time, I will continue to look for better ways to design and implement databases in my applications.  As always, any project decision has to match with the user and business requirements.  For new projects, I believe, we now have genuine non-relational alternatives on the table (pun intended).  Are you using a non-relational database?  Have you given up on SQL and relational databases?  Are you moving your data to a public or private cloud?  Post a comment.

The end of SQL and relational databases?

Part 1  ||  Part 2  ||  Part 3

Programming is Life!

Recent news for developers:

David Intersimone (David I) is the Vice President of Developer Relations and Chief Evangelist for Embarcadero Technologies. My company blog is at http://blogs.embarcadero.com/davidi

Computerworld's IT Salary Survey 2017 results
Shop Tech Products at Amazon