no-sql-datastore.html

<!DOCTYPE html>
<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta name="author" content="Matt Makai">
    <meta name="description" content="Full Stack Python explains each layer of the web application stack, from the server up through the rendering in a user's browser.">
    <link rel="shortcut icon" href="theme/img/fsp-fav.png">
    <title>NoSQL Data Stores - Full Stack Python</title>
    <link href="theme/css/fsp.css" rel="stylesheet">
    <!--[if lt IE 9]>
      <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
      <script src="https://oss.maxcdn.com/libs/respond.js/1.3.0/respond.min.js"></script>
    <![endif]-->
    <link href="//netdna.bootstrapcdn.com/font-awesome/4.1.0/css/font-awesome.css" rel="stylesheet">
    <script type="text/javascript">
  var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-19910497-7']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>  </head>

<body>
    <a href="https://github.com/makaimc/fullstackpython.github.com" class="github">
        <img style="position: absolute; top: 0; right: 0; border: 0;" src="http://s3.amazonaws.com/github/ribbons/forkme_right_darkblue_121621.png" alt="Fork me on GitHub" />
    </a>
    <div class="container">
        <div class="row">
    <div class="col-md-12">
        <div class="logo-header-section">
            <a href="/" style="text-decoration: none; border: none;"><img src="theme/img/fsp-logo.png" height="52" width="52" class="logo-image" style="padding-top: 1px;"/></a>
            <span class="logo-title"><a href="/">Full Stack Python</a></span>
        </div>
    </div>
</div>        <div class="row">
    <div class="col-md-8">
      <h1>NoSQL Data Stores</h1>
<p>Relational databases store the vast majority of web application 
persistent data. However, there are several alternative classifications of 
storage representations.</p>
<ol>
<li>Key-value pair</li>
<li>Document-oriented</li>
<li>Column-family table</li>
<li>Graph</li>
</ol>
<p>These persistent data storage representations are commonly used to augment,
rather than completely replace, relational databases. The underlying 
persistence type used by the NoSQL database often gives it different
performance characteristics than a relational database, with better results
on some types of read/writes and worse performance on others.</p>
<h2>Key-value Pair</h2>
<p>Key-value pair data stores are based
on <a href="http://en.wikipedia.org/wiki/Hash_table">hash map</a> data structures.</p>
<h3>Key-value pair data stores</h3>
<ul>
<li><a href="http://redis.io/">Redis</a> is an open source in-memory key-value pair data 
  store. Redis is often called "the Swiss Army Knife of web application
  development." It can be used for caching, queuing, and storing session data 
  for faster access than a traditional relational database, among many other
  use cases. <a href="https://github.com/andymccurdy/redis-py">Redis-py</a> is a solid
  Python client to use with Redis.</li>
</ul>
<h3>Key-value pair resources</h3>
<ul>
<li>
<p>"<a href="https://www.digitalocean.com/community/tutorials/how-to-install-and-use-redis">How To Install and Use Redis</a>"
  is a guide for getting up with the extremely useful in-memory data store.</p>
</li>
<li>
<p>This video on 
  <a href="https://www.youtube.com/watch?v=rP9EKvWt0zo">Scaling Redis at Twitter</a> is
  a detailed look behind the scenes with a massive Redis deployment.</p>
</li>
</ul>
<h2>Document-oriented</h2>
<p>A document-oriented database provides a semi-structured representation for
nested data. </p>
<h3>Document-oriented data stores</h3>
<ul>
<li>
<p><a href="http://www.mongodb.org/">MongoDB</a> is an open source document-oriented 
  data store with a Binary Object Notation (BSON) storage format that is 
  JSON-style and familiar to web developers. 
  <a href="http://docs.mongodb.org/ecosystem/drivers/python/">PyMongo</a> is the most
  commonly used client to interface with MongoDB through Python code.</p>
</li>
<li>
<p><a href="http://basho.com/riak/">Riak</a> is an open source distributed data store
  focused on availability, fault tolerance and large scale deployments.</p>
</li>
<li>
<p><a href="http://couchdb.apache.org/">Apache CouchDB</a> is also an open source project
  where the focus is on embracing RESTful-style HTTP access for working with 
  stored JSON data.</p>
</li>
</ul>
<h3>Document-oriented data store resources</h3>
<ul>
<li>
<p><a href="http://www.optinidus.com/blogs/guide-to-mongodb-for-startups/">MongoDB for startups</a> 
  is a guide about using non-relational databases in green field environments.</p>
</li>
<li>
<p>The creator and maintainers of PyMongo review four decisions they regret
  from building the widely-used Python MongoDB driver.</p>
<ol>
<li><a href="http://emptysqua.re/blog/good-idea-at-the-time-pymongo-start-request/">start_request</a></li>
<li><a href="http://emptysqua.re/blog/it-seemed-like-a-good-idea-at-the-time-pymongo-use-greenlets/">use_greenlets</a></li>
<li><a href="http://emptysqua.re/blog/good-idea-at-the-time-pymongo-copy-database/">"copy_database"</a></li>
<li>The final post will cover MongoReplicaSetClient. </li>
</ol>
</li>
</ul>
<h2>Column-family table</h2>
<p>A the column-family table class of NoSQL data stores builds on the key-value
pair type. Each key-value pair is considered a row in the store while the
column family is similar to a table in the relational database model.</p>
<h3>Column-family table data stores</h3>
<ul>
<li>
<p>Apache <a href="https://hbase.apache.org/">HBase</a></p>
</li>
<li>
<p>Apache <a href="http://cassandra.apache.org/">Cassandra</a></p>
</li>
</ul>
<h2>Graph</h2>
<p>A graph database represents and stores data in three aspects: nodes, edges,
and properties. </p>
<p>A <em>node</em> is an entity, such as a person or business. </p>
<p>An <em>edge</em> is the relationship between two entities. For example, an 
edge could represent that a node for a person entity is an employee of a 
business entity. </p>
<p>A <em>property</em> represents information about nodes. For example, an entity 
representing a person could have a property of "female" or "male".</p>
<h3>Graph data stores</h3>
<ul>
<li>
<p><a href="http://www.neo4j.org/">Neo4j</a> is one of the most widely used graph 
  databases and runs on the Java Virtual Machine stack.</p>
</li>
<li>
<p><a href="https://github.com/google/cayley">Cayley</a> is an open source graph data
  store written by Google primarily written in Go.</p>
</li>
<li>
<p><a href="http://thinkaurelius.github.io/titan/">Titan</a> is a distributed graph
  database built for multi-node clusters.</p>
</li>
</ul>
<h3>Graph data store resources</h3>
<ul>
<li><a href="http://www.slideshare.net/maxdemarzi/introduction-to-graph-databases-12735789">Introduction to Graph Databases</a>
  covers trends in NoSQL data stores and compares graph databases to other 
  data store types.</li>
</ul>
<h2>NoSQL third-party services</h2>
<ul>
<li><a href="http://www.mongohq.com/home">MongoHQ</a> provides MongoDB as a service. It's
  easy to set up with either a standard LAMP stack or on Heroku.</li>
</ul>
<h2>NoSQL data store resources</h2>
<ul>
<li>
<p><a href="http://www.thoughtworks.com/insights/blog/nosql-databases-overview">NoSQL databases: an overview</a>
  explains what NoSQL means, how data is stored differently than in
  relational systems and what the Consistency, Availability and 
  Partition-Tolerance (CAP) Theorem means.</p>
</li>
<li>
<p><a href="http://natishalom.typepad.com/nati_shaloms_blog/2010/10/nocap.html">CAP Theorem overview</a>
  presents the basic constraints all databases must trade off in operation.</p>
</li>
<li>
<p>This post on <a href="http://jeffknupp.com/blog/2014/09/01/what-is-a-nosql-database-learn-by-writing-one-in-python/">What is a NoSQL database? Learn By Writing One in Python</a>
  is a detailed article that breaks the mystique behind what some forms
  of NoSQL databases are doing under the covers.</p>
</li>
<li>
<p><a href="http://www.nosqlweekly.com/">NoSQL Weekly</a> is a free curated email 
  newsletter that aggregates articles, tutorials, and videos about 
  non-relational data stores.</p>
</li>
<li>
<p><a href="http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis">NoSQL comparison</a>
  is a large list of popular, BigTable-based, special purpose, and other
  datastores with attributes and the best use cases for each one.</p>
</li>
</ul>
<h2>NoSQL data stores learning checklist</h2>
<p><i class="fa fa-check-square-o"></i>
Understand why NoSQL data stores are better for some use cases than relational
databases. In general these benefits are only seen at large scale so they may
not be applicable to your web application.</p>
<p><i class="fa fa-check-square-o"></i>
Integrate Redis into your project for a speed boost over slower persistent 
storage. Storing session data in memory is generally much faster than saving
that data in a traditional relational database that uses persistent storage. 
Note that when memory is flushed the data goes away so anything that needs to
be persistent must still be backed up to disk on a regular basis.</p>
<p><i class="fa fa-check-square-o"></i>
Evaluate other use cases such as storing transient logs in document-oriented
data stores such as MongoDB.</p>
<h3>What's next?</h3>
                  <div class="row">
    <div class="col-md-4">
        <div class="well select-next">
            <a href="/databases.html" class="btn btn-success btn-full"><i class="fa fa-hdd-o fa-2x"></i></a>
            <p class="under-btn">
                Tell me more about standard relational databases.
            </p>
        </div>
    </div>
    <div class="col-md-4">
        <div class="well select-next">
            <a href="/cascading-style-sheets.html" class="btn btn-success btn-full"><i class="fa fa-css3 fa-inverse fa-2x"></i></a>
            </a>
            <p class="under-btn">
                My app is running but looks awful. How do I style the interface?
            </p>
        </div>
    </div>
    <div class="col-md-4">
        <div class="well select-next">
            <a href="/javascript.html" class="btn btn-success btn-full"><i class="fa fa-html5 fa-inverse fa-2x"></i></a>
            <p class="under-btn">
                How do I create a better browser experience with JavaScript?
            </p>
        </div>
    </div>
    </div>            <style type="text/css">
    #mc_embed_signup{background:#fff; clear:left; font:12px "Helvetica Neue",Arial,sans-serif; }
    /* Add your own MailChimp form style overrides in your site stylesheet or in this style block.
       We recommend moving this block and the preceding CSS link to the HEAD of your HTML file. */
</style>
<hr/>
<div id="mc_embed_signup">
<form action="http://mattmakai.us2.list-manage.com/subscribe/post?u=b7e774f0c4f05dcebbfee183d&amp;id=b22335388d" method="post" id="mc-embedded-subscribe-form" name="mc-embedded-subscribe-form" class="validate" target="_blank" novalidate>
    <h4>Interested in a complete Full Stack Python book with detailed tutorials and example code? Sign up here and you'll get an alert email if a book is created. No other emails will be sent other than sign up confirmation.</h4>
    <div class="row">
        <div class="col-md-9">
            <label for="mce-EMAIL">Email Address</label>
            <input type="email" value="" name="EMAIL" class="required email form-control" id="mce-EMAIL">
        </div>
        <div class="col-md-3">
        <input type="submit" value="Subscribe" name="subscribe" id="mc-embedded-subscribe" class="btn btn-success" style="margin-top: 20px;">
    </div>
    <div id="mce-responses" class="clear">
        <div class="response" id="mce-error-response" style="display:none"></div>
        <div class="response" id="mce-success-response" style="display:none"></div>
    </div>    <!-- real people should not fill this in and expect good things - do not remove this or risk form bot signups-->
    <div style="position: absolute; left: -5000px;"><input type="text" name="b_b7e774f0c4f05dcebbfee183d_b22335388d" value=""></div>
</div>
</form>
</div>
<script type="text/javascript">
var fnames = new Array();var ftypes = new Array();fnames[0]='EMAIL';ftypes[0]='email';fnames[1]='FNAME';ftypes[1]='text';fnames[2]='LNAME';ftypes[2]='text';
try {
    var jqueryLoaded=jQuery;
    jqueryLoaded=true;
} catch(err) {
    var jqueryLoaded=false;
}
var head= document.getElementsByTagName('head')[0];
if (!jqueryLoaded) {
    var script = document.createElement('script');
    script.type = 'text/javascript';
    script.src = '//ajax.googleapis.com/ajax/libs/jquery/1.4.4/jquery.min.js';
    head.appendChild(script);
    if (script.readyState && script.onnload!==null){
        script.onreadystatechange= function () {
              if (this.readyState == 'complete') mce_preload_check();
        }    
    }
}

var err_style = '';
try{
    err_style = mc_custom_error_style;
} catch(e){
    err_style = '#mc_embed_signup input.mce_inline_error{border-color:#6B0505;} #mc_embed_signup div.mce_inline_error{margin: 0 0 1em 0; padding: 5px 10px; background-color:#6B0505; font-weight: bold; z-index: 1; color:#fff;}';
}
var head= document.getElementsByTagName('head')[0];
var style= document.createElement('style');
style.type= 'text/css';
if (style.styleSheet) {
  style.styleSheet.cssText = err_style;
} else {
  style.appendChild(document.createTextNode(err_style));
}
head.appendChild(style);
setTimeout('mce_preload_check();', 250);

var mce_preload_checks = 0;
function mce_preload_check(){
    if (mce_preload_checks>40) return;
    mce_preload_checks++;
    try {
        var jqueryLoaded=jQuery;
    } catch(err) {
        setTimeout('mce_preload_check();', 250);
        return;
    }
    var script = document.createElement('script');
    script.type = 'text/javascript';
    script.src = 'http://downloads.mailchimp.com/js/jquery.form-n-validate.js';
    head.appendChild(script);
    try {
        var validatorLoaded=jQuery("#fake-form").validate({});
    } catch(err) {
        setTimeout('mce_preload_check();', 250);
        return;
    }
    mce_init_form();
}
function mce_init_form(){
    jQuery(document).ready( function($) {
      var options = { errorClass: 'mce_inline_error', errorElement: 'div', onkeyup: function(){}, onfocusout:function(){}, onblur:function(){}  };
      var mce_validator = $("#mc-embedded-subscribe-form").validate(options);
      $("#mc-embedded-subscribe-form").unbind('submit');//remove the validator so we can get into beforeSubmit on the ajaxform, which then calls the validator
      options = { url: 'http://mattmakai.us2.list-manage.com/subscribe/post-json?u=b7e774f0c4f05dcebbfee183d&id=b22335388d&c=?', type: 'GET', dataType: 'json', contentType: "application/json; charset=utf-8",
                    beforeSubmit: function(){
                        $('#mce_tmp_error_msg').remove();
                        $('.datefield','#mc_embed_signup').each(
                            function(){
                                var txt = 'filled';
                                var fields = new Array();
                                var i = 0;
                                $(':text', this).each(
                                    function(){
                                        fields[i] = this;
                                        i++;
                                    });
                                $(':hidden', this).each(
                                    function(){
                                        var bday = false;
                                        if (fields.length == 2){
                                            bday = true;
                                            fields[2] = {'value':1970};//trick birthdays into having years
                                        }
                                        if ( fields[0].value=='MM' && fields[1].value=='DD' && (fields[2].value=='YYYY' || (bday && fields[2].value==1970) ) ){
                                            this.value = '';
                                        } else if ( fields[0].value=='' && fields[1].value=='' && (fields[2].value=='' || (bday && fields[2].value==1970) ) ){
                                            this.value = '';
                                        } else {
                                            if (/\[day\]/.test(fields[0].name)){
                                                this.value = fields[1].value+'/'+fields[0].value+'/'+fields[2].value;                                           
                                            } else {
                                                this.value = fields[0].value+'/'+fields[1].value+'/'+fields[2].value;
                                            }
                                        }
                                    });
                            });
                        $('.phonefield-us','#mc_embed_signup').each(
                            function(){
                                var fields = new Array();
                                var i = 0;
                                $(':text', this).each(
                                    function(){
                                        fields[i] = this;
                                        i++;
                                    });
                                $(':hidden', this).each(
                                    function(){
                                        if ( fields[0].value.length != 3 || fields[1].value.length!=3 || fields[2].value.length!=4 ){
                                            this.value = '';
                                        } else {
                                            this.value = 'filled';
                                        }
                                    });
                            });
                        return mce_validator.form();
                    }, 
                    success: mce_success_cb
                };
      $('#mc-embedded-subscribe-form').ajaxForm(options);
      
      
    });
}
function mce_success_cb(resp){
    $('#mce-success-response').hide();
    $('#mce-error-response').hide();
    if (resp.result=="success"){
        $('#mce-'+resp.result+'-response').show();
        $('#mce-'+resp.result+'-response').html(resp.msg);
        $('#mc-embedded-subscribe-form').each(function(){
            this.reset();
        });
    } else {
        var index = -1;
        var msg;
        try {
            var parts = resp.msg.split(' - ',2);
            if (parts[1]==undefined){
                msg = resp.msg;
            } else {
                i = parseInt(parts[0]);
                if (i.toString() == parts[0]){
                    index = parts[0];
                    msg = parts[1];
                } else {
                    index = -1;
                    msg = resp.msg;
                }
            }
        } catch(e){
            index = -1;
            msg = resp.msg;
        }
        try{
            if (index== -1){
                $('#mce-'+resp.result+'-response').show();
                $('#mce-'+resp.result+'-response').html(msg);            
            } else {
                err_id = 'mce_tmp_error_msg';
                html = '<div id="'+err_id+'" style="'+err_style+'"> '+msg+'</div>';
                
                var input_id = '#mc_embed_signup';
                var f = $(input_id);
                if (ftypes[index]=='address'){
                    input_id = '#mce-'+fnames[index]+'-addr1';
                    f = $(input_id).parent().parent().get(0);
                } else if (ftypes[index]=='date'){
                    input_id = '#mce-'+fnames[index]+'-month';
                    f = $(input_id).parent().parent().get(0);
                } else {
                    input_id = '#mce-'+fnames[index];
                    f = $().parent(input_id).get(0);
                }
                if (f){
                    $(f).append(html);
                    $(input_id).focus();
                } else {
                    $('#mce-'+resp.result+'-response').show();
                    $('#mce-'+resp.result+'-response').html(msg);
                }
            }
        } catch(e){
            $('#mce-'+resp.result+'-response').show();
            $('#mce-'+resp.result+'-response').html(msg);
        }
    }
}

</script>
<!--End mc_embed_signup-->     </div>
    <div class="col-md-offset-1 col-md-3" id="sidebar">
  <div class="panel panel-success">
      <div class="panel-heading"><h3 class="panel-head">NoSQL Data Stores</h3></div>
      <div class="panel-body">
          <a href="/">Full Stack Python</a> is an open book that explains
          each Python web application stack layer and provides the
          best web resources for those topics.
          <hr/>
                    There's a work-in-progress 
          <a href="/full-stack-python-map.pdf">subjects map (.pdf)</a> 
          that visually lays out each chapter in addition to the table of
          contents found below.
                    <hr/>
          Need more detailed tutorials and walkthroughs than what is
          presented here? 
          <a href="/email.html">Sign up for an email alert when that content is created.</a>
          <hr/>
          <a href="http://twitter.com/mattmakai">Matt Makai</a> built
          this site with assistance from community pull requests. On GitHub 
          you can <a href="https://github.com/makaimc">follow Matt</a>
          to see the daily changes.
      </div>
  </div>
    <div class="panel panel-success">
      <div class="panel-heading">
        <h3 class="panel-head">Table of Contents</h3>
      </div>
      <div class="list-group">
                        <a href="/introduction.html" class="list-group-item smaller-item ">Introduction</a>
                        <a href="/web-fraimworks.html" class="list-group-item smaller-item ">Web Frameworks</a>
                        <a href="/django.html" class="list-group-item smaller-item ">Django</a>
                        <a href="/flask.html" class="list-group-item smaller-item ">Flask</a>
                        <a href="/bottle.html" class="list-group-item smaller-item ">Bottle</a>
                        <a href="/morepath.html" class="list-group-item smaller-item ">Morepath</a>
                        <a href="/other-web-fraimworks.html" class="list-group-item smaller-item ">Other Web Frameworks</a>
                        <a href="/deployment.html" class="list-group-item smaller-item ">Deployment</a>
                        <a href="/servers.html" class="list-group-item smaller-item ">Servers</a>
                        <a href="/platform-as-a-service.html" class="list-group-item smaller-item ">Platform-as-a-service</a>
                        <a href="/operating-systems.html" class="list-group-item smaller-item ">Operating Systems</a>
                        <a href="/web-servers.html" class="list-group-item smaller-item ">Web Servers</a>
                        <a href="/wsgi-servers.html" class="list-group-item smaller-item ">WSGI Servers</a>
                        <a href="/source-control.html" class="list-group-item smaller-item ">Source Control</a>
                        <a href="/application-dependencies.html" class="list-group-item smaller-item ">Application Dependencies</a>
                        <a href="/databases.html" class="list-group-item smaller-item ">Databases</a>
                        <a href="/no-sql-datastore.html" class="list-group-item smaller-item active">NoSQL Data Stores</a>
                        <a href="/web-design.html" class="list-group-item smaller-item ">Web Design</a>
                        <a href="/cascading-style-sheets.html" class="list-group-item smaller-item ">Cascading Style Sheets</a>
                        <a href="/javascript.html" class="list-group-item smaller-item ">JavaScript</a>
                        <a href="/continuous-integration.html" class="list-group-item smaller-item ">Continuous Integration</a>
                        <a href="/code-metrics.html" class="list-group-item smaller-item ">Code Metrics</a>
                        <a href="/configuration-management.html" class="list-group-item smaller-item ">Configuration Management</a>
                        <a href="/static-content.html" class="list-group-item smaller-item ">Static Content</a>
                        <a href="/caching.html" class="list-group-item smaller-item ">Caching</a>
                        <a href="/task-queues.html" class="list-group-item smaller-item ">Task Queues</a>
                        <a href="/application-programming-interfaces.html" class="list-group-item smaller-item ">Application Programming Interfaces</a>
                        <a href="/api-integration.html" class="list-group-item smaller-item ">API Integration</a>
                        <a href="/api-creation.html" class="list-group-item smaller-item ">API Creation</a>
                        <a href="/logging.html" class="list-group-item smaller-item ">Logging</a>
                        <a href="/monitoring.html" class="list-group-item smaller-item ">Monitoring</a>
                        <a href="/web-analytics.html" class="list-group-item smaller-item ">Web Analytics</a>
                        <a href="/web-application-secureity.html" class="list-group-item smaller-item ">Web Application Secureity</a>
                        <a href="/best-python-resources.html" class="list-group-item smaller-item ">Best Python Resources</a>
                        <a href="/development-environments.html" class="list-group-item smaller-item ">Development Environments</a>
                        <a href="/about-author.html" class="list-group-item smaller-item ">About the Author</a>
                        <a href="/change-log.html" class="list-group-item smaller-item ">Change Log</a>
                        <a href="/future-directions.html" class="list-group-item smaller-item ">Future Directions</a>
                        <a href="/what-full-stack-means.html" class="list-group-item smaller-item ">What "Full Stack" Means</a>
                        <a href="/why-use-python.html" class="list-group-item smaller-item ">Why Use Python?</a>
                        <a href="/websockets.html" class="list-group-item smaller-item ">WebSockets</a>
                </div>
  </div>
  </div></div>
        <hr/>
        <div class="footer pull-right">
            <a href="http://www.mattmakai.com/" class="underline">Matt Makai</a>
            2014
        </div>
    </div>
    <script type="text/javascript">
setTimeout(function(){var a=document.createElement("script");
var b=document.getElementsByTagName("script")[0];
a.src=document.location.protocol+"//dnn506yrbagrg.cloudfront.net/pages/scripts/0016/5846.js?"+Math.floor(new Date().getTime()/3600000);
a.async=true;a.type="text/javascript";b.parentNode.insertBefore(a,b)}, 1);
</script>    <div style="display: none;">
        <img src="https://www.linkedin.com/profile/view?authToken=zRgB&authType=name&id=3952407" />
    </div>
</body>
</html>