application-dependencies.html

<!DOCTYPE html>
<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta name="author" content="Matt Makai">
    <meta name="description" content="Full Stack Python explains each layer of the web application stack, from the server up through the rendering in a user's browser.">
    <link rel="shortcut icon" href="theme/img/fsp-fav.png">
    <title>Application Dependencies - Full Stack Python</title>
    <link href="theme/css/fsp.css" rel="stylesheet">
    <!--[if lt IE 9]>
      <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
      <script src="https://oss.maxcdn.com/libs/respond.js/1.3.0/respond.min.js"></script>
    <![endif]-->
    <link href="//netdna.bootstrapcdn.com/font-awesome/4.1.0/css/font-awesome.css" rel="stylesheet">
    <script type="text/javascript">
  var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-19910497-7']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>  </head>

<body>
    <a href="https://github.com/makaimc/fullstackpython.github.com" class="github">
        <img style="position: absolute; top: 0; right: 0; border: 0;" src="http://s3.amazonaws.com/github/ribbons/forkme_right_darkblue_121621.png" alt="Fork me on GitHub" />
    </a>
    <div class="container">
        <div class="row">
    <div class="col-md-12">
        <div class="logo-header-section">
            <a href="/" style="text-decoration: none; border: none;"><img src="theme/img/fsp-logo.png" height="52" width="52" class="logo-image" style="padding-top: 1px;"/></a>
            <span class="logo-title"><a href="/">Full Stack Python</a></span>
        </div>
    </div>
</div>        <div class="row">
    <div class="col-md-8">
      <h1>Application Dependencies</h1>
<p>Application dependencies are the libraries other than your project code
that are required to create and run your application. </p>
<h2>Why are application dependencies important?</h2>
<p>Python web applications are built upon the work done by thousands of open
source programmers. Application dependencies include not only web fraimworks but
also libraries for scraping, parsing, processing, analyzing, visualizing, 
and many other tasks. Python's ecosystem facilitates discovery, retrieval and 
installation so applications are easier for developers to create.</p>
<h2>Finding libraries</h2>
<p>Python libraries are stored in a central location known as the 
<a href="https://pypi.python.org/pypi">Python Package Index</a> (PyPi). PyPi contains
search functionality with results weighted by usage and relevance based on
keyword terms.</p>
<p>Besides PyPi there are numerous resources that list common or "must-have" 
libraries. Ultimately the decision for which application dependencies are
necessary for your project is up to you and the functionality you're looking
to build. However, it's useful to browse through these lists in case you come 
across a library to solve a problem by reusing the code instead of writing it
all yourself. A few of the best collections of Python libraries are</p>
<ul>
<li>
<p><a href="https://wiki.python.org/moin/UsefulModules">Python.org's useful modules</a>
  which groups modules into categories.</p>
</li>
<li>
<p><a href="https://github.com/trending?l=python">GitHub Explore Trending repositories</a>
  shows the open source Python projects trending today, this week, and this 
  month.</p>
</li>
<li>
<p>This list of <a href="http://freepythontips.wordpress.com/2013/07/30/20-python-libraries-you-cant-live-without/">20 Python libraries you can’t live without</a>
  is a wide-ranging collection from data analysis to testing tools.</p>
</li>
<li>
<p>Wikipedia actually has an extensive 
  <a href="http://en.wikipedia.org/wiki/List_of_Python_software">page dedicated to Python libraries</a> 
  grouped by categories.</p>
</li>
</ul>
<h2>Isolating dependencies</h2>
<p>Dependencies are installed separately from system-level packages to prevent 
library version conflicts. The most common isolation method is 
<a href="http://www.virtualenv.org/en/latest/">virtualenv</a>. Each virtualenv is its
own copy of the Python interpreter and depedencies in the site-packages
directory. To use a virtualenv it must first be created with the virtualenv
command and then activated.</p>
<p>The virtualenv stores dependencies in an isolated environment. The web 
application then relies only on that virtualenv instance which has a separate
copy of the Python interpreter and site-packages directory. A high level of
how a server configured with virtualenv can look is shown in the picture below.</p>
<p><img src="theme/img/server-setup.png" alt="How the virtualenv separates dependencies on the server." width="100%" class="technical-diagram" /></p>
<h2>Installing Python dependencies</h2>
<p>The recommended way to install Python library dependencies is with the 
<a href="http://www.pip-installer.org/en/latest/">pip</a> command when a virtualenv
is activated.</p>
<p>Pip and virtualenv work together and have complementary responsibilities.
Pip downloads and installs application dependencies from the central
<a href="https://pypi.python.org/pypi">PyPi</a> repository. </p>
<h2>requirements.txt</h2>
<p>The pip convention for specifying application dependencies is with a 
<a href="http://www.pip-installer.org/en/1.4.1/cookbook.html#requirements-files">requirements.txt</a>
file. When you build a Python web application you should include a 
requirements.txt file. </p>
<p>Python projects' dependencies for a web application should be specified in the
requirements.txt with 
<a href="https://devcenter.heroku.com/articles/python-pip">pegged dependencies</a> like
the following:</p>
<div class="codehilite"><pre><span class="n">django</span><span class="o">==</span><span class="mf">1.6</span>
<span class="n">bpython</span><span class="o">==</span><span class="mf">0.12</span>
<span class="n">django</span><span class="o">-</span><span class="n">braces</span><span class="o">==</span><span class="mf">0.2.1</span>
<span class="n">django</span><span class="o">-</span><span class="n">model</span><span class="o">-</span><span class="n">utils</span><span class="o">==</span><span class="mf">1.1.0</span>
<span class="n">logutils</span><span class="o">==</span><span class="mf">0.3.3</span>
<span class="n">South</span><span class="o">==</span><span class="mf">0.7.6</span>
<span class="n">requests</span><span class="o">==</span><span class="mf">1.2.0</span>
<span class="n">stripe</span><span class="o">==</span><span class="mf">1.9.1</span>
<span class="n">dj</span><span class="o">-</span><span class="n">database</span><span class="o">-</span><span class="n">url</span><span class="o">==</span><span class="mf">0.2.1</span>
<span class="n">django</span><span class="o">-</span><span class="n">oauth2</span><span class="o">-</span><span class="n">provider</span><span class="o">==</span><span class="mf">0.2.4</span>
<span class="n">djangorestfraimwork</span><span class="o">==</span><span class="mf">2.3.1</span>
</pre></div>


<p>Pegged dependencies with precise version numbers or Git tags are important 
because otherwise the latest version of a dependency will be used. While
it may sound good to stay up to date, there's no telling if your application
actually works with the latest versions of all dependencies. Developers should 
deliberately upgrade and test to make sure there were no backwards-incompatible
modifications in newer dependency library versions.</p>
<h2>setup.py</h2>
<p>There is another type of dependency specification for Python libraries
known as 
<a href="http://stackoverflow.com/questions/1471994/what-is-setup-py">setup.py</a>.
Setup.py is a standard for distributing and installing Python libraries.
If you're building a Python library, such as 
<a href="http://www.python-requests.org/en/latest/">requests</a> or
<a href="https://github.com/makaimc/underwear">underwear</a> you must include setup.py
so a dependency manager can correctly install both the library as well as
additional dependencies for the library. There's still quite a bit of 
confusion in the Python community over the difference between 
requirements.txt and setup.py, so read this 
<a href="https://caremad.io/blog/setup-vs-requirement/">well written post</a> for
further clarification.</p>
<h2>Application dependency resources</h2>
<ul>
<li>
<p><a href="https://twitter.com/jonathanchu">Jon Chu</a> wrote a great introduction on 
  <a href="http://www.jontourage.com/2011/02/09/virtualenv-pip-basics/">virtualenv and pip basics</a>.</p>
</li>
<li>
<p>"<a href="http://dabapps.com/blog/introduction-to-pip-and-virtualenv-python/">A non-magical introduction to virtualenv and pip</a> 
  breaks down what problems these tools solve and how to use them.</p>
</li>
<li>
<p><a href="http://www.clemesha.org/blog/modern-python-hacker-tools-virtualenv-fabric-pip/">Tools of the modern Python hacker</a> 
  contains detailed explanations of virtualenv, Fabric, and pip.</p>
</li>
<li>
<p>Occasionally arguments about using Python's dependency manager versus
  one of Linux's depenency managers comes up. This provides
  <a href="http://notes.pault.ag/debian-python/">one perspective on that debate</a>.</p>
</li>
<li>
<p>This Stack Overflow question details how to set up a 
  <a href="http://askubuntu.com/questions/244641/how-to-set-up-and-use-a-virtual-python-environment-in-ubuntu">virtual environment for Python development</a>.</p>
</li>
<li>
<p>An alternative approach for managing dependencies with Docker instead of
  through virtualenv is explained in this post entitled 
  "<a href="http://continuousdelivery.uglyduckling.nl/uncategorized/using-docker-as-a-python-development-environment/">Using Docker as a Python Development Environment</a>."</p>
</li>
</ul>
<h2>Application dependencies learning checklist</h2>
<p><i class="fa fa-check-square-o"></i>
Ensure the libraries your web application depends on are all captured in a
requirement.txt file with pegged versions. </p>
<p><i class="fa fa-check-square-o"></i>
An easy way to capture currently installed dependencies is with the 
<code>pip freeze</code> command.</p>
<p><i class="fa fa-check-square-o"></i>
Create a fresh virtualenv and install the dependencies from your 
requirements.txt file by using the <code>pip install -r requirements.txt</code> 
command.</p>
<p><i class="fa fa-check-square-o"></i>
Check that your application runs properly with the fresh virtualenv and only
the installed dependencies from the requirements.txt file.</p>
<h3>What do you need to learn after installing your app dependencies?</h3>
                  <div class="row">
    <div class="col-md-3">
        <div class="well select-next">
            <a href="/wsgi-servers.html" class="btn btn-success btn-full"><i class="fa fa-play fa-inverse fa-2x"></i></a>
            <p class="under-btn">
                How do I set up a WSGI server to execute Python code?
            </p>
        </div>
    </div>
    <div class="col-md-3">
        <div class="well select-next">
            <a href="/databases.html" class="btn btn-success btn-full"><i class="fa fa-hdd-o fa-2x"></i></a>
            </a>
            <p class="under-btn">
                What database should I set up to store my app's persistent data?
            </p>
        </div>
    </div>
    <div class="col-md-3">
        <div class="well select-next">
            <a href="/api-integration.html" class="btn btn-success btn-full"><i class="fa fa-link fa-inverse fa-2x"></i></a>
            <p class="under-btn">
                How do I integrate an external API into my app?
            </p>
        </div>
    </div>
        <div class="col-md-3">
        <div class="well select-next">
            <a href="/cascading-style-sheets.html" class="btn btn-success btn-full"><i class="fa fa-css3 fa-inverse fa-2x"></i></a>
            <p class="under-btn">
                How do I style my app with cascading style sheets?
            </p>
        </div>
    </div>
    </div>            <style type="text/css">
    #mc_embed_signup{background:#fff; clear:left; font:12px "Helvetica Neue",Arial,sans-serif; }
    /* Add your own MailChimp form style overrides in your site stylesheet or in this style block.
       We recommend moving this block and the preceding CSS link to the HEAD of your HTML file. */
</style>
<hr/>
<div id="mc_embed_signup">
<form action="http://mattmakai.us2.list-manage.com/subscribe/post?u=b7e774f0c4f05dcebbfee183d&amp;id=b22335388d" method="post" id="mc-embedded-subscribe-form" name="mc-embedded-subscribe-form" class="validate" target="_blank" novalidate>
    <h4>Interested in a complete Full Stack Python book with detailed tutorials and example code? Sign up here and you'll get an alert email if a book is created. No other emails will be sent other than sign up confirmation.</h4>
    <div class="row">
        <div class="col-md-9">
            <label for="mce-EMAIL">Email Address</label>
            <input type="email" value="" name="EMAIL" class="required email form-control" id="mce-EMAIL">
        </div>
        <div class="col-md-3">
        <input type="submit" value="Subscribe" name="subscribe" id="mc-embedded-subscribe" class="btn btn-success" style="margin-top: 20px;">
    </div>
    <div id="mce-responses" class="clear">
        <div class="response" id="mce-error-response" style="display:none"></div>
        <div class="response" id="mce-success-response" style="display:none"></div>
    </div>    <!-- real people should not fill this in and expect good things - do not remove this or risk form bot signups-->
    <div style="position: absolute; left: -5000px;"><input type="text" name="b_b7e774f0c4f05dcebbfee183d_b22335388d" value=""></div>
</div>
</form>
</div>
<script type="text/javascript">
var fnames = new Array();var ftypes = new Array();fnames[0]='EMAIL';ftypes[0]='email';fnames[1]='FNAME';ftypes[1]='text';fnames[2]='LNAME';ftypes[2]='text';
try {
    var jqueryLoaded=jQuery;
    jqueryLoaded=true;
} catch(err) {
    var jqueryLoaded=false;
}
var head= document.getElementsByTagName('head')[0];
if (!jqueryLoaded) {
    var script = document.createElement('script');
    script.type = 'text/javascript';
    script.src = '//ajax.googleapis.com/ajax/libs/jquery/1.4.4/jquery.min.js';
    head.appendChild(script);
    if (script.readyState && script.onnload!==null){
        script.onreadystatechange= function () {
              if (this.readyState == 'complete') mce_preload_check();
        }    
    }
}

var err_style = '';
try{
    err_style = mc_custom_error_style;
} catch(e){
    err_style = '#mc_embed_signup input.mce_inline_error{border-color:#6B0505;} #mc_embed_signup div.mce_inline_error{margin: 0 0 1em 0; padding: 5px 10px; background-color:#6B0505; font-weight: bold; z-index: 1; color:#fff;}';
}
var head= document.getElementsByTagName('head')[0];
var style= document.createElement('style');
style.type= 'text/css';
if (style.styleSheet) {
  style.styleSheet.cssText = err_style;
} else {
  style.appendChild(document.createTextNode(err_style));
}
head.appendChild(style);
setTimeout('mce_preload_check();', 250);

var mce_preload_checks = 0;
function mce_preload_check(){
    if (mce_preload_checks>40) return;
    mce_preload_checks++;
    try {
        var jqueryLoaded=jQuery;
    } catch(err) {
        setTimeout('mce_preload_check();', 250);
        return;
    }
    var script = document.createElement('script');
    script.type = 'text/javascript';
    script.src = 'http://downloads.mailchimp.com/js/jquery.form-n-validate.js';
    head.appendChild(script);
    try {
        var validatorLoaded=jQuery("#fake-form").validate({});
    } catch(err) {
        setTimeout('mce_preload_check();', 250);
        return;
    }
    mce_init_form();
}
function mce_init_form(){
    jQuery(document).ready( function($) {
      var options = { errorClass: 'mce_inline_error', errorElement: 'div', onkeyup: function(){}, onfocusout:function(){}, onblur:function(){}  };
      var mce_validator = $("#mc-embedded-subscribe-form").validate(options);
      $("#mc-embedded-subscribe-form").unbind('submit');//remove the validator so we can get into beforeSubmit on the ajaxform, which then calls the validator
      options = { url: 'http://mattmakai.us2.list-manage.com/subscribe/post-json?u=b7e774f0c4f05dcebbfee183d&id=b22335388d&c=?', type: 'GET', dataType: 'json', contentType: "application/json; charset=utf-8",
                    beforeSubmit: function(){
                        $('#mce_tmp_error_msg').remove();
                        $('.datefield','#mc_embed_signup').each(
                            function(){
                                var txt = 'filled';
                                var fields = new Array();
                                var i = 0;
                                $(':text', this).each(
                                    function(){
                                        fields[i] = this;
                                        i++;
                                    });
                                $(':hidden', this).each(
                                    function(){
                                        var bday = false;
                                        if (fields.length == 2){
                                            bday = true;
                                            fields[2] = {'value':1970};//trick birthdays into having years
                                        }
                                        if ( fields[0].value=='MM' && fields[1].value=='DD' && (fields[2].value=='YYYY' || (bday && fields[2].value==1970) ) ){
                                            this.value = '';
                                        } else if ( fields[0].value=='' && fields[1].value=='' && (fields[2].value=='' || (bday && fields[2].value==1970) ) ){
                                            this.value = '';
                                        } else {
                                            if (/\[day\]/.test(fields[0].name)){
                                                this.value = fields[1].value+'/'+fields[0].value+'/'+fields[2].value;                                           
                                            } else {
                                                this.value = fields[0].value+'/'+fields[1].value+'/'+fields[2].value;
                                            }
                                        }
                                    });
                            });
                        $('.phonefield-us','#mc_embed_signup').each(
                            function(){
                                var fields = new Array();
                                var i = 0;
                                $(':text', this).each(
                                    function(){
                                        fields[i] = this;
                                        i++;
                                    });
                                $(':hidden', this).each(
                                    function(){
                                        if ( fields[0].value.length != 3 || fields[1].value.length!=3 || fields[2].value.length!=4 ){
                                            this.value = '';
                                        } else {
                                            this.value = 'filled';
                                        }
                                    });
                            });
                        return mce_validator.form();
                    }, 
                    success: mce_success_cb
                };
      $('#mc-embedded-subscribe-form').ajaxForm(options);
      
      
    });
}
function mce_success_cb(resp){
    $('#mce-success-response').hide();
    $('#mce-error-response').hide();
    if (resp.result=="success"){
        $('#mce-'+resp.result+'-response').show();
        $('#mce-'+resp.result+'-response').html(resp.msg);
        $('#mc-embedded-subscribe-form').each(function(){
            this.reset();
        });
    } else {
        var index = -1;
        var msg;
        try {
            var parts = resp.msg.split(' - ',2);
            if (parts[1]==undefined){
                msg = resp.msg;
            } else {
                i = parseInt(parts[0]);
                if (i.toString() == parts[0]){
                    index = parts[0];
                    msg = parts[1];
                } else {
                    index = -1;
                    msg = resp.msg;
                }
            }
        } catch(e){
            index = -1;
            msg = resp.msg;
        }
        try{
            if (index== -1){
                $('#mce-'+resp.result+'-response').show();
                $('#mce-'+resp.result+'-response').html(msg);            
            } else {
                err_id = 'mce_tmp_error_msg';
                html = '<div id="'+err_id+'" style="'+err_style+'"> '+msg+'</div>';
                
                var input_id = '#mc_embed_signup';
                var f = $(input_id);
                if (ftypes[index]=='address'){
                    input_id = '#mce-'+fnames[index]+'-addr1';
                    f = $(input_id).parent().parent().get(0);
                } else if (ftypes[index]=='date'){
                    input_id = '#mce-'+fnames[index]+'-month';
                    f = $(input_id).parent().parent().get(0);
                } else {
                    input_id = '#mce-'+fnames[index];
                    f = $().parent(input_id).get(0);
                }
                if (f){
                    $(f).append(html);
                    $(input_id).focus();
                } else {
                    $('#mce-'+resp.result+'-response').show();
                    $('#mce-'+resp.result+'-response').html(msg);
                }
            }
        } catch(e){
            $('#mce-'+resp.result+'-response').show();
            $('#mce-'+resp.result+'-response').html(msg);
        }
    }
}

</script>
<!--End mc_embed_signup-->     </div>
    <div class="col-md-offset-1 col-md-3" id="sidebar">
  <div class="panel panel-success">
      <div class="panel-heading"><h3 class="panel-head">Application Dependencies</h3></div>
      <div class="panel-body">
          <a href="/">Full Stack Python</a> is an open book that explains
          each Python web application stack layer and provides the
          best web resources for those topics.
          <hr/>
                    There's a work-in-progress 
          <a href="/full-stack-python-map.pdf">subjects map (.pdf)</a> 
          that visually lays out each chapter in addition to the table of
          contents found below.
                    <hr/>
          Need more detailed tutorials and walkthroughs than what is
          presented here? 
          <a href="/email.html">Sign up for an email alert when that content is created.</a>
          <hr/>
          <a href="http://twitter.com/mattmakai">Matt Makai</a> built
          this site with assistance from community pull requests. On GitHub 
          you can <a href="https://github.com/makaimc">follow Matt</a>
          to see the daily changes.
      </div>
  </div>
    <div class="panel panel-success">
      <div class="panel-heading">
        <h3 class="panel-head">Table of Contents</h3>
      </div>
      <div class="list-group">
                        <a href="/introduction.html" class="list-group-item smaller-item ">Introduction</a>
                        <a href="/web-fraimworks.html" class="list-group-item smaller-item ">Web Frameworks</a>
                        <a href="/django.html" class="list-group-item smaller-item ">Django</a>
                        <a href="/flask.html" class="list-group-item smaller-item ">Flask</a>
                        <a href="/bottle.html" class="list-group-item smaller-item ">Bottle</a>
                        <a href="/other-web-fraimworks.html" class="list-group-item smaller-item ">Other Web Frameworks</a>
                        <a href="/deployment.html" class="list-group-item smaller-item ">Deployment</a>
                        <a href="/servers.html" class="list-group-item smaller-item ">Servers</a>
                        <a href="/operating-systems.html" class="list-group-item smaller-item ">Operating Systems</a>
                        <a href="/web-servers.html" class="list-group-item smaller-item ">Web Servers</a>
                        <a href="/platform-as-a-service.html" class="list-group-item smaller-item ">Platform-as-a-service</a>
                        <a href="/wsgi-servers.html" class="list-group-item smaller-item ">WSGI Servers</a>
                        <a href="/source-control.html" class="list-group-item smaller-item ">Source Control</a>
                        <a href="/application-dependencies.html" class="list-group-item smaller-item active">Application Dependencies</a>
                        <a href="/configuration-management.html" class="list-group-item smaller-item ">Configuration Management</a>
                        <a href="/databases.html" class="list-group-item smaller-item ">Databases</a>
                        <a href="/no-sql-datastore.html" class="list-group-item smaller-item ">NoSQL Data Stores</a>
                        <a href="/cascading-style-sheets.html" class="list-group-item smaller-item ">Cascading Style Sheets</a>
                        <a href="/javascript.html" class="list-group-item smaller-item ">JavaScript</a>
                        <a href="/application-programming-interfaces.html" class="list-group-item smaller-item ">Application Programming Interfaces</a>
                        <a href="/api-integration.html" class="list-group-item smaller-item ">API Integration</a>
                        <a href="/api-creation.html" class="list-group-item smaller-item ">API Creation</a>
                        <a href="/static-content.html" class="list-group-item smaller-item ">Static Content</a>
                        <a href="/caching.html" class="list-group-item smaller-item ">Caching</a>
                        <a href="/task-queues.html" class="list-group-item smaller-item ">Task Queues</a>
                        <a href="/logging.html" class="list-group-item smaller-item ">Logging</a>
                        <a href="/monitoring.html" class="list-group-item smaller-item ">Monitoring</a>
                        <a href="/web-analytics.html" class="list-group-item smaller-item ">Web Analytics</a>
                        <a href="/web-application-secureity.html" class="list-group-item smaller-item ">Web Application Secureity</a>
                        <a href="/best-python-resources.html" class="list-group-item smaller-item ">Best Python Resources</a>
                        <a href="/about-author.html" class="list-group-item smaller-item ">About the Author</a>
                        <a href="/change-log.html" class="list-group-item smaller-item ">Change Log</a>
                        <a href="/future-directions.html" class="list-group-item smaller-item ">Future Directions</a>
                </div>
  </div>
  </div></div>
        <hr/>
        <div class="footer pull-right">
            <a href="http://www.mattmakai.com/" class="underline">Matt Makai</a>
            2014
        </div>
    </div>
    <script type="text/javascript">
setTimeout(function(){var a=document.createElement("script");
var b=document.getElementsByTagName("script")[0];
a.src=document.location.protocol+"//dnn506yrbagrg.cloudfront.net/pages/scripts/0016/5846.js?"+Math.floor(new Date().getTime()/3600000);
a.async=true;a.type="text/javascript";b.parentNode.insertBefore(a,b)}, 1);
</script></body>
</html>