Table of Contents

Mastering Clojure

Credits

About the Author

About the Reviewer

www.PacktPub.com

eBooks, discount offers, and more

Why subscribe?

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Working with Sequences and Patterns

Defining recursive functions

Thinking in sequences

Using the seq library

Creating sequences

Transforming sequences

Filtering sequences

Lazy sequences

Using zippers

Working with pattern matching

Summary

2. Orchestrating Concurrency and Parallelism

Managing concurrent tasks

Using delays

Using futures and promises

Managing state

Using vars

Using refs

Using atoms

Using agents

Executing tasks in parallel

Controlling parallelism with thread pools

Summary

3. Parallelization Using Reducers

Using reduce to transform collections

What's wrong with sequences?

Introducing reducers

Using fold to parallelize collections

Processing data with reducers

Summary

4. Metaprogramming with Macros

Understanding the reader

Reading and evaluating code

Quoting and unquoting code

Transforming code

Expanding macros

Creating macros

Encapsulating patterns in macros

Using reader conditionals

Avoiding macros

Summary

5. Composing Transducers

Understanding transducers

Producing results from transducers

Comparing transducers and reducers

Transducers in action

Managing volatile references

Creating transducers

Summary

6. Exploring Category Theory

Demystifying category theory

Using monoids

Using functors

Using applicative functors

Using monads

Summary

7. Programming with Logic

Diving into logic programming

Solving logical relations

Combining logical relations

Thinking in logical relations

Solving the n-queens problem

Solving a Sudoku puzzle

Summary

8. Leveraging Asynchronous Tasks

Using channels

Customizing channels

Connecting channels

Revisiting the dining philosophers problem

Using actors

Creating actors

Passing messages between actors

Handling errors with actors

Managing state with actors

Comparing processes and actors

Summary

9. Reactive Programming

Reactive programming with fibers and dataflow variables

Using Reactive Extensions

Using functional reactive programming

Building reactive user interfaces

Introducing Om

Summary

10. Testing Your Code

Writing tests

Defining unit tests

Using top-down testing

Testing with specs

Generative testing

Testing with types

Summary

11. Troubleshooting and Best Practices

Debugging your code

Using tracing

Using Spyscope

Logging errors in your application

Thinking in Clojure

Summary

A. References

Index

Mastering Clojure

Mastering Clojure

Copyright © 2016 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: March 2016

Production reference: 1180316

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78588-974-5

www.packtpub.com

Credits

Author

Akhil Wali

Reviewer

Matt Revelle

Commissioning Editor

Neil Alexander

Acquisition Editor

Aaron Lazar

Content Development Editor

Aishwarya Pandere

Technical Editor

Tanmayee Patil

Copy Editor

Merilyn Pereira

Project Coordinator

Nidhi Joshi

Proofreader

Safis Editing

Indexer

Rekha Nair

Graphics

Jason Monteiro

Production Coordinator

Melwyn Dsa

Cover Work

Melwyn D'sa

About the Author

Akhil Wali is a software developer. He has been writing code as a hobbyist since 1997 and professionally since 2010. He completed his post graduation from Santa Clara University in 2010, and he graduated from Visvesvaraya Technological University in 2008. His areas of work include business intelligence systems, ERP systems, search engines, and document collaboration tools. He mostly works with Clojure, JavaScript, and C#. Apart from computers, his interests include soccer, guitar solos, and finding out more about the universe.

I would like to thank two important women in my life for supporting me and inspiring me to write this book—my mother Renuka and my wife Megha. I also thank Matt Revelle and the Clojure community for their fantastic input and ideas.

About the Reviewer

Matt Revelle is a doctoral candidate in computer science at George Mason University, where he works on machine learning and social network dynamics. He started using Clojure in 2008 and it continues to be his preferred language for many projects and his daily work.

I would like to thank Akhil Wali and Nidhi Joshi for helping me to understand the purpose of this book and encouraging me to provide feedback in a timely fashion.

www.PacktPub.com

eBooks, discount offers, and more

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Preface

Ever since the dawn of computers decades ago, there have been a number of programming languages created for the purpose of writing software. One of the earliest of these languages is Lisp, whose name is an abbreviation of the term list processing. Lisp has evolved greatly over time, and there are now several dialects of Lisp. Each of these dialects emphasizes its own set of ideas and features. Clojure is one among these Lisps, and it focuses on immutability, concurrency, and parallelism. It emphasizes being simple, practical, and intuitive, which makes it easy to learn. It is said that you have never realized how a language can be powerful until you have programmed in a Lisp, and Clojure is no exception to this rule. A skilled Clojure programmer can easily and quickly create software that is both performant and scalable.

With the recent rise of parallel data processing and multicore architectures, functional programming languages have become more popular for creating software that is both provable and performant. Clojure brings functional programming to the Java Virtual Machine (JVM), and also to web browsers through ClojureScript. Like other functional programming languages, Clojure focuses on the use of functions and immutable data structures for writing programs. Clojure also adds a hint of Lisp through the use of symbolic expressions and a dynamic type system.

This book will walk you through the interesting features of the Clojure language. We will also discuss some of the more advanced and lesser known programming constructs in Clojure. Several libraries from the Clojure ecosystem that we can put to practical use in our own programs will also be described. You won't need to be convinced any more about the elegance and power of the Clojure language by the time you've finished this book.

This book wouldn't have materialized without the feedback from the technical reviewers and the effort of the content and editing teams at Packt Publishing.

What this book covers

Chapter 1, Working with Sequences and Patterns, describes several elementary programming techniques, such as recursion, sequences, and pattern matching.

Chapter 2, Orchestrating Concurrency and Parallelism, explains the various constructs available in the Clojure language for concurrent and parallel programming.

Chapter 3, Parallelization Using Reducers, introduces reducers, which are abstractions of collection types for parallel data processing.

Chapter 4, Metaprogramming with Macros, explains how we can use macros and quoting to implement our own programming constructs in Clojure.

Chapter 5, Composing Transducers, describes how we can define and compose data transformations using transducers.

Chapter 6, Exploring Category Theory, explores algebraic data structures, such as functors, monoids, and monads, from the pure functional programming world.

Chapter 7, Programming with Logic, describes how we can use logical relations to solve problems.

Chapter 8 , Leveraging Asynchronous Tasks, explains how we can write code that is executed asynchronously.

Chapter 9, Reactive Programming, describes how we can implement solutions to problems using asynchronous event streams.

Chapter 10, Testing Your Code, covers several testing libraries that are useful in verifying our code. This chapter describes techniques such as test-driven development, behavior-driven development, and generative testing.

Chapter 11, Troubleshooting and Best Practices, describes techniques to debug your code as well as several good practices for developing Clojure applications and libraries.

What you need for this book

One of the pieces of software required for this book is the Java Development Kit (7 or above), which you can obtain from http://www.oracle.com/technetwork/java/javase/downloads/. JDK is necessary to run and develop applications on the Java platform. The other major software that you'll need is Leiningen (2.5.1 or above), which you can download and install from http://github.com/technomancy/leiningen.

Leiningen is a tool for managing Clojure projects and their dependencies. Throughout this book, we'll use a number of Clojure libraries. Leiningen will download these libraries, and also the Clojure language itself, for us, as required.

You'll also need a text editor or an integrated development environment (IDE). If you already have a text editor that you prefer, you can probably use it. Navigate to http://dev.clojure.org/display/doc/Getting+Started for a list of environment-specific plugins to write code in Clojure. If you don't have a preference, it is suggested that you use Eclipse with Counterclockwise (http://doc.ccw-ide.org/) or Light Table (http://lighttable.com/).

Some examples in this book will also require a web browser, such as Chrome (42 or above), Firefox (38 or above), or Microsoft Internet Explorer (9 or above).

Who this book is for

This book is for programmers or software architects who are familiar with Clojure and want to learn about the language's features in detail. It is also for readers who are eager to explore popular and practical Clojure libraries.

This book does not describe the syntax of the Clojure language. You are expected to be familiar with the language, but you need not be a Clojure expert. You are also expected to know how functions are used and defined in Clojure, and have some basic knowledge about Clojure data structures such as strings, lists, vectors, maps, and sets. You must also be able to compile ClojureScript programs to JavaScript and run them in an HTML page.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: Hence, for trees that are represented as sequences, we should use the seq-zip function instead.

A block of code is set as follows:

(f/defun fibo

  ([0] 0N)

  ([1] 1N)

  ([n] (+ (fibo (- n 1))

          (fibo (- n 2)))))

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

(f/defun fibo

  ([0] 0N)

  ([1] 1N)

  ([n] (+ (fibo (- n 1))           (fibo (- n 2)))))

Any command-line input or output is written as follows:

$ lein repl

Another simple convention that we use is to always show the Clojure code that's entered in the REPL (read-evaluate-print-loop) starting with the user> prompt. In practice, this prompt will change depending on the Clojure namespace that we are currently using. However, for simplicity, code in the REPL always starts with the user> prompt in this book, as follows:

user> (cons 0 ())

(0)

user> (cons 0 nil)

(0)

user> (rest (cons 0 nil))

()

For convenience, the REPL output in this book is pretty-printed (using the clojure.pprint/pprint function). Objects that are printed in the REPL output are enclosed within the #< and > symbols. We must note that the output of the time form in your own REPL may not completely match the output shown in the code examples of this book. Rather, the use of time forms is meant to give you an idea of the scale of the time taken to execute a given expression. Similarly, the output of the code examples that use the rand-int function may not exactly match the output in your REPL.

Some examples in this book use ClojureScript, and the files for these examples will have a .cljs extension. Also, all macros used in these examples will have to be explicitly included using the :require-macros clause of the ns form. The HTML and CSS files associated with the ClojureScript examples in this book will not be shown in this book, but can always be found in the book's code bundle.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail <feedback@packtpub.com>, and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

You can download the code files by following these steps:

Log in or register to our website using your e-mail address and password.

Hover the mouse pointer on the SUPPORT tab at the top.

Click on Code Downloads & Errata.

Enter the name of the book in the Search box.

Select the book for which you're looking to download the code files.

Choose from the drop-down menu where you purchased this book from.

Click on Code Download.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

WinRAR / 7-Zip for Windows

Zipeg / iZip / UnRarX for Mac

7-Zip / PeaZip for Linux

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

Piracy

Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at <copyright@packtpub.com> with a link to the suspected pirated material.

We appreciate your help in protecting our authors and our ability to bring you valuable content.

Questions

If you have a problem with any aspect of this book, you can contact us at <questions@packtpub.com>, and we will do our best to address the problem.

Chapter 1. Working with Sequences and Patterns

In this chapter, we'll revisit a few basic programming techniques, such as recursion and sequences, with Clojure. As we will see, Clojure focuses on the use of higher-order functions to abstract computation, like any other functional programming language. This design can be observed in most, if not all, of the Clojure standard library. In this chapter, we will cover the following topics:

Exploring recursion

Learning about sequences and laziness

Examining zippers

Briefly studying pattern matching

Defining recursive functions

Recursion is one of the central methodologies of computer science. It allows us to elegantly solve problems that have cumbersome non-recursive solutions. Yet, recursive functions are discouraged in quite a few imperative programming languages in favor of non-recursive functions. Clojure does no such thing and completely embraces recursion along with all its pros and cons. In this section, we will explore how to define recursive functions.

Note

The following examples can be found in src/m_clj/c1/recur.clj of the book's source code.

In general, a function can be made recursive by simply calling it again from within the body of the function. We can define a simple function to return the first n numbers of the Fibonacci sequence as shown in Example 1.1:

(defn fibo

  ([n]

  (fibo [0N 1N] n))

  ([xs n]

  (if (<= n (count xs))

    xs

    (let [x' (+ (last xs)

                (nth xs (- (count xs) 2)))

          xs' (conj xs x')]

      (fibo xs' n)))))

Example 1.1: A simple recursive function

Note

The Fibonacci sequence is a series of numbers that can be defined as follows:

The first element F0 is 0 and the second element F1 is 1.

The rest of the numbers are the sum of the previous two numbers, that is the nth Fibonacci number Fn = Fn-1 + Fn-2.

In the previously defined fibo function, the last two elements of the list are determined using the nth and last functions, and the sum of these two elements is appended to the list using the conj function. This is done in a recursive manner, and the function terminates when the length of the list, determined by the count function becomes equal to the supplied value n. Also, the values 0N and 1N, which represent BigInteger types, are used instead of the values 0 and 1.This is done because using long or integer values for such a computation could result in an arithmetic overflow error. We can try out this function in the REPL shown as follows:

user> (fibo 10)

[0N 1N 1N 2N 3N 5N 8N 13N 21N 34N]

user> (last (fibo 100))

218922995834555169026N

The fibo function returns a vector of the first n Fibonacci numbers as expected. However, for larger values of n, this function will cause a stack overflow:

user> (last (fibo 10000))

StackOverflowError  clojure.lang.Numbers.lt (Numbers.java:219)

The reason for this error is that there were too many nested function calls. A call to any function requires an additional call stack. With recursion, we reach a point where all of the available stack space in a program is consumed and no more function calls can be performed. A tail call can overcome this limitation by using the existing call stack for a recursive call, which removes the need for allocating a new call stack. This is only possible when the return value of a function is the return value of a recursive call made by the function, in which case an additional call stack is not required to store the state of the function that performs the recursive call. This technique is termed as tail call elimination. In effect, a tail call optimized function consumes a constant amount of stack space.

In fact, the fibo function does indeed make a tail call, as the last expression in the body of the function is a recursive call. Still, it consumes stack space for each recursive call. This is due to the fact that the underlying virtual machine, the JVM, does not perform tail call elimination. In Clojure, tail call elimination has to be done explicitly using a recur form to perform a recursive call. The fibo function we defined earlier can be refined to be tail recursive by using a recur form, as shown in Example 1.2:

(defn fibo-recur

  ([n]

  (fibo-recur [0N 1N] n))

  ([xs n]

  (if (<= n (count xs))

    xs

    (let [x' (+ (last xs)

                (nth xs (- (count xs) 2)))

          xs' (conj xs x')]

      (recur xs' n)))))

Note

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed to you.

Effectively, the fibo-recur function can perform an infinite number of nested recursive calls. We can observe that this function does not blow up the stack for large values of n, shown as follows:

user> (fibo-recur 10)

[0N 1N 1N 2N 3N 5N 8N 13N 21N 34N]

user> (last (fibo-recur 10000))

207936...230626N

We should note that a call to fibo-recur can take quite a while to terminate for large values of n. We can measure the time taken for a call to fibo-recur to complete and return a value, using the time macro, as follows:

user> (time (last (fibo-recur 10000)))

Elapsed time: 1320.050942 msecs

207936...230626N

The fibo-recur function can also be expressed using the loop and recur forms. This eliminates the need for using a second function arity to pass the [0N 1N] value around, as shown in the fibo-loop function defined in Example 1.3:

(defn fibo-loop [n]

  (loop [xs [0N 1N]

        n n]

    (if (<= n (count xs))

      xs

      (let [x' (+ (last xs)

                  (nth xs (- (count xs) 2)))

            xs' (conj xs x')]

        (recur xs' n)))))

Example 1.3: A recursive function defined using loop and recur

Note that the loop macro requires a vector of bindings (pairs of names and values) to be passed as its first argument. The second argument to the loop form must be an expression that uses the recur form. This nested recur form calls the surrounding expression recursively by passing in the new values for the declared bindings in the loop form. The fibo-loop function returns a value that is equal to that returned by the fibo-recur function, from Example 1.2, shown as follows:

user> (fibo-loop 10)

[0N 1N 1N 2N 3N 5N 8N 13N 21N 34N]

user> (last (fibo-loop 10000))

207936...230626N

Another way to handle recursion is by using the trampoline function. The trampoline function takes a function as its first argument, followed by the values of the parameters to be passed to the supplied function. A trampoline form expects the supplied function to return another function, and in such a case, the returned function will be invoked. Thus, a trampoline form manages recursion by obtaining a return value, and invoking the returned value again if it's a function. Thus, the trampoline function avoids using any stack space. Each time the supplied function is invoked, it returns and the result gets stored in the process heap. For example, consider the function in Example 1.4 that calculates the first n numbers of the Fibonacci sequence using a trampoline:

(defn fibo-trampoline [n]

  (letfn [(fibo-fn [xs n]

            (if (<= n (count xs))

              xs

              (let [x' (+ (last xs)

                          (nth xs (- (count xs) 2)))