Fr

ee

Julia is a high performance, high-level dynamic language

designed to address the requirements of high-level
numerical and scientific computing. Julia brings solutions
to the complexities faced by developers while developing
elegant and high performing code.
Julia High Performance will take you on a journey to
understand the performance characteristics of your Julia
programs, and enables you to utilize the promise of near
C levels of performance in Julia.
You will learn to analyze and measure the performance of
Julia code, understand how to avoid bottlenecks, and design
your program for the highest possible performance. In this
book, you will also see how Julia uses type information
to achieve its performance goals, and how to use multiple
dispatch to help the compiler to emit high performance
machine code. Numbers and their arrays are obviously the
key structures in scientific computing you will see how
Julia's design makes them fast. The last chapter will give
you a taste of Julia's distributed computing capabilities.

What you will learn from this book

Get a sense of the possibilities and limitations

of Julia's performance
Analyze the performance of Julia programs
Measure the time and memory taken by
Julia programs
Create fast machine code using Julia's
type information

C o m m u n i t y

Define and call functions without

compromising Julia's performance

E x p e r i e n c e

D i s t i l l e d

Understand number types in Julia

This book is for beginner and intermediate Julia programmers

who are interested in high performance technical computing.
You will have a basic familiarity with Julia syntax, and have
written some small programs in the language.

Get an overview of Julia's distributed

computing capabilities

$ 34.99 US
22.99 UK

Julia High Performance

Avik Sengupta

Use Julia arrays to write high performance

code

P U B L I S H I N G

pl

Discover the secrets behind Julia's speed

Who this book is written for

community experience distilled

Julia High Performance

Julia High Performance

Sa
m

Design and develop high performing programs with Julia

Prices do not include

local sales tax or VAT
where applicable

Visit www.PacktPub.com for books, eBooks,

code, downloads, and PacktLib.

Avik Sengupta

In this package, you will find:

The author biography

A preview chapter from the book, Chapter 1 'Julia is Fast'
A synopsis of the books content
More information on Julia High Performance

About the Author

Avik Sengupta has worked on risk and trading systems in investment banking

for many years, mostly using Java interspersed with snippets of the exotic R and K
languages. This experience left him wondering whether there were better things out
there. Avik's quest came to a happy conclusion with the appearance of Julia in 2012.
He has been happily coding in Julia and contributing to it ever since.

Preface
When I first learned about Julia in early 2012, it was clear to me that this is a
language that I've wanted for many years. The use of multiple dispatch made it
very easy to express mathematical concepts, while the speed of the language made
it feasible to express them in the Julia. I came for the elegance and stayed for the
performance. On the other hand, some users come to Julia for the performance and
stay for the elegance. Either way, in order to fully appreciate the power and beauty
of the language, it needs to live up to its promise of high performance.
I hope this book will help Julia programmers at all levels to learn the design
techniques and paradigms that produce fast Julia code. One of the nice things
about Julia is that its performance characteristics are simple and easy to reason out.
I hope this book will provide you with a framework to think about and analyze the
performance of your own code.

What this book covers

Chapter 1, Julia is Fast, discuses some of the design underpinning the language and its
focus on high performance.
Chapter 2, Analyzing Julia Performance, provides the tools and techniques you can use
to measure and analyze the performance of your own programs.
Chapter 3, Types in Julia, describes the type system and discusses why writing
type-stable code is crucial to high performance.
Chapter 4, Functions and Macros Structuring Julia Code for High Performance, discusses
techniques to use dispatch and code generation to structure high-performance
programs.
Chapter 5, Fast Numbers, discusses the basic numeric types and why they are fast.

Preface

Chapter 6, Fast Arrays, describes ways to use multidimensional arrays in the fastest
possible way.
Chapter 7, Beyond the Single Processor, provides an introduction to Julia's distributed
computing facilities.

Julia is Fast
In many ways, the history of programming languages has often been driven by, and
certainly intertwined, with the needs of numerical and scientific computing. The first
high-level programming language, Fortran, was created with scientific computing
in mind, and continues to be important in the field even to this day. In recent years,
the rise of data science as a specialty has brought additional focus to scientific
computing, particularly for statistical uses. In this area, somewhat counterintuitively,
both specialized languages such as R and general-purpose languages such as Python
are in widespread use. The rise of Hadoop and Spark has spread the use of Java and
Scala respectively among this community. In the midst of all this, Matlab has had
a strong niche within engineering and communities, while Mathematica remains
unparalleled for symbolic operations.
A new language for scientific computing therefore has a very high barrier to overcome.
It's been only a few short years since the Julia language was introduced into the
world. In this time, it's innovative features, which make it a dynamic language, based
on multiple dispatch as its defining paradigm, has created growing niche within the
numerical computing world. However, it's the claim of high performance that excited
its early adopters the most.
This, then, is a book that celebrates writing high-performance programs. With Julia,
this is not only possible, but also reasonably straightforward, within a low-overhead,
dynamic language.

[1]

Julia is Fast

As a reader of this book, you have likely already written your first few Julia
programs. We will assume that you have successfully installed Julia, and have a
working programming environment available. We expect you are familiar with
very basic Julia syntax, but we will discuss and review many of those concepts
throughout the book as we introduce them.

Julia fast and dynamic

Designed for speed

How fast can Julia be?

Julia fast and dynamic

It is a widely believed myth in programming language communities that
high-performance languages and dynamic languages are completely disjoint sets.
The perceived wisdom is that, if you want programmer productivity, you should
use a dynamic language, such as Ruby, Python or R. On the other hand, if you want
fast code execution, you should use a statically typed language such as C or Java.
There are always exceptions to this rule. However, for most mainstream programmers,
this is a strongly held belief.
This usually manifests itself in what is known as the "two language" problem. This is
something that is especially prominent in scientific computing. This is the situation
where the performance-critical inner kernel is written in C, but is then wrapped and
used from a dynamic, higher-level language. Code written in traditional, scientific
computing environments such as R, Matlab or NumPy follows this paradigm.
Code written in this fashion is not without its drawbacks however. Even though it
looks like this gets you the best of both worlds fast computation, while allowing
the programmer to use a high-level language this is a path full of hidden dangers.
For one, someone will have to write the low-level kernel. So, you need two different
skillsets. If you are lucky to find the low level code in C for your project, you are
fine. However, if you are doing anything new or original, or even slightly different
from the norm, you will find yourself writing both C and a high-level language.
This severely limits the number of contributors that your projects or research will
get: to be really productive, they have to be familiar with two languages.

[2]

Chapter 1

Secondly, when running code routinely written in two languages, there can be severe
and unforeseen performance pitfalls. When you can drop down to C code quickly,
everything is fine. However, if, for whatever reason, your code cannot call into a C
routine, you'll find your program taking hundreds or even thousands of times more
longer than you expected.
Julia is the first modern language to make a reasonable effort to solve the "two
language" problem. It is a high-level, dynamic, language with powerful features
that make for a very productive programmer. At the same time, code written in Julia
usually runs very fast, almost as fast as code written in statically typed languages.
The rest of this chapter describes some of the underlying design decisions that
make Julia such a fast language. We also see some evidence of the performance
claims for Julia.
The rest of the book shows you how to write your Julia programs in a way that
optimizes its time and memory usage to the maximum. We will discuss how to
measure and reason performance in Julia, and how to avoid potential performance
pitfalls.
For all the content in this book, we will illustrate our point individually with small
and simple programs. We hope that this will enable you grasp the crux of the issue,
without getting distracted by unnecessary elements of a larger program. We expect
that this methodology will therefore provide you with an instinctive intuition about
Julia's performance profile.
Julia has a refreshingly simple performance model and thus writing fast Julia
code is a matter of understanding a few key elements of computer architecture,
and how the Julia compiler interacts with it. We hope that, by the end of this book,
your instincts are well developed to design and write your own Julia code with the
fastest possible performance.
Versions of Julia
Julia is a fast moving project, with an open development process.
All the code and examples in this book are targeted at version 0.4
of the language, which is the currently released version at the time
of publication. Check Packt's website for changes and errata for
future versions of Julia.

[3]

Julia is Fast

Designed for speed

When the creators of Julia launched the language into the world, they said the
following in a blog post entitled Why We Created Julia, which was published in
early 2012:
"We want a language that's open source, with a liberal license. We want the speed
of C with the dynamism of Ruby. We want a language that's homoiconic, with true
macros like Lisp, but with obvious, familiar mathematical notation like Matlab. We
want something as usable for general programming as Python, as easy for statistics
as R, as natural for string processing as Perl, as powerful for linear algebra as
Matlab, as good at gluing programs together as the shell. Something that is dirt
simple to learn, yet keeps the most serious hackers happy. We want it interactive
and we want it compiled.
(Did we mention it should be as fast as C?)"
High performance, indeed nearly C-level performance, has therefore been one of
the founding principles of the language. It's built from the ground up to enable a
fast execution of code.
In addition to being a core design principle, it has also been a necessity from the early
stages of its development. A very large part of Julia's standard library, including very
basic low-level operations, is written in Julia itself. For example, the + operation to add
two integers is defined in Julia itself. (Refer to: https://github.com/JuliaLang/
julia/blob/1986c5024db36b4c921130351597f5b4f9f81691/base/int.jl#L8).
Similarly, the basic for loop uses the standard iteration mechanism available to all
user-defined types. This means that the implementation had to be very fast from the
very beginning to create a usable language. The creators of Julia did not have the
luxury of escaping to C for even the core elements of the library.
We will note throughout the book many design decisions that have been made with
an eye to high performance. But there are three main elements that create the basis
for Julia's speed.

JIT and LLVM

Julia is a Just In Time (JIT) compiled language, rather than an interpreted one.
This allows Julia to be dynamic, without having the overhead of interpretation.
This compilation infrastructure is build on top of Low Level Virtual Machine
(LLVM) (http://llvm.org).

[4]

Chapter 1

The LLVM compiler without infrastructure project originated at University of

Illinois. It now has contributions from a very large number of corporate as well as
independent developers. As a result of all this work, it is now a very high-quality,
yet modular, system for many different compilation and code generation activities.
Julia uses LLVM for its JIT compilation needs. The Julia runtime generates LLVM
Intermediate Representation (IR) and hands it over to LLVM's JIT compiler, which
in turn generates machine code that is executed on the CPU. As a result, sophisticated
compilation techniques that are built into LLVM are ready and available to Julia, from
the simple (such as Loop Unrolling or Loop Deletion) to state-of-the-art (such as SIMD
Vectorization) ones. These compiler optimizations form a very large body of work, and
in this sense, the existence is of LLVM is very much a pre-requisite to the existence of
Julia. It would have been an almost impossible task for a small team of developers to
build this infrastructure from scratch.
Just-In-Time compilation
Just-in-Time compilation is a technique in which the code in a highlevel language is converted to machine code for execution on the CPU
at runtime. This is in contrast to interpreted languages, whose runtime
executes the source language directly. This usually has a significantly
higher overhead. On the other hand, Ahead of Time (AOT) compilation
refers to the technique of converting source language into machine code
as a separate step prior to running the code. In this case, the converted
machine code can usually be saved to disk as an executable file.

[5]

Julia is Fast

Types
We will have much more to say about types in Julia throughout this book. At this
stage, suffice it to say that Julia's concept of types is a key ingredient in its performance.
The Julia compiler tries to infer the type of all data used in a program, and compiles
different versions of functions specialized to particular types of its arguments. To take
a simple example, consider the sqrt function. This function can be called with integer
or floating-point arguments. Julia will compile two versions of the code, one for integer
arguments, and one for floating point arguments. This means that, at runtime, fast,
straight-line code without any type checks will be executed on the CPU.
The ability of the compiler to reason about types is due to the combination of a
sophisticated dataflow-based algorithm, and careful language design that allows
this information to be inferred from most programs before execution begins. Put
in another way, the language is designed to make it easy to statically analyze.
If there is a single reason for Julia is being such a high-performance language, this
is it. This is why Julia is able to run at C-like speeds while still being a dynamic
language. Type inference and code specialization are as close to a secret sauce as Julia
gets. It is notable that, outside this type inference mechanism, the Julia compiler is
quite simple. It does not include many advanced Just in Time optimizations that
Java and JavaScript compilers are known to use. When the compiler has enough
information about the types within the code, it can generate optimized, straight-line,
code without many of these advanced techniques.
It is useful to note here that unlike some other optionally typed dynamic languages,
simply adding type annotations to your code does not usually make Julia go any faster.
Type inference means that the compiler is, in most cases, able to figure out the types
of variables when necessary. Hence you can usually write high-level code without
fighting with the compiler about types, and still achieve superior performance.

[6]

Chapter 1

How fast can Julia be?

The best evidence of Julia's performance claims is when you write your own code.
However, we can provide an indication of how fast Julia can be by comparing a
similar algorithm over multiple languages.
As an example, let's consider a very simple routine to calculate the power sum for a
series, as follows:

1000
n =1

1
n2

The following code runs this computation in Julia 500 times:

function pisum()
sum = 0.0
for j = 1:500
sum = 0.0
for k = 1:10000
sum += 1.0/(k*k)
end
end
sum
end

You will notice that this code contains no type annotations. It should look quite
familiar to any modern dynamic language. The same algorithm implemented in
C would look something similar to this:
double pisum() {
double sum = 0.0;
for (int j=0; j<500; ++j) {
sum = 0.0;
for (int k=1; k<=10000; ++k) {
sum += 1.0/(k*k);
}
}
return sum;
}

[7]

Julia is Fast

Downloading the example code

You can download the example code files for this book
from your account at http://www.packtpub.com. If you
purchased this book elsewhere, you can visit http://www.
packtpub.com/support and register to have the files
e-mailed directly to you.
You can download the code files by following these steps:

Log in or register to our website using your e-mail

address and password

Let the mouse pointer hover on the SUPPORT tab at

the top

Click on Code Downloads & Errata

Enter the name of the book in the Search box

Select the book for which you're looking to download

the code files

Choose from the drop-down menu where you

purchased this book from

Click on Code Download

You can also download the code files by clicking on the Code
Files button on the book's webpage at the Packt Publishing
website. This page can be accessed by entering the book's
name in the Search box. Please note that you need to be
logged in to your Packt account.
Once the file is downloaded, please make sure that you unzip
or extract the folder using the latest version of:

WinRAR/7-Zip for Windows

Zipeg/iZip/UnRarX for Mac

7-Zip/PeaZip for Linux

By timing this code, and its re-implementation in many other languages (all of which
are available at https://github.com/JuliaLang/julia/tree/master/test/
perf/micro), we can note that Julia's performance claims are certainly borne out in
this limited test. Julia can perform at a level similar to C and other statically typed
and compiled languages.
This is of course a micro benchmark, and should therefore not be extrapolated
too much. However, I hope you will agree that it is possible to achieve excellent
performance in Julia. The rest of the book will attempt to show how you can
achieve performance close to this standard in your code.

[8]

Chapter 1

Summary
In this chapter, you noted that Julia is a language that is built from the ground up
for high performance. Its design and implementation have always been focused on
providing the highest possible performance on the modern CPU.
The rest of the book will show you how to use the power of Julia to the maximum,
to write the fastest possible code in this language. In the next chapter, we will discuss
how to measure the speed of Julia code, and identify performance bottlenecks.
You will learn some of the tools that are built into Julia for this purpose.

[9]

Get more information Julia High Performance

Where to buy this book

You can buy Julia High Performance from the Packt Publishing website.
Alternatively, you can buy the book from Amazon, BN.com, Computer Manuals and most internet
book retailers.
Click here for ordering and shipping details.

www.PacktPub.com

Stay Connected: