Introduction to Julia:

Why are we doing this to you?

(Spring 2019)
Steven G. Johnson, MIT Applied Math

MIT classes 18.06, 18.303, 18.330, 18.08[56],

18.335, 18.337, …

1
 What language for teaching
scientific computing?
For the most part, these are not hard-core programming courses,
and we only need little “throw-away” scripts and toy numerical experiments.

Almost any high-level, interactive (dynamic) language

with easy facilities for linear algebra (Ax=b, Ax=λx),
plotting, mathematical functions, and working with
large arrays of data would be fine.

And there are lots of choices…

2
Lots of choices for interactive math…

3
Image courtesy of Viral Shah. Used with permission.
 Just pick the most popular?
Matlab or Python or R?

We feel guilty pushing a language on

you that we
are starting to abandon ourselves.

Traditional HL computing languages

hit a performance wall in “real” work
… eventually force you to C, Cython, …
4
 A new programming language?
Jeff Bezanson

julialang.org
Viral Shah

Alan Edelman (MIT) [begun 2009, “0.1” in 2013, ~40k commits,

1.0 release in Aug. 2018, 1.1 in Jan. 2019 ]

Stefan Karpinski
[ 30+ developers with 100+ commits,
1000+ external packages, 6th JuliaCon in 2019 ]

As high-level and interactive as Matlab or Python+IPython,

as general-purpose as Python,
as productive for technical work as Matlab or Python+SciPy,
but as fast as C.
5
Performance on synthetic benchmarks
[ loops, recursion, etc., implemented in most straightforward style ]

(normalized so that C speed = 1) 6

 Special Functions in Julia
Special functions s(x): classic case that cannot be vectorized well
… switch between various polynomials depending on x

Many of Julia’s special functions come from the usual C/Fortran libraries,
but some are written in pure Julia code.

Pure Julia erfinv(x) [ = erf–1(x) ]

3–4× faster than Matlab’s and 2–3× faster than SciPy’s (Fortran Cephes).
Pure Julia polygamma(m, z) [ = (m+1)th derivative of the ln Γ function ]
~ 2× faster than SciPy’s (C/Fortran) for real z
… and unlike SciPy’s, same code supports complex argument z

Julia code can actually be faster than typical “optimized”

C/Fortran code, by using techniques
[metaprogramming/codegen generation] that are
hard in a low-level language. 7
 Pure-Julia FFT performance
double-precision complex, 1d transforms
powers of two
14000
intel-mkl-dfti in-place
13000 intel-mkl-dfti out-of-place

12000
(FFTW, MKL: fftw3 out-of-place
fftw3 in-place
“unfair” factor of ~2 fftw3-no-simd out-of-place
11000
from manual SIMD) fftw3-no-simd in-place
dfftpack
10000
emayer
9000 julia
bloodworth
speed (mflops)

8000 cross
cwplib
7000 esrfft

6000
5000 K FFTW already comparable to FFTPACK
TPAC w/o SIMD
FF
4000 Ju lia
3000
[ probably some tweaks to
inlining will make it better ]
2000
1000
0
FFTW 1.0-like code generation
+ recursion in Julia
2
4
8
16
32
64
128
256
512
1024
2048
4096
8192
16384
32768
65536
131072
262144

~ 1/3 lines of code compared to

FFTPACK, more functionality
8
 Generating Vandermonde matrices
given x = [α1, α2, …], generate: NumPy (numpy.vander): [follow links]

Python code …wraps C code

… wraps generated C code

type-generic at high-level, but

low level limited to small set of types.

Writing fast code “in” Python or Matlab = mining the standard library
for pre-written functions (implemented in C or Fortran).

If the problem doesn’t “vectorize” into built-in functions,

if you have to write your own inner loops … sucks for you.

9
 Generating Vandermonde matrices
given x = [α1, α2, …], generate: NumPy (numpy.vander): [follow links]

Python code …wraps C code

… wraps generated C code

type-generic at high-level, but

low level limited to small set of types.
Julia (type-generic code):
function vander(x, n=length(x))
m = length(x)
V = Array(eltype(x), m, n)
for j = 1:m
V[j,1] = one(x[j])
end
for i = 2:n
for j = 1:m
V[j,i] = x[j] * V[j,i-1]
end
end
return V
end
10
Generating Vandermonde matrices
function vander(x, n=length(x))
m = length(x)
V = Array(eltype(x), m, n)
note: works for any container
for j = 1:m
V[j,1] = one(x[j]) of any type with “*” operation
end … performance ≠ inflexibility
for i = 2:n
for j = 1:m
V[j,i] = x[j] * V[j,i-1]
end
end
return V
end

11
 But I don’t “need” performance!
For lots of problems, especially “toy” problems in courses,
Matlab/Python performance is good enough.

But if use those languages for all of your “easy” problems,

then you won’t be prepared to switch when you hit a hard
problem. When you need performance, it is too late.

You don’t want to learn a new language at the same time

that you are solving your first truly difficult computational
problem.

12
 Just vectorize your code?
= rely on mature external libraries,
operating on large blocks of data,
for performance-critical code

Good advice! But…

• Someone has to write those libraries.

• Eventually that person will be you.

— some problems are impossible or
just very awkward to vectorize.

13
 But everyone else is using
Matlab/Python/R/…

Julia is still a young, niche language. That

imposes real costs — lack of familiarity,
rough edges, continual language changes.
These are real obstacles.

But it also gives you advantages that

Matlab/Python users don’t have.

14
But I lose access to all the libraries
available for other languages?

Very easy to call C/Fortran libraries from

Julia, and also to call Python…

15
 Julia leverages Python…
Directly call Python libraries (PyCall package),
e.g. to plot with Matplotlib (PyPlot package), and also…

via IPython/Jupyter:

Modern multimedia
interactive notebooks
mixing code, results,
graphics, rich text,
equations, interaction

“IJulia”
© Project Jupyter. All rights reserved. This content is excluded from our Creative 16
Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use.
 goto live IJulia notebook demo…

Go to juliabox.org for install-free IJulia on the Amazon cloud

See also julialang.org for more tutorial materials…

17
MIT OpenCourseWare
https://ocw.mit.edu

18.335J Introduction to Numerical Methods

Spring 2019

For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.