University of Maryland,Baltimore

County

Signal Processing for Big Data

Class Project

MRI Image Segmentation

Author:
Krishna Dontaraju

May 28, 2017

Contents
1 Introduction 2

2 Related Work and Motivation 2

3 Methodology 2

4 Problem Formulation 3
4.1 KSVD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
4.2 kernal KSVD . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

5 Dataset 4

6 Implementation 5

7 Results 5

8 Discussion 8

9 Appendix 9
9.1 Proof of D= Φ(Y )A . . . . . . . . . . . . . . . . . . . . . . . . 9
9.2 KOMP algorithm[5] . . . . . . . . . . . . . . . . . . . . . . . . 9

1
 Abstract
In this project, two algorithms have been implemented to address
the issue of brain image segmentation. The include a KSVD based
dictionary learning algorithm and its kernelized version.

1 Introduction
Identifying tumors in brain scan is currently being done manually by radiol-
ogists. According to Lee et al [1] the manual segmentation could take upto
3.5 hours. The authors Vaidyanathan et al [2] contend that it is often hard
and unreliable due to multiple interpretation of MRI by different people.
Thus, automating this process of brain tumor segmentation could be benefi-
cial to the diagnosis and treatment of tumors. There are multiple challenges
involved in this process. The tumor tissue and normal tissue have similar
pixel intensities. Also the tumors vary in the shape, size and locations of the
tumors making it further challenging [1].

2 Related Work and Motivation

A lot of research has been conducted in this area which is briefly summarized
in BRATS[2]. According to the paper, the algorithms currently being used
can be classified into two groups-Generative and discriminative. The former
use “domain specific prior knowledge” to segment the images, which are
difficult to model. The latter, however, focuses on developing localized image
features in identifying tumors directly from data, making them sensitive to
variations in MRI scanning methods. The authors in paper [1] have obtained
good results using discriminative Dictionary Learning(DL) algorithm. Per
them, the plain DL Algorithms can be improved by using a kernel to deal
with non-linear features. The papers [3], [4] and [6] provide some insights in
designing a kernel for the DL algorithm. This project tries to investigate the
performance of dictionary learning algorithms on MRI scans.

3 Methodology
The goal of the project is to evaluate the performance of kernel KSVD al-
gorithm over standard DL algorithm. Initially the datasets weree tested on

2
KSVD agorithm described by Aharon et al[4] then on kernel KSVD algorithm
presented by Nguyen et al[].

4 Problem Formulation
4.1 KSVD
Given yi N signals of set Y = [y1 , ...yN ]
The Goal of the algorithm is to minimize the representational error.

(D̂, X̂) = arg min kY − DXk2F s.t.

D,X

kxi k0 ≤ T0 , ∀i = 1...N
where xi i-th column of X, T0 Sparsity level.
This problem is solved in alternating stages of Sparse coding and Dictionary
Update. In the first stage the sparse coefficients are computed using a pursuit
algorithm like OMP. Then the dictionary is updated one atom at a time
based on singular value decomposition. The problem with this algorithm is
it cannot represent non linear data structures. To overcome this problem a
kernal version of the same algorithm has been proposed by Nguyen et al[5]

4.2 kernal KSVD

The input signal is mapped to higher dimensional space.
Given signal Φ ∈ Rn 7→ F ⊂ Rñ
Goal of the algorithm is to learn a non-linear dictionary D = [d1 , ..., dK ] in
the feature space F

arg min kΦ(Y ) − DXk2F s.t.

D,X

kxi k0 ≤ T0 , kdj k2 = 1, ∀i, j.

Where Φ(Y ) = [Φ(y1 ), .., Φ(yn )], D ∈ RñxK
X ∈ RKxN is a matrix whose i-th column is the sparse vector xi corresponding
to the sample Φ(yi ) with sparsity T0 . F represents dot product feature space(
Hilbert space)
This problem cannot be solved using KSVD since F can be infinitely large.
To solve this problem a kernel trick is employed. which is, the inner products

3
of a pair of signal in feature, is equal to, the output of ”a non-linear positive
semidefinte kernel function” applied on the corresponding signals in the input
space. This means the inner products of the feature space can be calculated
without actually mapping the data into feature space, by just finding the
values of kernel in the input space.
So, now by alternating between sparse coding and dictionary update stages,
we can solve this problem. However, there is another hurdle in solving this
problem. We can not find the reverse mapping between the updated inner
products and the input space directly. So, the dictionary D is split into a fixed
part Φ(Y ) and a variable part A. The variable part is called the ”coefficient
dictionary”[6]. This gives us a new formulation to our problem

arg min kΦ(Y ) − Φ(Y )AXk2F s.t.

A,X

kxi k0 ≤ T0 , kdj k2 = 1, ∀i, j.

First X is updated using a pursuit algorithm like KOMP by fixing A , and
then by applying svd, A is updated using a fixed X.

5 Dataset
Two datasets have been used to evaluate the performance of the algorithm.
The first dataset contains 60,000 training samples and 10,000 testing samples
of handwritten digits from 0 to 9. The second dataset contains fully anno-
tated MRI scans of 30 patients taken in four different modalities FLAIR, T1,
T1c and T2. This includes 15 patients with High grade glioma and another
fifiteen with low grade glioma. It also contains 50 synthetic MRI scan sets,
developed by a software. The annotations for real MRI scans include 5 labels
indicating ”biological interpretations” of the region[7] which are 1.necrosis,
2.edema, 3.non enhancing tumor, 4.enhancing tumor, 5.everything else. Dif-
ferent modalities provide specific ”biological information”[7] of a particular
region of the brain.

4
6 Implementation
All initial experiments have been performed on MNIST datasets to bench-
mark the algorithms. 50,000 images have been used for training, 10,000 for
validation and remaining 10,000 for testing. Subsequently tests were done on
the brats database. For the purpose of this project, owing to the enormity of
each MRI scan and time taken to evaluate each algorithm, only limited parts
of the database have been utilized. This includes FLAIR MRI of 3 patients
with high grade glioma for training, one patients scan for validation and one
patient scan for testing. Overlapping patches of size 5*5*5=125 around each
voxel are formed as feature vector. Then kmeans was applied on the training
set to get 2000 centroids. These are then input to the algorithms. 4000 vox-
els from each class label of the test MRI were used for testing the algorithm.
For the MNIST dataset Dictionary sizes(K) of 500, 1000, 1500 and sparsity
values(T) of 3, 5, and 10 have been tested. While for the BRATS dataset
dictionary size of 500, 1000 and sparsity value of 5 were tested.

7 Results
Reconstructive accuracy was used as parameter for performance evaluation.
The results of the MNIST dataset have been tabulated in table 1. From the
table we can see that kernal ksvd clearly outperforms ksvd in all the runs.
Increasing dictionary size improved performance of both algorithms, while
with increasing sparsity, the performance flatten after 5. The results of kernal
ksvd have been tabulated in table 2. For ksvd algorithm the total accuracy is
around 50 percent, however, if we take individual class accuracy values into
account, the accuracy of class1 and class 2 together is around 80 percent.
This is expected since, only one modality was used for testing only certain
classes of the tumor would flare up, hence those classes show higher accuracy.
For kernal ksvd only one class seems to form leading to an accuracy of 25
percent. This could mean it requires more tuning of parameters to validate
the results.

5
 run ksvd kernel ksvd
K=500,T=3 94.8 97.3
K=500 T=5 96.4 98.4
K=500 T=10 96.3 98.9
K=1000,T=3 94.9 97.8
K=1000 T=5 96.2 98.5
K=1000 T=10 96.4 98.3

Table 1: Variation of accuracy for different values of K and T for MNIST

run ksvd(1+2,3+4) kernel ksvd(1+2,3+4)

K=500,T=5 51.91(79.75,24.08) 25(50,0)
K=1000 T=5 52.13(78.65, 25.60) 25(50,0)

Table 2: Variation of accuracy for different values of K for BRATS

6
7
8 Discussion
Due to the heavy computational burden of the algorithm on the system, a
comprehensive evaluations could not be conducted on MRI data. To avoid
this situations faster versions of the algorithms like ”online kmeans” and
”online kernel dictionary learning” need to be explored.

8
9 Appendix
9.1 Proof of D= Φ(Y )A
Using the standard orthogonal decomposition, we can write D∗

D∗ = D̃ + D⊥

where D̃ and D⊥ are matrices whose columns lie within and orthogonal to col-
umn subspace of Φ(Y ), respectively. In other words, D̃ = Φ(Y )AandΦ(Y )T D⊥ =
0

kΦ(Y )−D∗ X ∗ k2F = tr[(I−AX ∗ )T Φ(Y )T Φ(Y )(I−AX ∗ )]+tr[(X ∗ )T D⊥

T
D⊥ X ∗ ]

since the second term is a ”positive semidefinite matrix”, it will only increase
the cost function which means D⊥ = 0, D∗ = D̃ = Φ(Y )A is an optimal
solution

9.2 KOMP algorithm[5]

9
References
[1] Lee, Jeon, et al. Brain tumor image segmentation using kernel dictionary
learning. Engineering in Medicine and Biology Society (EMBC), 2015
37th Annual International Conference of the IEEE. IEEE, 2015.

[2] Vaidyanathan, M., et al. ”Monitoring brain tumor response to therapy

using MRI segmentation.” Magnetic resonance imaging 15.3 (1997): 323-
334.

[3] Kim, Seung-Jun. ”Online kernel dictionary learning.” In Signal and In-
formation Processing (GlobalSIP), 2015 IEEE Global Conference on, pp.
103-107. IEEE, 2015.

[4] Michal Aharon, Michael Elad, and Alfred Bruckstein. ”K-SVD: An Al-
gorithm for Designing Overcomplete Dictionaries for Sparse Representa-
tion.” IEEE Transactions on Signal Processing 54, no. 11 (2006): 4311-
23.

[5] Van Nguyen, Hien, et al. ”Design of non-linear kernel dictionaries for ob-
ject recognition.” IEEE Transactions on Image Processing 22.12 (2013):
5123-5135.

[6] Golts, Alona, and Michael Elad. ”Linearized kernel dictionary learning.”
IEEE Journal of Selected Topics in Signal Processing 10.4 (2016): 726-
739.

[7] Ayache, N., N. Cordier, and H. Delingette. ”The Multimodal Brain Tu-
mor Image Segmentation Benchmark (BRATS).” (2014).

[8] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. ”Gradient-based learning

applied to document recognition.” Proceedings of the IEEE, 86(11):2278-
2324, November 1998.

[9] code adapted from ”http://www.cs.technion.ac.il/ elad/software/”

[10] code adapted from ”http://www.cs.technion.ac.il/ elad/software/”

10