EECE 5642

Data Visualization

Data Representation

Y. Raymond Fu
Professor
Electrical and Computer Engineering (ECE), COE
College of Computer and Information Science (CCIS)
Northeastern University
 Data Visualization
exists everywhere!

Pictures from Google Images

Courtesy of Zhiqiang Tao

Example 1: Visualizing a Zipcode

http://benfry.com/zipdecode/

Ben Fry, http://benfry.com/

9****

0****
02***

021**
 Example 2: Visualizing a Music

Viva la Vida
(MUSIC VIDEO)

Mp3 wma rm …

Dancing Particles – Reflection

http://www.youtube.com/watch?v=-bdKMT1znJ0
http://www.whitedots.net/2013/07/29/tutorial-dancing-particles-
reflection/ http://www.youtube.com/watch?v=9yeGHyRavAA&feature=related
 Dancing Particles – Reflection

https://www.youtube.com/watch?v=-bdKMT1znJ0
 Viva la Vida (MUSIC VIDEO)

https://www.youtube.com/watch?v=LSNyWGc0gsU
Example 3: Visualizing 200 countries, 200 years

https://www.youtube.com/watch?v=jbkSRLYSojo
Example 4: Visualizing TV Series

https://www.youtube.com/watch?v=7DemM7UGmIg
 Visualization Pipeline
Collection Data table can be Rendering components
• a table of figures, network, graphs draw the contents of the
• a social network graph, and trees visual abstraction into any
• a file directory structure, number of interactive
• or any other data set. views

mouse and
keyboard
feedback
varying
• a data model that
Internal representations perspectives
includes visual
• reading in the data from a panning and
features such as
formatted file or database, zooming,
spatial layout, color,
• could potentially involve snapshots
size, and shape
transformations.
• Information to
draw

http://prefuse.org/doc/manual/introduction/structure/
 Seven Stages of Visualizing Data

• Acquire: Obtain the data, whether from a file on a disk or a

source over a network.
• Parse: Provide some structure for the data’s meaning, and
order it into categories.
• Filter: Remove all but the data of interest.
• Mine: Apply methods from statistics or data mining as a way
to discern patterns or place the data in mathematical context.
• Represent: Choose a basic visual model, such as a bar graph,
list or tree.
• Refine: Improve the basic representation to make it clearer
and more visually engaging.
• Interact: Add methods for manipulating the data or
controlling what features are visible.

Visualizing Data, Ben Fry, O‘Reilly (2007)

The Visualization Pipeline - Recall

any kind formatted filtered spatial 2D

of data data data data image
Dataset Process Dataset Process Dataset Process Dataset Process Dataset

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides

The Visualization Pipeline - Recall

any kind formatted filtered spatial 2D

of data data data data image
Dataset Input Dataset Filtering Dataset Mapping Dataset Rendering Dataset

1. Input data
• your primary “raw” source of information
• can be anything (measurements, simulations, databases, …)
2. Formatted data
• converted to points, cells, attributes (discussed next in this module)
• Ready to use for visualization algorithms
3. Filtered data
• eliminates the unneeded data, adds the needed information
• read and written by visualization algorithms
4. Spatial (mapped) data
• has spatial embedding can be drawn
5. 2D Image
• final image you look at to get your answers

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides

 Data Display Requires Planning

• Too Much Information

• Data Collection
• Thinking About Data
• Data Never Stays the Same
• What is the Question?
• A Combination of Many Disciplines
• Process

Visualizing Data, Ben Fry, O‘Reilly (2007)

Subway map

http://www.mbta.com/schedules_and_maps/subway/
 Two Fundamental Forms of Data
--Bertin (1977)

• Entities (data values)

– Objects of interest.
– People, hurricanes, fish, fishponds
– A group of things can be considered as a single entity
• Relationships (data structure)
– Structures and patterns that relate entities
– Structural, physical, conceptual, causal, temporal, etc.
• Attributes of Entities or Relationships
– Both entities and relationships can have attributes
– A property of some entity and cannot be thought of
independently
– Color of apple, temperature of water, duration of a journey, etc.

Information Visualization (perception for design) (2nd Edition), Colin Ware, Elsevier Press.
50 Great Examples of Data Visualization
http://www.webdesignerdepot.com/2009/06/50-great-examples-of-data-visualization/
50 Great Examples of Data Visualization
http://www.webdesignerdepot.com/2009/06/50-great-examples-of-data-visualization/
Entity–relationship model ---software engineering

Wikipedia
http://en.wikipedia.org/wiki/Entity%E2%80%93relationship_model
Attribute Quality - Four Measurements
--S.S. Stevens (1946)

• Nominal (Labeling function)

– Fruits: Apples, Oranges, Banana, etc.
– Numbers: On the front of a bus to identify the route
• Ordinal (Ordering things in a sequence)
– Rank a group of courses in order of preference
– Display a gift pool from lowest price to highest
• Quantitative: Interval (Gap between data values)
– Departure time and arrival time of an aircraft
– Temperature of a day
• Quantitative: Ratio (A real number for a ratio scale)
– Object A is twice as large as object B
– Money is defined on a ratio scale

Information Visualization (perception for design) (2nd Edition), Colin Ware, Elsevier Press.
 Typical Data Classes in Visualization

• Three Typical Levels of Measurement

– Category Data: Nominal class
– Integer Data: Ordinal class (discrete and ordered)
– Real-number Data: Combine Interval and ratio scales
• Some Tricks
– Using graphic size to display category information is misleading
– If we map measurements to color, we can perceive nominal or ordinal
values with a few discrete steps
– Perceiving metric intervals using color is not effective
– Many visualization techniques are capable of conveying only nominal
or ordinal data qualities

Information Visualization (perception for design) (2nd Edition), Colin Ware, Elsevier Press.
 Attribute Dimensions and Orders

• Dimensions
– 1D: scalar
– 2D: two-dimensional vector
– 3D: three-dimensional vector
– >3D: multi-dimensional vector
• Orders
– scalars
– vectors
– tensors (high-order) 1-st order 2-nd order
An attribute of an entity can have multiple dimensions.
1D--We can have a single scalar quantity, such as the weight of a
person.
2D/3D-- We can have a vector quantity, such as the direction in which
that person is traveling. vector matrix

Information Visualization (perception for design) (2nd Edition), Colin Ware, Elsevier Press.
 Earth's gravity field as captured by the GOCE satellite gives an unprecedented view of how the force acts on our planet. The
differences in gravitational force are represented using colours that show -100 metres up to 100 metres.

Tensors are higher-order quantities that describe both direction and shear forces.
The gravitational field of the Earth is a three-dimensional attribute of the Earth. In fact, it is a three-dimensional
vector field attribute. If we are interested only in the strength of gravity at the Earth’s surface, it is a two-
dimensional scalar field attribute.

Dimension: Attributes’ coordinates, 3D field, 2D field at the surface

Order: attributes’ value, vector field attribute (direction and strength)

Extraordinary map reveals Earth‘s gravity field for the first time, By DAILY MAIL REPORTER
Data Table

www.many-eyes.com
Courtesy of Prof. Hanspeter Pfister, Harvard University.
 Common Data Operations
• Mathematical operations (multiplication, division, etc.)
• Merging two lists to create a longer list
• Inverting a value to create its opposite
• Bringing an entity or relationship into existence (such as the
mean of a set of numbers)
• Deleting an entity or relationship (a marriage breaks up)
• Transforming an entity in some way (the chrysalis turns into a
butterfly)
• Forming a new object out of other objects (a pie is baked from
apples and pastry)
• Splitting a single entity into its component parts (a machine is
disassembled)
Information Visualization (perception for design) (2nd Edition), Colin Ware, Elsevier Press.
 Metadata
• Metadata is "data about other data", of any sort in any media.
• Describe an individual datum, or content item, or a collection
of data with multiple content items and hierarchical levels.
• About the context, quality and condition, or characteristics
– Data elements or attributes, (name, size, data type, etc.)
– Records or data structures (length, fields, columns, etc.)
– About data (location, how it is associated, ownership, etc.)
• Example: A digital audio file, such as an MP3 of a song, might
include the album name, song title, genre, year, composer,
contributing artist, track number and album art.

http://en.wikipedia.org/wiki/Metadata
 Quantities of Data
• Intrinsically Continuous and Intrinsically Discrete
• Continuous: pressure, temperature, position, speed, density,
force, color, light intensity, and electromagnetic radiation.
• Discrete: hypermedia, e.g. text and image, contents of web
pages, software source code, plain text, database records.
• Continuous data are often manipulated in finite approximated
form— Sampled Data.
• Sampled data are also discrete with finite set of elements.
• Difference
– Sampled data have corresponding continuous
approximation of the original intrinsically continuous data
– Intrinsically discrete data has no counterpart in the
continuous world

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.

 Continuous Data

Cauchy definition of continuity

A function f is continuous iff

C0 discontinuous (graph of function has “holes”)

C1 first-order continuous (graph of function has “kinks”)
Ck first k derivatives of the function are continuous

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides

 Sampled Data
• We do not always process data in its continuous, functional,
representation.
• Operations, such as filtering, simplification, denoising,
analysis, and rendering, are not easy, nor efficient, to perform
on continuous data representation.
• Two operations relate sampled data and continuous data
– Sampling
– Reconstruction
• The reconstruction operation is operated by interpolation.

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.

 Visualization data properties

Sampling
(data importing) Reconstruction

Sampling Reconstruction

Continuous data Measurements (samples) Continuous data, as close as possible to input

at discrete set of points

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides

 Requirements for Sampled Data
• Accurate: control the production of a sampled set so that the
continuous set can be reconstructed with arbitrarily small error.
• Minimal: the sampled set contains the least number of sample
points needed to ensure a reconstruction with desired error.
• Generic: can easily replace the data processing operations for
the continuous set with equivalent counterparts for the
sampled set.
• Efficient: both reconstruction and data processing performed
on the sampled set can be done efficiently.
• Simple: can design simple software implementation of
sampled set and the operations performed on it.

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.

Color mapping

Basic idea
•Map each scalar value f ÎR at a point to a color via a function c : [0,1]  [0,1]3

Color tables
•precompute (sample) c and save results into a table {ci }i=1.. N
•index table by normalized scalar values

input data scalar value f

determine input range scalar value range [fmin , fmax]

f - fmin
normalize input to [0,N] i=N
fmax - fmin

æiö
Color mapping ci = c ç ÷ c :[0,1] ®[0,1]3
èNø desired color
color table transfer function
f is color-mapped to ci

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides

 Colormap design

What makes a good colormap?

•map scalar values to colors intuitively…

•…so we can visually invert the mapping to tell scalar values from colors

Recall example in Module 1

Data values mapped to RGB colors via a colormap

x,y Invert mapping:

1.look at some point (x,y) in the image  color c
2.locate c in colormap at some position p
3.use the colormap legend to derive data value s from p

blue=0 red=100
p answer: s = 90

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides

Rainbow colormap

•probably the most (in)famous in data visualization

•intuitive ‘heat map’ meaning
• cold colors = low values
• warm colors = high values

Simple to implement
(see Sec. 5.2)

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides

Gray-value colormap
•brightness = value
•natural in some domains (X-ray, angiography)
2D slice in 3D CT dataset
Scalar value: tissue density

Gray-value colormap Rainbow colormap

•white = hard tissues (bone) •red = hard tissues (bone)
•gray = soft tissues (flesh) •blue = air
•black = air •other colors = soft tissues
Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides
Colormap comparison

2D slice in 3D hydrogen atom potential field

Heat colormap Heat colormap Gray-value colormap

•maxima highlighted well • maxima not prominent •maxima are highlighted well
•lower values better • lower values better •lower values are unclear
separable than with • separable
gray-value colormap

Which is the better colormap? Depends on the application context!

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides
Colormap comparison

2D slice in 3D pressure field in an engine

A. Gray-value colormap B. Purple-to-green colormap
•maxima highlighted well •maxima highlighted well
•low-contrast •good high-low separation

C. Red-to-green colormap D. ‘Random’

•luminance not used •equal-value zones visible
•color-blind problems.. •little use for the rest

Which is the better colormap? Depends on the application context!

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides
Colormap design techniques

We cannot give universal design rules

•but some technical guidelines/tricks still exist

1. Fully use the perceptual spectrum

•colormap entries should differ in more, rather than less, HSV components

scalar value ~ V; H,S not used

scalar value ~ H; S,V not used

scalar value ~ H,V; S not used

2. Colormap should be easily invertible

•avoid colormap entries with
• similar HSV entries
• which are perceived as similar (see color blindness issues)
• which are hard to perceive (e.g. dark or strongly desaturated colors)
Good design guidelines: www.colorbrewer.org
Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides
Colormap design techniques

3. Design based on what you need to emphasize

•specific value ranges
•specific values
•value change rate (1st derivative of scalar data)
•… -10( x 4 +y 4 )
2D function f (x, y) = e

Gray-scale colormap Zebra colormap

•highlights plateaus •highlights value variations (1st derivative)
•value transitions hard •dense, thin bands: fast variation
to see •thick bands: slow variation

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides

Colormap implementation details

Where to apply the colormap?

• per grid-cell vertex

2D periodic high-frequency function

64x64 points 32x32 points 16x16 points

As we decrease the sampling frequency, strong colormapping artifacts appear

Why is this so?

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides

 Colormap implementation details

Where to apply the colormap?

•per pixel drawn – better results than per-vertex colormapping
•done using 1D textures
2D periodic high-frequency function

64x64 points 32x32 points 16x16 points

Explanation
•per-vertex: f  c( f ) interpolation(c( f )) color interpolation can fall outside colormap!
•per-pixel: f interpolation( f ) c(interpolation( f )) colors always stay in colormap

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides

Color banding
How many distinct colors N to use in a color table?
•more colors: better sampled c thus smoother results
•fewer colors: color banding appears

color banding

Question
•can we see sharp color banding
with per-vertex colormapping?
Why (not)?

Alexandru C. Telea , http://www.cs.rug.nl/svcg/DataVisualizationBook/Slides

 Reconstruction Function
Definition

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.

 Example

Gaussian function reconstruction with constant basis functions

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.

 Characteristics of Visualization Data
• Data has a topological dimension
– Determines methods for visualization
– Determines data representation
– Examples
0D - points
1D - lines/Curves
2D - surfaces
3D – volumes

• Data is organized into datasets for visualization

– Datasets consist of two pieces
Organizing structure
cells (topology)
points (geometry)
Data attributes associated with the structure
– File format derived from organizing structure

Ray Gasser, Boston University, SCV Visualization Workshop – Fall 2008

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.
 Organizing Structure
• Topology
– The way in which constituent parts are interrelated or arranged
– Specified by one or more cells
– vertex - (0D) point
– polyvertex - (0D) arbitrarily ordered list of points
– line - (1D) defined by two points
– polyline - (1D) ordered list of one or more connected lines
– triangle - (2D) ordered list of three points
– triangle strip - (2D) ordered list of n + 2 points (n is the number of triangles)
– polygon - (2D) ordered list of three or more points lying in a plane
– pixel - (2D) ordered list of four points with geometric constraints
– quadrilateral - (2D) - ordered list of four points lying in a plane
– tetrahedron - (3D) ordered list of four nonplanar points
– voxel - (3D) ordered list of eight nonplanar points with geometric constraints
– hexahedron - (3D) ordered list of eight nonplanar points

Ray Gasser, Boston University, SCV Visualization Workshop – Fall 2008

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.
 Reconstruction with Grids
• Reconstruct continuous functions from sampled data provided
at the vertices of the cells of a given grid,
– A grid in terms of a set of cells defined by a set of sample
points
– Some sampled values at the cell centers or cell vertices
– A set of basis functions
• We can define a piecewise continuous reconstruction of the
sampled signal on this grid and work with it in similar ways to
what we do with the continuous signal.

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.

Cell Types

Ray Gasser, Boston University, SCV Visualization Workshop – Fall 2008

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.
 Four Types of Grids
• Uniform Grids
– Cell edges are uniformly sampled and axis-aligned
– Simple and efficient
– But limited modeling power
• Rectilinear Grids
– Still axis-aligned, but relax the constraint of equal sampling distances for a given axis
– The sample point density can be changed only one axis at a time
– Visualization capability is still limited
• Structured Grids
– Allow explicit placement of every sample point
– Preserve the matrix-like ordering of the sample points
– Can be seen as the free deformation of a uniform or rectilinear grid
– Can not model the shapes with different topologies
• Unstructured Grids
– The most general and flexible grid type
– Allow us to define both sample points and cells explicitly
– Define every cell separately and independently of the other cells
– Cells of different type and even dimensionality can be freely mixed in the same grid

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.

Uniform Grids

Google.com, image search

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.
Rectilinear Grids

Google.com, image search

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.
Structured Grids

Google.com, image search

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.
Unstructured Grids

Google.com, image search

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.
 Data Attributes
• Data attributes associated with the organizing structure
– Scalars
• single valued
• examples: temperature, pressure, density, elevation
– Vectors
• magnitude and direction
• examples: velocity, momentum
– Normals
• direction vectors (magnitude of 1) used for shading
– Texture Coordinates
• used to map a point in Cartesian space into 1, 2, or 3D texture space
• used for texture mapping
– Tensors (generalizations of scalars, vectors and matrices)
• rank 0 ( scalar), rank 1 (vector), rank 2 (matrix), rank3 (3D rectangular array)
• examples: stress, strain

Ray Gasser, Boston University, SCV Visualization Workshop – Fall 2008

Data Visualization (principles and practice), Alexandru C. Telea., A K Peters, Ltd.
 Representation learning bridge data and task
Raw features Data cluster analysis
Various Data
raw features, time-series data, clicked-through rates,
log history, visual data, graph-structured, etc

User activities User modeling

Representation Learning action

prediction
vectorizes raw data as compact representations to recommendation
facilitate downstream tasks user profiling

Image data sign

Visual understanding
traffic lights
sign
Downstream Tasks
traffic
car person
car lights tre
tree
person
e
classification, clustering, regression, prediction,
person
person person person
curb

car phone person perso

detection, ranking, retrieval, etc curb n
phone
person
hydrant hydran curb
t

57
Courtesy of Zhiqiang Tao
 Representation learning by neural networks and labels

Label is the most common way for conveying the guidance

from tasks. ImageNet (image-level)

Raw features multi-layer

perceptron label guidance
CIFAR-10 (60K of10 classes)
(MLP)
convolutional neural MS COCO (pixel-level)
Images / Videos network (CNN)

recurrent neural
Encoder
Time-series network (RNN)

graph convolutional
Graph-structured network (GCN)
Representation Kinetics (650K of 700 classes) Visual Genome (relationship)
Different Data
s
Parameterize encoder with different Large-scale dataset with human annotations
neural networks to handle data variety

Label is the most common way for conveying the guidance

from task.
Challenges for fully-supervised representation learning ImageNet (image-level)

Raw features multi-layer

perceptron label guidance
CIFAR-10 (60K of10 classes)
(MLP)
convolutional neural MS COCO (pixel-level)
Images / Videos
Expensive network (CNN) Human
Encoder
Lack generalized Limited
Time-series human
recurrent neural knowledge
network (RNN) representations regression tasks
annotations Upper-bounded
graph convolutional
Graph-structured network (GCN)
Representation Kinetics (650K of 700 classes) Visual Genome (relationship)
Different Data
s
58
Courtesy of Zhiqiang Tao
 Representation learning with data reconstruction

Raw features multi-layer multi-layer Raw features

perceptron perceptron
(MLP) (MLP)
convolutional neural convolutional neural
Images / Videos network (CNN) network (CNN)
Images / Videos

recurrent neural
Encoder Decoder recurrent neural
Time-series network (RNN) network (RNN)
Time-series

graph convolutional Inner Product decoder

Graph-structured network (GCN) Graph-structured

Different Data Reconstruction

ℎ
𝑥→ Encoding Representation Decoding →𝑥

59
Courtesy of Zhiqiang Tao
 Reconstruction

60
Generation

Translation
Projection

Approximation

Label/Pseudo label of interests

My research: build and design the reconstruction network

high-jump
basketball
volleyball
3. Visual Understanding

fake
Inner Product decoder
convolutional neural

recurrent neural
network (RNN)
network (CNN)
multi-layer
perceptron
(MLP)

Higher-level information
with different higher-level guidance

Copy The Selection To A New Layer Press

Open your photo that contains the Press Ctrl+J / Command+J to quickly

Ctrl+J / Command+J to quickly copy the

Copy The Selection To A New Layer

copy the couple to their own layer in

couple to their own layer in the Layers panel.

Use step 8 to make everything nicely square.
duplicate_image

Step 3 Duplicate this layer and flip it

horizontally by choosing Edit > Transform >
Flip Horizontal.
2. Interpretable User Modeling

Semantics from auxiliary source

…
the Layers panel.

A dialogue box will appear, click OK. Notice

how you'll now be working on the duplicate
layer.
Step 2 Create a new shape below the first one
with the Pen Tool (P).
So download this tiger and open the image in
Photoshop.
Step 19 First off, let's open the Butterfly stock
in Photoshop.
…

We will be using the Butterfly from this stock

open

image so we will need to transfer it to our main

canvas.
seg-vis
…

Open your image and go to Layer>Duplicate

Decoder

Layer.
couple.

Open the image in Photoshop and move it into

your document using the Move Tool (V).
To select it, I'll go up to the Edit menu in the
Press Ctrl+T (Win) / Command+T

with the keyboard shortcut: Go to

Menu Bar along the top of the screen and

choose Free Transform.
(Mac) to access Free Transform

Press Ctrl+T (Win) / Command+T (Mac) to

access Free Transform with the keyboard
free_transform

shortcut: Go to Edit > Free Transform.

…
Edit > Free Transform.

Now download this texture

http://www.sxc.hu/photo/1176171 and again,
re-size it so it fits on top of your image.
…

Select the splash layer and the 2 adjustments

layer you just added and press Ctrl + G to make
a group.
Use the Free Transform Tool to reduce the size
and reposition in the bottom right-hand corner
over the larger pink gradient.
Then, with your mouse button still

With my mouse button still held down, I'll hold

held down, drag straight up to the

down my Shift key on my keyboard and drag

very top: Click and drag out the

down a short distance towards the boat.

transition area for the gradient.

This moves yellow into the spot originally held

by green: Click on the yellow color stop, then
enter 85% for the Location.
gradient_tool

…
Double-click the black color stop below the
gradient ramp, set the color to #ab82bc, and
click OK to close the Color Picker.

…
Then, with your mouse button still held down,
drag straight up to the very top: Click and drag
out the transition area for the gradient.
For our example, we are using a color gradient
Φ ℎ

of dark blue to lighter blue.

Annotations Actions Attentions Memory Slots

Structured information of partitions

1. Learnable Ensemble Clustering

partitions
Encoder

→
Regularizations

raw features
convolutional neural

graph convolutional
recurrent neural
network (RNN)
network (CNN)

network (GCN)
multi-layer

adversarial training
perceptron
(MLP)

KL-divergence

Courtesy of Zhiqiang Tao

Input
Graph-structured

Different Data
Images / Videos
Raw features

nuclear norm
Time-series

ℓ0 , ℓ1 norm

low-rank
sparse