Briefing Doc: Hough Transform and Homography

This document reviews key concepts from "lect09_HoughTransform.pdf," focusing on Hough

Transform and Homography.
Hough Transform

 Purpose: Detect curves (analytic and non-analytic) from boundary information in images.
 Mechanism:Voting Scheme: Features "vote" for parameters of curves they could belong to.
 Parameter Space: Duality is exploited between points on a curve and the curve's parameters.
Image space is transformed into a parameter space where peaks represent likely curves.
 Line Detection:Parameterization: Lines can be represented using slope-intercept (m, c) or polar
coordinates (θ, ρ).

1. Procedure:Discretize parameter space into bins.

2. Each edge point votes for all lines (parameter combinations) it could lie on.
3. Bins with the most votes correspond to detected lines.

 Generalization:The Hough Transform can be extended to detect arbitrary shapes using a

reference point and displacement vectors from boundary points.
 This approach is used in object recognition by indexing displacements with "visual codewords"
instead of gradient orientation.
 Advantages:Handles occlusion and non-locality of features.
 Detects multiple instances in a single pass.
 Robust to noise to some extent.
 Disadvantages:Computationally expensive, complexity increases with the number of
parameters.
 Non-target shapes can generate spurious peaks.
 Choosing an appropriate grid size for discretization is crucial.

Illustrative Quotes:

 "A powerful method for detecting curves from boundary information."

 "Exploits the duality between points on a curve and parameters of the curve."
 "For each feature point in the image, put a vote in every bin in the parameter space that could
have generated this point."

Homography

 Definition: A transformation mapping points between two views of a planar surface or between
images from cameras sharing the same center.
 Representation: A 3x3 matrix with 8 degrees of freedom (scale is arbitrary).
 Estimation:Homogeneous coordinates are used to represent points.
 Direct Linear Transform (DLT) algorithm:
 Each point correspondence provides two linearly independent equations.
 Minimum four correspondences are needed to solve for H.
 More correspondences lead to a least-squares solution.
 Application: Panorama stitching.

Illustrative Quotes:

 "The transformation between two views of a planar surface."

 "H has 8 degrees of freedom (9 parameters, but scale is arbitrary)."
 "Four matches needed for a minimal solution (null space of 8x9 matrix)."

Panorama Recognition

 Algorithms can automatically identify images that form a panorama from a collection.
 This enables automatic panorama stitching from unordered image sets.

Overall, the document highlights Hough Transform as a robust method for shape detection
and Homography as a fundamental tool for image alignment, particularly in panorama
creation.

Computer Vision: Hough Transform and Homography

Study Guide
Quiz
Instructions: Answer the following questions in 2-3 sentences.

1. What is the fundamental principle behind the Hough Transform, and how does it relate to
detecting curves in images?
2. Explain the parameter space representation of a line in the Hough Transform, using the (m, c)
parameterization.
3. What are the advantages of using the polar coordinate system (ρ, θ) over the slope-intercept form
(m, c) in the Hough Transform for line detection?
4. How does incorporating image gradient information improve the efficiency and accuracy of the
Hough Transform for line detection?
5. Describe the impact of noise on the Hough Transform output and how it can affect the detection
of lines.
6. List three advantages and three disadvantages of the Hough Transform.
7. How many dimensions are present in the parameter space when applying the Hough Transform
to detect circles?
8. What is the main concept behind the Generalized Hough Transform, and how does it differ from
the standard Hough Transform?
9. Explain the concept of a homography and its application in computer vision.
10. Briefly describe the Direct Linear Transform method for fitting a homography.

Answer Key

1. The Hough Transform leverages the duality between points on a curve and the parameters
defining that curve. It maps image points to a parameter space, where peaks in the voting
scheme represent potential curves present in the image.
2. In the Hough Transform using (m, c) space, a line in the image space is represented as a point in
the parameter space. Each image point (x, y) votes for all lines (m, c) satisfying the equation c = -
xm + y, forming a line in the parameter space.
3. The polar representation (ρ, θ) avoids the unbounded parameter domain and the issue of infinite
slope for vertical lines encountered in the (m, c) representation. It uses the perpendicular
distance (ρ) from the origin to the line and the angle (θ) between this perpendicular and the x-
axis.
4. Incorporating image gradients helps by directly determining the line's orientation (θ) from the
gradient at the edge point. This reduces the voting process to a single (ρ, θ) pair for each edge
point, increasing efficiency and accuracy.
5. Noise can lead to spurious peaks in the Hough parameter space, making it challenging to
distinguish true lines. The peaks become fuzzy and harder to locate, potentially leading to false
detections.
6. Advantages: Handles occlusion and non-locality, detects multiple instances in one pass, some
noise robustness. Disadvantages: Exponential complexity with increasing parameters, spurious
peaks from non-target shapes, difficulty in choosing grid size.
7. The parameter space for detecting circles using the Hough Transform has three dimensions: the
circle's center (x, y) and its radius (r).
8. The Generalized Hough Transform extends the standard Hough Transform to detect arbitrary
shapes by using a reference point and a table of displacement vectors based on gradient
orientation or visual codewords. This allows for the detection of more complex shapes beyond
simple lines and circles.
9. A homography is a projective transformation that maps points from one plane to another. In
computer vision, it's commonly used for image stitching, perspective correction, and camera pose
estimation.
10. The Direct Linear Transform (DLT) method for fitting a homography involves setting up a system
of linear equations from point correspondences between two images. By solving this system, the
homography matrix can be estimated, enabling the transformation between the two views.

Essay Questions

1. Compare and contrast the Hough Transform and the Generalized Hough Transform, focusing on
their applications, strengths, and limitations.
2. Discuss the problem of noise in the Hough Transform and analyze the different approaches to
mitigate its impact on line detection.
3. Explain how the Hough Transform can be used to detect circles in an image. Describe the
parameter space and the voting procedure.
4. Discuss the application of homography in panorama stitching. Explain the steps involved,
including feature matching, homography estimation, and image warping.
5. Describe the concept of the "visual codeword" used in the Implicit Shape Model for object
recognition. How is it related to the Generalized Hough Transform?

Glossary of Key Terms

 Hough Transform: A feature extraction technique used in image analysis, computer vision, and
digital image processing to detect shapes like lines, circles, and more complex forms.
 Parameter Space: A mathematical space representing the range of possible values for the
parameters defining a specific shape (e.g., slope and intercept for a line).
 Voting Scheme: In the Hough Transform, each image point "votes" for the parameters of
potential shapes passing through it. Parameters with the highest votes indicate the likely
presence of those shapes in the image.
 Polar Coordinate System (ρ, θ): A coordinate system representing points in a plane using a
distance (ρ) from the origin and an angle (θ) relative to a reference direction.
 Image Gradient: A directional change in image intensity, indicating edges and boundaries in the
image.
 Generalized Hough Transform: An extension of the Hough Transform that allows the detection
of arbitrary shapes by using a reference point and a table of displacement vectors.
 Homography: A projective transformation that maps points from one plane to another,
preserving straight lines.
 Direct Linear Transform (DLT): A method for estimating the homography matrix by solving a
system of linear equations derived from point correspondences between two images.
 Panorama Stitching: The process of combining multiple images with overlapping fields of view
to create a wider, panoramic image.
 Implicit Shape Model: A method for object recognition that learns a statistical model of the
object's shape based on local features and their spatial relationships.
 Visual Codeword: A quantized representation of local image features, used to index
displacement vectors in the Implicit Shape Model.

This study guide is designed to help you review your understanding of the Hough Transform,
homography, and related concepts. Use it effectively to prepare for your exams or assignments.
Good luck!
Hough Transform FAQ
1. What is the Hough Transform?
The Hough Transform is a feature extraction technique used in image analysis, computer vision,
and digital image processing. It's primarily used to isolate features of a particular shape within an
image. This is achieved by converting the image space representation into a parameter space
representation. The transform is particularly effective for detecting simple shapes like lines,
circles, and ellipses.
2. How does the Hough Transform work for line detection?
The Hough Transform utilizes a voting scheme to detect lines. First, it represents each edge point
in the image space as a line in the parameter space. This line represents all possible lines that
could pass through that point. Then, it discretizes the parameter space into bins. Each point in
the image space "votes" for bins in the parameter space that correspond to lines passing through
it. Bins with the most votes indicate the presence of lines in the image.
3. Can the Hough Transform handle noise in images?
The Hough Transform is robust to a certain degree of noise. Random noise points are unlikely to
consistently vote for the same bin in the parameter space. However, as noise increases, the
peaks in the parameter space become less distinct, making line detection more challenging.
4. What are the advantages of using the Hough Transform?
The Hough Transform has several advantages, including:

 Handles occlusion: Can detect shapes even if they are partially hidden.
 Detects multiple instances: Can find multiple occurrences of a shape in a single pass.
 Noise robustness: Offers some resistance to noise due to its voting mechanism.

5. What are the limitations of the Hough Transform?

While powerful, the Hough Transform has limitations:

 Computational complexity: Search time increases exponentially with the number of model
parameters.
 Spurious peaks: Non-target shapes can create false peaks in the parameter space.
 Grid size selection: Choosing an appropriate grid size for the parameter space can be
challenging.

6. How is the Hough Transform used for circle detection?

 For circle detection, the Hough Transform uses a three-dimensional parameter space
(representing the circle's center coordinates and radius). Edge points in the image vote for
potential circles they could lie on. Peaks in this 3D space indicate the presence of circles.
7. What is the Generalized Hough Transform?
The Generalized Hough Transform extends the concept to detect arbitrary shapes. It defines a
shape by its boundary points and a reference point. By using a table of displacement vectors
from the reference point to boundary points, indexed by gradient orientation, it can identify
instances of the shape in an image.
8. How is the concept of homography related to the Hough
Transform?
While both homography and the Hough Transform are used in computer vision, they serve
different purposes. The Hough Transform is primarily used for feature extraction, especially
shape detection. Homography, on the other hand, deals with the geometric transformation
between two views of a planar surface, often used in applications like panorama stitching.
Timeline of Events
This document focuses on explaining the Hough Transform, primarily for line detection, but also
mentioning applications for circle and arbitrary shape detection. There isn't a specific timeline of
events presented. Instead, the document explains the algorithm and its variations in a logical
order, building upon concepts as it progresses.
Here's a breakdown of the concepts in the order they are presented:

1. Introduction to Voting Schemes: The document starts by introducing the idea of voting
schemes in image processing, where features "vote" for potential models.
2. The Hough Transform for Line Detection: This section details the classic Hough Transform
algorithm using the slope-intercept (m, c) parameterization, then introduces the polar (ρ, θ)
parameterization due to the limitations of the former.
3. Implementation and Improvements: It delves into the pseudocode for the Hough Transform,
followed by an improvement incorporating image gradients to refine the voting process.
4. Effects of Noise: This part analyzes how noise affects the Hough Transform results, potentially
creating spurious peaks and making peak detection challenging.
5. Advantages and Disadvantages: The document presents a balanced view by listing the pros
and cons of the Hough Transform.
6. Hough Transform for Circle Detection: It extends the Hough Transform concept to circle
detection, highlighting the change in parameter space dimensions.
7. Generalized Hough Transform: A further generalization is introduced for detecting arbitrary
shapes by utilizing a displacement vector table indexed by gradient orientation.
8. Application in Recognition: A practical application of the Generalized Hough Transform is
presented, where "visual codewords" replace gradient orientation for indexing.
9. Homography and Panorama Stitching: The document shifts focus to homography, explaining
its application in panorama stitching and outlining the Direct Linear Transform method for fitting a
homography.
10. Recognizing Panoramas: The final section briefly mentions research on automatically
identifying images suitable for panorama creation.

Cast of Characters
This document primarily focuses on explaining concepts and algorithms rather than highlighting
specific individuals. However, it does mention the following researchers and their contributions:
1. P.V.C. Hough: The inventor of the Hough Transform. The document references a publication
by Hough on the machine analysis of bubble chamber pictures, likely the origin of the transform.
 2. D. Ballard: Contributed to the development of the Generalized Hough Transform, extending its
applicability to arbitrary shapes. The document references a publication by Ballard on this topic.
3. B. Leibe, A. Leonardis, and B. Schiele: These researchers applied the Generalized Hough
Transform for object categorization and segmentation using an "Implicit Shape Model". Their
work, utilizing "visual codewords", is referenced in the document.
4. M. Brown and D. Lowe: Conducted research on automatically recognizing panoramas within a
collection of images. The document cites their work published in ICCV 2003.
5. Svetlana Lazebnik: While not explicitly mentioned, the document states that some slides are
from Lazebnik. This implies that Lazebnik is a researcher or professor who has contributed to the
field, likely with material related to homography or related image transformations.
A Deep Dive into Neural Radiance Fields (NeRFs)
This briefing document summarizes key themes and concepts presented in "lec07-08_nerf.pdf"
focusing on NeRFs as a revolutionary approach to inverse rendering in computer vision.
From Classical Methods to Deep Learning
Traditionally, 3D scene reconstruction relied on techniques like stereo or multi-view stereo. This
document explores how deep learning, specifically neural networks and GPU-accelerated
gradient descent, can be leveraged for this task. The central question is:
“How can we leverage powerful tools of deep learning?”
NeRF and its Core Ideas
NeRFs, or Neural Radiance Fields, represent a novel way to approach 3D reconstruction. The
core idea lies in:

 Differentiable rendering: Crafting a loss function and a scene representation that can be
optimized using gradient descent.
 Neural scene representation: Employing neural networks to learn and represent complex scene
properties.

Multiplane Images (MPIs): A Stepping Stone

Before diving into NeRFs, the document discusses MPIs. While effective for view synthesis and
modeling certain effects, they have limitations:

 "MPIs are easy to differentiate, but only allow for rendering a small range of views"

NeRF: Differentiable Rendering Meets Neural Volumetric Representation

NeRFs build upon the "render-and-compare" paradigm of inverse rendering, choosing a
volumetric representation for the scene. Key characteristics include:

 Volumetric rendering: The scene is treated as "colored fog" with color (c) and opacity (alpha)
defined at each point.
 Neural network representation: An MLP maps 3D position (x, y, z) to color and density (σ),
enabling efficient storage and manipulation of scene data.
 Positional encoding: Input coordinates are passed through a high-frequency mapping to enable
the network to capture high-frequency details.

Rendering with NeRFs

The rendering process involves shooting rays through the "fog" for each pixel. The final color is

"Rendering model for ray (𝑡) = 𝐨 + 𝑡𝐝: 𝐜 ≈ ∑ 𝑖=1 𝑛 𝑇𝑖𝛼𝑖𝐜𝑖"

computed by integrating the color and opacity along the ray, considering occlusion effects:

Training and Results

The network is trained using gradient descent, minimizing the difference between rendered
images and ground truth images captured from different viewpoints. NeRFs excel at:
 kView-dependent effects: By incorporating viewing direction, NeRFs can accurately render
shiny surfaces and other view-dependent phenomena.
 Detailed geometry: NeRFs effectively capture intricate geometric details and occlusion effects.

Beyond Shape and Color: NeRF in the Wild (NeRF-W)

NeRF-W extends the capabilities of NeRFs to handle real-world scenarios, successfully
reconstructing scenes from unconstrained images and even allowing for relighting of the
reconstructed models.
Summary
NeRFs represent a significant leap in 3D scene reconstruction, offering a powerful and flexible
framework for inverse rendering. Combining volumetric rendering, neural networks, and clever
encoding schemes, NeRFs achieve impressive results in capturing detailed geometry, view-
dependent effects, and even complex real-world scenes. This technology opens up exciting
possibilities for applications ranging from virtual reality to robotics and beyond.
Briefing Doc: Two-View Geometry and Stereo Vision
This briefing doc reviews key concepts from two sources on stereo vision and two-view geometry,
summarizing the main ideas and highlighting important facts with supporting quotes.
Source 1: "lec05_stereo.pdf"
Key Theme: Stereo Vision for Depth Perception
This presentation introduces the concept of stereo vision, explaining how using two viewpoints
allows us to perceive depth. It details the process of stereo matching, its challenges, and various
techniques employed to achieve it.
Important Ideas and Facts:

 Depth from disparity: The difference in position of a point in two stereo images, called disparity,
is inversely proportional to the point's depth in the scene. This is visualized by holding a finger
close to your face and alternatingly closing each eye. "Disparity = inverse depth... (Or, hold a
finger in front of your face and wink each eye in succession.)"
 Stereo Matching: This process involves identifying corresponding points in two images.
Assuming brightness constancy, we compare pixels (or windows of pixels) along epipolar lines to
find the best match. "Your basic stereo matching algorithm... Match Pixels in Conjugate Epipolar
Lines – Assume brightness constancy."
 Challenges in stereo matching: This problem is computationally challenging due to factors like
occlusions, violations of brightness constancy, and low-contrast regions. "What will cause
errors?... Occlusions... Violations of brightness constancy (specular reflections)... Low-contrast
image regions."
 Window size effects: The size of the window used for matching impacts the results. Smaller
windows capture more detail but are susceptible to noise, while larger windows are robust to
noise but may blur finer details. "Window size – Smaller window + more detail - more noise –
Larger window + less noise"
 Stereo reconstruction pipeline: The complete process involves calibrating cameras, rectifying
images for horizontal alignment, computing disparity, and finally estimating depth. "Stereo
reconstruction pipeline... Steps – Calibrate cameras – Rectify images – Compute disparity –
Estimate depth"
 Variants of stereo: Beyond passive stereo, the document discusses active stereo with structured
light (projecting patterns to simplify correspondence), laser scanning (a precise structured light
technique), and real-time stereo for applications like robot navigation. "Variants of stereo... Active
stereo with structured light... Laser scanning... Real-time stereo – Used for robot navigation (and
other tasks)"
 Source 2: "lec06_twoview.pdf"
Key Theme: Epipolar Geometry and the Fundamental Matrix
This lecture delves into the theoretical foundation of stereo vision, focusing on epipolar geometry
and the fundamental matrix. It explains how these concepts relate the geometry of two views to
simplify stereo matching.
Important Ideas and Facts:

 Epipolar geometry: A 3D point projects onto a line (epipolar line) in each image plane. These
lines intersect at epipoles, the projections of each camera center onto the other image. "Where
do epipolar lines come from?... 3D point lies somewhere along r (projection of r) ... epipolar
plane"
 Fundamental matrix (F): This 3x3 matrix encapsulates the epipolar geometry, mapping a point
in one image to its corresponding epipolar line in the other. "Fundamental matrix... Epipolar
geometry, very special 3x3 fundamental matrix... maps (homogeneous) points in image 1 to lines
in image 2!"
 Epipolar constraint: The fundamental matrix imposes a constraint: corresponding points in two
images must lie on their respective epipolar lines. This reduces the search space for matching
significantly. "Epipolar constraint on corresponding points: p^T * F * q = 0"
 Derivation of F: The document outlines the derivation of the fundamental matrix for both
calibrated (known intrinsic and extrinsic parameters) and uncalibrated cases. "Fundamental
matrix – calibrated case... Fundamental matrix –uncalibrated case"
 Rectified case: Rectification simplifies stereo matching by transforming images so that epipolar
lines become horizontal. "Rectified case... Stereo image rectification – Reproject image planes
onto a common plane – Pixel motion is horizontal after this transformation"
 Sparse and dense correspondence: The lecture touches upon both sparse correspondence,
where matching focuses on specific features like edges or contours, and dense correspondence,
aiming to match all pixels for applications like 3D modeling. "Sparse correspondence... Dense
correspondence"

Connecting the Sources

Both sources emphasize the importance of understanding two-view geometry for effective stereo
vision. Source 1 focuses on practical applications and challenges of stereo matching, while
Source 2 provides the theoretical framework with the fundamental matrix and epipolar geometry.
Together, they provide a comprehensive understanding of how two images can be used to extract
depth information.
Briefing Doc: Edge Detection, Light, and Perception in
Computer Vision
This briefing doc reviews key themes and ideas from two lectures on computer vision: "Lecture 2:
Edge Detection" and "Lecture 3: Light & Perception."
Lecture 2: Edge Detection
This lecture focuses on the fundamental task of identifying edges in digital images, a crucial step
in understanding and processing visual information.
Key Concepts:

 What are Edges?: Edges represent rapid changes in image intensity, often caused by depth,
surface color, illumination, or surface normal discontinuities. They can be visualized as "steep
cliffs" in the image intensity function.
 Detecting Edges: Edges are detected using image derivatives. A discrete derivative (finite
difference) can be implemented using linear filters, such as the Sobel operator. The gradient of
 the image, calculated from these derivatives, gives both the edge strength (magnitude) and
direction.
 Noise Reduction: Directly applying derivatives to noisy images can lead to spurious edge
detections. Smoothing the image with a Gaussian filter before differentiation reduces noise and
allows for more accurate edge detection. This process leverages the associative property of
convolution: (f * h)' = f' * h, where f is the image and h is the Gaussian kernel.
 Canny Edge Detector: This popular algorithm provides a robust way to find edges. The key
steps are:

1. Smoothing: Filter the image with a derivative of Gaussian.

2. Gradient Calculation: Find the magnitude and orientation of the gradient.
3. Non-Maximum Suppression: Thin the edges by suppressing non-maximum gradient values
along the gradient direction.
4. Hysteresis Thresholding: Use two thresholds (high and low) to identify strong and weak edges.
Strong edges are always kept, while weak edges are retained only if connected to strong edges.

Important Facts:

 The choice of the Gaussian kernel width (σ) in the Canny edge detector influences the scale of
edges detected. Larger σ detects larger-scale edges, while smaller σ detects finer edges.
 The Sobel operator is a commonly used approximation of the derivative of Gaussian filter.

Quotes:

 "An edge is a place of rapid change in the image intensity function."

 "The Canny edge detector is still a widely used edge detector in computer vision."

Lecture 3: Light & Perception

This lecture delves into the complex relationship between light, perception, and digital imaging. It
covers the physics of light, human color perception, and how these factors are modeled in
computer vision.
Key Concepts:

 Light and Color: Light is electromagnetic radiation. The human visual system perceives light
within the visible spectrum. We have three types of cones (S, M, L), sensitive to different
wavelengths, allowing us to perceive color.
 Color Perception Challenges:
 The mapping from radiance to perceived color is nonlinear and influenced by factors like
brightness contrast and constancy.
 We can only represent the full spectrum with three values (from our cones), leading to metamers
- different spectra that appear indistinguishable.
 Reflectance Models:
 Lambertian Reflectance: A simplified model for matte surfaces where reflected light is
proportional to the cosine of the angle between the surface normal and light direction.
Importantly, the perceived intensity is viewpoint-independent.
 Shape from Shading: Using shading variations to infer the shape of an object, often relying on
assumptions about lighting and surface reflectance. This is an ill-posed problem, but additional
constraints (known normals, smoothness assumptions) can help. Deep learning is also showing
promise in this area.
 Cameras and Color: Cameras use Bayer filters (mosaics of color filters) to capture color
information. This raw data is then demosaiced to create a full-color image.
 Important Facts:

 Rods are responsible for intensity perception, while cones are responsible for color vision.
 The fovea, a small central region of the retina, has the highest density of cones and provides the
sharpest vision.
 Our visual system adapts to different light levels, allowing us to see both light and dark areas
simultaneously.

Quotes:

 "There is ambiguity between shading and reflectance."

 "Light response is nonlinear."
 "The same is true for cameras - but we have tools to correct for these effects (Computational
Photography)."

Connecting the Lectures:

Edge detection relies on understanding how light interacts with surfaces and how this interaction
is captured in images. The concept of illumination discontinuities as a source of edges is directly
tied to the material properties and lighting conditions discussed in the lecture on light and
perception.
Further Exploration:

 Explore computational photography techniques used to correct for camera limitations and achieve
specific visual effects.
 Investigate how deep learning is being used to improve shape from shading algorithms.
 Examine the challenges and advancements in color constancy, which aims to perceive colors
accurately under different lighting conditions.