https://doi.org/10.2352/ISSN.2470-1173.2018.11.

IMSE-326
© 2018, Society for Imaging Science and Technology

A Distance Measurement Method Using A Time-of-Flight CMOS

Range Image Sensor with 4-Tap Output Pixels and Multiple
Time-Windows
Kohei Yamada† Komazawa Akihito† Taishi Takasawa† Keita Yasutomi† Keiichiro Kagawa† and Shoji Kawahito†
†Research Institute of Electronics, Shizuoka University, 3-5-1 Johoku, Naka-ku, Hamamatsu, Shizuoka, 432-8011, Japan
E-mail: kyama@idl.rie.shizuoka.ac.jp TEL: +81-53-478-1342 FAX: +81-53-412-5481

Abstract conventional range measurement method, a single time window is

In this study, advanced range measurement method using a used for range measurement. In the proposed technique, switchable
TOF sensor with 4-tap output pixels and small-duty short light multiple time windows based on the range-shifting operation and
pulse is presented. A CMOS TOF range imager with pinned- automatic range switching with thresholding to the multi-tap pixel
photodiode high-speed charge modulator pixels using lateral outputs are used. The effectiveness of the proposed technique is
electric field (LEF) control has been implemented by a 0.11-μm experimentally confirmed by applying the technique to an
CIS process with high near infrared sensitivity. In order to implemented CMOS TOF imager.
improve the range resolution while maintaining the measurable
range, multiple time-windows are used for range measurements. Range Measurement with Double Light Pulse
Compared with the conventional single time window, the use of N The timing diagram for range shift operation with a small duty-ratio double
time-windows theoretically improve the range resolution by a light pulse is shown in Fig. 1. The gate pulse width of G1, G2, G3 and G4
factor of N1.5 if the back ground light shot noise is dominant. In the is given by RDTC, where TC is the cycle time and RD is duty ratio of the gate
pulse to the cycle time. Then, the gate pulse width of GD is given by (1-
measurement for N=2, the range resolution is improved by a factor
4RD) TC. The signal light pulse width and the time of flight of the received
of 2.8 compared with the case of N=1 while maintaining the same light are denoted by T0 and Td, respectively. The signals from FD1, FD2,
distance range. FD3 and FD4 are denoted by S1, S2 S3 and S4, respectively. In TOF
method, the equation for estimating the range is given by
Introduction 1
(1)
Indirect-type Time-of-Flight (TOF) range imagers recently L = cTd
2
have been paid much attentions to applications for outdoor use, where L is the range, c is the speed of light.
particularly for smart devices, robot eyes, VR/AR, security In the case of received light (2) in Fig. 1, the difference of the amount of
systems and automobiles. Various Indirect-type TOF range charges between the two FDs (FD1, FD2) reflects the
imagers have been reported [1-8], though, the tolerance to large time-of-flight of light pulse Td. The range is calculated by the TOF-
dependent charges. During G1 is activated and the other gates are
ambient light is not always well considered. In general, there are deactivated, a part of the signal light pulse is received, and the signal
two modulation methods of light; continuous wave (CW) and short charge Q1 is stored in FD1. The Q1 is expressed as
pulse (SP) modulations. The CW modulation is better for high
Q1 = IS (T0 − Td ) (2)
linearity of range measurements, and by increasing the modulation
where IS is the photo current generated by the signal light pulse. In the next
frequency, the CW modulation has high range resolution if the phase where G2 gates are activated and the other gates are deactivated, the
ambient light is small enough. However, in the CW modulation, rest part of the light pulse is received and the signal charge Q2 stored in
because the duty-ratio of the modulated light is 50%, the tolerance FD2 is expressed as
to ambient light is hard to be comparable of better than that of the Q2 = IS Td (3)
indirect-type TOF range imagers which use short light pulse where If the background light exists, Eq. 2 and Eq. 3 are modified as
the light energy is concentrated at very short instance. Concerning Q1 = IS (T0 − Td ) + IBL R D TC (4)
the tolerance to the ambient light, the SP modulation method in the
indirect TOF range imaging should has a better performance than Q2 = IS Td + IBL R D TC (5)
the CW method. However, in the conventional SP modulation where IBL is photo current generated by the background light and R DTC is
the gate pulse width.
method, there is a tradeoff between the range resolution and In order to cancel the background light, FD3 and FD4 are used for taking
maximum measurable range. background light charges only. The electrons generated by background
To address the trade-off problem between maximum light are transferred from pinned photo diode to the FDs. The signal
measurement range and range resolution, in this paper, a CMOS charges Q3 and Q4 stored in FD3 and FD4 are expressed as
TOF range imaging technique using the SP modulation, multiple Q3 = Q4 = IBL R D TC (6)
(>2)-tap demodulation pixels and a range-shifting operation is The background light cancelling is done by the subtraction of back-ground-
presented. A 4-tap lateral electric field charge modulator (LEFM) only signals from the signals with received signal lights. In this case, Q1
pixels with high-speed charge draining structure based on and Q2 are TOF-dependent signals and Q3 and Q4 are background-only
photodiode CMOS image sensor technology [9,10] is used. The signals. The equation for estimating the range in each pixel is given by
proposed lock-in pixel structure using lateral electric field (LEF) 1 1 Q2 − Q4
control is suitable for implementing a multiple-tap charge L = cTd = cT0 ∙ (7)
2 2 Q1 + Q2 − 2Q4
modulator while achieving high-speed charge transfer and high
demodulation contrast for high time resolution [12]. In the

IS&T International Symposium on Electronic Imaging 2018 326-1

Image Sensors and Imaging Systems 2018
 be left. While the conventional method needs three signals to
remove background light components, the advanced method uses
two signals for it. So, the time of calculating ADC and the area of
analog circuit will be reduced.
Also, the equation for estimating the range is given by
D = Dmax0 {D1 Z1 + D2 Z2 + D3 Z3 }E (11)
where Dmax0 is the maximum range at one time-window, D1
and D2 and D3 are each range equation, Z1 and Z2 and Z3 and E
are the digital value to determine correct regions. The timing
diagram in the advanced range shift method is shown in Fig.3. And,
the equations and values of determining the regions automatically
is shown in Fig.4. The ingenuity is required to distinguish between
the vicinity of the distance of zero and the vicinity of the maximum
distance when distance is calculated using S13 and S24 by the range
shift method. Then, we have setting threshold voltage to vanish the
ambiguity. Therefore, effective measurement range will be reduced
by using threshold voltage. But, it is very important to vanish the
ambiguous regions. So, we need to set threshold voltage T to be
sufficiently small value. While the conventional method is divided
into 4 regions and there are no methods to determine correct
regions automatically, the advanced method is reduced to 3 regions
Fig 1: Timing diagram for the lock-in pixel operation using double pulses and is able to determine correct regions automatically.
𝐐𝟏 − 𝐐𝟑
𝟎 < 𝐓𝐎𝐅 < 𝐑 𝐃 𝐓𝐂 𝐓𝐎𝐅 = 𝐓𝟎
𝐐𝟏 + 𝐐𝟒 − 𝟐𝐐𝟑
𝐐𝟐 − 𝐐𝟒
𝐑 𝐃 𝐓𝐂 < 𝐓𝐎𝐅 < 𝟐𝐑 𝐃𝐓𝐂 𝐓𝐎𝐅 = 𝐑 𝐃 𝐓𝐂 + 𝐓𝟎
𝐐𝟐 + 𝐐𝟏 − 𝟐𝐐𝟒
𝐐𝟑 − 𝐐𝟏
𝟐𝐑 𝐃 𝐓𝐂 < 𝐓𝐎𝐅 < 𝟑𝐑 𝐃𝐓𝐂 𝐓𝐎𝐅 = 𝟐𝐑 𝐃 𝐓𝐂 + 𝐓𝟎
𝐐𝟑 + 𝐐𝟐 − 𝟐𝐐𝟏
𝐐𝟒 − 𝐐𝟐
𝟑𝐑 𝐃 𝐓𝐂 < 𝐓𝐎𝐅 < 𝟒𝐑 𝐃𝐓𝐂 𝐓𝐎𝐅 = 𝟑𝐑 𝐃 𝐓𝐂 + 𝐓𝟎
𝐐𝟒 + 𝐐𝟑 − 𝟐𝐐𝟐
Fig 2: Range shift operation for extending the measurement range using shifted
short double light pulses

Range Measurement with Range Switching by

Thresholding
The timing diagram for the advanced range-shift operation is
shown in Fig. 3. To produce range image, it is divided into 3
measurable ranges which are R1, R2 and R3. The proposed method
calculates the difference of the 2 outputs in the 4-tap pixel outputs
as given by equation (8) and (9). The two differential signals S13
and S24 changes like triangular wave to the TOF. By using this two
signals, it is possible to measure the distance with the simple
equations while automatically detecting the range zone. Also,
equation (10) is the total of the absolute value of equation (8) and
equation (9). S𝐴 is the constant value proportional to the signal
light intensity at the effective range zones. At the all zones, it will
be calculated the range images by using the relative value of
S13 , S24 for SA . Also, setting the boundary of the range zone to a
linearly changing point makes it possible to prevent an error in the
distance even if an error occurs in the zone detection. Therefore, it
will be expected high range linearity at wide range.
Fig3:Timing diagram for the lock-in pixel operation using advanced range shif t
S13 = S1 − S3 (8) method
S24 = S2 − S4 (9)
S𝐴 = |𝑆13 | + |𝑆24 | (10)
By calculating the difference value, the background light
components can be removed and effective signal components can

IS&T International Symposium on Electronic Imaging 2018

326-2 Image Sensors and Imaging Systems 2018
 offset is calibrated by the time offset. As the result, it confirms
𝐒𝟏𝟑 𝐒𝟐𝟒 𝐒𝟏𝟑 high range linearity even at the boundary of each zone. And, the
𝐃𝟏 = 𝟏 − 𝐃𝟐 = 𝟐 − 𝐃𝟑 = 𝟑 + nonlinearity error which is the difference of the measured range
𝐒𝐀 𝐒𝐀 𝐒𝐀
from real range is less than 2.2% of the full scale between 0.8m
𝐙𝟏 = ̅̅̅
𝐏𝟐 𝐏𝟑 𝐙𝟐 = 𝐏𝟐 𝐙𝟑 = ̅̅̅
𝐏𝟐 𝐏𝟒 and 8.6m.
Fig.7 shows the measured range resolution by using advanced
𝐏𝟐 𝐏𝟑 𝐏𝟑 range shift method and conventional method without range shift. In
𝟏(𝐢𝐟 𝐑 𝟏 ≥ 𝐓𝟐 ) 𝟏(𝐢𝐟 𝐑 𝟏 ≥ 𝐓𝟑 ) 𝟏(𝐢𝐟 𝐑 𝟏 ≥ 𝐓𝟑 ) the conventional method, both the gate pulse width and the light
={ ={ ={ pulse width set up 60ns and the results are measured at the only
𝟎(𝐢𝐟 𝐑 𝟏 < 𝐓𝟐 ) 𝟎(𝐢𝐟 𝐑 𝟏 < 𝐓𝟑 ) 𝟎(𝐢𝐟 𝐑 𝟏 < 𝐓𝟑 )
one zone of Gate1 and Gate2. In the advanced method, both the
𝟏(𝐢𝐟 𝐒𝐀 ≤ 𝐓𝟐 )
𝐄={ gate pulse width and the light pulse width set up 15ns and the
𝟎(𝐢𝐟 𝐒𝐀 > 𝐓𝟐 ) results are measured at the 3zone of Gate1, Gate2, Gate3 and
Fig4:The equations of selecting the measurement range regions using advanced Gate4. The range resolution is the standard derivation of the
range shift method temporal deviation of measured range. The high range resolution
of 95.9mm is achieved at 8.6m by using advanced range shift
Implemented TOF sensor method. And, the range resolution of the advanced method is 3
The block diagram of the TOF range imager is shown in Fig. 5. The TOF times lower than that of the conventional method without range
range imager consists of a 160×240 pixel array, vertical and horizontal shift shift.
register, ADC, and LEF charge modulator driver. Each pixel has four SF
outputs which are shared by twelve charge modulators. Each of the four
outputs from each pixel are connected each of the column ADC and are
converted to digital codes in parallel using high-resolution folding
integration/cyclic ADCs [11].

Fig6:Measured linearity and nonlinearity

Fig5:Sensor architecture

Measurement Results
The signal light source is composed of 96 near-infrared LEDs
with the 870nm wavelength, and the signal light pulse width T0 is
set to 15ns. An average power of the light source is 250mW. The
cycle time of exposure TC and the duty ratio RD are 1.0μs and
0.015, respectively. The gate pulse width RDTC for G1, G2, G3
and G4 is 15ns. In this measurement, T0 of 15ns for the purpose of
demonstration of high range resolution due to high-speed response
of the pixel. The frame rate is 15.7fps. A lens with a focal length of
12.5mm and a F-number of 2.0 are mounted on the sensor. An
infrared bandpass filter is placed in front of the lens. The High
level for gate voltage of G1, G2, G3 and G4 is set to 2.5V, and the
Low level is set to -1.0V. The High level for gate voltage of GD is
set to 3.6V, and the Low level is set to -1.0V. The measured results Fig7:Measured range resolution
are obtained by the average of 30 frames of raw data in 10 × 10
pixels in the center of the pixel array. We have used a white flat
panel as an object. Conclusion
Fig.6 shows a measured distance linearity and nonlinearity This paper presented the advanced distance measurement
error. A white flat panel is located between 0.8m and 8.6m with method with range switching by thresholding. The proposed
the step of 0.2m. The results are compensated for the slop and method uses the difference value of two pixel outputs and makes it
offset. The slop is calibrated by the demodulation contrast, the possible to detect range zone by using the threshold voltage

IS&T International Symposium on Electronic Imaging 2018 326-3

Image Sensors and Imaging Systems 2018
automatically. Therefore, the background light components can be References
removed and effective signal components can be left before [1] R. Lange, P. Seitz, A. Biber and S. Lauxtermann, “Demodulation
measuring distance. Also, it has succeeded high range linearity at Pixels in CCD and CMOS technologies for Time-of-Flight Ranging,”
the wide range. The nonlinearity error which is the difference of Proc. SPIE, vol.3965, pp. 177-188, 2000.
the measured range from real range is less than 2.2% of the full [2] S. Kawahito, I. A. Halin, T. Ushinaga, T. Sawada, M. Homma, and Y.
scale between 0.8m and 8.6m. The high range resolution of Maeda, “A CMOS time-of-flight range image sensor with gates-on-
95.9mm is achieved at 8.6m by using advanced range shift method. field-oxide structure,” IEEE Sensors J., vol. 7, no.12, pp. 1578-1586,
Compared with the conventional method without range shift, the Dec. 2007.
range resolution of the advanced method is 3 times better than that
[3] D. Stoppa, N. Massari, L. Pancheri, M. Malfatti, M. Perenzoni, and L.
of the conventional method. Table 1 summarizes the specification
Gonzo, “A Range Image Sensor Based on 10um Lock-In Pixels in
and characteristics of this work.
0.18um CMOS Imaging Technology,” IEEE J. Solid-State Circuits,
vol. 46, no.1, pp. 248-258, Jan. 2011.
Acknowledgements [4] S. J. Kim, B. Kang, K. D. K. Kim, K. Lee, C. Y. Kim, and K. Kim,
This study was partly supported by A JSPS KAKENHI Grant “A 1920x1080 3.65μm-pixel 2D/3D image sensor with split and
Number JP25220905, 25109003 and the Center of Innovation binning pixel structure in 0.11 μm standard CMOS,” ISSCC Dig.
Program and Regional Innovation Ecosystem Program. The chip Tech. Papers, pp. 396-397, Feb. 2012.
design was also supported by VLSI Design and Education Center [5] O. Shcherbakova, L. Pancheri, G.-F. Dalla Betta, N. Massari, and D.
(VDEC), the University of Tokyo in collaboration with Mentor Stoppa, “3D camera based on linear-mode gain-modulated avalanche
Graphics, Inc. The authors appreciate Dongbu HiTek for CIS photodiodes,” ISSCC Dig. Tech. Papers, pp. 490- 491, Feb. 2013.
fabrication. [6] A. Payne, A. Daniel, A. Mehta, B. Thompson, C. S. Bamji, D. Snow,
H. Oshima, L. Prather, M. Fenton, L. Kordus, P. O’Connor, R.
McCauley, S. Nayak, S. Acharya, S. Mehta, T. Elkhatib, T. Meyer, T.
Table 1: Sensor specification O’Dwyer, T. Perry, V.-H. Chan, V. Wong, V. Mogallapu, W. Qian,
and Z. Xu, “A 512×424 CMOS 3D Time-of-Flight image sensor with
multi-frequency photo-demodulation up to 130MHz and 2GS/s ADC,”
Process 0.11 μm 1P4M CIS ISSCC Dig. Tech. Papers, pp. 134-135, Feb. 2014.
[7] Dal Mutto, C.; Zanuttigh, P.; Cortelazzo, G.M. “Time-of-Flight
Total area 9.0 mm (H) x 9.3 mm (V) Cameras and Microsoft KinectTM,” Springer Science & Business
Media: New York, NY, USA, Jan. 2013.
1.5 V (Digital),
Power supplies [8] Y.Kato, T.Sano, Y.Moriyama, S.Maeda, T.Yamazaki, A.Nose,
3.3 V (Analog, Digital)
K.Shina, Y,Yasu, W.Tempel, and A.Ercan, “320x240 Back-
96 LED array Illuminated 10μm CAPD Pixels for High Speed Modulation
Emitter
with 870 nm wavelength Time-of-Flight CMOS Image Sensor,” VLSI Circuits, 2017
Symposium on, Jon. 2017.
Lens F# 2.0 / focal length of 12.5mm [9] S. Kawahito, G. Beak, Z. Li, S.-M. Han, M.-W. Seo, K. Yasutomi,
and K. Kagawa, “CMOS Lock-in Pixel Image Sensors with Lateral
Number of pixels 160 (H) x 240 (V) (4-Tap pixels) Eelectric Field Cotrol for Time-Resolved Imaging,” Proc. 2013 Int.
Image Sensor Workshop (IISW), pp. 361-364, Jun. 2013.
Pixel size 22.4 μm (H) x 16.8 μm (V) [10] S.-M. Han, T. Takasawa, T. Akahori, K. Yasutomi, K. Kagawa, and S.
Kawahito, “A 413×240-pixel sub-centimeter resolution Time-of-
Conversion Gain 1.1 μV/e- Flight CMOS image sensor with in-pixel background canceling using
lateral-electric-field charge modulators,” ISSCC Dig. Tech. Papers,
Fill factor 41 % (with micro lens) pp. 130-131, Feb. 2014.
[11] M.-W. Seo, S.-H. Suh, T. Iida, T. Takasawa, K. Isobe, T. Watanabe,
Frame rate 15.7 fps (exposure time of 50ms) S. Itoh, K. Yasutomi, and S. Kawahito, “A Low-Noise High
Intrascene Dynamic Range CMOS Image Sensor With a 13 to 19b
15 ns (Light width) Variable-Resolution Column-Parallel Folding-Integration/Cyclic
Modulation speed
15 ns (Gate width) ADC,” IEEE J. Solid-State Circuits, vol.47, no.1, pp. 272-283, Jan.
2012.
Demodulation
68.7% @15ns (Gate width) [12] T.Kasugai,et al., ”A time-of-flight CMOS range image sensor using
Contrast
4-tap output pixels with lateral-electric-field control,” Electronic
Measured range 0.8 ~ 8.6 m Imaging 2016, Feb. 2016

Non linearity 2.2 %FS

Author Biography
Range resolution 95.9mm @ 8.6m
Mr.Kohei Yamada was received the B.E. degrees in from
Shizuoka University, Hamamatsu Japan, in 2017. He is now a
master course student of Shizuoka University.

IS&T International Symposium on Electronic Imaging 2018

326-4 Image Sensors and Imaging Systems 2018