IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL.

70, 2021 8502609

Models of Direct Time-of-Flight Sensor Precision

That Enable Optimal Design and
Dynamic Configuration
Lucas J. Koerner , Member, IEEE

Abstract— Direct time-of-flight (dToF) sensors that measure (dToF) sensor probes the scene with a brief laser pulse and
depth by pulsing a laser and timing the photon return are measures the time for the reflected photons to return. This
used in many applications, including consumer electronics for approach produces a histogram of photon return times from
proximity sensing and depth map generation. A histogram of
photon return times is measured and then processed to estimate which the depth and reflectivity of objects in the scene are
object depth. By collecting many photons that span multiple bins extracted. Direct ToF examples include a 4096-pixel sensor
of the histogram the final depth estimate interpolates between that was demonstrated as an altimeter for spacecraft land-
time-to-digital converter (TDC) bins to produce a result that ing [1] and the miniaturized ST VL53L1X, a 16 × 16 array
is more precise than the converter resolution. The precision with a 4-m range and 1.5-mm precision for mobile phone
of this interpolation depends on the temporal spread of the
measurement, the resolution of the TDC, and the number of applications [2]. Direct ToF technology continues to improve
signal and background photons measured. There is a need suggesting further capability growth. Recent developments
for dToF depth precision models to guide design and predict include a 3-D stacked single-photon avalanche photodiode
and tune performance during use. In this article, we present (SPAD) chip [3], a 256 × 256 SPAD array chip connected by
models that estimate sensor depth precision versus dToF design 3-D Cu-Cu stacking to a processing chip [4], and a 1-Mpixel
parameters and photons measured. We use Monte Carlo simu-
lations and experimental measurements to prove the accuracy time-gated SPAD array (1200 × 900 SPAD array with 9.4-μm
of the models. With proven models in hand, we investigate pixel pitch) [5].
a dToF sensor design by first presenting the dependence of A single-photon dToF sensor signal chain consists of a
precision upon the TDC resolution and the signal-to-noise ratio. pulsed laser diode, an SPAD receiver array, one or more time-
Second, we experimentally measure the depth precision versus to-digital converters (TDCs) that digitize the photon flight
the intensity of background illumination. The models closely
match the measurements of background susceptibility and locate time, and memory to build a histogram of the flight times.
a transition point of background intensity below which pre- The pulsed laser, SPAD, and TDC each have an inherent
cision is constant and above which the precision continuously timing jitter, which, when combined with other sources of
degrades. Finally, experimental measurements demonstrate how timing spread that include the distribution of photon travel
the modeling enables dynamic tuning: from a single histogram we lengths and weather effects such as fog [6], produce a total
estimate precision, thus enabling sensor exposure time tuning for
a target precision or prediction of the precision given a change in temporal spread. Recent work has demonstrated a laser diode
object distance or background illumination. This work presents driver capable of 64-ps rms [7]. SPAD miniaturization and
straightforward models verified by simulation and measurement. integration has progressed with [8] demonstrating an SPAD
These models guide dToF design and enable dynamic adjustments in 45-nm CMOS image sensor technology with 46-ps rms
that benefit power-constrained usage scenarios. timing jitter and no apparent exponential tail due to charge
Index Terms— Direct time-of-flight (dToF), distance measure- carrier diffusion such that the timing response is well-modeled
ment, histograms, light detection and ranging (LiDAR), robot by a single Gaussian distribution. Similarly, an SPAD array
sensing systems, single-photon avalanche diode (SPAD), time of fabricated in 180-nm CMOS with a 4-μm pixel pitch and a
flight (ToF).
timing jitter of 88-ps rms has no apparent exponential tail [9].
I. I NTRODUCTION A TDC with a resolution of 100 ps and a timing jitter of 44-ps
rms was demonstrated previously and is representative of cur-
O PTICAL time-of-flight (ToF) systems consist of a trans-
mitter and a receiver that directly or indirectly time
photons to determine the distance to objects. A direct ToF
rent TDC capabilities [10]. The timing jitter and quantization
noise of these three elements may be added in quadrature
for a composite rms uncertainty of ∼120-ps rms, equivalent
Manuscript received January 20, 2021; revised March 19, 2021; accepted to a distance precision of 18 mm. Yet, considerably more
April 6, 2021. Date of publication April 26, 2021; date of current version precise measurements are available from both commercial
April 30, 2021. This work was supported by the School of Engineering,
University of St. Thomas. The Associate Editor coordinating the review (1.5-mm rms [2]) and academic devices (1.4-mm rms [11]).
process was Dr. Christoph Baer. To achieve a depth measurement more precise than the TDC
The author is with the Department of Electrical and Computer Engi- resolution or system temporal spread, the TDC histogram is
neering, University of St. Thomas, St. Paul, MN 55105 USA (e-mail:
koerner.lucas@stthomas.edu). processed using super-resolution localization methods. Here,
Digital Object Identifier 10.1109/TIM.2021.3073684 we ask: what analytical models best predict the dependence of
1557-9662 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 30,2021 at 04:15:46 UTC from IEEE Xplore. Restrictions apply.
8502609 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 70, 2021

the depth precision on instrument timing jitter, TDC resolution, LiDAR systems include a number of configurable parameters
and the number of signal and background photons collected? for which users and designers need guidance. Our models help
Localization of a peak in a histogram or a spot in an fulfill this need by guiding the tuning of parameters for optimal
image in the presence of noise (Poisson photon and Gaussian performance.
instrument) and resolution limits has been studied in several In this work, we present models for the precision of single-
contexts. These contexts include spacecraft star tracking [12], photon dToF sensors that link hardware design parameters
spot localization in fluorescence microscopy (see [13] and ref- and the number of signal and background photons collected
erence therein), and processing of light detection and ranging to the depth precision, thus guiding design and configuration
(LiDAR) histograms of photon return times [14]–[16]. Our optimization. Our studies present three analytical models of
work draws upon localization methods from both imaging progressively increasing rigor and verify the applicability of
and LiDAR with analogies between images and timing his- the models using Monte Carlo (MC) simulations and exper-
tograms of: 1) position ⇔ time; 2) point spread function imental measurements. Questions pursued include: 1) how
(PSF) ⇔ instrument response function (IRF); and 3) pixel many signal photons are needed for a particular precision?
size ⇔ histogram bin width. As established in [17] for images, 2) how does precision depend on the background photon rate?
the fundamental
√ precision limit without noise nor pixelization and 3) what is the optimal TDC resolution given the temporal
is (σPSF / NIMG ), where σPSF is the standard deviation of spread of the IRF? These models predict the precision from a
a Gaussian PSF and NIMG is the number of signal photons single histogram and thus support the deployment of adaptive
measured in the image. The work of Winick [18] establishes dToF usage including energy-aware algorithms that adjust
the Cramér–Rao bound of the precision of spot localization to the environmental scenario in real time to enable longer
given a Gaussian PSF, pixelization, and pixel background battery life in handheld applications. Our contribution uniquely
noise. This previous scenario is analogous to depth estimation presents a straightforward model for timing precision that
from a dToF histogram with the TDC resolution mapping to guides dToF design and configuration.
the image sensor pixelization.
Localization precision has been extensively studied in the II. M ETHODS
field of single-molecule fluorescence microscopy. Similar to
dToF depth estimation, spot localization in microscopy esti- A. Analytical Modeling
mates the position of a fluorophore in an image captured by A dToF sensor measures an unknown photon return time, t0 ,
an array of pixels that is corrupted by photon shot noise, in the presence of timing jitter from the laser, SPAD, and TDC.
background photons, pixelization, and image sensor read- Previous SPAD technologies demonstrated a time response
noise in 2-D [19] or even in 3-D dimensions [20]. Spot with an exponential diffusion tail following an initial Gaussian
localization to subpixel precision (hence the term super- response. However, miniaturization and device engineering
resolution) has been demonstrated using least-squares (LS) have eliminated this charge diffusion-based slow exponential
fits [19], maximum-likelihood estimation (MLE) [21], or non- tail and justify an assumption of a Gaussian distribution of
fitting methods such as iterative centroiding of the intensity response times [8], [9], [26]. The temporal spread may be
[22]. MLE approaches the theoretical limit of precision as further broadened by details of photon propagation, such as
set by the Cramér–Rao bound (CRB) when a model of the measurement of a range of photon path lengths due to object
microscope PSF and noise is available [13]. This theoretical size and the field-of-view (FoV) of the sensor [27], [28]
precision limit has been derived in the presence of photon shot and atmospheric effects (including fog [6]). We model the
noise, background photons, and detector pixelization while combined effects from the sensor and optical propagation as
considering an Airy disk PSF [23]. a Gaussian spread of return times with a standard deviation
Return-time processing methods for a dToF system may be σ . The timing spread of σ may be estimated from a fit
separated into distinct steps of peak detection and character- to the histogram of return times and used in subsequent
ization [15]. A dToF system may measure multiple distinct modeling as shown in our experimental validation. The models
object distances in the scene with more than one and possibly consider the number of signal photons collected to be Poisson
overlapping histogram peaks. In this scenario preprocessing distributed with mean value N. This parameter depends on
steps, such as Richardson–Lucy deconvolution, may first be object reflectivity and depth (d), laser intensity, and sensor
used to minimize the timing spread due to the instrument and exposure time. The timing digitization resolution of the TDC
recover the resolution of the object [16]. Multiple peaks may is represented as a. The Poisson distributed background counts
be detected using methods, such as bump hunting on smoothed per TDC bin are denoted as b. This parameter depends on the
derivatives of the photon count data [15] or the reversible jump ambient illumination in the environment, the sensor optical
Markov chain Monte Carlo (RJMCMC) [24]. Once peaks are and detector design, and the SPAD dark count rate. For a
detected, MLE using a Poisson distribution can be used for given measurement scenario, with a background photon rate
the best estimate of the actual return time [15]. These works set by the ambient illumination intensity and ToF receiver
present optimal processing approaches given specific hardware parameters, b scales with TDC resolution. This scaling makes
and/or data as well as innovative approaches to extract multiple our analysis distinct from other analyses of microscopy local-
peaks in a single histogram. Our goal differs in that we seek to ization where pixel noise may be dominated by electronics
provide models for the localization precision of a single return- noise (i.e., read noise) and does not scale with pixel size.
time peak. Furthermore, as discussed by Chen et al. [25], We note that modeling of optical propagation in a LiDAR

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 30,2021 at 04:15:46 UTC from IEEE Xplore. Restrictions apply.
KOERNER: MODELS OF dToF SENSOR PRECISION THAT ENABLE OPTIMAL DESIGN AND DYNAMIC CONFIGURATION 8502609

system is beyond the scope of this work (see [27], [28]); in

this work, we experimentally extract these parameters from
the sensor histogram to estimate depth precision.
With these parameters in place, an expression for the
probability distribution function (PDF) of the continuous-time
signal is given as
1
√ e−(t−t0 ) /2σ .
2 2
f (t, t0 ) = (1)
σ 2π
The signal and background accumulated into a TDC bin
that spans a time interval of t1 to t2 with a = t2 − t1 is then
given as
 t2
N
√ e−(t−t0 ) /2σ dt + b.
2 2
Q(t1 , t2 ) = (2)
t1 σ 2π
Fig. 1. Example histogram created using MC simulations with 176 TDC bins,
Often instruments have a limit of one-photon arrival time a background level of b = 30.0, total signal photons of N = 300, a temporal
spread of σ = 150 ps, and a mean signal arrival time of 12 ns. A Gaussian
digitized per laser pulse such that the histogram signal is built fit to the histogram data is overlaid in magenta. The plot is zoomed onto a
from many laser pulses. subset of the TDC bins centered around the arrival time of 12 ns.
The dToF localization performance is analytically modeled
using three methods. For the case of a Gaussian distributed
IRF, a centroid calculation has been shown to be an unbiased B. MC Simulations
MLE with a one standard deviation uncertainty of
We verified the analytical models using MC simulations.
√ The MC simulation develops an array of Poisson distributed
δ = σ/ N (3)
background photon arrival times given a specified average rate
when TDC resolution and background noise are ignored [29]. that corresponds to b photons per bin. The number of signal
This model is referred to as the fundamental limit. A second photons is determined by drawing from a Poisson distribution
localization method presented by Thompson et al. minimizes with a mean value of N. The arrival time of each signal photon
the LS error of a centroid calculation to data that has had a is determined by sampling from a Gaussian distribution with
Gaussian mask applied [19]. This method localizes a discretely an average arrival time t0 and a standard deviation set by the
sampled 1-D Gaussian distribution with a precision of temporal spread (σ ). With the continuous-time data available,
the photon arrival times are then quantized into bins to

√ emulate the digitization of a TDC. Throughout the simulations,
σ 2 + a 2 /12 4 πσ 3 b
δ= + . (4) the average arrival time is randomly varied by the width of a
N a N2 histogram bin to remove systematic quantization effects. Fifty
The third analytical model evaluates the CRB, which is histograms are generated for each set of configurations. Note
the lowest possible variance of an unbiased estimator of an that photon pileup caused by an instrument limit on the number
unknown value [30]. The CRB for 1-D localization given the of detected photons per laser pulse, typically one photon per
PDF of (1) in the presence of quantization, and background pulse, is not considered in these simulations. Fig. 1 shows a
noise [18] is zoom-in around the signal peak of an MC simulation result
for the scenario of b = 30.0, a TDC resolution a = 150 ps,
 − t0 )2 ] = 
 0 ( Q) 1 and a total of N = 300 signal photons.
E[(t [N f i (t0 )]2
(5)
The generated histograms are localized using multiple
i N f i (t0 )+b
approaches. First, the peak is located by finding the mode
where i is the i th TDC bin, f i is the derivative with respect of the histogram (alternative initial peak finding methods are
to time of the PDF from (1), Q represents the measured count more capable (see [16]) and may locate a peak buried in the
values in each TDC bin, and E represents the expected value. noise). Next, the peak is more accurately localized by the
The actual return time may fall anywhere within the TDC bins following methods.
and, as such, the estimated variance must be averaged over a 1) A center of mass (CoM) calculation spanning ±3σ bins
full TDC bin of width a to produce a precision estimate of from the mode. For a Gaussian distributed instrument
response, a CoM calculation is a minimum variance
  a/2 1/2
1 estimator.
δ=  0 ( Q)
E[(t  − t0 )2 ]dt0 . (6)
a −a/2 2) A nonlinear Gaussian fit to the background-subtracted
signal with initial guesses from the known temporal
This analytical method is referred to as the Cramér–Rao spread (σ ) and the amplitude (N) as determined from the
bound (or CRB). We have implemented the CRB calculation mode. The background level is calculated as an average
in Python and verified concordance with the results in [18]. of a set of TDC bins away from the peak.

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 30,2021 at 04:15:46 UTC from IEEE Xplore. Restrictions apply.
8502609 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 70, 2021

3) A CoM calculation after processing with a Gaussian mask

(of standard deviation σ ) that is iteratively positioned
to follow the best estimate of the peak position as per
Thompson et al. [19].
From the peak fits, the standard deviation of the difference
of the estimated arrival time and the actual arrival time is
calculated to be the localization precision (δ). The computer
code to run MC simulations, evaluate the analytical models,
and create the figures in this article is available on GitHub [31].

C. Experimental Verification
We captured experimental measurements with the ST
VL53L1X dToF sensor [2], [10] to verify the analytical models
and MC simulations. The ST VL53L1X is a dToF senor with
a 16 × 16 array of SPAD pixels and TDCs that create a Fig. 2. Lines are analytical calculations of depth precision as the number of
timing histogram of photon arrivals. The sensor system uses signal photons is varied. Markers are results from MC simulations processed
with the three different methods to localize the histogram peak. The temporal
a 940-nm Class 1 pulsed vertical-cavity surface-emitting laser spread is σ = 100 ps, and the TDC resolution is a = 1.5σ with a background
(VCSEL) that fills the FoV of the dToF sensor. The FoV is per TDC bin of b = 18.75.
programmable from 15◦ to 27◦ . The VL53L1X implements
signal processing on the sensor to extract object depth, yet
in this work, we download the photon arrival histogram and
implement peak localization using a Gaussian fit as described
in Section II-B. The dToF sensor was tripod mounted with
a flat target of uniform reflectivity that filled the FoV as
in [32]. For controlled background illumination, a set of
IR light-emitting diodes (LEDs) with a center wavelength
of 940 nm and a full-width at half-maximum of 80 nm
(LST1-01F09-IR04-00) was driven by a programmable dc
supply automated using instrument control software [33]. The
sensor output data include a histogram of photon return times.
Precision was analyzed using a Gaussian fit to the histogram
after first locating the mode. The fit extracts the arrival time
and the temporal spread (σ ), while the average background
counts per TDC bin (b) were determined from reference TDC
bins that record immediately before the laser pulses. The Fig. 3. Lines show analytical calculations, while markers show MC results of
the depth precision as the number of background photons per bin (b) is varied
precision was calculated from the standard deviation of the with a signal level of N = 1000 and the same instrument configurations as
peak position of the Gaussian fit to 98 nominally identical in Fig. 2.
measurements.
The mask CoM, however, degrades at higher signal levels,
III. R ESULTS which in our case is primarily due to an insufficient number of
Fig. 2 compares the three analytical calculations of depth iterations and target error in the iterative routine. Based on the
precision to the results from MC simulations processed with close correspondence with the MC results, the CRB analytical
the three different methods as the number of signal photons approach is deemed the most accurate, whereas the analytical
is varied. The temporal spread is σ = 100 ps, and the TDC model of Thompson is useful for estimates that provide design
resolution is a = 1.5σ with a background rate per TDC bin of intuition. For the remainder of this article, histograms from
b = 18.75. The fundamental limit ignores TDC quantization MC simulations and experimental measurements are localized
and background noise and estimates the smallest precision using a Gaussian fit as this method shows the best precision
noise. The CRB consistently estimates a slightly greater error over a range of SNRs. Note that our goal is not to evaluate
than that of Thompson with the deviation greatest at low the best peak localization method(s), but rather to determine
signal levels (see Fig. 4). The MC results closely track the the dependence of localization precision upon instrument and
CRB calculation and provide confidence in the implementation experiment parameters.
of this model. For the MC localization, the precision of the Fig. 3 assesses the accuracy of the second and third analyt-
Gaussian fit peak processing method outperforms the CoM ical models compared with MC simulations as the number
and the mask CoM. The CoM calculation does not filter the of background photons per bin increases with a constant
tails of the IRF and so degrades in low signal-to-noise ratio signal level of N = 1000. Again, the CRB more closely
(SNR) scenarios, whereas the mask CoM performs better in tracks the results from the MC simulation. Two regimes
these scenarios since the bins in the tails are deemphasized. are evident. At b < 10, the background contribution is

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 30,2021 at 04:15:46 UTC from IEEE Xplore. Restrictions apply.
KOERNER: MODELS OF dToF SENSOR PRECISION THAT ENABLE OPTIMAL DESIGN AND DYNAMIC CONFIGURATION 8502609

Fig. 5. MC simulations as the TDC resolution (a) is swept with a constant

temporal spread (σ = 100 ps). The ratio of total signal photons to background
photons per bin (N/b) is indicated in the legend for a/σ = 1. The background
signal per bin is scaled by TDC bin size. The solid lines are the CRB for
Fig. 4. Percent difference between the CRB timing precision and the each of the four N/b√levels. The dashed line indicates the limit as set by
Thompson model as the SNR (N/b) and a/σ is varied. In all cases, the CRB TDC resolution (= a/ 12).
precision estimate is greater than the Thompson model.

TABLE I
insignificant such that the precision is dominated by the TDC B IN R ESOLUTION (a) IN T ERMS OF THE T EMPORAL S PREAD (σ )
temporal spread, TDC resolution, and the number of signal T HAT P RODUCES A G IVEN P RECISION D EGRADATION FOR VARIOUS
L EVELS OF SNR ( N/b). T HE P RECISION AT a = 0 I S R EPRESENTED
photons
 captured and, as given by (4), plateaus to around AS δ0 (N O D EGRADATION ). T HE S ECOND C OLUMN S HOWS
((150 ps)2 + (100 ps)2 /12)/1000 = 3.4 ps. At high back- THE a/σ VALUES FOR A 10% D EGRADATION OF P RECISION
ground levels, the precision noise increases as the square F ROM THE Z ERO TDC B IN W IDTH BASELINE , W HEREAS
THE T HIRD C OLUMN S HOWS THE S AME FOR
root of the number of background photons as predicted by A 41% D EGRADATION
the second term of (4). Section IV-A further verifies these
results with experimental measurements.
The CRB model most closely tracks the MC results, par-
ticularly when the MC simulations are localized using a
Gaussian fit to the histogram data. The Thompson model
underestimates the precision error because the background
noise contribution assumes an infinitesimally small TDC bin
size [19]. The deviation of the Thompson model from the
CRB and simulations is most significant at high background noise as a/σ is varied for four levels of SNR (N/b). Solid
levels and large TDC bins. Using N/b as the SNR, Fig. 4 lines show the CRB calculation, while markers represent the
shows the percent difference between the CRB model and the results from the MC simulations.
Thompson model for N/b from 0.5 to 100 and a/σ from Fig. 5 shows that the smallest precision noise for a given
0.25 to 4. For a/σ ≤ 1.2, the Thompson model deviates from SNR (δ0 in Table I) is found at a/σ = 0, which impractically
the CRB model but at most by 11%. The most significant requires a zero width TDC bin. To determine a realistic TDC
deviations are found at low SNR with large TDC bins. The resolution, we set an allowable precision degradation due to
CRB model most accurately tracks the MC simulations, yet finite TDC resolution. To do so, the CRB model is searched
the Thompson model beneficially provides a straightforward for an a/σ value that produces a 10% and 41% increase in
precision estimate with accuracy within ≈ 20% of the CRB precision noise at a particular SNR. The results summarized
model except for scenarios with both low SNR (< 10) and in Table I suggest an optimum around a = 1.5σ if a precision
large TDC bins (a/σ > 2). degradation due to TDC resolution of ≤ 10% is targeted.
With confidence in the CRB model and the MC simulations, A higher resolution TDC (a < 1.5σ ) does not limit depth
we investigate the optimal TDC resolution for a given temporal precision but is overdesigned and may cost area or power. The
spread. Previous work for imaging arrays [18] shows an CRB projects a continuously growing precision degradation as
optimal value of a to be a ≈ 1.5σ for the case of 1-D the TDC bin width becomes much greater than the temporal
localization. However, in these scenarios, the background noise spread. However, as is shown by the MC results of Fig. 5,
is assumed constant independent of pixel size. For the case the precision plateaus at the quantization noise limit (dashed
of a dToF sensor, in a given environmental scenario with a line). In this scenario, all the signal photons are confined
certain level of ambient illumination, the background rate (r ) is to a single TDC bin so that interpolation between bins is
constant so that the background counts per TDC bin (b) scale not possible, and thus, the precision noise is set only by the
with TDC bin width (b = r a). Fig. 5 shows the localization resolution of the TDC.

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 30,2021 at 04:15:46 UTC from IEEE Xplore. Restrictions apply.
8502609 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 70, 2021

These values are shown as stars in Fig. 6 and lie 13% below
the CRB curve since the Thompson estimation underestimates
the contribution from background photons to the precision
noise. When the temporal spread dominates quantization noise
(σ 2 √ a 2 /12), the background rate (r ) that increases the noise
by 2 follows the number of signal photons received per
pulse and is√inversely proportional to the temporal spread:
r = N/(4P πσ ). The ambient rejection of a dToF sensor
(often reported in lux) is improved by a decrease in the
temporal spread and an increase in laser power or detector
collection efficiency.

B. Tuning Laser Intensity and Exposure Time

How should laser power and exposure time be adjusted for a
Fig. 6. Experimental measurements of precision versus background photons given target precision? Both the signal and background photon
for exposure times of T = 5, 10, 33, and 100 ms. For each exposure time, terms in (4) can be written as the product of a rate of arrival
the CRB model is indicated as a dashed line. Experimental measurements
extracted σ = 745 ps and√a TDC resolution of a = 1067 ps. The point where and the number of laser pulses (P). To clarify the sources
precision is degraded by 2 of the zero background baseline is indicated by of precision noise, we use N = s P, where s is the signal
 as calculated by (4). returned per pulse and b = raP with r the background rate in
photons/s, and find
IV. C ASE S TUDIES ⎡ ⎤
Here, we leverage the established models to provide guid- √ 3 
⎢ 2 a 2
4 πσ r a ⎥
ance relevant to potential dToF design decisions and experi- δ∝ ⎢ σ + + ⎥ 1 (8)
⎣ 12 ⎦
mentally demonstrate dynamic tuning of sensor parameters to spread   a  s  s P
reach a target precision. resolution background

which shows that the precision noise decreases with the inverse
A. Ambient Susceptibility of the square root of the number of laser pulses. The ratio of
At what level of ambient intensity does the precision of a (r a/s) is the inverse of the SNR: SNR = s/(r a) = (N/b),
dToF sensor degrade and how does this ambient susceptibility which depends on the laser intensity, the object distance and
depend on design parameters? Fig. 6 shows the experimental reflectivity, and the ambient illumination. The first two terms
measurements of the timing precision at an object distance in brackets are independent of the SNR, while the impact of
of 1.5 m overlaid with the CRB model versus background the third term upon precision is reduced as the SNR increases.
photons per bin. The measurement is repeated for four expo- At a ratio of total signal photons to background photons per
sure times at varying levels of interfering IR LED intensity. bin of
√
Gaussian fits to the histogram data determined σ = 745 ps N 48 πσ 3
and a TDC resolution of a = 1067 ps. For each exposure = (9)
b 12σ 2 a + a 3
time, the CRB model uses a constant N, σ , and a as
determined from a data set measured with the interfering the background contribution to the noise is equal to the con-
IR LEDs off, while the background signal (b) is determined tributions from the temporal spread and the TDC resolution.
from the experimental data. The measured data cover an SNR This value of N/b is 2.66 when the TDC resolution is given
range of 97.8–0.38 through which the CRB model tracks as a = 2σ . Fig. 7 shows this transition point as  and
the experimental measurements accurately. At the shortest demonstrates two regimes of the precision noise. At low N,
exposure time of 5 ms, the method to localize the histogram the noise is dominated by the background and scales as 1/N,
peak fails at the highest levels of background photons. whereas at high
√ N, the noise is limited by the signal and
The Thompson noise expression provides intuition to deter- scales as 1/ N . Since the power consumption of the laser
mine the level of background beyond which the depth pre- is the product of the number of pulses (P) and the laser
cision is degraded. By equating the left-hand side (temporal photons per pulse (s), an increase in laser power and an
spread and TDC resolution) of (4) to the right-hand side increase in laser pulses (i.e., exposure time) impact the power
(background noise), we find an expression for the number consumption equally. For power-efficient operation with the
of background photons needed to increase the timing noise best possible precision, the laser intensity should be increased
√ so that the background term in (8) contributes minimally. Once
by 2 from the zero background baseline. This number of
background photons per bin or equivalently the product of this signal rate is reached, a reduction in precision noise is
background rate (r ), TDC resolution, and the number of laser accomplished
√ nearly equally by an increase in laser intensity,
√
pulses (P) is found to be δ ∝ 1/ s, or by an increase in exposure time, δ ∝ 1/ P.
The proper parameter to adjust can then be determined by
(σ 2 + a 2 /12)a N constraints, such as motion blurring, laser capabilities, or eye
b = raP = √ . (7)
4 πσ 3 safety.

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 30,2021 at 04:15:46 UTC from IEEE Xplore. Restrictions apply.
KOERNER: MODELS OF dToF SENSOR PRECISION THAT ENABLE OPTIMAL DESIGN AND DYNAMIC CONFIGURATION 8502609

TABLE II
R ESULTS F ROM E XPERIMENTAL M EASUREMENTS T HAT U SE THE CRB M ODEL TO P REDICT P RECISION . F OR E ACH P ROBE M EASUREMENT, THE
P RECISION I S E STIMATED F ROM S INGLE H ISTOGRAMS U SING THE CRB M ODEL (E ST. δ). N , b, AND σ/a A RE THE AVERAGE M EASUREMENTS
FOR THE VALUES D ESCRIBED IN THE T EXT. T HE M EASURED P RECISION (M EAS . δ) I S THE S TANDARD D EVIATION OF THE H ISTOGRAM
L OCALIZATION , W HEREAS THE P REDICTED P RECISION (P RED . δ) I S THE AVERAGE OF THE P REDICTIONS M ADE F ROM E ACH OF
THE P ROBE H ISTOGRAMS W ITH N AND b S CALED AS D ESCRIBED IN THE T EXT. B OTH E ST. δ AND P RED . δ H AVE A O NE
S TANDARD D EVIATION E RROR R ANGE I NDICATED

could be tuned to a shorter exposure time. A single-shot

probe measurement captured at T = 33 ms was used to
determine this updated exposure time by an exhaustive search
of the CRB model with the signal and background photons
scaled by a ratio of the two exposure times. This search
finds the closest available exposure time of T = 10 ms.
At this exposure time, the model predicted a precision of
δ = 3.80 mm and we measured a precision of 4.89 mm. In this
example, the adjustment to the exposure time reduces the
laser power by a factor of ×3.3, thus saving power. Similarly,
a second hypothetical scenario may require more precision
(δ = 1.0 mm) than predicted by the probe measurement,
and thus, the exposure time is tuned to 150 ms (see case 1,
dynamic B). Case 2 extends this dynamic approach to a
scenario with the object distance changing from d to d  , thus
scaling the signal photons received by a factor of (d/d  )2 .
Fig. 7. Demonstration of the two SNR regimes. The number of signal The signal and background values measured at the probe
photons is varied, while the background photons are held constant (b = 30)
with σ = 100 ps and a = 2σ . The precision noise is shown as calculated by position allow for predictions of N and b values at the new
the Thompson expression (4) and the CRB (6). The two regimes are separated object distance. We input these values to the CRB model and
by a dashed line with the background limited regime to the left and the signal predict the updated precision (case 2, dynamic A) or deter-
photon limited regime to the right. The slopes of linear fits to log(N ) versus
log(δ) of the CRB estimate are annotated as m = −0.98 and m = −0.54. mine the required exposure time to maintain the precision
(case 2, dynamic B). The example of case 3 uses the CRB
C. Experimental Demonstration of Dynamic Precision Tuning model to predict precision performance at a longer range with
The CRB precision model can be used to tune sensor con- minimal background given a measurement with considerable
figuration and predict performance. From a single histogram, background at a closer range. Here, the prediction at the
the signal, background, and temporal spread can be extracted updated distance scales the signal photons by (d/d  )2 and
and used as inputs to the model. Table II shows this possibility assumes the background signal to be that measured with no
using experimental measurements captured with the VL53L1X interfering IR intensity. The dynamic A entry of case 3 shows
dToF sensor. Each row of the table represents a specific set of the predicted and measured precisions of 4.71 and 4.84 mm,
distance (d), exposure time (T ), and intensity of interfering IR respectively. Table II shows how the CRB noise modeling
background. For the configuration of each row, 98 measure- allows measurements from a single histogram measurement to
ments were captured with the estimated precision calculated predict precision given configuration adjustments or changes
from single histograms using the CRB model (Est. δ) and the in the environment, such as object distance or background
measured precision evaluated as the standard deviation of the intensity.
histogram localization using a Gaussian fit (Meas. δ). For each
of the three cases in the table, we demonstrate how the CRB V. C ONCLUSION AND D ISCUSSION
precision estimate from a probe measurement can be used Experimental evaluation of dToF sensors often reports depth
to dynamically tune the dToF sensor configuration or predict precision and allowable ambient intensity, yet models that
performance after an environmental change. predict these results from design parameters and the mea-
Consider case 1 in Table II with an object depth of 800 mm sured signal and background are not readily available. The
and an exposure time of T = 33 ms. A hypothetical power- straightforward expression of (4) for depth precision allows for
aware use case (dynamic A) that requires only 0.5% precision dToF design and usage insight and correctly predicts transition

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 30,2021 at 04:15:46 UTC from IEEE Xplore. Restrictions apply.
8502609 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 70, 2021

points between signal limited and background limited pre- ACKNOWLEDGMENT

cision. However, this model underestimates the localization The authors would like to thank undergraduate students
noise because the derivation of the background noise term Ryan Jans and Jacob Komarek for ToF experimental and
considers the pixel size to be infinitesimal. The CRB model analytical support, respectively, and Sarah A. Munro for useful
of (6) integrates the background noise over the pixel size discussions.
and, thus, tracks the MC simulation results more closely
than the Thompson model. As shown in Figs. 2, 3, and 5,
MC simulations with the histogram location extracted by a R EFERENCES
Gaussian fit accurately track the CRB precision estimate ver- [1] M. Perenzoni, D. Perenzoni, and D. Stoppa, “A 64 × 64-pixels digital
sus signal photons, background photons, and TDC resolution, silicon photomultiplier direct TOF sensor with 100-M photons/s/pixel
background rejection and imaging/altimeter mode with 0.14% precision
respectively. The models recommend a TDC resolution of up to 6 km for spacecraft navigation and landing,” IEEE J. Solid-State
a ≈ 1.5σ , determine the background signal beyond which Circuits, vol. 52, no. 1, pp. 151–160, Jan. 2017, doi: 10.1109/JSSC.
precision degrades, and show that an increase in laser intensity 2016.2623635.
[2] VL53L1X Datasheet, 3rd ed., ST, Geneva, Switzerland, Nov. 2018.
is more impactful than an equivalent increase in exposure time [3] M.-J. Lee et al., “High-performance back-illuminated three-dimensional
when the precision is background limited. stacked single-photon avalanche diode implemented in 45-nm CMOS
In addition to design intuition, models that predict preci- technology,” IEEE J. Sel. Topics Quantum Electron., vol. 24, no. 6,
pp. 1–9, Nov. 2018.
sion enable efficient usage of dToF sensors. From a single [4] R. K. Henderson et al., “A 256× 256 40 nm/90 nm CMOS 3D-stacked
return-time histogram, the depth precision can be estimated 120 dB dynamic-range reconfigurable time-resolved SPAD imager,”
to enable sensor configuration adjustments that target a spe- in IEEE Int. Solid-State Circuits Conf. (ISSCC) Dig. Tech. Papers,
Dec. 2019, pp. 106–108.
cific precision. Future work may incorporate precision esti- [5] K. Morimoto et al., “Megapixel time-gated SPAD image sensor for
mation into the embedded system of a dToF sensor used 2D and 3D imaging applications,” Optica, vol. 7, no. 4, pp. 346–354,
for battery-powered applications, such as smart canes for Apr. 2020.
[6] W. Song et al., “The effect of fog on the probability density distribution
visually impaired people [34]. As the sensor moves and object of the ranging data of imaging laser radar,” AIP Adv., vol. 8, no. 2,
distances change, the models can predict the current precision Feb. 2018, Art. no. 025022.
and find an optimal exposure time that meets the required [7] S. Jahromi, J.-P. Jansson, P. Keranen, and J. Kostamovaara, “A 32 × 128
SPAD-257 TDC receiver IC for pulsed TOF solid-state 3-D imaging,”
precision and no better. This tuning may reduce the power IEEE J. Solid-State Circuits, vol. 55, no. 7, pp. 1960–1967, Jul. 2020,
consumption of the system. doi: 10.1109/JSSC.2020.2970704.
Factors related to the sensor optical system, photon prop- [8] A. R. Ximenes, P. Padmanabhan, M.-J. Lee, Y. Yamashita, D.-N. Yaung,
and E. Charbon, “A modular, direct time-of-flight depth sensor in
agation, and object scattering are not explicitly included 45/65-nm 3-D-stacked CMOS technology,” IEEE J. Solid-State Circuits,
in our models. These factors have well-known relationships vol. 54, no. 11, pp. 3203–3214, Nov. 2019, doi: 10.1109/JSSC.2019.
to our model parameters: the received signal photons (N), 2938412.
[9] K. Morimoto and E. Charbon, “High fill-factor miniaturized SPAD
the background photons (b), and the instrument timing spread arrays with a guard-ring-sharing technique,” Opt. Exp., vol. 28, no. 9,
(σ ). This approach of using generalized terms ensures that p. 13068, Apr. 2020, doi: 10.1364/OE.389216.
the models are widely applicable. One could determine a [10] N. A. W. Dutton, T. A. Abbas, F. M. D. Rocca, N. Finlayson, B. Rae, and
R. K. Henderson, “Time of flight imaging and sensing for mobile appli-
specific factor’s effect through the relationship of the factor cations,” in Low-Power Analog Techniques, Sensors for Mobile Devices,
to N, b, and σ . For example, consider the SPAD photon and Energy Efficient Amplifiers. Cham, Switzerland: Springer, 2019,
detection efficiency (PDE), which ranges from 0 to 1. The pp. 221–249.
PDE scales both the signal photons and background photons [11] C. Zhang, S. Lindner, I. M. Antolovic, J. M. Pavia, M. Wolf, and
E. Charbon, “A 30-frames/s, 252 × 144 SPAD flash LiDAR with 1728
detected.
√ By referring to (4), we find δ(PDE)/δ(PDE = 1) = dual-clock 48.8-ps TDCs, and pixel-wise integrated histogramming,”
(1/PDE). Another factor, object reflectively (ρ), directly IEEE J. Solid-State Circuits, vol. 54, no. 4, pp. 1137–1151, Apr. 2019,
impacts the signal photons collected as N ∝ ρ, while doi: 10.1109/JSSC.2018.2883720
[12] E. W. Dennison and R. H. Stanton, “Ultra-precise star tracking using
object reflectivity does not modify the background nor the charge coupled devices (CCDs),” in Smart Sensors II, D. F. Barbe, Ed.
timing spread. A third important parameter, object distance Bellingham, WA, USA: SPIE, Jan. 1980, doi: 10.111712.959483.
(r ), causes the signal photons to fall off as N ∝ 1/r 2 . Object [13] A. Small and S. Stahlheber, “Fluorophore localization algorithms
for super-resolution microscopy,” Nature Methods, vol. 11, no. 3,
distance may impact the timing spread depending on the object pp. 267–279, Feb. 2014, doi: 10.1038/NMETH.2844.
FoV and atmospheric propagation. Two relevant pulsed laser [14] A. M. Wallace, R. C. W. Sung, G. S. Buller, R. D. Harkins,
parameters could be evaluated using our models. The laser R. E. Warburton, and R. A. Lamb, “Detecting and characterising returns
in a pulsed ladar system,” IEE Proc.-Vis., Image Signal Process.,
intensity is directly proportional to the signal photons and vol. 153, no. 2, pp. 160–172, Apr. 2006.
the laser pulsewidth is one contributor to the total system [15] A. M. Wallace, J. Ye, N. J. Krichel, A. McCarthy, R. J. Collins, and
timing spread. Integration of the detector precision model- G. S. Buller, “Full waveform analysis for long-range 3D imaging laser
radar,” EURASIP J. Adv. Signal Process., vol. 2010, no. 1, pp. 1–12,
ing of this work with physics-based optical modeling [27], Dec. 2010.
[28] is a valuable future extension. Other extensions include [16] Xing et al., “A depth-adaptive waveform decomposition method for air-
incorporation of photon pile-up [35], systematic timing errors borne LiDAR bathymetry,” Sensors, vol. 19, no. 23, p. 5065, Nov. 2019,
doi: 10.3390/s19235065.
inherent to merging multiple SPADs or TDCs [36], and linear [17] N. Bobroff, “Position measurement with a resolution and noise-limited
APD LiDAR systems [37], [38]. This work provides dToF instrument,” Rev. Sci. Instrum., vol. 57, no. 6, pp. 1152–1157, Jun. 1986,
designers and users an accessible yet accurate tool to optimize doi: 10.1063/1.1138619.
[18] K. A. Winick, “Cramér–Rao lower bounds on the performance of charge-
measurement performance, including in scenarios with power coupled-device optical position estimators,” JOSA A, vol. 3, no. 11,
constraints. pp. 1809–1815, 1986.

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 30,2021 at 04:15:46 UTC from IEEE Xplore. Restrictions apply.
KOERNER: MODELS OF dToF SENSOR PRECISION THAT ENABLE OPTIMAL DESIGN AND DYNAMIC CONFIGURATION 8502609

[19] R. E. Thompson, D. R. Larson, and W. W. Webb, “Precise nanometer [31] L. J. Koerner. dToF Sensor Noise Simulations and Analysis.
localization analysis for individual fluorescent probes,” Biophys. J., Accessed: Apr. 25, 2021. [Online]. Available: https://github.com/
vol. 82, no. 5, pp. 2775–2783, 200, doi: 10.1016/S0006-3495(02)75618- lucask07/dtof-sim
X. [32] R. M. Jans, A. S. Green, and L. J. Koerner, “Characterization of a
[20] Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal miniaturized IR depth sensor with a programmable region-of-interest
point spread function design for 3D imaging,” Phys. Rev. Lett., vol. 113, that enables hazard mapping applications,” IEEE Sensors J., vol. 20,
no. 13, Sep. 2014, Art. no. 133902. no. 10, pp. 5213–5220, May 2020.
[21] H. Babcock, Y. M. Sigal, and X. Zhuang, “A high-density 3D local- [33] L. Koerner, “Instrbuilder: A Python package for electrical instrument
ization algorithm for stochastic optical reconstruction microscopy,” Opt. control,” J. Open Source Softw., vol. 4, no. 36, p. 1172, Apr. 2019.
Nanoscopy, vol. 1, no. 1, p. 6, 2012. [34] W. Elmannai and K. Elleithy, “Sensor-based assistive devices for
[22] A. J. Berglund, M. D. McMahon, J. J. McClelland, and J. A. Liddle, visually-impaired people: Current status, challenges, and future direc-
“Fast, bias-free algorithm for tracking single particles with variable size tions,” Sensors, vol. 17, no. 3, p. 565, Mar. 2017, doi: 10.3390/
and shape,” Opt. Exp., vol. 16, no. 18, pp. 14064–14075, 2008. s17030565.
[23] R. J. Ober, S. Ram, and E. S. Ward, “Localization accuracy in single- [35] A. Gupta, A. Ingle, and M. Gupta, “Asynchronous single-photon 3D
molecule microscopy,” Biophys. J., vol. 86, no. 2, pp. 1185–1200, imaging,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2019,
Feb. 2004. pp. 7909–7918.
[24] S. Hernandez-Marin, A. M. Wallace, and G. J. Gibson, “Bayesian [36] J. Holma, I. Nissinen, J. Nissinen, and J. Kostamovaara, “Characteriza-
analysis of lidar signals with multiple returns,” IEEE Trans. Pattern tion of the timing homogeneity in a CMOS SPAD array designed for
Anal. Mach. Intell., vol. 29, no. 12, pp. 2170–2180, Dec. 2007. time-gated Raman spectroscopy,” IEEE Trans. Instrum. Meas., vol. 66,
[25] G. Chen, C. Wiede, and R. Kokozinski, “Data processing approaches no. 7, pp. 1837–1844, Jul. 2017.
on SPAD-based d-TOF LiDAR systems: A review,” IEEE Sensors J., [37] K. Pasquinelli, R. Lussana, S. Tisa, F. Villa, and F. Zappa, “Single-
vol. 21, no. 5, pp. 5656–5667, Mar. 2021. photon detectors modeling and selection criteria for high-background
[26] D. Bronzi et al., “Low-noise and large-area CMOS SPADs with timing LiDAR,” IEEE Sensors J., vol. 20, no. 13, pp. 7021–7032, Jul. 2020.
response free from slow tails,” in Proc. Eur. Solid-State Device Res. [38] D. Li, M. Liu, R. Ma, and Z. Zhu, “An 8-ch LIDAR receiver based
Conf. (ESSDERC), Sep. 2012, pp. 230–233. on TDC with multi-interval detection and real-time in-situ calibration,”
[27] P. Padmanabhan, C. Zhang, and E. Charbon, “Modeling and analysis IEEE Trans. Instrum. Meas., vol. 69, no. 7, pp. 5081–5090, Jul. 2020.
of a direct time-of-flight sensor architecture for LiDAR applications,”
Sensors, vol. 19, no. 24, p. 5464, Dec. 2019, doi: 10.3390/s19245464.
[28] M. Beer, J. F. Haase, J. Ruskowski, and R. Kokozinski, “Back-
ground light rejection in SPAD-based LiDAR sensors by adaptive
photon coincidence detection,” Sensors vol. 18, no. 12, p. 4338,
Dec. 2018. Lucas J. Koerner (Member, IEEE) received the B.A. degree (Hons.) in
[29] J. D. Valentine and A. E. Rana, “Centroid and full-width at half max- integrated science, physics, and mathematics from Northwestern University,
imum uncertainties of histogrammed data with an underlying Gaussian Chicago, IL, USA, in 2003, and the Ph.D. degree in physics from Cornell
distribution—The moments method,” IEEE Trans. Nucl. Sci., vol. 43, University, Ithaca, NY, USA, in 2010.
no. 5, pp. 2501–2508, Oct. 1996. Since 2017, he has been an Assistant Professor of electrical and computer
[30] C. R. Rao, “Information and the accuracy attainable in the estimation engineering at the University of St. Thomas, St. Paul, MN, USA. His
of statistical parameters,” in Breakthroughs in Statistics. New York, NY, research interests include electrical instrumentation development, software for
USA: Springer, 1992, pp. 235–247. reproducible research, and image sensors.

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 30,2021 at 04:15:46 UTC from IEEE Xplore. Restrictions apply.