3.1 Achievement of major science objectives
A primary and fundamental accomplishment in the past 3 years has been
the successful achievement of all major science objectives of the
instrument. These objectives include the detection of lightning during
day and night, with storm-scale spatial resolution, millisecond timing,
and high flash detection efficiency without a land/ocean bias. ISS LIS
was designed to measure radiant energy and provide background
images/intensity, and its deployment on the ISS enables the delivery of
realtime lightning data. These objectives were achieved by successfully
meeting several instrument/platform requirements. Most of these are
discussed in Appendix A. Below we focus on validation of timing and
geolocation accuracy, detection efficiency, and false alarm rate. Note
that, as detailed in Appendix A, we expect an improvement of 2-5% in
many of the instrument performance parameters discussed below based on
planned future improvements to ISS LIS data processing.
Like TRMM LIS, ISS LIS has a frame rate of 500 s-1.
This implies a native 2-ms timing precision. The actual on-orbit timingaccuracy of ISS LIS was determined by comparison against multiple
ground-based and spaceborne reference datasets. The ground-based
reference datasets were the EarthNetworks Global Lightning Network
(ENGLN) [e.g., Marchand et al. , 2019], which operates
wideband sensors (1 Hz to 12 MHz), and the Vaisala Global Lightning
Dataset 360 (GLD360) [e.g., Rudlosky et al. , 2017], which
detects waveforms in the very low frequency range (VLF;
~500 Hz to ~50 kHz). Both of these are
global datasets. The spaceborne reference datasets were the
Geostationary Operational Environmental Satellite (GOES) 16 and 17
Geostationary Lightning Mappers (GLM-16 and GLM-17) [Goodman et
al. , 2013, Rudlosky et al. , 2018]. GLM is built on the LIS/OTD
optical detection heritage and is sensitive to both intracloud (IC) and
cloud-to-ground (CG) lightning.
The timing data as initially received from ISS exhibited offsets up to ±
1 s with respect to the reference data, with an alternating drift
pattern that cycled approximately every 9 days. This drift was
accurately characterized based on careful analysis. On the basis of
this, timing correction variables, and an additional constant offset,
were applied to produce the current timing accuracy (Fig. 3 left).
After correction, ISS LIS has modal temporal offset of +1 ms, or
approximately one-half the LIS frame duration, when compared with
GLM-16/17 (Fig. 3 left). The standard deviation in the offset is less
than 2 ms in either direction, compared to GLM-16/17. This comparison
was based on group timings between ISS LIS and the GLM datasets.
Relative to the ground-based reference datasets, there is zero modal
offset, and the standard deviation is also below 2 ms. Thus, after
correction for the ISS timing errors, the ISS LIS timing accuracy is
less than the native timing precision of the instrument itself (2 ms).
This is similar to the independent analysis performed by Erdmann
et al. [2020], using a different reference dataset, and is
consistent with (though not as precise as) the timing analysis for TRMM
LIS by Bitzer and Christian [2015].
ISS LIS geolocation accuracy was analyzed through a coordinate system
transform technique that allowed LIS location errors with respect to the
reference data to be displayed in the native LIS field-of-view. This
analysis revealed that the ISS navigation variables used for LIS
geolocation vary systematically during each orbit, creating location
errors of up to 25 km. This particular issue was unrelated to the TRMM
LIS geolocation issue discussed by Zhang et al. [2019]. The
ISS LIS team worked diligently to troubleshoot the initial issues with
geolocation (and timing), as these posed complex interconnected problems
that required focused analysis and skilled interpretations to resolve.
An iterative tuning process resolved these initial problems and produced
corrected geolocation data and the current analysis of ISS LIS spatial
accuracy (Fig. 3 right). Relative to GLM-16/17, corrected ISS LIS
spatial offsets are almost entirely less than 10 km, with the vast
majority below 5 km. Offsets relative to ENGLN and GLD360 are
distributed more broadly, but for each of the spaceborne and
ground-based reference datasets the modal offset is approximately 2-3
km. This means that ISS LIS has achieved sub-pixel (< 4 km)
location accuracy. The independent analysis by Erdmann et al.[2020] supports this assessment.
Timing, location, flash DE, and false alarm rate (FAR) have been stable
during most of the mission to date. Figure 4 shows the time series of
these parameters through early 2020. ISS LIS temporal accuracy offset
shifted by about 1 ms on 16 December 2018, such that LIS now slightly
leads the reference data (Fig. 4 top). A few larger deviations occurred
during two periods in early 2019, related to atypical ISS maneuvers.
However, in most circumstances the absolute magnitude of the offsets
remain less than 2 ms. LIS geolocation accuracy also has been stable
throughout the mission (Fig. 4 middle), with the modal peak of the
offset normally less than 5 km. The detection performance of ISS LIS is
also stable over time (Fig. 4 bottom), aside from the known deviations
in early 2019 mentioned above. The flash DE (calculated using a 50-km,
200-ms matching window) is stable with respect to each of the reference
datasets (64% relative to GLM, and 56-57% relative to ENGLN/GLD360),
and the FAR, calculated over the Americas domain where the greatest
quantity and best quality reference data are available, is under 5% on
average. This is a higher FAR than that published for TRMM LIS
[Boccippio et al. , 2002], but that study did not include the
impact of specular reflections (e.g., cloud and ocean glint) like this
analysis did. The flash DE is comparable to the values computed
independently for ISS LIS by Erdmann et al. [2020].
Detection efficiency was also examined as a function of time of day
(Fig. 5). As expected for optical instruments like ISS LIS, flash DE is
maximized during local nighttime (64-75% near local midnight, depending
on reference dataset) and minimized during local daytime (51-65% around
1700 LT, just before sunset/dusk). There is another apparent minimum in
flash DE against GLM-16, around sunrise (~0600 LT);
however, the analysis includes a period of time before the GLM-16
blooming filter was implemented to reduce the impact of solar glint.
Since a similar sunrise decrease is not observed in the glint-filtered
GLM-17 data, we infer that this reduction in DE against GLM-16 is
primarily caused by increased GLM-16 false alarms during local sunrise.
Analysis of TRMM LIS DE versus the ground-based reference datasets
indicates that ISS LIS DE is approximately 4-7% lower overall compared
to TRMM LIS (using 2014-2015 ENGLN and GLD360 data; not shown). As
mentioned previously (and explained in Appendix A), we expect increases
of ~2-5% in flash DE after planned ISS LIS dataset
processing improvements. In addition, the ground networks often improve
each year [e.g., Rudlosky et al. , 2017], and this may also
explain some of the DE discrepancy between the ISS LIS and TRMM LIS eras
(i.e., the ground networks are likely detecting more lightning now
versus 5+ years ago). Finally, there is a possibility that ISS LIS is
slightly less sensitive than TRMM LIS. Future work will attempt to
quantify any LIS instrument sensitivity differences, if they exist.
Note that none of the above flash comparisons follows the Bayesian
approach of Bitzer et al. [2016]. That study found a
similarly low TRMM LIS detection efficiency against ground networks
(~53%), but when corrected within a Bayesian
probability framework the estimated detection efficiency increased to
80%. Bayesian analysis of ISS LIS detection efficiency is planned in
the future.