Analysis Ready Data workshop - OGC presentation

®
Data Cubes and ARD
Remote Sensing and GIS
based on open standards
George Percivall
OGC CTO and Chief Engineer
14 August 2018
Copyright © 2018 Open Geospatial Consortium

OGC
®
The OGC Mission
Global forum of developers and users
of spatial data products and services
Open international standards for
geospatial interoperability.
Source: 3d Stadtmodell Berlin
Source: One GeologySource: Space Time Toolkit

OGC
®
Geoscience Australia Data Cube
•Paradigm Shift from Scenes to Pixels
and Arrays
•Data Cube = Time-series multi-
dimensional (space, time, data type)
stack of spatially aligned pixels for
efficient data access and analysis.
•Proven by Geoscience Australia (GA)
and the Australian CSIRO Agency
•Analysis Ready Data (ARD) ...
Dependent on processed products to
reduce technical burden on users
•Discrete Global Grid Systems (DGGS)
TIME
http://www.datacube.org.au
IGARSS 2013 – Melbourne Australia
OGC TC 2015 – Sydney, Australia

OGC
®
EarthServer: Datacubes At Your Fingertips
• Operational Agile Analytics
– Earth Science (3D sat image timeseries, 4D weather); Planetary Science
– EU rasdaman + US NASA WorldWind
– Strictly standards: WMS + WCS + WCPS
• Massive datacubes
– 2.5 PB  220 PB
• Intercontinental initiative, 6y:
EU+US+AUS
• www.earthserver.eu,
www.planetserver.eu

OGC
®
Geospatial Data Cubes
• Objectives
– Interoperability and Federation
• Information model
– Geospatial Coverages
• Data preparation
– Analysis Ready Data
• Data cube analytics
– Access and queries
• Interoperability
– APIs and Encodings

OGC
®
Interoperability of Remote Sensing and GIS
Interoperability:
By visualization-> Maps -> WMS
By semantics -> Coverages -> WCS
GIS
Abstraction of real world
Discrete Geometry/Planar
“Naïve Physics”
Remote Sensing
Measurement of real world
Grids on a Sphere
Physics
WMS
WCS
HAYES, P. J. The naivephysics manifesto.
In ExpertSystems inthe Micro-Electronic Age, D. Michie, Ed.,
Edinburgh UniversityPress,Edinburgh, UK,1979,pp. 242–270.

GeospatialInteroperabilityOffice OGC WMTS access to Landsat, 2004
• WMS Global Mosaic
– 1st global, systematically compiled, ortho-rectified,
generally cloud-free Landsat dataset
– 8,500 orthorectified Landsat 7 ETM+ scenes: NASA-
Stennis and Earthsat, Inc.
– Mosaiced at JPL
• OnEarth WMS/WMTS Mosaic Server
– Developed by Lucian Plesea, JPL
– Demonstrated at United Nations, Dec 2002
– Operational until 2011, replaced by NASA GIBS
Original slide source: George Percivall, NASA/GIO, 200

OGC
®
NASA’s Global Imagery Browse System GIBS
Source: Ryan Boller, NASA
http://earthdata.nasa.gov/wiki/main/index.php/Global_Imagery_Browse_Services
Access Methods
• OGC WMTS
• Tiled WMS
• OGC KML
Supported
Clients
• OpenLayers
• Google Earth
• NASA World
Wind
Open to any
client using
OGC open
standards

OGC
®
OGC Tile Matrix Set Standard
• Indexes space as regular grids
with scales in a CRS
• XML and JSON encodings for
tile matrix sets
• For global projections and
specific regions.
Tile Matrix Set originally defined in WMTS
• Being finalized as a stand alone spec
• Reuse in several standards, e.g., GeoPackage
Example of OGC Building Block approach

OGC
®
Geospatial Coverages
• Geospatial Coverage
– “spatial function” or “field”
– From spatial domain to values range
• Continuous Coverage
– Positions return a value
– May involve interpolation
– e.g. predictive model outputs
• Discrete Coverage
– Associates a value to positions
within a Geometry or Grid
– e.g. imagery, land use
OGC Abstract Specification – Topic 6: Coverage Geometry and Functions
Also published as ISO 19123-1

OGC
®
Web Coverage Service (WCS)
• WCS Core: Simple & efficient access to spatio-temporal
coverages, in any suitable format
– subset = trim | slice
• WCS Extensions: additional, optional functionality facets
• WCS Application Profiles: domain-oriented bundling
11Copyright © 2018 Open Geospatial Consortium

OGC
®
Coverage Analytics – OGC WCPS
• Web Coverage Processing Service (WCPS)
= spatio-temporal datacube analytics language
for $c in ( M1, M2, M3 )
where some( $c.nir > 127 )
return encode( $c.red - $c.nir, “image/tiff“ )
• "From MODIS scenes M1, M2, M3:
difference between red & nir, as TIFF"
•…but only those where nir exceeds 127 somewhere

OGC
®
GeoPlatform – WMS/WCS datasets
An example of WMS/WCS
implementations from US Data.Gov

OGC
®
Evolution of OWS Standards
• OWS standards: WMS, WMTS, WFS, WCS, SOS
– Initially adopted by OGC and ISO more than 10 years ago
– Widely implemented with compliant implementations
• API and Resource-oriented techniques
– OGC API White Paper, USGS API Study, W3C Spatial Data on Web
– Proliferation of public, proprietary APIs degrades interoperability
– OpenAPI tools, Geospatial Resources, OGC Building Blocks
• WFS 3.0 revised as open consensus standard
– No longer requires GML; Defined using OpenAPI on GitHub
– WFS 3.0 Hackathon with STAC at USGS Ft Collins, March 2018
– WFS 3.0 draft call for comments to 31 December 2018
• http://www.opengeospatial.org/pressroom/pressreleases/2830

OGC
®
Discrete Global Grid Systems
“…a spatial reference system that uses a hierarchical
tessellation of cells to partition and address the globe.
DGGS are characterized by the properties of their cell
structure, geo-encoding, quantization strategy and
associated mathematical functions.”
– OGC DGGS Candidate Standard

OGC
®
Notes about ARD based on this workshop
• ARD as an endpoint
– ARD is what ever needs to be done to your data to get it ready for
Analysis
– High variability of what is required for ARD; No single ARD
• ARD as a starting point
– ARD is ready if you can do the analysis you need
– Algorithmic framework to define what ARD needs to support
– Consistency across sensors (Environmental Data Records?)
– Geophysical parameter names and meaning
• Remote sensing has yet to become a commodity
– Exquisite processing of individual scenes is the norm
– Data is sensor focused; needs to be natural process measurements

OGC
®
Separation of ARD and Analytics
eric
[3].
ata
ate
for
een
y as
orm
the
for
the
hus
e’s
orm
ing
on,
Fig. 3. Reference architecture for a generalized Earth Observation analytics
system.
III. EARTHDATA CLOUD ANALYTICS PLATFORM
ARCHITECTURE
Despite the diversity of users and use cases, it is nonetheless
possible to define a platform that provides some value to all of
Fig. 2. Abstract workflow for remote sensing data analytics.
“Generalizing a Data Analysis Pipeline in the Cloud to Handle Diverse
Use Cases in NASA's EOSDIS” Chris Lynnes NASA at IGARSS 2018

OGC
®
EO Exploitation Platforms Working Group
• Several Platforms for Exploitation of Earth Observation
(EO) data are being developed by public and private
companies
• New OGC Domain Working Group
– Open forum for discussion of interoperability requirements, use
cases, pilots, and implementations
– Consider the application of OGC standards on existing and future
platforms
– DWG will not define a new platform or a new architecture
• Leadership includes Cristiano Lopes, ESA

OGC
®
Geospatial Data Cubes
• Geospatial Data Cube Objectives
– Interoperability and Federation
• Information model
– Geospatial Coverages
• Data preparation
– Analysis Ready Data
• Data cube analytics
– Resource access and queries
• Interoperability
– APIs and Encodings

OGC
®
Innovation and Standards
OGC as a Standards Developing Organization (SDO)
1. OGC Community Standards adoption process
2. OGC Innovation Program and OGC Standards Program
No
Standard
Public, but
Proprietary
Standard
Open
Consensus
Standard
2. Anticipatory Standard
developed by an SDO
0. De Facto Standard
in the marketplace
1. Intellectual Property assigned
to an SDO for management
In complex systems, standards are stable intermediate forms
necessary for innovation and evolution.
Herbert Simon, in The Architecture of Complexity as found in
http://www.opengeospatial.org/blog/2559

Analysis Ready Data workshop - OGC presentation

More Related Content

What's hot

Similar to Analysis Ready Data workshop - OGC presentation

More from George Percivall

Recently uploaded

Analysis Ready Data workshop - OGC presentation