KEMBAR78
Setting up and testing NSG and DGIWG Profiles | PPTX
LUIS BERMUDEZ
Executive Director Innovation Program
April 24, 2018
SETTING UP AND TESTING
NSG AND DGIWG PROFILES
Outline
• OGC Introduction and Standards Overview
• NSG OGC Profiles
• OGC Validation Tools
• OGC Certification
OGC Introduction and
Standards Overview
Data can come from different
places
7
21 Billion Connected Devices in the
Global Internet of Things in 2020
Variety'L'Models'
Short Term Long Term
Variety'L'Models'
Short Term Long Term
What do they have in common?
Variety'L'Models'
Short Term Long Term
Geospatial LocationGeospatial Location
Each layer is like a source
Each source
can be provided
as a service
14
Heterogeneous Services
15
Heterogeneous Services
16
Heterogeneous Services
17
Agreement
18
OGC open standards provide
the common interfaces
e.g. Web Map Service (WMS)
19
Interoperability
Increase ability to access, fuse and
apply diverse content when and
where needed is critical to situational
awareness and disaster planning/
response in cross-boundary and
cross-domain settings.
20
Interoperability in the context of Intelligence,
Surveillance, and Reconnaissance (ISR)
J28 division chief from
http://www.jfcom.mil/newslink/storyarchive/2007/pa071607.htm
doesn’t
have to
worry about
where his
information
is coming
from
plug and
play ISR
capabilities
OGC
22
Open Geospatial Consortium (OGC)
Leading International consortium for advancing open geospatial standards
23
Open Geospatial Consortium (OGC)
• 70 standards
• 550 members
• 5000 users in the portal
• 100 innovation projects
• 230 compliant products
• 900 implementations
24
StandardDefines
operations
Rules
Formal rules
Consensus
and
approved
Open Standards
• Freely and publicly available
• Non discriminatory
• No license fees
• Vendor neutral
• Data neutral
• Based on consensus
26
Standard
… and are widely implemented
For WMS
41 organizations
92 compliant products
3 reference implementations
Randomly selected complete list at
http://www.opengeospatial.org/resource/products
Standards by Purpose
30
WMS
• The OGC Web Map Service Interface Standard
(WMS) defines a set of interfaces (e.g. the rule of
how to encode the URL) for requesting map
images over the Internet.
• WMS makes it easy for a client to request
images on demand changing parameters such
as size and coordinate reference systems.
31
GetMap - Request Example
Link
32
Web Feature Service (WFS)
Web Service used to:
• Publish Features
• Access Features
• Edit Features
33
GetFeature Request
Link
34
Geographic Markup Language (GML)
• Defines an abstract feature model
• Provides XML Schemas
• Provides primitives:
– Geometry
– Feature Types
– Coordinate Reference
Systems
• Topology
35
OGC SensorThings API
Interface for Internet of Things
http://developers.sensorup.com/docs/
36
Querying data from bike stations
• Endpoint: http://toronto-bike-
snapshot.sensorup.com/v1.0/
GeoPackage
What do we want?
http://www.spatiallyadjusted.com/2012/12/20/ogc-draft-geopackage-specification/
Small, Fast, Reliable
GeoPackage Vector Support
Support for Raster
NSG Profiles
NSG Profiles Introduction
• Is National System for Geospatial Intelligence
• …the combination of technology, policies, capabilities, doctrine,
activities, people, data, and communities necessary to produce
geospatial intelligence in an integrated multi-intelligence, multi-
domain environment. (NSG Statement of Strategic Intent, March
2007)
• NSG Defines an enterprise architecture
• The architecture is based on standards
Profile
• is a subset of a standard (selected subset of conformance classes)
• imposes additional constraints on an existing standard
• profile can also be extended to offer specialized functionality (called
profile with extensions)
National Spatial Agency (NSG) Profiles
• NSG GeoPackage 2.1 (raster and vector data), based on
the OGC GeoPackage 1.1
• NSG WMS 1.0 (raster data), based on the OGC WMS 1.3
• NSG WMTS 1.1 (raster data), based on the OGC WMTS
1.0
• NSG WFS 1.0 (vector data), based on the OGC WFS 2.0
Note: These are the only ones that have an OGC Test
Highlights of implementing profiles
• Report: OGC Testbed-13: Executable Test Suites and Reference
Implementations for NSG WMTS 1.0 and WFS 2.0 Profiles with
Extension http://docs.opengeospatial.org/per/17-043.html
NSG WFS 1.0 Operations
• GetCapabilities
• GetPropertyValue
• GetFeature
• DescribeFeatureType
• ListStoredQueries
• DescribeStoredQueries
• LockFeature
• GetFeatureWithLock
• Transaction
• CreateStoredQuery
• DropStoredQuery
• PageResults
NSG WFS 1.0 – new operation: PageResults
• Better navigation
support (e.g. Big Data)
• Requires Servers to
store interim results
Current Pagination Support
Requires sequential
navigation
NSG WFS - PageResults
• 1) get resultSetID
NSG WFS - PageResults
• 2) use the resultSetID to navigate the results
NSG – WFS – Feature Versioning
• restricts the notion of versioning to a time-based versioning
approach
• getting changes that happened since their last update
• Two type of identifiers
– Instance Resource Identifier - uniquely identifies each version of a
certain feature (use by GetResourceByID)
– Entity Resource Identifier - identifies the entity that is represented by
the feature. Multiple versions of an entity should have the same entity
identifier. (entity identifier should be encoded in the name attribute of
the AbstractGML element
NSG WMTS Profile
• Requires REST interface to be exposed
• Coordinate systems CRS:84 WGS84 and EPSG:4326 WGS84
• Support for projections EPSG:3395, EPSG:5041 and EPSG:5042.
Scales for World Mercator EPSG:3395
• MIME types, image formats: image/png, image/jpeg and image/gif
• GetFeatureInfo request must
support text/xml, text/html and application/gml+xml; version=3.2
• Caching information (expiration data) for the data must be provided
using HTTP control headers.
NSG WMS Profile
• A DGIWG WMS server shall be able to provide all mandatory and
optional service metadata elements based on NSG Metadata
Foundation (NMF) .
• Coordinate systems CRS:84 WGS84 and EPSG:4326 WGS84
• MIME types for raster
formats: image/png, image/jpeg and image/gif
• If vector based server should provide image/svg+xml
• Keywords lists …
NSG WMS Profile
• A DGIWG WMS server shall be able to provide all mandatory and
optional service metadata elements based on NSG Metadata
Foundation (NMF) .
• Coordinate systems CRS:84 WGS84 and EPSG:4326 WGS84
• MIME types for raster
formats: image/png, image/jpeg and image/gif
• If vector based server should provide image/svg+xml
• Keywords lists …
NSG WMS Profile
• A DGIWG WMS server shall always provide at least one style element
and that style shall be advertised even if it's only the default style.
• The MaxWidth and MaxHeight shall be greater or equal to 800 pixels.
• Each vector data layer's style shall have an associated legend,
available as an image in one of the following formats: PNG, GIF or
JPEG and available via LegendURL.
• A DGIWG WMS service should be configured to serve requests that
range from at least 4x to at least 0.25x the native scale(s) or
resolution(s) of the underlying datasets".
NSG GeoPackage
• NSG tile matrix sets
• NSG adopted GeoPackage extensions
• Metadata compliant with the NSG Metadata Foundation (NMF)
Version 2.2
Reference Implementations
• Demo setting up reference implementations
Compliance Testing - Tests
• https://github.com/opengeospatial/ets-wfs20-nsg
• https://github.com/opengeospatial/ets-wms13-nsg
• https://github.com/opengeospatial/ets-wmts10-nsg/
• https://github.com/opengeospatial/ets-gpkg12-nsg
Compliance Testing – Web validator
Beta Environment
• http://cite.opengeospatial.org/te2/
Demo Using Docker
• Link and detail about the demo can be found here:
https://goo.gl/HG67wd
• Main page with information about running the OGC validation tool
in Docker: https://github.com/opengeospatial/teamengine-docker
OGC Certification
63
64
65
How long this it take?
• Once the application is passing the test (1-2
months), organizations asking for compliance can:
• Make an online request 5-10 minutes.
• OGC will verify and respond in 1-3 days.
66
What if your application is not passing the test?
• Use the community forum
citeforum@lists.opengeospatial.org
• About 300 members participate in the forum:
– Each test suite has a lead
– OGC has alliances with groups supporting the
test (answering questions, fixing bugs, etc.)
67
68
Tests available
for certification
Validation
portal
http://cite.open
geospatial.org/t
eamengine/
69
Hos to ask for Compliance
“ Purchaser seeks geospatial products that provide
maximum interoperability with purchaser’s and data
sharing partner’s systems. To accomplish this, purchaser
seeks to make maximum use of open standards provided
by the Open Geospatial Consortium (OGC), ISO ...”
OGC Compliance Overview - Guide for Software Acquisition White paper (15-002r5)
http://docs.opengeospatial.org/wp/15-002r5/15-002r5.html
70
How to
verify that
a product
is
compliant?
71
Links
• OGC validation portal:
https://github.com/opengeospatial/teamengine
• Source code Test code repositories
https://github.com/opengeospatial?page=2&query=ets
• Building the testing facility on the cloud
https://github.com/opengeospatial/teamengine-builder/blob/master/build-amazon-instance.rst
• Virtualization Guide
https://github.com/opengeospatial/teamengine/blob/master/src/site/asciidoc/virt-guide.adoc
Contact Information
• Luis Bermudez
• lbermudez@opengeospatial.org
• Twitter: @berdez
• https://www.linkedin.com/in/bermudez

Setting up and testing NSG and DGIWG Profiles

  • 2.
    LUIS BERMUDEZ Executive DirectorInnovation Program April 24, 2018
  • 3.
    SETTING UP ANDTESTING NSG AND DGIWG PROFILES
  • 4.
    Outline • OGC Introductionand Standards Overview • NSG OGC Profiles • OGC Validation Tools • OGC Certification
  • 5.
  • 6.
    Data can comefrom different places
  • 7.
    7 21 Billion ConnectedDevices in the Global Internet of Things in 2020
  • 8.
  • 10.
    Variety'L'Models' Short Term LongTerm What do they have in common?
  • 11.
  • 12.
  • 13.
    Each layer islike a source Each source can be provided as a service
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
    18 OGC open standardsprovide the common interfaces e.g. Web Map Service (WMS)
  • 19.
    19 Interoperability Increase ability toaccess, fuse and apply diverse content when and where needed is critical to situational awareness and disaster planning/ response in cross-boundary and cross-domain settings.
  • 20.
    20 Interoperability in thecontext of Intelligence, Surveillance, and Reconnaissance (ISR) J28 division chief from http://www.jfcom.mil/newslink/storyarchive/2007/pa071607.htm doesn’t have to worry about where his information is coming from plug and play ISR capabilities
  • 21.
  • 22.
    22 Open Geospatial Consortium(OGC) Leading International consortium for advancing open geospatial standards
  • 23.
    23 Open Geospatial Consortium(OGC) • 70 standards • 550 members • 5000 users in the portal • 100 innovation projects • 230 compliant products • 900 implementations
  • 24.
  • 25.
    Open Standards • Freelyand publicly available • Non discriminatory • No license fees • Vendor neutral • Data neutral • Based on consensus
  • 26.
    26 Standard … and arewidely implemented For WMS 41 organizations 92 compliant products 3 reference implementations Randomly selected complete list at http://www.opengeospatial.org/resource/products
  • 27.
  • 30.
    30 WMS • The OGCWeb Map Service Interface Standard (WMS) defines a set of interfaces (e.g. the rule of how to encode the URL) for requesting map images over the Internet. • WMS makes it easy for a client to request images on demand changing parameters such as size and coordinate reference systems.
  • 31.
    31 GetMap - RequestExample Link
  • 32.
    32 Web Feature Service(WFS) Web Service used to: • Publish Features • Access Features • Edit Features
  • 33.
  • 34.
    34 Geographic Markup Language(GML) • Defines an abstract feature model • Provides XML Schemas • Provides primitives: – Geometry – Feature Types – Coordinate Reference Systems • Topology
  • 35.
    35 OGC SensorThings API Interfacefor Internet of Things http://developers.sensorup.com/docs/
  • 36.
    36 Querying data frombike stations • Endpoint: http://toronto-bike- snapshot.sensorup.com/v1.0/
  • 37.
  • 38.
    What do wewant? http://www.spatiallyadjusted.com/2012/12/20/ogc-draft-geopackage-specification/
  • 39.
  • 40.
  • 41.
  • 42.
  • 43.
    NSG Profiles Introduction •Is National System for Geospatial Intelligence • …the combination of technology, policies, capabilities, doctrine, activities, people, data, and communities necessary to produce geospatial intelligence in an integrated multi-intelligence, multi- domain environment. (NSG Statement of Strategic Intent, March 2007) • NSG Defines an enterprise architecture • The architecture is based on standards
  • 44.
    Profile • is asubset of a standard (selected subset of conformance classes) • imposes additional constraints on an existing standard • profile can also be extended to offer specialized functionality (called profile with extensions)
  • 45.
    National Spatial Agency(NSG) Profiles • NSG GeoPackage 2.1 (raster and vector data), based on the OGC GeoPackage 1.1 • NSG WMS 1.0 (raster data), based on the OGC WMS 1.3 • NSG WMTS 1.1 (raster data), based on the OGC WMTS 1.0 • NSG WFS 1.0 (vector data), based on the OGC WFS 2.0 Note: These are the only ones that have an OGC Test
  • 46.
    Highlights of implementingprofiles • Report: OGC Testbed-13: Executable Test Suites and Reference Implementations for NSG WMTS 1.0 and WFS 2.0 Profiles with Extension http://docs.opengeospatial.org/per/17-043.html
  • 47.
    NSG WFS 1.0Operations • GetCapabilities • GetPropertyValue • GetFeature • DescribeFeatureType • ListStoredQueries • DescribeStoredQueries • LockFeature • GetFeatureWithLock • Transaction • CreateStoredQuery • DropStoredQuery • PageResults
  • 48.
    NSG WFS 1.0– new operation: PageResults • Better navigation support (e.g. Big Data) • Requires Servers to store interim results
  • 49.
  • 50.
    NSG WFS -PageResults • 1) get resultSetID
  • 51.
    NSG WFS -PageResults • 2) use the resultSetID to navigate the results
  • 52.
    NSG – WFS– Feature Versioning • restricts the notion of versioning to a time-based versioning approach • getting changes that happened since their last update • Two type of identifiers – Instance Resource Identifier - uniquely identifies each version of a certain feature (use by GetResourceByID) – Entity Resource Identifier - identifies the entity that is represented by the feature. Multiple versions of an entity should have the same entity identifier. (entity identifier should be encoded in the name attribute of the AbstractGML element
  • 53.
    NSG WMTS Profile •Requires REST interface to be exposed • Coordinate systems CRS:84 WGS84 and EPSG:4326 WGS84 • Support for projections EPSG:3395, EPSG:5041 and EPSG:5042. Scales for World Mercator EPSG:3395 • MIME types, image formats: image/png, image/jpeg and image/gif • GetFeatureInfo request must support text/xml, text/html and application/gml+xml; version=3.2 • Caching information (expiration data) for the data must be provided using HTTP control headers.
  • 54.
    NSG WMS Profile •A DGIWG WMS server shall be able to provide all mandatory and optional service metadata elements based on NSG Metadata Foundation (NMF) . • Coordinate systems CRS:84 WGS84 and EPSG:4326 WGS84 • MIME types for raster formats: image/png, image/jpeg and image/gif • If vector based server should provide image/svg+xml • Keywords lists …
  • 55.
    NSG WMS Profile •A DGIWG WMS server shall be able to provide all mandatory and optional service metadata elements based on NSG Metadata Foundation (NMF) . • Coordinate systems CRS:84 WGS84 and EPSG:4326 WGS84 • MIME types for raster formats: image/png, image/jpeg and image/gif • If vector based server should provide image/svg+xml • Keywords lists …
  • 56.
    NSG WMS Profile •A DGIWG WMS server shall always provide at least one style element and that style shall be advertised even if it's only the default style. • The MaxWidth and MaxHeight shall be greater or equal to 800 pixels. • Each vector data layer's style shall have an associated legend, available as an image in one of the following formats: PNG, GIF or JPEG and available via LegendURL. • A DGIWG WMS service should be configured to serve requests that range from at least 4x to at least 0.25x the native scale(s) or resolution(s) of the underlying datasets".
  • 57.
    NSG GeoPackage • NSGtile matrix sets • NSG adopted GeoPackage extensions • Metadata compliant with the NSG Metadata Foundation (NMF) Version 2.2
  • 58.
    Reference Implementations • Demosetting up reference implementations
  • 59.
    Compliance Testing -Tests • https://github.com/opengeospatial/ets-wfs20-nsg • https://github.com/opengeospatial/ets-wms13-nsg • https://github.com/opengeospatial/ets-wmts10-nsg/ • https://github.com/opengeospatial/ets-gpkg12-nsg
  • 60.
    Compliance Testing –Web validator Beta Environment • http://cite.opengeospatial.org/te2/
  • 61.
    Demo Using Docker •Link and detail about the demo can be found here: https://goo.gl/HG67wd • Main page with information about running the OGC validation tool in Docker: https://github.com/opengeospatial/teamengine-docker
  • 62.
  • 63.
  • 64.
  • 65.
    65 How long thisit take? • Once the application is passing the test (1-2 months), organizations asking for compliance can: • Make an online request 5-10 minutes. • OGC will verify and respond in 1-3 days.
  • 66.
    66 What if yourapplication is not passing the test? • Use the community forum citeforum@lists.opengeospatial.org • About 300 members participate in the forum: – Each test suite has a lead – OGC has alliances with groups supporting the test (answering questions, fixing bugs, etc.)
  • 67.
  • 68.
  • 69.
    69 Hos to askfor Compliance “ Purchaser seeks geospatial products that provide maximum interoperability with purchaser’s and data sharing partner’s systems. To accomplish this, purchaser seeks to make maximum use of open standards provided by the Open Geospatial Consortium (OGC), ISO ...” OGC Compliance Overview - Guide for Software Acquisition White paper (15-002r5) http://docs.opengeospatial.org/wp/15-002r5/15-002r5.html
  • 70.
    70 How to verify that aproduct is compliant?
  • 71.
    71 Links • OGC validationportal: https://github.com/opengeospatial/teamengine • Source code Test code repositories https://github.com/opengeospatial?page=2&query=ets • Building the testing facility on the cloud https://github.com/opengeospatial/teamengine-builder/blob/master/build-amazon-instance.rst • Virtualization Guide https://github.com/opengeospatial/teamengine/blob/master/src/site/asciidoc/virt-guide.adoc
  • 72.
    Contact Information • LuisBermudez • lbermudez@opengeospatial.org • Twitter: @berdez • https://www.linkedin.com/in/bermudez

Editor's Notes

  • #8 Global Internet of Things Gartner group
  • #21 The goal is that we get ISR to the point where the warfighter doesn’t have to worry about where his information is coming from, he just knows that we have the capability to “plug and play” ISR capabilities in to the mix so that his information needs are met – with timeliness and precision
  • #39 What is this guy talking about? - How can I share something that is SQLlite contained That i based on one of the most popular vector data models - Simple Features That can support raster and tiles