Download to read offline
![Resource Description Framework (RDF)
• Represent information on the Web
• Consists of three-element directed graphs a.k.a triples
6.9.2017
12
Ev
e1
ep:
EventObject
:loc1
geo:
lat
60.158776
geo:long
24.881490
ep:hasEventObjectSamplingTime
2014-01-07T09:18:21 geo:alt
122.37
<Eve1> a ep:EventObject;
ssn:Sensor :loc1;
geo:Point [ geo:lat 60.158776 ; geo:long 24.881490 ; ] ;
ep:hasEventObjectSamplingTime "2014-01-07T09:18:21"^^xsd:dateTime .
P I,
P III
Turtle
serialization
format](https://image.slidesharecdn.com/eventprocessinglectiov1-170906085504/85/Event-Processing-Using-Semantic-Web-Technologies-12-320.jpg)
![Remote SPARQL Endpoint for
Enriching Events
• Remote access to
background knowledge
6.9.2017
20
P III
PREFIX omgeo: <http://www.ontotext.com/owlim/geo#>
PREFIX ff: <http://factforge.net/>
INSERT { GRAPH ?translated {
?event :locationName ?label .
?s ?p ?o } }
WHERE { GRAPH ?g {
FILTER ( strStarts(str(?g),concat(str(<>),"Input-Eve")) )
?s ?p ?o .
?event a ep:EventObject ;
geo:Point [ geo:lat ?lat; geo:long ?long ] .
SERVICE <http://factforge.net/sparql> {
?location omgeo:nearby(?lat ?long "1km");
ff:preferredLabel ?label }
BIND (IRI(concat("Translated-",strAfter(str(?event),str(<>))))
AS ?newgraph) } }
5) Linking Open Data cloud diagram 2017, by Andrejs Abele,
John P. McCrae, Paul Buitelaar, Anja Jentzsch and Richard
Cyganiak. http://lod-cloud.net/
5)
Enrich](https://image.slidesharecdn.com/eventprocessinglectiov1-170906085504/85/Event-Processing-Using-Semantic-Web-Technologies-20-320.jpg)
![Resource Description Framework (RDF)
• Represent information on the Web
• Consists of three-element directed graphs a.k.a triples
6.9.2017
12
Ev
e1
ep:
EventObject
:loc1
geo:
lat
60.158776
geo:long
24.881490
ep:hasEventObjectSamplingTime
2014-01-07T09:18:21 geo:alt
122.37
<Eve1> a ep:EventObject;
ssn:Sensor :loc1;
geo:Point [ geo:lat 60.158776 ; geo:long 24.881490 ; ] ;
ep:hasEventObjectSamplingTime "2014-01-07T09:18:21"^^xsd:dateTime .
P I,
P III
Turtle
serialization
format](https://image.slidesharecdn.com/eventprocessinglectiov1-170906085504/75/Event-Processing-Using-Semantic-Web-Technologies-12-2048.jpg)
![Remote SPARQL Endpoint for
Enriching Events
• Remote access to
background knowledge
6.9.2017
20
P III
PREFIX omgeo: <http://www.ontotext.com/owlim/geo#>
PREFIX ff: <http://factforge.net/>
INSERT { GRAPH ?translated {
?event :locationName ?label .
?s ?p ?o } }
WHERE { GRAPH ?g {
FILTER ( strStarts(str(?g),concat(str(<>),"Input-Eve")) )
?s ?p ?o .
?event a ep:EventObject ;
geo:Point [ geo:lat ?lat; geo:long ?long ] .
SERVICE <http://factforge.net/sparql> {
?location omgeo:nearby(?lat ?long "1km");
ff:preferredLabel ?label }
BIND (IRI(concat("Translated-",strAfter(str(?event),str(<>))))
AS ?newgraph) } }
5) Linking Open Data cloud diagram 2017, by Andrejs Abele,
John P. McCrae, Paul Buitelaar, Anja Jentzsch and Richard
Cyganiak. http://lod-cloud.net/
5)
Enrich](https://image.slidesharecdn.com/eventprocessinglectiov1-170906085504/75/Event-Processing-Using-Semantic-Web-Technologies-20-2048.jpg)
More Related Content
Similar to Event Processing Using Semantic Web Technologies
Event Processing Using Semantic Web Technologies
- 1. Event Processing Using SemanticWeb Technologies lectio praecursoria Mikko Rinne, 1st of September 2017
- 2. The Real-Time SemanticWeb? • The availability of networked sensors is increasing rapidly • Fast stream processing of large quantities of data is becoming a critical competitive advantage • Example application areas e.g. retail, investment market, fraud detection, emergency response, health services and farming • Highly distributed, loosely coupled solutions based on common standards are required • Challenge to proprietary platforms • Semantic web standards RDF, SPARQL and OWL offer a good base for interoperability • How would they work for event processing? 6.9.2017 2
- 3.
- 4. Events And EventObjects • Event = Anything that happens, or that can be contemplated as happening1) • Event Object = A record of observations in a system environment 6.9.2017 4 Measurements Observations Event Event Object 1) Luckham, D., Schulte, R.: Event processing glossary – version 2.0 (Jul 2011)
- 5. Event Object Categories 6.9.2017 5 (Simple) Event (Simple) EventComplex Event (Simple) Event CompositeEvent Seppo came in Seppo, Mikko and Esko are in. Meeting started in time (Simple) Event (Simple) Event Mikko came in (Simple) Event (Simple) Event Esko came in It is 9 a.m. Summarizes, represents, or denotes a set of other events1) 1) Luckham, D., Schulte, R.: Event processing glossary – version 2.0 (Jul 2011) A derived event created by combining a set of other simple or complex events in a separable form1)
- 6. Events, States andBackground knowledge • Both timepoint and interval semantics are used for events in literature • In this study single timepoint semantics have been chosen to support immediate streaming of event objects - Note: Does not preclude attaching multiple timestamps to an event, when time is measured by different agents during processing • “Any condition, which has a beginning and an end, either of which may stretch to infinity” is hereby referred to as a “state” • Statements without an associated time of validity are referred to as “background knowledge” 6.9.2017 6
- 7. Example of Events,States and Background knowledge 6.9.2017 7 friendOf a symmetricProperty Jim friendOf Alan Jim location1 Alan location1 Jim location2 Jim closeTo Alan Events States Background Alan location2 Jim location3 Alan location3 Jim is nearby! Alan is nearby! Moving away Simple Complex P I
- 8. Database vs. StreamProcessing 6.9.2017 8 2. Database Traditional database processing 1. Data Customer 3.Query 4.Response Stream Processing Customer 1. Data
- 9. Data Stream Processingvs. Event Processing 6.9.2017 9 Event ProcessingData Stream Processing Query Input stream Stream window Output streamQueryQuery Event Processing Network Note: Event processing platforms typically support data stream processing.
- 10. Complex Event Processing(“CEP”) • “Computing that performs operation on complex events”1) • Simple lower-layer events abstracted into more tangible higher-layer events • Recognition of event patterns often of importance 6.9.2017 10 1) Luckham, D., Schulte, R.: Event processing glossary – version 2.0 (Jul 2011) Event Producer Event Processing Agent (EPA) EPA Event Consumer Storage Event Processing Network (EPN) EPA Event Channel Event Channel
- 11. World-Wide Web vs.Semantic Web 6.9.2017 11 Semantic Web, “The Web of Data” World Wide Web, “The Web of Documents” http://www.nokia.com/en_int/ about-us Networks Innovation We create the technology to connect the world Nokia is shaping the technologies at the heart of our connected world, to transform the human experience Networks Innovation http://www.nokianka upunki.fi/matkailu/fo reign_visitors/welco me_to_nokia/ Welcome to Nokia! http://dbpedia.o rg/resource/Cat egory:Nokia http://www.wikid ata.org/entity/Q 7215352 owl:sameAs rdfs:label “Category:Nokia” rdfs:label “Nokia” skos:broader http://dbpedia.org/r esource/Category: Electronics_compa nies rdfs:label “Electronics companies” Short prefix for ontology URI. rdfs:comment “The property that determines that two given individuals are equal.”
- 12. Resource Description Framework(RDF) • Represent information on the Web • Consists of three-element directed graphs a.k.a triples 6.9.2017 12 Ev e1 ep: EventObject :loc1 geo: lat 60.158776 geo:long 24.881490 ep:hasEventObjectSamplingTime 2014-01-07T09:18:21 geo:alt 122.37 <Eve1> a ep:EventObject; ssn:Sensor :loc1; geo:Point [ geo:lat 60.158776 ; geo:long 24.881490 ; ] ; ep:hasEventObjectSamplingTime "2014-01-07T09:18:21"^^xsd:dateTime . P I, P III Turtle serialization format
- 13. SELECT ?a WHERE {?x rdfs:subClassOf ?y . ?a a ?x } SPARQL Protocol and RDF Query Language • Query language for RDF data 6.9.2017 13 INSERT{ ?a a ?y } WHERE { ?x rdfs:subClassOf ?y . ?a a ?x } 1. Match basic graph pattern against the data graph P V Rule known as “rdfs9” in RDFS entailment, “cax-sco” in OWL 2 RL Animal (?y)Rabbit (?x)?a 2. Select and output result 2. Add result to output graph • SPARQL Update (2013 in v. 1.1) added maintenance operations like DELETE and INSERT • Used in all included publications to build rules
- 14. Contribution of Publications inDissertation
- 15. INSTANS Research PlatformPositioning 6.9.2017 15 Semantic Web No Semantic Web Static Database Data Stream Processing (Complex) Event Processing INSTANS2) 2) Incremental eNgine for STAnding SPARQL EP- SPARQL CQELS Esper Jena C-SPARQL RDFox Stardog SQL NoSQL WSO2 Aurora STREAM Publication I Publication IV Publication V
- 16. Proof of Concept •Layered event processing application “Close Friends” • Collaboration of specification-compliant SPARQL Query and Update • Continuous, asynchronous processing using a Rete-network • Use of the local triple store as execution- time memory • Structured RDF events consisting of multiple triples • Event processing benefits in processing event patterns • Comparison with a data stream processing platform 6.9.2017 16 P I Mapping Event Processing Network elements to RDF / SPARQL domain Event Producer RDF input stream to the current system Event Consumer RDF output stream to console Storage RDF triples in local storage EPA (A Network of) SPARQL queries and rules
- 17. Complex Event Processingto RDF • Support for complex and composite events • Definitions to separate header and payload elements • Discussion on matching complex and composite events using SPARQL • Suggested methods, limitations • Support for multiple timestamps of an event object • Sampling time, time of entering data stream, arrival in event processing system, expiration time 6.9.2017 17 P II
- 18. Validation of TheApproach for CEP • Examples of all types of event processing agents found in related literature3), 4) • Named RDF graphs as event channels 6.9.2017 18 P III 3) Etzion, O., Niblett, P., & Luckham, D. (2010). Event Processing in Action (p. 325). Manning Publications. 4) Taylor K., Leidinger L. (ESWC 2011). Ontology-driven complex event processing in heterogeneous sensor networks. Event Processing Agent Filter Transformation Pattern Detect Translate Aggregate Split Compose Enrich Project Figure reproduced with publisher’s permission.3)
- 19. Query Network Example •Aggregate computes a multiple-in single-out function of incoming events • Typical examples count, min, max, average, sum • Calculate number of events per hour • Uses an auxiliary query to extract data utilized by three other queries 6.9.2017 19 P III memhour, counter, eventhour Initialize Extract Hour Increase Event Counter Output Event Counts Reset Memory Events In Construct Out memhour, counter eventhour eventhour = memhour eventhour != memhour Aggregate
- 20. Remote SPARQL Endpointfor Enriching Events • Remote access to background knowledge 6.9.2017 20 P III PREFIX omgeo: <http://www.ontotext.com/owlim/geo#> PREFIX ff: <http://factforge.net/> INSERT { GRAPH ?translated { ?event :locationName ?label . ?s ?p ?o } } WHERE { GRAPH ?g { FILTER ( strStarts(str(?g),concat(str(<>),"Input-Eve")) ) ?s ?p ?o . ?event a ep:EventObject ; geo:Point [ geo:lat ?lat; geo:long ?long ] . SERVICE <http://factforge.net/sparql> { ?location omgeo:nearby(?lat ?long "1km"); ff:preferredLabel ?label } BIND (IRI(concat("Translated-",strAfter(str(?event),str(<>)))) AS ?newgraph) } } 5) Linking Open Data cloud diagram 2017, by Andrejs Abele, John P. McCrae, Paul Buitelaar, Anja Jentzsch and Richard Cyganiak. http://lod-cloud.net/ 5) Enrich
- 21. Turtle vs. TriG 6.9.2017 21 Block2 Block 1 Block 1 • RDF has challenges with event object boundaries6) – Turtle grammar rule #6 specifies “triples”, but the order of triples impacts the composition of the block 6) Keskisärkkä, R., Blomqvist, E.: Event Object Boundaries in RDF Streams: A Position Paper. In: ISWC 2013 Workshop: Proceedings of OrdRing 2013 - 2nd International Workshop on Ordering and Reasoning. CEUR workshop proceedings, Sydney, Australia (October 2013) • TriG specifies the Turtle format for sending datasets – Each event in a separate graph _:5 geo:lat 60.158775; geo:long 24.88149 . <Eve1> rdf:type ep:EventObject; ssn:Sensor :loc1; geo:Point _:5; ep:hasEventObjectSamplingTime "2014-01-07T09:18:21"^^xsd:dateTime . <Eve1> rdf:type ep:EventObject; ssn:Sensor :loc1; geo:Point _:5; ep:hasEventObjectSamplingTime "2014-01-07T09:18:21"^^xsd:dateTime . _:5 geo:lat 60.158775; geo:long 24.88149 . <Eve1> { _:5 geo:lat 60.158775; geo:long 24.88149 . <Eve1> rdf:type ep:EventObject; ssn:Sensor :loc1; geo:Point _:5; ep:hasEventObjectSamplingTime "2014-01-07T09:18:21"^^xsd:dateTime . } vs. TriG P III
- 22. Individual And Combined Testing 6.9.2017 22 P III Sourc e 1: Stateless Post- stateless 2:Stateful Post- stateful 3: Translate Memory 4: Project 5: Split Geo- Events Sensor- Events 6: Aggregate Event- Counts 7: Compose Combined Transform Directions Compare Pattern Reset Detect Pattern- detect 8: Pattern Detect TranslatedProjected
- 23. Performance Comparison with APractical Example • Counterfeit and theft detection in pharmaceutical manufacturing • INSTANS compared with Esper event processing platform • New execution environment coded for Esper in Scala • RDF events converted to XML • Event processing application separately coded for INSTANS in SPARQL and Esper in EPL (“Event Processing Language”) • Identical results verified 6.9.2017 23 P IV
- 24. Example Segment ofTask • All products are identified with an Electronic Product Code (EPC) • EPC are scanned during Commissioning and Packing operations • If a commissioned EPC is not found in packing within a specified delay, it should be reported stolen • If a non-commissioned EPC is found in packing, it should be reported as counterfeit 6.9.2017 24 P IV c(1) Commissioning <reader102> c(cp) p(1) Packing <reader103> p(2) Δtcp EPC1 EPCec c(2) Δt c p(ps) Δt p
- 25. Different Solutions 6.9.2017 25 EPCc, evenc, tc EPCp, evenp, tp EPCc = EPCp truetrue delete delete truetrue c() p() EPCc = EPCp true delete delete EPCc not in winm EPCp not in winm true winc winf report lost report counterfeit EPCc, evenc winm EPCp, evenp remove stream remove stream report lost EPCcp true report counterfeit INSTANS Esper c() p() tegtc tegtp tc + twinc ≤ te tp + twinf ≤ te P IV
- 26. Qualitative Comparison • Movefrom RDF to XML collapses all semantic web benefits • No connection to globally accessible ontologies • No semantic web tools can operate on the stream • No reasoning over an XML stream • Logistics chain scenario • With semantic web tools the manufacturer could create a graph or dataset of shipped items for the warehouse • Using only a link the warehouse could access a federated SPARQL endpoint to compare received items with the shipping manifest • Without these tools a dedicated solution with more task-specific coding would be needed 6.9.2017 26 P IV
- 27. Support for Reasoning •Generate new knowledge out of facts and rules • Survey of rule-based entailment regimes for RDF data • RDF, RDFS, ρdf, D*, P-Entailment, OWL 2 RL, SWCLOS2 • Implementation on INSTANS using SPARQL Query and Update • Altogether 100 unique rules and 21 unsatisfiability conditions • Compliance-tested using entailment tests of the SPARQL 1.1 Test Suite • Performance tested using the Lehigh University Benchmark (LUBM) 6.9.2017 27 P V ρdf RDF(S) OWL2RL SWCLOS2 D* 8 9 2 48 21 6 18 3 P-ent.2 3
- 28. Customisation of Rules •Rules are written in the same language (SPARQL) as the queries, and can be packaged together • Full control over the version of rules used with a particular query • A complete entailment regime is rarely needed • Current compliance test set only covers 17-47% of the rules (depending on the regime) • No LUBM query requires more than three rules, a total of eight rules passes all 14 LUBM queries - Some require rules only present in OWL 2 RL, having 58 rules 6.9.2017 28 P V
- 29. Event-Based Memory Handling •Infinite streams of data eventually fill any memory or database • Resources need to be released, when no longer required • Publications I and III demonstrate explicit “cleanup-rules”, which compare timestamps and DELETE any events except the latest one • INSTANS provides an operational policy, which removes incoming events after all dependencies have been computed to the Rete- network - Not applicable to multi-input EPA:s (Publication III, EPA 7) • Publication V demonstrates an approach, where background knowledge and related materialisations are directed to a named graph and each event and the related materialisations are deleted after processing 6.9.2017 29 P V 1. Static Static 2. Static Event 3. Static End
- 30.
- 31. Concepts Developed DuringStudy • RDF definitions for • Composite and complex events • Header and payload • Timestamps for different phases of processing path • Structured, heterogeneous events streamed as datasets using TriG format • (Complex) event processing using networks of asynchronously operating, specification-compliant SPARQL Update rules • Mapping of all event processing network elements to SPARQL building blocks • Operations on nested composite and complex events • Recognition of event patterns • Computation of aggregate values • Configurable approaches to memory handling • Materialisation-based reasoning 6.9.2017 31
- 32. Event Processing UsingRDF and SPARQL • Tested with • Structured, heterogeneous events • Asynchronously operating networks of SPARQL queries and update rules • Different types of event processing agents presented in literature • Verified using an executable platform • All tests with explicit sampling time in events - Query answers are deterministic and repeatable • Identical results with comparison platforms • Code, queries, data and documentation openly available for independent verifications 6.9.2017 32
- 33. Performance of INSTANSPlatform • Average notification delay 12ms*) (P I) • Considerably faster than a feasible window repetition rate in typical data stream processing systems • Real-life target of 2.83 EPC/s exceeded 950 times while processing commissioning, packing and shipping on a regular laptop (P IV) • Average speeds of up to 1,810 events / second (P III, EPA 6) and 21,195 triples / second (P V, LUBM Q1) demonstrated • Using event-based memory handling and optimised sets of rules all LUBM queries were completed up to a dataset of 100 universities (13,405,383 triples) 6.9.2017 33 *) Result obtained with older laptop HW using 1st generation INSTANS.
- 34. Performance Improvement Suggestions forINSTANS • Support for multi-core processor architectures • Current Common Lisp implementation only using a single core • Modular Rete-engine • The whole EPN of P III was tested in one Rete-engine • Running each EPA in a separate module would reduce complexity and unnecessary verifications • Parallel HW • Indexing of comparisons • P IV, experiment 4, expiration of up to one million EPC codes is checked every time the time moves forward • Major improvements achievable by indexing the comparisons, preliminarily tested as 86 times faster for Exp4 100k EPC case (from 20 EPC/s to 1,736 EPC/s) 6.9.2017 34
- 35. Identified SPARQL Enhancementsfor Event Processing • Allow combinations of DELETE + INSERT + ( SELECT or CONSTRUCT ) • SPARQL 1.1 grammar does not allow combined Query and Update • Complexity reduction for cases where both result output and data processing are needed • Rule priorities to specify execution order • Wild card matching for GRAPH statements • Reduce steps for matching individually named events on event channels • Extensions for geographical coordinates and time arithmetics • Dataset output capability for CONSTRUCT to enable TriG output (syntax identical to INSERT) • Extension for converting xsd:dateTime timestamps into integer values 6.9.2017 35 }Implemented In INSTANS
- 36. Challenges of TheApproach • Implementation efficiency • No a priori assumptions about the structure of an incoming event can be made - Every event could be different, but this flexibility is typically unnecessary and also complex for writing queries - In more rigid approaches event object schemas are pre-defined, and event objects can be directly assigned to programming-language objects • Management of concepts between ontologies is unnecessary in closed, well- defined environments • Unnecessary complexity for closed environments requiring only a few event types 6.9.2017 36
- 37. Conclusions • RDF andSPARQL offer a flexible framework for layered processing of structured, heterogeneous events • Best suited for distributed, loosely coupled environments - Support for compatibility between ontologies, multiple event types, reasoning • Simple, well-defined stream processing tasks in closed environments requiring maximum performance can be executed more optimally using other means • Performance measured for a variety of cases on a test platform • Sufficient for many real-world stream processing tasks 6.9.2017 37