Oracle NoSQL Database

Overview & Use Cases

Dave Segleau, Dir. Product Mgmt
1 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
 The following is intended to outline our general product
direction. It is intended for information purposes only, and may
not be incorporated into any contract. It is not a commitment to
deliver any material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole
discretion of Oracle.

2 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.
 Agenda

• Oracle NoSQL Database Overview

• Release 2.0 Features
• NoSQL DB Use Cases

3 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.
 Oracle Big Data Platform

4 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.
 Big Data Lifecycle

DECIDE ACQUIRE

Acquire all
available data

ANALYZE ORGANIZE

5 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.
 Big Data Storage Choices
Hadoop Distributed File System
Oracle NoSQL Database
(HDFS)
File System Database

Parallel scanning Indexed storage

No inherent structure Simple data structure

High volume writes High volume random reads and writes

Real-Time, “Last Mile to the customer”

Batch Oriented
applications
6 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
 Where does Oracle NoSQL DB fit?

Best database for the “last mile to the customer “ applications

In the Big Data Acquisition phase

Complements Hadoop/HDFS (batch), Oracle Database

Integrates with OEP, RDF, others

7 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.
 Typical Big Data/NoSQL Solution Architecture
Endeca Information Oracle Business
Application Discovery Intelligence EE
NoSQL DB Driver
Big Data
Real Time Access
Connectors
Oracle Exalytics

Big Data ORE, OEP, Endeca

Acquisition
Oracle Advanced Analytics

OLH, ODC, ODI,

Batch Processing External Tables

HDFS, Hadoop, CDH

Map Reduce Map Reduce Map Reduce
ORCH - Stats Pig - Sessionize Hive - Activities

Oracle Big Data Appliance Oracle Exadata

8 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Sample of Big Data Use Cases Today
AUTOMOTIVE COMMUNICATIONS CONSUMER FINANCIAL EDUCATION &
PACKAGED SERVICES RESEARCH
Auto sensors Location-based
reporting advertising GOODS Risk & portfolio Experiment
location, Sentiment analysis analysis sensor analysis
problems of what’s hot, New products
problems

HIGH TECHNOLOGY / LIFE MEDIA/ ON-LINE HEALTH CARE

INDUSTRIAL MFG. SCIENCES ENTERTAINMENT SERVICES / Patient sensors,
Mfg quality Clinical trials Viewers / advertising SOCIAL monitoring, EHRs
Genomics effectiveness MEDIA
Warranty analysis Quality of care
People & career
matching
Web-site
optimization
OIL & GAS RETAIL TRAVEL & LAW
TRANSPORTATION UTILITIES ENFORCEMENT
Drilling Consumer
exploration sentiment Sensor analysis for Smart Meter & DEFENSE
sensor analysis optimal traffic flows analysis for Threat analysis -
Optimized network
marketing Customer sentiment social media
capacity, monitoring, photo
analysis

Challenged by: Data Volume, Velocity, Variety

Oracle NoSQL Database is typically a component of a Big Data Solution

9 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.
 Oracle NoSQL Database Use Cases
Use Cases
High-throughput event processing
Customer profile management SIMPLE QUERIES
Click-through data processing
DYNAMIC SCHEMA
Sensor & statistics data capture
Social networks
HIGH VOLUME
Personalization
Mobile application backend infrastructure REAL TIME DATA ACCESS
Authentication & Content management
Archiving “Last mile” problems
10 Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
 Oracle NoSQL Database
Scalable, Highly Available, Key-Value Database
Application Application
Application Application
Features
NoSQL DB Driver NoSQL DB Driver

 Simple Key-Value Data Model

 Horizontally Scalable
 Highly Available
 Elastic Configuration
 Simple administration
 Transparent load balancing
 Commercial grade software and
support
Storage Nodes Storage Nodes
Datacenter A Datacenter B

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

 Oracle NoSQL Database
Technical Summary
• Scalable • C and Java APIs
– Dynamic data partitioning and • JSON Schemas
distribution
– Automatic Query Load Balancing
• ACID Transactions
– High Performance and Throughput • Easy to install and manage
• Highly Available • Requirements
– One or more replicas – Java SE 6 (JDK 1.6.0 u25)+
– No single point of failure – Solaris 10 or Linux
• Optimized, intelligent database driver
• Simple Key-Value data model

12 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Oracle NoSQL Database
Key Features

NoSQL DB Feature User Benefit

Distributed storage and queries Scalability, Performance

Intelligent database driver Performance, Scalability, Reliability

No Single Point of Failure Reliability, 99.999% availability

Simple Key-Value storage Performance, flexibility, ease of use

ACID transactions Reliability, data integrity, ease of use

Simple Administration Low OPEX, ease of use

13 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Simple Data Model
Key-value pairs
• Simple data model – key-value pair (major+minor-key paradigm)
• Simple operations – read/insert/update/delete, RMW support
• Scope of transaction – records within a major key, single API call
• Unordered scan of all data (non-transactional)
Major key: userid
Strings
Minor key: subscriptions address

Byte Array  Value: expiration date phone # email id

14 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Key Structure Design - Examples

Social Network Email CDR

Major Key /john/smith/ Major Key /steve.jobs/ Major Key /calling#/

Minor Key tweets birthday Minor Key inbox sent Minor Key today yesterday

Value tweet001 tweet002 1970-1-1 Value thread001 thread002 thread002 Value CDR001 CDR002 CDR003

15 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.
 Simple Data Model
ACID Transactions
• ACID transactions by default

• Transaction Scope
– Single API call
– All records must have the same major key
– Support for multiple operations within a transaction

• Can be relaxed for increased performance on a per-

operation basis
16 Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
 Simple Data Model
ACID Transactions – Configurability
• Configurable Durability Policy

• Configurable Consistency Policy

17 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Easy to use APIs
C or Java
• Simple CRUD operations
• Conditionalized to reduce client/server round trips

• Iteration operations
• Returns multiple results in a single API call
• Get full records or just the keys

• Multi and Sub-Key operations

• Multiple operations and/or Multiple records in a single transaction

18 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Scalability and Availability
Application Perspective
Application

NoSQL DB Driver

Shard 1 Shard 2 Shard N

Master Master Master

Replicas Replicas Replicas

19 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.
 Scalability and Availability

• Replicated Application Servers

• Driver is linked into each Application
• Storage Nodes kept current via
replication (Berkeley DB Java Edition HA)
• Storage Nodes across Data Centers
• Automatic SN failure handling
– Graceful degradation
– Automatic recovery

 No Single Point of Failure

20 Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
 Oracle NoSQL Database Differentiation
Integrates seamlessly with Oracle Stack (ODI, CEP, OLH)

Commercial Grade Scalability and Simple

Software and Support Availability Simple Data Model Administration

• General Purpose • Intelligent Oracle • Simple Major + • Web-based Console

NoSQL DB Driver Minor Key-Value and CLI commands
• Reliable – Based • Evenly distributes Data data structure
on proven Berkeley • Sends operation to • Manages and Monitors:
DB JE HA fastest node • ACID transactions • Topology
• Bounded network hops • Load
for all operations • Performance
• Easy to Install & • Configurable
Configure consistency and • Events
• Automatic replication • Alerts
durability
and failover

21 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 YCSB Benchmark Results

Mixed Throughput
•1.25M ops/sec 1,400,000

Throughput (ops/sec)

Average Latency (ms)

1,200,000 4

• 2 billion records 1,000,000

3
800,000
• 2 TB of data 600,000 2

400,000
1
• 95% read, 5% update 200,000
0 0
• Low latency 6 (2x3) 12 (4x3) 24 (8x3) 30 (10x3)
Cluster Size
• High Scalability
Throughput (ops/sec) Write Latency (ms)
Read Latency (ms)

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

 Agenda

• Oracle NoSQL Database Overview

• Release 2.0 Features
• NoSQL DB Use Cases

23 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 What’s New?
Release 2 Feature Summary
R2 Features

Scalability & Integration &

New APIs
Manageability Monitoring

Elasticity JSON schemas External Tables

Oracle Event
Rebalancing C-API
Processing

Smart Topology Large Object Support RDF Adapter

SNMP/JMX

24 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.
 Configurable Smart Topology

• Storage nodes have indication of “capacity”

• System may allocate multiple replication nodes per
storage node
• Intelligent node balancing
• Efficient use of system resources
• Reduce time for recovery of disk media failure

25 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 JSON Data Format
Avro based Serialization/Deserialization for the Value (KV)
• Why Avro?
– Compact, highly efficient serialization
– Synergy with Hadoop
• Schema
– DDL allows schema creation through Avro JSON definition
– Supports serialization from/to JSON strings
• Schema evolution
– Easy to use mechanism for schema evolution
– Schema versions can be opaque to readers
26 Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
 Support for Large Objects

• Efficient storage and retrieval of large objects

• Client side streaming interface for low memory
consumption
• Server side splitting and distribution of object chunks
across nodes for better read/write latency

27 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Enhanced System Monitoring

• SNMP (Enterprise Edition Only)

– MIBs in SNMPv2 format
– Storage node as SNMP agent – Proxies to admin and replication nodes
– Performance metrics – Via polling
– Service change events – Via traps/notifications (UDP) or polling
– Notifications (Traps) – Latency and throughput limits
• Java JMX
– Performance metrics – Via subscriptions
– Service changes events – Via subscriptions
– Notifications – Via subscriptions

28 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 External Table Support

• Available with Oracle NoSQL DB Enterprise Edition

• Oracle Database SQL access to NoSQL Database data
• Steps:
1. Create NoSQL DB table formatter (use sample template)
2. Define External Table in SQL
3. Define Configuration file (use sample XML template)
4. Use NoSQL Database Publish utility
5. Use SQL to access NoSQL data

29 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Integration with other Oracle Products

• Available in Oracle NoSQL Database Enterprise Edition only!

• Oracle Event Processing (OEP)
– OEP NoSQL Database cartridge reads data from NoSQL Database
– Flexible Java serialization utilized for values
• Oracle RDF
– Jena Adapter for NoSQL Database to be released by the RDF product
team
– Enables RDF data storage and SPARQL queries on NoSQL DB

30 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Agenda

• Oracle NoSQL Database Overview

• Release 2.0 Features
• NoSQL DB Use Cases

31 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 High Profile Oracle Key-Value customers

32 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Oracle NoSQL Database
Success Stories
• provides PaaS for deploying applications
over the cloud
– Oracle NoSQL Database exposed as a service through their PaaS web
application server infrastructure

• , Oracle Platinum Partner, built an online gaming application

for their customer (Passoker) using Oracle NoSQL Database

• CGBU launches Elastic Charging Engine based on NoSQL DB as

part of Billing application

33 Copyright © 2012, Oracle and/or its affiliates. All rights Oracle Confidential
reserved.
 Passoker Sales Win
Global Sports Betting application
• Challenges
– Provide a platform for real-time sports data feeds
– Deploy scalable, efficient, and highly technology for incoming data stream
– Establish a technology to receive XML files and rapidly process them for relevance
– Reduce development times and simplify day-to-day administration
– Scale-out the platform as demand grows and players take up gaming in greater numbers
• Benefits of Oracle NoSQL Database
– Reduced development and implementation time by 75%
– Simplified day-to-day administration
– Simplified rapid correlation between events and gaming
– Improved operational efficiency due to ongoing commercial support
– Eliminated daily maintenance related to single node point-of-failure
– Enabled more application development and integration, better testing, shorter time to market

34 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 NoSQL Example: On-line Ads and Content
Goal: Optimize Internet Advertising to Increase Revenues

Scenario
• User clicks on a web page
• Cookie used to look up user profile
• Appropriate ad placed on page
Ad to be
Look Up User Profile Displayed
Key Features
• Fast response times and high throughput
NoSQL Expert
• High availability and scalability System
DB
• Simple data, flexible data model
• Connectivity to Big Data Analytics and Discovery

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

 NoSQL Example: Remote Patient Monitoring
Goal: Better Patient Care at Lower Cost

Scenario
• Patient uses multiple devices at home
• Medical data periodically sent to NoSQL DB
• App monitors and alerts patient state
• Alerts sent to medical or emergency
personnel, recorded in profile Capture Patient
Monitoring Data
Key Features
• High performance and high availability NoSQL Alerting
• High throughput event capture DB System
• Huge volumes of data
• Simple data, flexible data model

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

 Example: Credit Card Transaction Authorization
Goal: Better, Faster Fraud Detection and Response

Scenario
• Credit card swipe at point of sale (PoS)
• Look up customer profile (interactive)
• Authorization systems determines approval
• Approval sent to PoS and recorded Look Up

Authorization Approval
Key Features Server Status
• Fast response time, high throughput
Record
• High availability and scalability NoSQL DB
Transaction
• Huge transaction rates, simple data
• Connectivity to Big Data Analytics and Discovery
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
 Use Case – Online Display Advertising

• Problem
– Very low latency requirements – Publishers require < 75 ms response time from the ad
serving platform
– Extreme data velocity – Multi-millions of requests per second
– Highly available – 24/7 sites
– Revenue maximization – Deliver the most relevant ad to maximize revenue

• Solution – Where to use a NoSQL Database?

– Cookie store – NoSQL database used to store cookies and associated behavioral segments
– Track behavioral data – Beacons utilized during browsing to store timestamp, frequency, and
behavioral segments by cookie
– Optimize ad delivery – Recency, frequency, and behavioral segments used to determine
optimal ad to deliver to user

38 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Use Case – Online Display Advertising
Architecture
RDBMS Hadoop Cluster

Ad Server Application

NoSQL DB Driver

Multi-Dimensional
Reporting

39 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Online Display Advertising
Using the right tool for the right job
• Oracle NoSQL Database
– Low latency high volume
• Millions of ad serving requests per minute or second
• Stringent latency requirements from publishers
– Loose consistency
• Cookie data used for ad targeting – Increase probability that user will click on ad

• Oracle Database
– Campaign booking information – hundreds of users
– Real time business metrics for publishers and advertisers
– Business financials for ad serving company
• Year to date revenue, quarter over quarter etc., Billing, SOX reporting for public companies

• Hadoop
– Unique visits (select count(distinct)) over many terabytes of data
– Inventory forecasting across behavioral segments

40 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Use Case – Rich Web 2.0 Applications

• Problem
– Low latency requirements – Page loads in less than one second to retain users. Potentially
hundreds of queries to populate page data.
– Ease of scale – Automatic partitioning/scaling as user base grows
– Highly available – 24/7 operations (e.g. Amazon.com)
– Efficient state management – Application state info must extremely efficient
• Solution
– Cookie store – A NoSQL database is utilized to store state and recommendation information
(derived offline from Hadoop) by cookie
– Profile management – rich user profile and history stored in NoSQL
– Recommendations and initial state – Offline recommendations as well as initial state (e.g.
recent products browsed) are read at page load time
– State updated – The user’s state is updated in real time as browsing occurs

41 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Rich Web 2.0 Applications - Overall Solution
Architecture
Hadoop Cluster
App State Store

Oracle Database

Order Management & Financial

Reporting

42 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.
 Rich Web2.0 Applications
Using the right tool for the right job
• Oracle NoSQL Database
– Low latency high volume
• Millions of store browsing requests per minute
• Predictable page rendering times - Less than 1 secs
– Loose consistency
• Cookie data used for UI customization

• Oracle Database
– Order booking – Customer already “acquired”
– Master product catalog
– Business financial reporting
• Year to date revenue, quarter over quarter etc., Quarterly auditing, SOX reporting for public companies

• Hadoop
– Product recommendations
– Probability of purchase

43 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 MoviePlex Demo

• Goal
– Deliver a personal experience to every user
– Each user profile must be retrieved and
updated with minimal latency

• Challenge
– Need to service this at web scale
– 100k’s customers buying 100k’s movies

• Value
– millisecond latency, on-demand scalability,
high-throughput, lowest $/ops

44 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Why Oracle NoSQL Database?

Easy to use, easy to manage

Scalable, Available, Predictable Latency

A NoSQL Database from a vendor you trust

45 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Oracle NoSQL DB Resources

• NoSQL DB Use Cases, White Papers, Data Sheets, Benchmarks

http://www.oracle.com/technetwork/products/nosqldb/overview/index.html
• NoSQL DB Documentation
http://www.oracle.com/technetwork/products/nosqldb/documentation/index.html
• NoSQL DB Downloads
http://www.oracle.com/technetwork/products/nosqldb/downloads/index.html
• NoSQL DB OTN Forum
http://forums.oracle.com/forums/forum.jspa?forumID=1388
• NoSQL DB version 2.0 Features
http://bit.ly/UKn5Sc
• OU Training Classes
http://bit.ly/V5qbmY

46 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Oracle NoSQL DB Resources

• Great Application Development article on OTN:

http://www.oracle.com/technetwork/articles/bigdata/oracle-nosqldb-appdev-
1891870.html

47 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Questions & Discussion

Q&A

48 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 APPENDIX

49 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Big Data Acquisition Characteristics
Where should we put all that data?

Batch-Oriented Real-Time

Process data to use Deliver a service

Bulk storage Fast access to specific record

Write once, read all Read, write, delete, update

Distributed, parallel bulk processing “Last Mile” to the customer

50 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 How much throughput do you need?
NoSQL DB has throughput even for the largest players

51 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
 Oracle NoSQL Database
Major-Minor Key Paradigm
Master-1 /major/key/components/ - /minor/key/components
/Smith/Bob/-/birthdate
/Smith/Bob/-/phonenumber
/Smith/Bob/-/image
/Smith/Bob/-/userID
Master-2

Oracle NoSQL Driver

/Smith/Richard/-/birthdate
/Smith/Richard/-/phonenumber
/Smith/Richard/-/image
/Smith/Richard/-/userID
Master-3 /Wong/Bill/-/birthdate
/Wong/Bill/-/phonenumber
/Wong/Bill/-/image
/Wong/Bill/-/userID

52 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.
 Partitions and Shards
Partition300
Major Key /home/john

Minor Key images birthday

Value 001.jpg 002.jpg 1970-1-1

Partition# = Hash(“Major Key”)

Partition001 Partition002 Partition100 Partition101 Partition102 Partition200 Partition201 Partition202 Partition300

K-V K-V K-V K-V K-V K-V K-V K-V K-V

K-V K-V K-V K-V K-V K-V K-V K-V K-V
Shard001 Shard002 Shard003

53 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.
 Simple Administration

• Web-based console and

CLI commands
• Manages and Monitors
– Configuration changes
– Load: Number of operations,
data size
– Performance: Latency, throughput. Min, max,
average, trailing, …
– Events: Failover, recovery, load distribution
– Alerts: Failure, poor performance, …
54 Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
 Elasticity
On-Demand Cluster Expansion
Application

On Demand NoSQL DB Driver

 Increase Data Capacity

– Add more storage nodes
Master Master
– New shards automatically created
Replica Replica
 Increase Data Throughput
– More shards = better write
throughput Replica Replica
– More replicas/shard = better read Shard-1 Shard-2
throughput

StorageNode StorageNode StorageNode

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

 Rebalance an Unbalanced Store
Master-1 Master-2 Master-3
Improve Performance

 Replication nodes move from

over-utilized to under-utilized
storage nodes
 Number of shards and
replication factor remain
unchanged

Represents a partition

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

 Oracle NoSQL DB Value Proposition

• Designed for the “last mile” problem

• Production-ready NoSQL database
• Built on industry-tested storage and replication
technology (Berkeley DB)
• Integrates with Oracle and Open Source technology
• Supported by Oracle

57 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.
58 Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.