hybris Marketing

Marketing Data Management

Data Inbound
May 2015

Overview

Objectives

At the end of this presentation, you will be able to:

Explain how interaction contacts are loaded
Describe how CRM business partners are extracted to
interaction contacts
Add additional data sources for interaction contacts

Customer

The Digital Marketing Process

Events & interactions

from all channels

Explore &
gain insights

Intelligence
through Predictive
Analytics

Personalized
targeting &
orchestration

Campaign execution
in all channels

Market data & events

Digital & social channel

Sales & service data

Personalized commerce

Financial data

Interaction center

Big data industry

Lead, opportunity

Social media, web

E-mail, sms, traditional

Capturing response

Plan, measure & optimize

Customer

2nd level details for inbound integration as of 1505

Connected Solution

Market data & events

Sales & service data

Financial data
Big data industry

Social media, Web &

Commerce

Content

Technology

DMPs like Turn, Krux

Cookie based user data & interactions

OData

Lists of ext. providers, event participation

Prospects: contact data

CSV upload

SAP C4C (& C4Service*)

Contacts, leads, opportunities (Servicetickets*)

SOAP

SAP CRM

1. Customers, consumers & contacts, Sales

BizDocs, e.g. Orders, Opportunities,
2. Interaction Center generated BizDocs
3. Unstructured mail content (EML, IC)*

SLT

SAP ERP

Customer & Contacts, Sales related BizDocs,

e.g. Orders

SLT

hybris Commerce

Customers & sales Sales related BizDocs*

Non-SAP solutions

Contacts & Interactions

Biz Docs

CSV, oData
DS, part of rapid data load RDS

SAP ERP

WBS spend for campaigns, COPA data

SLT

SAP BRIM

BSS BizDocs

SLT

SAP Convergent Mediation

OSS Data

Digital Root Connector (partner

sol.)

SAP CAR/Retail

POS Data

HANA Co-Deployment

Twitter Public API

Social Posts

DS, REST

Facebook Fan pages

Google+

1. Social Posts
2. Likes & Comments*

DS, part of rapid data load RDS

Datasift (Harvesting Partner)

Social Posts

DS

Sprinklr

Social Posts*

OData

hybris Commerce

Clickstreams

ESP

Adobe

Clickstreams

DS, part of rapid data load RDS

hybris Conversion (SeeWhy)

Clickstreams (analyzed)

SOAP

SLT
DS, part of rapid data load RDS

* Planned for
2. H 2015
Customer

Integration Technologies
Technology

Description

SOAP

SOAP is a protocol specification for exchanging structured information in the implementation of web services in computer networks. The
message format is based on XML. Message transfer is based on other web protocols, usually HTTP(S).

REST

Representational State Transfer (REST) is a architecture style for creating scalable web services. REST services are usually based on
HTTP(S). They use HTTP URIs for resource identification and HTTP methods for service operations. It is used widely as an alternative to
SOAP, as REST services usually provide better performance, scalability and simpler interfaces.

OData

OData provides a protocol for queryable and interoperable RESTful APIs. It provides an entity-based data model and a query language.
Create, read, update and delete methods expressed using HTTP methods.

RFC

Call of a function module that runs in a different system (destination) from the calling program. Connections are possible between different AS
ABAP and between an AS ABAP and a non-SAP system. In non-SAP systems, instead of function modules, special programmed functions
are called, whose interface simulates a function module.

CSV

A comma-separated values (CSV) (also sometimes called character-separated values) file stores tabular data (numbers and text) in plain-text
form. CSV files are widely used as import or export format and can be down- and uploaded to many systems.

ESP

SAP Event Stream Processor is a high performance complex event processing engine that collects and analyzes streams of events
(messages) in real-time, as fast as they arrive. It can be used in conjunction with SAP HANA to collect streams of data, process the data,
capture it in HANA, and monitor the data to generate alerts or immediate response.

SLT

System Landscape Transformation or short SLT is a tool for data migration and data replication: replicates tables in (near) real time from SAP
systems to SAP HANA.

DS

Data Services (DS) provides a data integration platform to integrate and transform heterogeneous data (e.g. Web, RDBMS, XML, Flat files)
using ETL processes.

SAP MW

SAP CRM Middleware replicates, synchronizes and distributes data between different components of a SAP CRM solution. It links together
the various types of data producers (such as ERP back end, SAP NetWeaver Business Intelligence, SAP APO, hybris Marketing and the
CRM Server applications) to provide all participants with the information they require. Its main part is provided by the CRM Server.

HANA Co-Deployment

HANA co-deployment refers to multiple applications that are deployed to the same HANA instance. They can integrate by consuming HANA
information models which integrate data of both applications.
Customer

Inbound Interfaces for Interaction Contact + Interaction

C4C Marketing
Lead

CRM Contact &

Consumer &
Marketing Prospect

OData Service

Inbound
Processing

SAP Systems

CEI

ERP Contact &

Consumer

Interaction
Contact

Web Service

Hybris, Omniture,
RDS, 3rd Party

Interaction
RFC

Social Data
(BS_FND)

CSV Upload

DataSift

Social Data Source

Customer

MDS-Related Steps During Interaction Contact Inbound

Processing
Matching

Merge

BAdI w/ default implementation

Standard logic + BAdI + API

Example: collect tweet for new Twitterer

=> find existing interaction contact by email
address

Example: business partner record is updated

in CRM with additional email address => 2 CEI
interaction contacts are merged on the fly

Validation
BAdI w/ example implementation + standard logic

Example: validate email address for

pattern and characters
Example: transform email address to lower
case
Enrichment
BAdI w/ default implementation

Split
Standard logic + BAdI + API

Example: multiple business partners in CRM

share the same email address info@sap.com
=> email address is separated from the
corresponding CEI interaction contacts; an
additional interaction contact is created for
info@sap.com

Example: rule to determine contact level

Customer

Inbound Interfaces for Interaction Contact + Interaction

Customer

WEB INTERFACES
Interactions
Interaction Contacts

Marketing Data Management

Contact Engagement Odata Web Service for Interaction Contacts
Entity type contact

Odata Import of Interaction Contacts

To import Interaction Contacts from different sources, it is possible to use an

Odata service, which is delivered with SAP Hybris Marketing.
Name of the Odata Service CUAN_IMPORT_SRV
Entity types to import Interaction Contacts:

ImportHeader describes the technical header of an import of Interaction Contacts.

Contact contains all attributes that are required to create an Interaction Contact. The
ID of the contact is provided by the external source system and can identify the
contact for later updates.
ContactFacet is useful to save additional facets, means identifiers from other sources
to identify a natural person.
ContactMarketingPermission describes the consent (opt-in/opt-out) information of
an contact for different communication channels.

Customer

Marketing Data Management

Contact Engagement Odata Web Service for Interaction Contacts

Code Snippet for Odata Import in JSON Format

"Id" : ",
"Timestamp" : "/Date(1406014140922)/", "UserName" : "USER", "SourceSystemId" : "HYBRIS",
"Contacts" : [
{ "Id" : "4711", "Timestamp" : "/Date(1406014140601)/", "City" : "Kiel", "CountryDescription" : "Germany", "CustomerName" : "Stahlbau GmbH",
"DepartmentDescription" : "Sales", "EMailAddress" : "otto.normalverbraucher@company.de", "EMailOptIn" : "Y",
"EMailAddress2" : "otto.normalverbraucher2@company.de","EMail2OptIn" : "N",
"EMailAddress3" : "otto.normalverbraucher3@company.de", "EMail3OptIn" : "",
"FacebookId" : "4711", "FacebookOptIn" : "Y", "FirstName" : "Otto", "FullName" : "Otto Normalverbraucher", "FunctionDescription" : "Director",
"GenderDescription" : "Male", "GooglePlusId" : "", "GooglePlusOptIn" : "", "HouseNumber" : "1", "IndustryDescription" : "Manufacturing",
"IsConsumer" : false, "IsContact" : true, LanguageDescription" : "German", "LastName" : "Normalverbraucher", "MaritalStatusDescription" : "Married",
"MobilePhoneNumber" : "+49119201412191", "MobilePhoneOptIn" : "N", "MobileSMSOptIn" : "Y", "Obsolete" : false, "PhoneNumber" : "+49115",
"PhoneOptin" : "", "PostalCode" : "24105", "PostalOptin" : "", "RegionDescription" : "Schleswig-Holstein", "SAPCRMBusinessPartnerId" : "", "SAPCRMMarketingProspectId" : "",
"SAPERPAccountId" : "12345", "SAPERPConsumerAccountId" : "", "SAPERPContactId" : "23456", "SAPHybrisConsumerAccountId" : "4711", "Street" : "Hauptstrasse",
"TitleDescription" : "Mr.", "TwitterId" : "", "TwitterOptIn" : "", "Facets" : [ { "Id" : "otto.normalverbraucher4@company.de", "IdOrigin" : "EMAIL", "Timestamp" : "/Date(1406014140601)/",
"OptIn" : "Y
},
{ "Id" : "otto.normalverbraucher5@company.de", "IdOrigin" : "EMAIL", "Timestamp" : "/Date(1406014140601)/", "OptIn" : "N"
}
],
"MarketingPermissions" : [
{"Id" : "otto.normalverbraucher3@company.de", "IdOrigin" : "EMAIL", "Timestamp" : "/Date(1406014140601)/", "OptIn" : "Y", "OutboundCommunicationMedium" : "EMAIL"
}
]
},
{ "Id" : "4712", "Timestamp" : "/Date(1406014140601)/", "City" : "Walldorf","CountryDescription" : "Germany", "CustomerName" : "", "DepartmentDescription" : "", "EMailAddress" : "erika.mustermann@privat.de",
"EMailOptIn" : "", "EMailAddress2" : "erika.mustermann2@privat.de", "EMail2OptIn" : "N",
"EMailAddress3" : "erika.mustermann3@privat.de", "EMail3OptIn" : "Y",
"FacebookId" : "4712", "FacebookOptIn" : "N", "FirstName" : "Erika", "FullName" : "Erika Mustermann", "FunctionDescription" : "", "GenderDescription" : "Female", "GooglePlusId" : "4712", "GooglePlusOptIn" : "Y",
"HouseNumber" : "1", "IndustryDescription" : "", "IsConsumer" : true, "IsContact" : false, "LanguageDescription" : "German", "LastName" : "Mustermann", "MaritalStatusDescription" : "Single",
"MobilePhoneNumber" : "+49119201412192", "MobilePhoneOptIn" : "Y", "MobileSMSOptIn" : "Y", "Obsolete" : false, "PhoneNumber" : "+49116", "PhoneOptin" : "", "PostalCode" : "69190",
"PostalOptin" : "Y", "RegionDescription" : "Baden-Wurttemberg", "SAPCRMBusinessPartnerId" : "", "SAPCRMMarketingProspectId" : "", "SAPERPAccountId" : "", "SAPERPConsumerAccountId" : "CAID-4712",
"SAPERPContactId" : "", "SAPHybrisConsumerAccountId" : "4712", "Street" : "Nebenstrasse", "TitleDescription" : "Ms.", "TwitterId" : "4712", "TwitterOptIn" : "N",
"Facets" : [
{ "Id" : "erika.mustermann4@privat.de", "IdOrigin" : "EMAIL", "Timestamp" : "/Date(1406014140601)/", "OptIn" : ""
},
{ "Id" : "erika.mustermann5@privat.de", "IdOrigin" : "EMAIL", "Timestamp" : "/Date(1406014140601)/", "OptIn" : "Y
}
]
}
]
}

Customer

CSV UPLOAD
Interactions
Interaction Contacts

Marketing Data Management

Contact Engagement File Import of Interactions

File Import of Interactions

For sources that do not warrant automated

integration for import of interactions, there is a
file import option
The csv file import of interactions serves the
same interface as the automated import.
Click on Download Example CSV File to
download a template incl. example data and
instructions

Customer

Marketing Data Management

Contact Engagement File Import of Interaction Contacts

File Import of Interactions

For sources that do not warrant automated integration

for import of interaction contacts, there is a file import
option
The csv file import of interaction contacts serves the
same interface as the automated import.
Click on Download Example CSV File to download a
template incl. example data and instructions
Note: When you upload a list of contacts the system
checks if the uploaded data is in correct format, e.g.
email-format
OK: john.smith@smith.com
Not ok: john.smith@.com.de
In this cases nothing will be uploaded and the system
throws an errror message that guides the user to the
line and column in the CSV file that is not correct

Customer

How to add New Data Sources for

Interaction Contacts

Marketing Data Management

Add New Data Sources for Interaction Contacts

The

following slides give you an overview of how to connect additional

data sources to extract interaction contacts. We have used our standard
delivery example of the SAP CRM business partner.
Note:

Please adapt the steps to your requirements.

If you want to load data periodically from a system (SAP or third-party) that is connected to a
SAP HANA system, it is useful to establish data extraction based on database trigger.

Connect any SAP systems with the SAP Landscape Transformation Replication
Server
Connect any third-party systems using SAP Data Services
You do not need the trigger mechanism (described in the following slide in steps 1 and 2) if
you do not need to extract data periodically.

Customer

Marketing Data Management

Dataflow and Main Entities

Necessary Steps
SAP System

Third Party System

Contact
Information

HANA System
Replicated

SAP Data Services

SAP LT Replication Server

Contact
Information

1.

The source data is replicated 1 to 1 from the source system

using replication tools like SAP Landscape Transformation
Replication Server or SAP Data Services.

2.

With the replication of source data to the SAP HANA

system, the database trigger in the SAP HANA system fills
the trigger log table.

3.

The extraction program evaluates whether the interaction

contact has to be inserted, updated, or deleted with the help
of the trigger log table and a SAP HANA view,.

4.

Save the interaction contacts with the help of the SAP

function module.

5.

Reset the trigger log table entries for the edited interaction
contacts.

DB Schema
Replicated
Contact
Information

Default
DB Schema

Trigger Log

Interaction
Contact

Extraction
Program

Customer

Marketing Data Management

Trigger-Based Delta Replication of Interaction Contacts
The trigger log table shows which data set has to be added, updated, or deleted. The table is filled with
the database triggers. The table is generated in the default database schema. The main table structure is
the primary key of your source and the operation mode.

Step 1: Create a trigger log table in ABAP-DDIC (SE11):

SAP HANA Modeler:

Note: Adjust table name, primary key, and attributes to your requirements.
Customer

Marketing Data Management

Trigger-Based Delta Replication of Interaction Contacts

Trigger Log Table

Application Table

e.g. Business Partner

e.g. Business Partner

CLIENT

Client (CEI)

CLIENT
Key

Key

CRM_CLIENT

PARTNER

CRM_PARTNER

<attributes>

TIMESTAMP

..

OPERATION

CLIENT is the current client (fixed part)

CRM_CLIENT & CRM_PARTNER is the application key with a prefix (e.g. CRM_)

TIMESTAMP is the timestamp of the DB operation (fixed part)

OPERATION is the database operation mode (fixed part) like create (=I), change (=U)
or delete (=D)
Customer

Marketing Data Management

Trigger-Based Delta Replication of Interaction Contacts

Root table
o

Contact Key

Attributes

..

..

Joined table (n)

Joined table (1)

o

Key

Attributes

..

Key

Attributes

..

A database trigger log table needs to be set up for every table when you have data sources
with multiple joined tables:
o

INSERT, UPDATE, DELETE

for root tables

INSERT, UPDATE

for all other subordinate tables

Customer

Marketing Data Management

Trigger-Based Delta Replication of Interaction Contacts
The database trigger creates corresponding entries for the log table.
Step 2: Generate database trigger for insert, update, and delete by entering appropriate SQL
statements in the SQL console of SAP HANA Modeler.

Example Insert DB Trigger

SAP HANA Modeler:

Example Update DB Trigger

Example Delete DB Trigger

Note: Adjust statements (schema name, table names, etc.) to your requirements.
Customer

Marketing Data Management

Creation of SAP HANA Views

Step 3: Create SAP HANA View(s)

Must be generated manually in SAP HANA Modeler

Should be generated in a customer package
Is needed to read data from a different schema
Helps to read the data easily
Can help to format the data correctly for the interaction contact

Customer

Marketing Data Management

Creation of SAP HANA Views
SAP Hana Attribute Views need to be created for:

Full Extraction
which is one view for the full object including all joined tables; see view
sap.hana-app.cuan.contact.extract/CA_CONS_EXTR_CRM_CONTENT

Delta Extraction
which are views for each of the subordinate, joined tables and determine the root node key
Example: Determine partner number (root node key) based on customer number;
See view: sap.hana-app.cuan.contact.extract/AT_CONS_DELT_CRM_B000
Trigger table

Root table

Customer

Marketing Data Management

Delta Extraction Background Information

Delta Extraction
Step 2

Step 1
Extract Keys
from Logging

Logging tables used in SP03

List of
Keys

Internal Call of
Full Extraction
with List of Keys

Tables

In the Delta Extraction a list of keys is extracted based

on the logging table views
These keys are candidates for the requested objects

In case of commonly used tables like business partner

or addresses only a small part of these candidates are
finally relevant

After delta extraction the processed logging table

entries are deleted

Source

Subject schema
SAP_CUAN_CRM

Subject table
(monitored table)
ADCP

Logging table
(trigger table)
CUAND_CE_IC_CADC

CRM
CRM

SAP_CUAN_CRM

ADCP

CUAND_CE_IC_CADC

CRM
CRM
CRM
CRM
CRM

SAP_CUAN_CRM
SAP_CUAN_CRM
SAP_CUAN_CRM
SAP_CUAN_CRM
SAP_CUAN_CRM

ADR2
ADR2
ADR6
ADR6
ADRC

CUAND_CE_IC_CAD2
CUAND_CE_IC_CAD2
CUAND_CE_IC_CAD6
CUAND_CE_IC_CAD6
CUAND_CE_IC_CADR

CRM

SAP_CUAN_CRM

ADRC

CUAND_CE_IC_CADR

CRM

SAP_CUAN_CRM

ADRP

CUAND_CE_IC_CADP

CRM
CRM
CRM
CRM
CRM
CRM

SAP_CUAN_CRM
SAP_CUAN_CRM
SAP_CUAN_CRM
SAP_CUAN_CRM
SAP_CUAN_CRM
SAP_CUAN_CRM

BUT000
BUT000
BUT051
BUT052
CRMD_MKTHV_MC
CRMD_MKTHV_MC_TC

CUAND_CE_IC_CBT0
CUAND_CE_IC_CBT0
CUAND_CE_IC_CBT1
CUAND_CE_IC_CBT2
CUAND_CE_IC_MC
CUAND_CE_IC_MCTC

CRM

SAP_CUAN_CRM

CRMD_MKTHV_MC_TC

CUAND_CE_IC_MCTC

CRM

SAP_CUAN_CRM

CRMKUNNR

CUAND_CE_IC_EKUN

CRM

SAP_CUAN_CRM

CRMM_BUT_FRG0041

CUAND_CE_IC_CFRG

ERP
ERP
ERP
ERP
ERP

SAP_CUAN_ERP
SAP_CUAN_ERP
SAP_CUAN_ERP
SAP_CUAN_ERP
SAP_CUAN_ERP

ADR2
ADR2
ADR6
ADR6
ADRP

CUAND_CE_IC_EAD2
CUAND_CE_IC_EAD2
CUAND_CE_IC_EAD6
CUAND_CE_IC_EAD6
CUAND_CE_IC_EADR

ERP

SAP_CUAN_ERP

ADRP

CUAND_CE_IC_EADR

ERP
ERP
ERP
ERP
SMI

SAP_CUAN_ERP
SAP_CUAN_ERP
SAP_CUAN_ERP
SAP_CUAN_ERP
SAP:CUAN

KNA1
KNA1
KNVK
KNVK
SMI_VOICE_CUST

CUAND_CE_IC_EKNA
CUAND_CE_IC_EKNA
CUAND_CE_IC_EKNV
CUAND_CE_IC_EKNV
CSAND_CE_IA_VOC

Customer

Marketing Data Management

Trigger-Based Delta Replication of Interaction Contacts
The extraction program reads the source data with help of the trigger log table.
The program extracts the data from a source system and saves the data to the interaction contact structure.
Step 4a: Create your own extraction program
Optional: Reuse delivered extraction program
Program name for full load: CUAN_IC_MASTERDATA_EXTR_FULL
Program name for delta load: CUAN_IC_MASTERDATA_EXTR_DELT
Use the program as a template
Copy and enhance the program

Customer

Marketing Data Management

Optional: Reuse of SAP Delivered Extraction Class
1.

2.

Create an empty class in

customer name space

Refer to superclass
CL_CUAN_IC_EXTRACT_COMMON

3.

Implement the CONSTRUCTOR

Call the constructor of the superclass

Set the source ID (1 .. 9)

Determine the database schema

Customer

Marketing Data Management

Optional: Reuse of SAP Delivered Extraction Class

4.

Redefine Methods

CONTENT_MAP_ATTRIBUTES
Map attributes of your FULL view to
structure CUAN_T_CE_IA_EXT

DELTA_INIT_METADATA
Define names of additional delta HANA
views (after CALL METHOD super>delta_init_metadata)

FULL_INIT_METADATA
Define names of additional full HANA
views (after CALL METHOD super>full_init_metadata)

Customer

Marketing Data Management

Additional report for delta extraction across sources

Report
CUAN_IC_MASTERDATA_EXTR_D
ELT
o

Delta-only variant of the

CUAN_IC_MASTERDATA_EXTR
_DELT report using the same
framework

Allows to extract deltas from all

sources in one job

One application log for all deltas

Customer

Marketing Data Management

Trigger-Based Delta Replication of Interaction Contacts
Save the data with the help of the delivered function module.
Step 4b: Use function module CUAN_CE_IC_POST_FLAT to upload the interaction contacts
Convert the source data to the target structure (see data structure: CUAN_S_CE_IC_EXT_FAFL)

Interaction
Contacts

Customer

Summary

Marketing Data Management

Sentiment Engagement Data Sources
Data can be harvested from public social media or
internal sources and is loaded into the inbound tables
of the Social Intelligence data model for further
processing by choosing one of the following options:

Implement own data crawling logic, e.g. use

script language to access provider APIs and pass
data via ODBC into HANA inbound tables or
ABAP program to pass data via respective API
classes.

Use example connectors of Social Contact

Intelligence based on SAP Business Objects
Data Services to access social media channel
data that has been harvested by DataSift as an
external data provider.
For more information, see SAP Help content as
well as the Installation Guide for Social Data
Harvesting Connector (Example DataSift) at the
SAP service market place.
For test purposes this direct twitter connection
can be used

Refer to the installation guide of SAP Customer

Engagement Intelligence for more details.

Customer

 2015 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate
company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.
Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.
National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its
affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and
services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as
constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop
or release any functionality mentioned therein. This document, or any related presentation, and SAP SEs or its affiliated companies strategy and possible future
developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time
for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forwardlooking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place
undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

Customer