KEMBAR78
SAP Data Intelligence Strategy | PDF | Cloud Computing | Metadata
0% found this document useful (0 votes)
106 views42 pages

SAP Data Intelligence Strategy

Uploaded by

ngneto
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
106 views42 pages

SAP Data Intelligence Strategy

Uploaded by

ngneto
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 42

SAP Business Technology Platform Analytics Strategy

Positioning SAP Data Intelligence


Cristiano Dias - SAP
Igor Alexandre Jakuboski – SAP

PUBLIC
PUBLIC
© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 1
Disclaimer

The information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission of SAP.
Except for your obligation to protect confidential information, this presentation is not subject to your license agreement or any other service
or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or any related
document, or to develop or release any functionality mentioned therein.
This presentation, or any related document and SAP's strategy and possible future developments, products and or platforms directions and
functionality are all subject to change and may be changed by SAP at any time for any reason without notice. The information in this
presentation is not a commitment, promise or legal obligation to deliver any material, code or functionality. This presentation is provided
without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement. This presentation is for informational purposes and may not be incorporated into a contract. SAP
assumes no responsibility for errors or omissions in this presentation, except if such damages were caused by SAP’s intentional or gross
negligence.
All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from
expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates,
and they should not be relied upon in making purchasing decisions.

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 2


Agenda

▪ Data Management with SAP BTP


▪ SAP Data Intelligence
• Overview
• Integration
• Processing
• Cataloging
• Hybrid Data Management
• Licensing
• Strategic Roadmap
• Q&A

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 4


Data Management with
SAP Business Technology Platform

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 5


Data and Analytics Solution Path
Data Democratization: Bring Autonomy and Value to the Entire Business

01 - Data Integration
• Connectivity
• Replication & Federation
• Orchestration & Processing
07 - Analytics & Planning • Business Semantics Inheritance
• Business Intelligence
• Enterprise Planning
• Augmented Analytics DevOps & Security &
Lifecycle Authorization
Management
02 - Data Storage
• Data Tiering
Data Strategy • Data Persistency
• Multi-model data platform
Data
Democratization
06 - Data Sharing
Business Machine

Data Marketplace Learning
Content
• Role-based Data Spaces Access

03 - Data Cataloging
• Data Discovery & Exploration
SAP One Domain Model &
Master Data Management • Data Profiling
• Data Lineage

05 - Data Modelling
• Semantics Modelling
• Data Layer Modelling
• Data Spaces 04 - Data Preparation
• Data Quality
• Data Transformation
14

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 6


Our strategic vision: SAP BTP data & analytics solutions
Enabling an end-to-end data fabric to drive business outcomes

SAP Analytics Cloud SAP BTP Services


SAP Business Technology Platform –

(Integration Suite,
Master Data
Business Intelligence Augmented Analytics Enterprise Planning
Governance, …)
data & analytics

SAP Data Warehouse Cloud SAP Data


Intelligence Cloud 3rd party processing
Business Layer Data Spaces
engines

Shared Services
Orchestration

3rd party data

Embedded
Data Catalog

engines
catalogs
SAP HANA Cloud

Multi-model Engines Tiered Data Storage Integration


3rd party data lakes

Non-SAP
SAP Applications Unstructured data Streaming data
Applications
© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 7
© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 8
SAP Data Intelligence
Overview

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 9


SAP Data Intelligence | Key Capabilities
Data Integration, processing and catalog in cloud

Distributed & External


SAP Applications Data Systems

SAP S/4HANA® SAP


ABAP Streaming (e.g. IoT)

SAP S/4HANA® CLOUD


NetWeaver
+ DMIS Addon
Integration
SAP Data Intelligence
Cloud Storages

BW Standard
SAP BW/4 HANA SAP BW Hadoop / HDFS
Integration Connectors
(open & native
Data Integration protocols)
REST APIs
Data Ingestion / Data Enrichment / Data Workflows
SAP C/4HANA
Cloud Data Databases
Integration

… Public Clouds

Data Processing 3rd Party


Exploration / Model Design / ML orchestration 3rd Party Applications
Connectors
Workflows
BW Process Data Services HANA
Chains Jobs Flowgraphs
SAP API
Business Hub
SAP Analytics Cloud SAC Push API
Data Catalog SAP BTP
Connectors
SCI for process
integration
Data Discovery / Data Profiling / Metadata Cataloging
SAP HANA® SAP HANA
SAP Open
(on-premise, cloud, multi cloud) Integration
Connectors

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 10


SAP Data Intelligence
Overview - Integration

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 11


SAP Data Intelligence – Data Integration

Integrate any kind of data (structured, unstructured or streaming), with


any pattern (real-time, near-real-time, batch), enrich it, refine it and
feed process it
© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 12
SAP Data Intelligence – Data Integration
Build Flow-based Applications using the Pipeline Modeler

• Data pipelines = Flow-based applications


• Drag-and-drop operators (independent computation units)
• Data (messages) flows between operators
• Combine and switch between integration styles:
streaming, batch, replication, virtualization, migration, etc.
• Extensible
• Over 250 pre-defined operators (data integration, connectivity,
processing, data quality, ML . . .)
• Custom and partner operators
• Microservices via APIs
• Wrap any scripting
• Scalable
• Containerized – Docker containers constitute the operators’
execution environments
• Distributed – Easy horizontal scaling
• Reusability
• Create complex, multistep, reusable data pipelines and
operators

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 14


SAP Data Intelligence | Out-of-the box operators
SAP HANA Cloud integration

SAP HANA Cloud


Metadata Profiling,
catalog discovery
SAP HANA
Virtual
Table Views Lineage Data
table extraction preparation
Create

Metadata explorer

SAP HANA SAP HANA SAP HANA SAP HANA SAP HANA

Read Delete Update

Modeler

SAP Data Intelligence

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 15


SAP Data Intelligence | Out-of-the box operators
SAP HANA integration

Predefined SAP HANA operators to consume and interact

Initialize HANA Table Operator that Initializes one or more tables on an SAP HANA database.

Read HANA Table Operator to read data from a table in SAP HANA.

Run HANA SQL Operator that executes user-provided SQL statements on an SAP HANA database.

Write to HANA Table Operator that executes user-provided SQL statements on an SAP HANA database.

Change Data Capture (CDC) is a Delta Capture Technique that uses triggers for Insert, Update, and
Table Replicator
Delete to track the change history for a specific table.

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 16


SAP Data Intelligence | Out-of-the box operators
SAP ABAP integration

Predefined ABAP operators to consume and interact

ABAP CDS Operator that supports initial load and delta replication of CDS views.
view reader

SLT connector Operator to communicate with SAP Landscape Transformation Replication Server;
operator leverage existing replication scenarios to bring data into SAP DI pipelines

Operator that can read from operational data provisioning out of SAP Business
ODP reader Warehouse or SAP BW/4HANA into SAP Data Intelligence

Custom ABAP Operator to implement your ABAP custom code that will be executed as part of a
operator pipeline in the connected ABAP system – for example, to call a function module

Integration requires a certain system level, planned for SAP S/4HANA 1909, SAP S/4HANA Cloud 1908, and SAP NetWeaver 7.00 with DMIS 2011/2018 Q4/2019 version, or higher.
Certain functionality can be made available for SAP Data Intelligence for certain release levels.

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 17


SAP Data Intelligence | Out-of-the box operators
Data Lake integration

Predefined operators to consume and interact

Read File Operator Operator that reads the content of files from various storage services.

Write File Operator Operator that writes files to various services.

Structured File Consumer Operator reads from any supported cloud storage.
The operator produces structured output, and you need to connect it to other operators from
Structured Data Operators category.

Operator receives data from any structured data operators and produces a file (CSV, ORC,
Structured File Producer
or PARQUET) in the specified storage.

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 18


SAP Data Intelligence | Out-of-the box operators
SAP Analytics Cloud integration

SAP Analytics Cloud

Create “on the fly”


Story Data set
Push result set to SAP Analytics Cloud
using a dedicated API for SAP Analytics
Create Create Cloud
Model

Specify host for


SAP Analytics
Cloud and
OAuth2 Client
information

Modeler

SAP Data Intelligence

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 19


© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 20
SAP Data Intelligence
Overview - Processing

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 21


SAP Data Intelligence – Data Processing

Explore your data assets, create and validate your models, and
orchestrate any mix of data processing engines, seamlessly within the
data pipelines that feed them
© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 22
SAP Data Intelligence – Data Processing

Search & Connection Data Data Model Integration into Automation &
Model Model Model
Browse /Storage Preparation Processing Validation Application Maintenance
Training Deployment
Data Management Creation

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 23


© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 25
SAP Data Intelligence
Overview - Cataloging

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 26


SAP Data Intelligence – Data Catalog
Metadata Management using the Meta Data Explorer

▪ Discovery and profiling


SAP Data Intelligence
▪ Business glossary Metadata explorer
▪ Collaborative rating and comments Discovery and Data Business
Search Lineage
profiling preparation rules
▪ Self-service data preparation
▪ Data lineage
Data
▪ Data quality and business rules catalog
▪ Tight integration with data pipelines to streamline
the end-to-end data-to-value workflow Metadata crawler Manual definition

Connected Sources

Data intelligence sources Other metadata repositories


(DBS, SAP HANA, SAP BW, (SAP Information Steward, SAP
object stores, EDWS, WS/APIs, Hadoop, PowerDesigner, SAP EA Designer,
NOSQL, enterprise applications, dev Atlas/Navigator, Hive, APIs …)
platforms, APIs, SDK, …)

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 27


SAP Data Intelligence | Enterprise Data Catalog
Data Discovery & Governance

Data discovery,
browsing and Business
searching Glossary

Data lineage Self-service data


preparation

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 28


SAP Data Intelligence | Enterprise Data Catalog
Data Quality Monitoring

Data quality
Data profiling rules

Data quality
dashboards User ratings

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 29


© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 30
SAP Data Intelligence
Hybrid Data Management

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 31


Hybrid Data Catalogs in enterprise landscapes
Federation and metadata exchange, to avoid rip and replace

In large enterprises a single catalog 3rd party


catalogs *
is not realistic (e.g. Collibra,
Alation)
SAP Data SAP Data
Warehouse
Metadata is spread among Cloud Intelligence
multiple physical catalogs

3rd party DBs


and data
Data objects and business terms are lakes
managed in heterogeneous systems SAP
Information
Steward
SAP
Applications
Federation is key​

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC *planned 32
Rolling out a catalog solution
Scaling the project scope along different axis
Rolling out to more USERS or LINE OF BUSINESS
or REGIONS or DEPARTMENTS

Adding further MODULES


or CAPABILITIES

Connecting and Integrating


more SYSTEMS or
APPLICATIONS

From Pilot to Production

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 33


SAP Data Intelligence enabling Hybrid Data Management
Transition to Cloud Data Management at your own pace

End-to-end data fabric across SAP Data


hybrid landscapes Services

3rd Party

SAP HANA
Safeguard past investments
by reusing existing assets

SAP Data Open Source


Modernize landscape SAP Intelligence
without disruption Information
Steward

SAP LT
SAP Replication
Integration Server
Cloud elasticity & agility
Suite

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 34


SAP Data Intelligence enabling Hybrid Data Management
Leverage existing SAP EIM tools

Data Discovery &


Orchestration
Governance

SAP Data SAP Data SAP Information SAP Data


Services Intelligence Steward Intelligence
Common Support Wide
Connections Span of Data
[Data Lake]

Data Quality, Enrichment,


Operationalization & Compliance

▪ Extend reach of Data Services ▪ Reuse Information Steward


assets across diverse system landscapes​ rules within Data Intelligence​
▪ Orchestrate Data Services jobs ▪ Shared glossary of terms​
within Data Intelligence pipelines ▪ Metadata shared across
▪ Enhance enterprise business applications, augmented by ML
processes with high value data and learning models​*
insights from external systems
*planned
© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 35
Data Flow and Data Intelligence – Feature comparison
Data Flow in Data Warehouse Cloud Data Intelligence Cloud
Simplicity and Ease of Use Flexibility and Power
User Persona: Business Users / Analysts User Persona: Data Engineers / Data Scientists / Developers

Included for free within DWC, with simplified resource Separate service, integrated ootb with DWC, with fine-
control grained resource control and autoscaling

Flexible support for any-to-any integration scenario,


Support data ingest from supported sources into DWC
including data distribution out of DWC

Simplified no-code UI for citizen users, powered by the same Full-blown low-code UI to leverage the whole
technology as Data Intelligence Cloud functionality of the underlying pipeline engine

Flexible support for any kind of structured, unstructured


Manage structured data and text-based unstructured data
or streaming data

Flexible support for any kind of data transform and data


Support for relational data transforms (union, join, merge,
processing (any Python, R, NodeJS, Go), ML and
project) and simple Python scripting (NumPy and Pandas)
orchestration of external processing engines
Full-blown enterprise data catalog, metadata
Predefined selective features, predefined quality rules,
management, agile data preparation features; flexible
DWC data catalog for immediate ease of use
data quality rules across any data source

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 36


SAP Data Intelligence
Licensing

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 37


SAP Data Intelligence
License models on Cloud
SAP BTP Plans and Pricing (link)
• Available in the cloud as subscription and consumption-based
model (link for details)
• A Data Intelligence Node is a compute node that includes up to 8
CPU cores and up to 32 GB RAM
• Capacity Units are the number of units consumed by the usage of
the services and are tied to the consumption of infrastructure
resources (compute and storage)
• Minimum volume: 3,000 Capacity Units per month

New Cost Management Features


• Hibernation
• Customers can now hibernate the DI Nodes when not in use and will
not incur costs during hibernation except for any persistent storage
• Hibernation can be scheduled* by customers

• Autoscaling
• Customers automatically scale their deployment based on resource
requirements and user defined resource limitations
• Billing and metering is recording system usage based on scale up /
down on an hourly basis

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC * Check the current limitations 38
SAP Data Intelligence
Estimating the required usage
Sizing Calculator: Link to online calculator
• Determine the required amount of Capacity Units based on the
chosen Data Intelligence services and usage volumes
• How to use?
• Data management & orchestration services:
• Users: Add the relevant amount of concurrent total and
Jupyter users
• Pipelines & Jobs: Add the relevant amount of active,
concurrent pipelines or jobs

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 39


Example: Data Management & Orchestration Services
Data Management & Orchestration Services Project
▪ Services for executing data pipelines or jobs for data preparation, data profiling or metadata extraction
▪ ML services are now included in the DI Node and pipelines making use of ML capabilities (R, Python, TensorFlow) are considered large
pipelines for the purposes of estimation calculations

Example: Execution of different pipelines and jobs to manage and orchestrate data
▪ 100 small pipelines (each 0.5 GB), 30 medium pipelines (each 1 GB) and 20 large pipelines (each 1.5 GB) -> Consumption of 100*0.5 GB
+ 30*1 GB + 20*1.5 GB = 110 GB memory
▪ 2 jobs extracting metadata into Metadata Explorer (each 1 GB), 2 jobs browsing and data profiling of connected systems (each 1 GB), 2
jobs executing data preparation processes of connected systems (each 1.5 GB) -> Consumption of 7 GB memory
▪ 3 concurrent users -> 8.7 GB of memory
▪ Memory consumption and nodes required
– Pipelines, data jobs and users: 125.7 GB memory
– Workload by basic system components (mandatory components to run the platform): 64 GB memory
– Total consumption of 189.7 GB memory -> Need for 6 Data Intelligence nodes (each 32 GB)

▪ Node hours per month: 6 * 720 hours (24 hours x 30 days) = 4,320 hours
▪ Capacity Units per month: 4,320h * 1.36 (Value of Data Intelligence Node) = 5,876 Capacity Units (rounded up)

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 40


SAP Data Intelligence
Strategic Roadmap

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 41


SAP Data Intelligence | Strategic vision
Unification of capabilities

DATA INTEGRATION DATA CATALOG DATA PROCESSING

SAP LT Replication Server SAP Information Steward


On-prem On-prem

SAP HANA Smart Data Integration SAP HANA Agile Data Preparation
On-prem On-prem

SAP Data Services SAP HANA Smart Data Quality


On-prem On-prem

SAP Data Intelligence


Cloud, on-prem and hybrid

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 42


SAP Data Intelligence – Longer-term roadmap priorities*

Deeper Application Business Content & Extend


Integration Templates Span of Connectivity
Align integration model Extend both SAP-led and partner-led content Support more cloud services (e.g. DWHs),
to consume and interact with Apps store for pre-packaged industry and LoB content and additional data sources

Synergy with HANA Get (even) Open Data Catalog and Unified & Hybrid
Cloud, DWC and SAC Smarter end-to-end Governance Technologies
Tighter integration of the BTP UD&A Apply ML and intelligence where possible. Extend metadata exchange and improve Fine tune consolidation and alignment
portfolio Augmented Data Integration & Active Metadata integration with 3rd party catalogs. Make data with on-prem EIM portfolio, enabling
& Data Governance, Automated Data quality rules smarter, and add enforcement for hybrid data fabric scenarios
Classification, Quality and Monetization data privacy rules. Improve collaboration

*This is the current state of planning and can be changed by SAP at any time
© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 43
SAP Data Intelligence
Q&A

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 48


Thank you!
Igor Alexandre Jakuboski
SAP Principal Solution Architect
igor.jakuboski@sap.com

© 2022 SAP SE or an SAP affiliate company. All rights reserved. PUBLIC 49

You might also like