Unity Catalog
Learning Objectives
u What is Unity Catalog
u 3-level namespace
u Security model for governing data objects
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
Unity Catalog
u Centralized governance solution across all your workspaces on any
cloud.
u Unify governance for all data and AI assets
u files, tables, machine learning models and dashboards
u based on SQL
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
Architecture
Before Unity Catalog With Unity Catalog
Workspace 1 Workspace 2 Unity Catalog
User/group User/group User/group UC Access
management management management Metastores Controls
Hive Metastore Hive Metastore
Access Controls Access Controls Workspace 1 Workspace 2
Compute Compute Compute Compute
resources resources resources resources
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
UC 3-level namespace
SELECT * FROM schema.table
SELECT * FROM catalog.schema.table
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
UC hierarchy
UC
Metastore
Catalog
Schema
(Database)
Table View Function
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
UC hierarchy
UC
Metastore
Storage External
Catalog Y Y
Credential Location
Schema
(Database)
Table View Function
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
UC hierarchy
UC
Metastore
Storage External
Catalog Share Recipient
Credential Location
Schema
(Database)
Table View Function
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
Identities
u Users: identified by e-mail addresses
u Account administrator
u Service Principles: identified by Application IDs
u Service Principles with administrative privilege
u Groups: grouping Users and Service Principles
u Nested groups
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
Identity Federation
Account
user1@company.com
Workspace 1 Workspace 2
user1@company.com user1@company.com
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
Privileges
u CREATE
u USAGE
u SELECT
u MODIFY
u READ FILES
u WRITE FILES
u EXECUTE
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
Security model
GRANT Privilege ON Securable_Object TO Principal
Privileges Principles
Securable objects
CREATE UC
Metastore
USAGE
Storage External User
SELECT Catalog Share Recipient
Credential Location
Service Principles
MODIFY Schema
(Database)
READ FILES group
Table View Function
WRITE FILES
EXECUTE
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
Accessing legacy Hive metastore
UC
Metastore
hive_metastore dev prod
Unity
Workspace Schema Schema Schema Catalog
Table Table Table
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
Features
u Centralized governance for data and AI
u Built-in data search and discovery
u Automated lineage
u No hard migration required
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
Account Console
u Log in as account administrator
u https://accounts.cloud.databricks.com
Derar Alhussein © Udemy | Databricks Certified Data Engineer Associate - Preparation
