PCF Architecture

BUILT FOR THE SPEED OF BUSINESS

Pivotal CF Architecture
Key Terminology, Architecture
Pivotal Korea, 2014
© Copyright 2013 Pivotal. All rights reserved. 2

목차
• PCF Key Concept
• PCF Components
• Elastic Runtime Architecture
• PCF Differentiator

PCF 용어집
• Applications
• Buildpacks
• Manifests
• Organizations
• Spaces
• Users and Roles
• Domains
• Routes
• Services

PCF Key Concepts
Application, Buildpack, Manifest

Cloud Foundry Platform as a Service
• Application : Deploy의 단위
– VM,middleware,Container,Process의 추상화
– Data as a Service
– Provision하고 배포하는 bottleneck을 제거
– PaaS는 Application을 효과적으로 배포하기 위해 존재
– 개발자는 app에만 집중하고 runtime환경이나 service,Infra는
신경쓰지 않도록 설계

VM 중심 배포에서 Application 중심 배포로

IaaS: VM 이 배포의 단위
Scale Out: VM의 신규 생성 혹은 Template을 통한 생성

PaaS: Application 중심의 배포
Scale Out: VM pool 내에 Container 를 생성

Cloud 환경상에서 Application 의 Deploy
Cloud Foundry is Development Agostic
 다양한 Application Source Code , Framework
Java/Spring, Ruby/Rails, Java Script for Node.js …
 Cloud Foundry 상에서는 모두 수행이 가능
 How?

Server Configuration
개발자에게 필요한 개발환경 제공 = 운영비용, Time To Market 
 OS
 Software Runtime(Java,Ruby,Python…)
 Container(Tomcat for JAVA, Apache HTTPD for PHP..)
 Framework(APM tool..)
 Application Binary
 자동화된 Script의 필요

Buildpack
CF상에서 수행될 수 있는 개발환경 생성을 위한 자동화된 Script
 Runtime, Container, Framework 과
개발 Application 을 Droplet 에 올리기 위한
script의 조합
 Droplet은 Warden Container상에서 구동.
 Warden Container 는 DEA상에서 구동.

Buildpack “builds” the “droplet” to run an
app.
= Application Staging
 Buildpack은 특정 application을 수행하기 위한 droplet을
어떻게 조합하는지를 정의

Buildpack 구조
일반적으로 ruby script 로 작성되며 3개 파트로 구성 .
 Detect : 해당 buildpack이 적용되어야 하는지 결정
 Compile: application의 필요에 따라 code 와 runtime,
framework, plugin등과 같이 compile 함
 Release: assign된 DEA 들에 application들을 deploy

Deploying CF

Staging
 Buildpack은 application을 위한 machine image를 준비함

가용한 Buildpack
 Buildpack은 CF에 설치되거나, push시 외부에서 load 됨
 PCF/CF Community 에서 제공된 buildpack을 사용, 혹은
custom buildpack을 작성해서 사용

External Buildpack
Heroku 및 기타 PaaS의 buildpack과 호환됨.

Buildpack
Runtimes Containers Frameworks

Manifest
 Application 배포의 “Blueprint”
 배포의 옵션 정보들을 기입
 Yaml형식으로 작성되며
App Push시 참조됨

PCF Key Concepts
Organization, Space, User, Role

Organization
• 최상위 관리 단위
• 일반적으로 회사,부서,App Suite,
큰 프로젝 단위로 생성
• 1개 이상의 Domain 정의 가능
• Quota 지정으로 resource 통제

Space
• Organization은 여러개의 space를 생성
• 예: 개발, staging, QA, 운영계
• 유저가 추가적으로 space 생성
• Application은 Space 내에서 수행 및 Scale In-Out 가능

User and Role
• User = Organization의 멤버
• Role = Domain과 Space에 대한 접근 권한 제어
어떤 유저가 Route 관리
Application Deploy
add/bind/remove service 권한이 있는지 관리

Role 의 종류
• Organization Role
– Organization Manager : user 초대/관리, plan 선택/변경, 사용량 조정
– Organization Auditor : 모든 org/space 접근 정보/셋팅,report에 대한 조회
• Application Space Role
– Space Manager : user 초대/관리, 해당 space에 대한 기능 관리
– Space Developer : application,service 생성/삭제/관리, report,log에 대한 조회
권한
– Space Auditor : space에 대한 셋팅정보,report,log에 대한 조회권한

PCF Key Concepts
Domain, Route

Domain
• Deploy된 Application 은 URL을 가짐
• Cloud Foundry Instance 는 Default Domain을
가짐(pcf.open-paas.com)
• Custom Domain을 추가해서 CF에서 관리 가능
• Subdomain: Application은 Unique한 Sub-domain을 가짐
– Application URL: subdomain.domain
– 예: http://myapp.pcf.open-paas.com

Route
• Application의 접근 방법
– 모든 space의 각 Application은 Unique한 Route 를 가짐
– CF는 load-balance를 사용하여 incoming request를 알맞은
application으로 route 시킴
• 동일한 Application이 여러 space에 mapping 가능
– 예: dev space route: http://myapptest.pcf.open-paas.com
– prod space route : http://myapp.pcf.open-paas.com

PCF Key Concepts
Service

Cloud Foundry Service
What is Service?
 외부의 application dependency 혹은 component
 Database
 Message Queue
 Monitoring App
 Hadoop Instance
 Security
 일반적인 Service Endpoint(Web Service)
 Other dependent applications

Cloud Foundry Service
Feature & Functionality
 Provide Functionality to your applications
 Application의 외부에 존재 – add-on provisioned aloneside
an application
 여러 application에서 공유되어 사용 가능(DB, Message
Queue)
 Application 에 binding – Service Broker 를 이용
 Application에 환경변수를 이용하여 접속정보를
제공(VCAP_SERVICES)

Clound Foundry Service
Why use Service?
 Application은 deploy의 단위
모든 application은 self-contained이여야 하며, 나머지 모두는 PaaS에서
제공되어야 함 : Service
 Service in a PaaS
과금이 될 수 있는 단위(IaaS에서 H/W 자원과는 상반됨)

Service Type
 Marketplace Service
– PCF설치후 곧장 이용할 수 있는 서비스들
– Marketplace catalog에서 이용
– PaaS에 instance 가 생성됨
 User Defined Service
– 외부의 서비스를 Cloud Foundry에서 이용
– Connection 정보를 저장하여 연결시 사용
 Custom Built Service

Marketplace Service
 Easily available via Marketplace
 Allow you to sign-up,
select plans, etc
 Once bound to application,
can be used easily
 2014년 4월기준 provider list

Pivotal CF Service
Standard out-of-box service : 계속 추가되고 있음

PCF Components
Ops Manager, Elastic Runtime..

Operations Manager
 업계 최초의 IaaS와 통합된 PaaS 운영
관리 UI
 Ops Manager=the core orchestrating
component.
 VM들의 생성, deployment, 기타 PCF
software와 service들의 lifecycle 관리.
 PaaS 셋업과 운영 툴 제공
 Application Downtime 없이 Live Update 제공
 Ops Manager Web Console

Elastic Runtime
 Scalable Runtime Environment
 Cloud Foundry itself
 Application 의 Runtime 환경관리
 Buildpack
 Component 별 Sizing /환경 설정
 Ops Manager를 사용해 IaaS
관리통제

Pivotal Services
 PCF 는 PHD, RabbitMQ, MySQL Dev 등의 Data Service 를
제공
 서비스들은 Message Broker를 통해 Application에 연결되고
Binding

Pivotal CF Architecture
Ops Manager UI
Ops Manager Director
Operations Manager
Service Broker
Service Nodes
Service
Service Broker
Service Nodes
Service
HA Proxy LB
Dynamic Router
Health Manager
Login Server
DEA Pool
Apps
Messaging (NATS)
App Log Aggregator
Cloud Controller
UAA
Apps
Metrics Collection
Elastic Runtime

Architectural
Components

Cloud Foundry
Architecture
• The Cloud Foundry platform
Architecture = 대형 분산 서비스 처리
아키텍쳐의 표준
• Cloud Foundry Bosh 를 사용하여
기반 IaaS 를 관리 및 통제함(e.g.,
VMware, Amazon AWS, OpenStack)
Underlying
Infrastructure
BOSH

BOSH
How It Works:
• BOSH = 대형 분산 시스템을
대상으로 한 배포, lifecycle 관리하기
위한 공개 S/W
• PCF는 BOSH를 사용하여 Application
Provisioning 및 infra 제어 등을
수행함.
• Cloud Foundry 관리및 배포를 위해
개발되었으나 일반적인 분산
시스템에서도 사용 가능
Responsible For:
• VM 생성과 관리
• Continuous and predictive updates
with minimal downtime
• BOSH 는 CPI (Cloud Provider
Interface) 를 사용하여 vSphere,
vCloud Director, Amazon Web
Services EC2, OpenStack 등과 같은
IaaS provider 상에 service를 deploy
할 수 있음 .
• Pivotal CF Operations Manager가
Bosh를 사용함. 직접적으로 access할
필요가 없음

BOSH : 분산시스템 관리
 BOSH를 통해 Application을 Provisioning 하고 인프라는 제어하는 등의 다양한
기능을 수행. 구성요소 설명
CLI BOSH와의 Interaction을 위한 Command Line Interface
Director VM 생성, Application Deployment, 기타 Lifercycle을
관리하는 BOSH의 core orchestrating component
CPI 개별 IaaS를 추상화 시킨 표준적 BOSH Engine
Blobstore BOSH가 수행하는 Release, Job, Package의 Content가
저장되는 공간
Agent Director로부터 명령을 받아 개별 작업을 수행하는, 모든
VM에 설치된 Agent
Stemcell 내장된 Agent를 포함한 VM Template
Health Monitor Agent를 통해 개별 VM의 상태를 체크하고, 이상 발견 시
조치하는 요소
NATS Lightweight Pub/Sub, Distributed Queuing Messaging
System
Operations
Staff
DB Blobstore
NATS
Director
Worker Health Monitor
Inner
Shell
CLI
Agents
Outer
Shell
CPI

BOSH Modules

PCF Architecture 의 특징
 Loosely Coupled 된 독립적인
System Component들로 구성
Idempotent Asynchronous
 표준적 Communication Model,
쉽게 측정되고 진단 가능
 Blocking을 유발하지 않는 Event-
Driven Interaction 모델
 특정 Operation에 의한 전체 시스템
성능저하 방지(Consistent)
 자동 Restart가 가능한 내부
Watchdog에 의한 지속적 Health
Checks
 No single point of failure
 모든 명령과 제어가 Message
기반으로 동작
 언어 독립적 Communication
 Application Load/Traffic에 대해
신속한 수평적 확장성 보장
Scalable Secure
 공유 인프라에 대한 Resource
Pooling
 Application은 분리된 안전한 영역의
Container에서 실행
 개별 사용자/그룹에 다른 레벨의
Security Role을 제공하는
Organizations & Spaces (CF Command)
Self
Healing
Message
Based

Pivotal CF Elastic Runtime Architecture

HA Proxy
Access
App
myapp.<mycfdomain.com>
DNS
HA Proxy
Resolve app and system domain
names to HA Proxy IP
Router Router
Software single-instance LB, for
non-production purposes
DEA DEA DEA
Router
Pivotal CF Elastic
Runtime (PaaS)

External Load Balancer(HA Proxy 대신 사용 가능)
Access
App
External LB, usually domain names to LB IP
Load Balancer
Load Balancer
Resolve app and system
Router Router
hardware-based (F5 or similar)
DEA DEA DEA
Router
DNS
Pivotal CF Elastic
Runtime (PaaS)

Router
How It Works:
• PCF 의 모든 유입 HTTP traffic 을
다른 component로 Route함
 System traffic(cf command)
 Application traffic
• 각각의 load balance된 app
instance 용 dynamic routing
table 를 관리함
• 여러 개의 router 를 사용 가능
Responsible For:
• Load balancing
• Maintaining an active routing table
• Access logs
• Supports web-sockets

Cloud Controller
How It Works:
• Client Interface(CF
CLI,WebUI,STS등), Account 및
Provisioning Control 등의 명령어 및
Control System 등을 관리
• Domain Object(apps, services,
organizations, spaces, service
instances, user roles, and more) 와
통신하기 위한 RESTful interface 를
제공함.
Responsible For:
• Expected App state, state transitions,
and desired convergence
• Permissions/Auth
• Orgs/Spaces/Users
• Services management
• App placement
• Blob storage

UAA and Login Servers
How It Works:
“User Authorization and
Authentication”
identity, security and authorization
services 를 제공
It manages 3rd party Oauth 2.0 access
credentials and can provide application
access and identity-as-a-service for
apps running on Cloud Foundry.
Composed of: UAA Server, Command
Line Interface, Library.
Responsible For:
• Token Server
• ID Server (User management)
• OAuth Scopes (Groups) and
SCIM
• Login Server
• UAA Database
• SAML support (for SSO integration)
and Active Directory support with
the VMWare SSO Appliance
• Access auditing

Health Manager
How It Works:
• Cloud Controller 에서 발행되는
Expected State 와 각 DEA 에서
발행되는Actual State 를 NATS
message bus 를 통해
전달받아서, 이를 비교함
• 문제가 있을 경우(상태값이
상이한 경우) 이를 Cloud
Controller 에게 전달함.
Responsible For:
• Maintains the actual state of apps
• Compares to expected state
• Sends suggestions to make actual
match expected (cannot make state
changes itself – only CC can do that!)

DEA
How It Works:
• “Droplet Execution Agents” 의
약자. are secure and fully
isolated containers.
• DEA는 Apps의 lifecycle을
관리함: building, starting and
stopping Apps as instructed.
• 주기적으로 현재 DEA의
상태(state)값을 NATS message
bus를 통해 broadcasting함
Responsible For:
• Linux containers (Warden) 관리
• Resource pools 모니터링
• Process
• File system
• Network
• Memory
• App lifecycle 을 관리
• App log and file streaming 처리
• DEA heartbeats (NATS to CC,
HM)

Messaging (NATS)
How It Works:
• Publish-Subscribe 메커니즘을
통해 CF 시스템의 각 component
간의 통신을 가능케 하는 Fast
internal messaging bus
Responsible For:
• Non-Persistent messaging
• Pub/Sub
• Queues (app events)

Service Broker
How It Works:
• Native(Managed Service) 혹은 3rd
party service 를 위한 interface를 제공
• Service processes run on Service
Nodes or with external as-a-service
providers (e.g., email, database,
messaging, etc.).
Responsible For:
• Advertising service catalog
• Makes create/delete/bind/unbind calls
to service nodes
• Requests inventory of existing
instances and bindings from cloud
controller for caching, orphan
management
• SaaS marketplace gateway

User Provided Service Instances
How It Works:
Service Broker 내에 meta data를
저장하여, Cloud Foundry가 외부의
CF에서 관리되지 않는 서비스(예:
OracleDB,DB2,SQLServer등..) 에 연결될
수 있게 함.
Responsible For:
• Metadata management

User Provided Service Instances
and Service Brokers
Synchronous Synchronous
Custom
Service
Broker
Service
Broker
Service
Broker
UPSI Service
Connector
IBM DB2
UPSI Service
Connector
ORACLE
DB
CF MySQL
MYSQL DB
Another
3rd Party
Synchronous
Provider (e.g.
AppDirect)
Send
Grid
INTERNET
Mongo Service
ClearDB
Lab

Creating and Binding a Service Developer
create service (HTTP) create service (HTTP) reserve resources
CLI Cloud
Controller
Service
Broker
bind service (HTTP) bind service (HTTP) obtain connection data
Router
Cloud Foundry
Runtime (PaaS)
DB
Service
credentials
Data
Service

PCF 구성 요소
Component 설명 Implementation
Router Incoming Traffic을 적절한 CF Component로 Routing GoRouter
Authentication Identity Management Service UAA
Cloud Controller Application Lifecycle 관리 CCNG, CC_DB, Blob Store
Health Manager Application Monitoring, Cloud Controller 제어 HM9000
Application Execution Warden Container 포함, Application Instance 관리 DEA
Blob Store Application Code, Buildpacks, Droplets 저장
Buildpacks 언어/프레임워크을 Detect 하고 소스코드를 실행파일로 Compile, DEA
로 Release
Ruby/Java/Node 등
Service Broker Application이 필요로 하는 Database 등의 Service의 Instance를 제공 MySQL 등
Message Bus 내부 Component간 통신을 위한 lightweight publish-subscribe and
distributed queueing messaging system
NATS
BOSH release engineering, deployment and lifecycle management of
large scale distributed services
Agent, Director, CPI, Name
Server, etc

Pivotal CF Elastic Runtime 상의 App Deploy
① Upload app
bits and
metadata
push app
Router
+ app MD
② Create and bind services
③ Stage application
④ Deploy application
⑤ Manage application health
Blobstore DB
Cloud Controller
Service
credentials
Service Broker
Node(s)
DEA
DEA
DEA
DEA
+ =
Pivotal CF
Elastic
Runtime (PaaS)

Application Access
• Deploy된 Application는 DEA(Droplet Execution Agent)의
Container에서 서비스
• Application 사용자는 Router를 통해 DEA로 접근하며,
접근경로(Routing 정보)는 NATS Message Bus를 통해
지속적으로 최신 정보로 Update됨

Operations Manager: Behind the Scenes (BOSH)
BOSH Director
Message Bus
IaaS
Health Monitor
Pivotal CF Operations
Manager Director
DB
Blobs
Deploy my
Services
Deployment
• Packages
• Jobs
• Blobs
• Source
• Manifest
Worker VMs
Messaging
Target VM Health Manager
Target VM Cloud Controller
Target VM

Access
api.<mycfdomain.com>
console.<mycfdomain.com>
External LB, hardware domain names to LB IP
Load Balancer
based (F5 or similar)
Router
Pivotal CF
Elastic
Runtime
Load Balancer
Resolve app and system
Router Router
DNS
Monitoring of platform
components health and KPIs
Monitoring Tool
JMX
Aggregation, storage, filtering
and analytics on system and
app logs
Enterprise Log
Mgr (e.g. Splunk)
DDEEAA DEA
UAA/Login Server
Health Manager
CClolouudd C Coonnttrroolllelerr
JMX Provider
Collector
Loggregator
Service Broker NATS
HTTP
Service Broker
Service Broker
HTTP
Git repository
TCP
Versioning /configuration
mgmt of buildpacks and apps
Custom Service lifecycle
management and binding

PCF High Availability

4 Levels of HA in PCF
Elastic Runtime (ERS):
 Distribution across availability zones
 Application health management and
recovery
BOSH (cluster management):
 Process monitoring, recovery and
alerting
 Virtual machine health monitoring,
recovery and alerting
Pivotal Confidential–Internal Use Only 67

• Application Instance를
분리된 각 Hardware에
deploy하여 가용성을
보장함.
• 최대 50%까지의 H/W
failure시에도 downtime
없이 서비스 가능
• 여러 data center에 쉽게
deploy 가능
DEAs
1. ERS: Availability Zones
AZ1
api.pcf.com
App
AZ2 AZ3 AZ4 AZ5 Availability Zones
Chassi_1 Chassi_2 Chassi_3

Application Instances and Availability Zones
Router
Zone 1 Zone 2
DEA
DEA
DEA
App Ops
Cloud Foundry
Elastic Runtime
DEA
DEA
DEA

Router
App Ops
Zone 1 Zone 2 Application instances
DEA
DEA
DEA
are evenly distributed
over two availability
zones.
Cloud Foundry
Elastic Runtime
DEA
DEA
DEA

Router
App Ops
Zone 1 Zone 2 Application instances
DEA
DEA
DEA
are evenly distributed
over two availability
zones.
Loosing an AZ keeps
instances running and
available.
Cloud Foundry
Elastic Runtime
DEA
DEA
DEA

2. Application Instance
 The Health Manager is essential to ensuring that apps
running on Cloud Foundry remain available.
– Application 의 상태state (e.g. running, stopped, crashed, etc.) 및
버전, instance 수를 모니터링
– Determine applications' expected state, version, and number of
instances.
– Reconcile the actual state of applications with their expected state.
– Application 상태 불안정시 Cloud Controller에게 수정작업을 지시

Failed Application Instances Replaced
Router
Blobstore
Cloud
Controller
Health Manager
Messaging
(NATS)
DEA DEA DEA
App Ops
Cloud Foundry
Elastic Runtime

Router
Blobstore
Cloud
Controller
Actual State
Health Manager
Messaging
(NATS)
DEA DEA DEA
App Ops
Cloud Foundry
Elastic Runtime

Router
Blobstore
Cloud
Controller
Desired State Actual State
Health Manager
Messaging
(NATS)
DEA DEA DEA
App Ops
Cloud Foundry
Elastic Runtime

Router
Blobstore
Cloud
Controller
Health Manager
Messaging
(NATS)
DEA DEA DEA
App Ops
Cloud Foundry
Elastic Runtime

3. Platform processes DB
Blob
Store
CLI Director
Worker NATS
Health
Monitor
CPI
Inner shell
Outer shell
Agents
Health Monitor
Health Monitor는 Agent 를 통해
받는 상태값과 lifecycle event 를 받으며,
운영자에게 notification plugin(예: email)
을 통해 alert을 보낼 수 있습니다.

ERS Processes are Monitored
Message Bus
Health Manager
DEA
PaaS Ops
Cloud Controller
Cloud Foundry BOSH IaaS
Health Monitor
Responders:
pager
email
monitoring
…

Message Bus
Health Manager
DEA
PaaS Ops
Cloud Controller
Health Monitor
Responders:
pager
email
monitoring
…

4. VM Monitoring: Health Monitor & Resurrector
• Health Monitor는 BOSH Agent 를 통해
들어오는 상태값과 lifecycle event를 사용하여
VM들의 health 상태를 check합니다. Health
Monitor가 VM의 이상 감지시 notification
plugin를 통해 alert을 보내던지, BOSH
Resurrector를 trigger할 수 있습니다.
• Enable 시 BOSH Resurrector plugin 는
자동으로 Health Monitor에 의해서 멈추거나
응답하지 않는 VM들을 재생성합니다.
NATS
Health
Monitor
✔Ressurector
Agents
Re-create
VMs

BOSH Director
Message Bus
VMs are Monitored
DEA
PaaS Ops
Cloud Controller
IaaS
Actual State
Health Monitor
Health Manager
Responders:
pager
email
monitoring
resurrector
…
Cloud Foundry BOSH

BOSH Director
Message Bus
VMs are Monitored
DEA
PaaS Ops
Cloud Controller
IaaS
Actual State
Desired State
Health Monitor
Health Manager
Responders:
pager
email
monitoring
resurrector
…
Cloud Foundry BOSH

BOSH Director
Message Bus
VMs are Monitored
DEA
PaaS Ops
Cloud Controller
IaaS
Health Monitor
Health Manager
Responders:
pager
email
monitoring
resurrector
…
Cloud Foundry BOSH

BOSH Director
Message Bus
VMs are Monitored
DEA
PaaS Ops
Cloud Controller
IaaS
Health Monitor
Health Manager
Responders:
pager
email
monitoring
resurrector
…
(CPI)
Cloud Foundry BOSH

Summary

Cloud Foundry Project
Private
Clouds
Open
Source
Micro
Clouds
Public
Clouds
CUSTOM
SERVICES
http://www.cloudfoundry.org

Abstracting and Standardize
Clouds Runtime/Frameworks
App Services
Buildpacks Service Brokers
… and Custom Runtimes … and Custom Services
Cloud Provider Interface
(CPI)
Open Source
… and Custom Clouds

References
• Cloud Foundry
• http://www.cloudfoundry.org
• http://network.pivotal.io/
• http://run.pivotal.io
• Installing CF on vSphere
• http://www.youtube.com/watch?v=TLhST6Hmiso
• Cloud Foundry demo
• http://www.youtube.com/watch?v=PFZPCwH_OcM
• Pivotal Academy(free e-training materials!)
• http://pivotal.biglms.com

Appendix: CF 대비 PCF 추가 기능
OSS Cloud Foundry 에 기업용 PaaS를 위한 Built-in 기능 추가
BOSH
Web Console
Ops
Manager
Elastic
Runtime
Data Services
Core Cloud Foundry
Components
Messaging Services
Mobile Services
Pivotal Support
PCF Add Value

PCF Architecture

More Related Content

What's hot

Similar to PCF Architecture

More from seungdon Choi

Recently uploaded

PCF Architecture

Editor's Notes