Migration to Huawei Cloud

Module 10: Using and Migrating Container

Services
 Objectives
⚫ Upon completion of this course, you will:
 Understand Huawei Cloud container services.
 Understand how to migrate data to Huawei Cloud.
 Be able to perform hands-on exercises.

2
 Contents
1. Huawei Cloud Container Services

2. Container Migration

3
 What Is a Container?
⚫ Containers are a lightweight virtualization technology. This technology allows user space on an OS to be
isolated and divided into several independent units that run in the kernel.

⚫ Such an independent space is called a container.

4
 Differences Between Containers (Using Docker as
an Example) and VMs
Application 1 Application 2 Application 3

Support Support Support

module module module

Container Container Container

Customer OS Customer OS Customer OS
image 1 image 2 image 3

Virtualization layer (hypervisor) Docker engine

Host OS Host OS

Hardware Hardware

Server Container

5
 Differences Between Cloud Containers (Using
Docker as an Example) and VMs
Container Container Container
image 1 image 2 image 3
Application 1 Application 2 Application 3
Docker engine
Support Support Support
module module module Support module

Customer OS Customer OS Customer OS Customer OS

Virtualization layer (hypervisor) Virtualization layer (hypervisor)

Host OS Host OS

Hardware Hardware

Server Cloud container

6
 Comparison Between Containers and VMs

◼ Low efficiency for using a ◼ Smaller resource segments for higher

large amount of resources server utilization
◼ Slow startup in minutes ◼ Fast startup in seconds or even
◼ Maintenance for VMs shorter
◼ Different OSs and ◼ Management and maintenance for
configurations of virtual images
servers ◼ Consistent environment, or
environment-agnostic
◆ Easier migration
VM Container

7
 Container Scheduling and Kubernetes
A scheduler needs to know:
◼ Where are the available resources? What is ◆ Kubernetes is an orchestration management platform
the utilization of resources? proposed by Google.
◼ Which containers need to be run? ◆ Kubernetes is the most popular container platform.
◼ How many container instances need to be run? ◆ Huawei Cloud is an initial participant and important
◼ What is the way to start or stop an expected contributor of the Kubernetes community.
container on a selected resource? ◆ Huawei Cloud provides industry-recognized, leading
Kubernetes service implementation.

Kubernetes Master
(master node)

Node 1 Node 2 Node 3

9
 Container Implementation on Huawei Cloud

Image registration after development

SoftWare Repository
for Container
(SWR)
Resource Formation Service
(RFS)
Auto scheduling of basic resources

Cloud Container
Engine
Heavy O&M:
(CCE)
Cloud Container Cloud Container security, patches, availability,
Instance Instance Cloud backup...
(CCI) (CCI)
server
10
 CCE Turbo: A Flagship Container Engine Running
on Cloud Native Infrastructure
User UI Lower costs
New console • Container storage and networking offloading based on
QingTian architecture to improve resource utilization by 30%
• Higher container density (up to 256 independent IP
O&M upgrade Cluster version Cluster patch Add-on Cloud native addresses) on a single node in passthrough networking
Custom images
upgrade upgrade upgrade monitoring
Higher efficiency
• Passthrough networking through trunkports (for VMs) and ENIs
Backup and DR Data caching
Cross-cloud
Data migration
Storage fault (for BMSs), 40% higher performance
Storage access chain • Faster auto scaling: 500 nodes within 1 minute and 3,000
containers within 30 seconds

Networking Network isolation Passthrough networking

High container density on Comprehensive security
a single node • Kata containers deployed on bare metal nodes to isolate
resources and services in VMs
• Container network access control and other advanced features
CCE Turbo cluster based on VPC security groups

Trunkport ENI Intelligent scheduling

• Intelligent, hybrid scheduling: application topology
awareness, time-based resource scheduling, oversubscription

Common
container
Common
container
... Common
container
Kata
container
• Ultra-fast scheduling (up to 10,000 pods/second); cache +
multi-scheduler shared view

Comprehensive experience
• New GUI with scenario-specific wizard
Cloud native infrastructure powered by QingTian • Multi-path and one-click cluster upgrade without service
interruption
11
 SoftWare Repository for Container (SWR)

 Container image hosting

 Complete lifecycle management of images with

multiple tags

 Free storage and traffic

SWR  Compatible with Docker Hub and Docker commands

 Integration with CCE and CCI

 Event triggers

12
 SWR: Unified Cloud Native Artifact Repository
with Dedicated Isolation and Global Distribution
Product features:
OCI Helm Operator • SWR Enterprise Edition is independently deployed for
each tenant with dedicated domain names, resources,
and flow control.
Image Helm Operator
Enterprise-
repository repository repository dedicated
artifact Enterprise A Enterprise Advantages:
dedicated B
repository 1. Dedicated isolation
repository dedicated
OCI Helm Operator repository ⚫ Enhances the security of enterprise-grade
cloud native artifacts with physical multi-
tenancy deployment and access control.
2. Unified management of cloud native artifacts
⚫ Unifies management and distribution of
artifacts such as images, Helm charts, and
Operators.
Enterprise Enterprise A Enterprise B
3. Global image distribution and accelerated pulls:
applications application application
⚫ Allows pulls from Huawei Cloud, Huawei
Cloud Stack, and edge cloud across all
regions. Accelerates pulls with P2P, image
buffer, and download-free approaches.

Kubernetes Kubernetes Kubernetes Kubernetes

Huawei Cloud Huawei Cloud Agile Edge cloud
Stack Online
13
 Cloud Container Engine (CCE)

 Huawei Cloud container orchestration and

scheduling service
 Supports Kubernetes and Docker.
 Manages container network in VPCs.
 Generates master nodes during creation and
supports high availability.
CCE  Installs probes on nodes to collect monitoring
data.
 Requires the functions of adding, managing, or
automatically scaling out (worker) nodes.
 Uses kubectl to manage clusters

14
 Hybrid Cloud Native Resource Pool
CCE
ClusterManager NodePoolController CCI

• Supports hybrid
deployment of
API BMSs, VMs, and
Gateway
serverless for
high density and
CCE cluster CCI resource
flexibility.
pool
Prometheus HPA AutoScaler
Virtual • BMSs provide fixed
Kubelet
pod resources and VMs
provide elastic
Fixed resource pool Elastic resource CCI resource pool
pool pod resources. The
Node Node Node Node VM node
AutoScaler enables
pod pod pod pod pod
fast and flexible
pod pod pod pod pod scaling of cluster
nodes.

Container Node scalability Cross-service scalability

scalability
15
 Application Service Mesh (ASM) Traffic
1
Extended definition of VirtualService and definition of
complex ingress traffic: Define complex ingress traffic to
3 Rest (microservice egress protocol) solve microservice non-configuration due to traffic rule
Graphical Protocol gRPC (inter-microservice protocol)
coupling and redundancy.
console extension and 2 Reliability
integration Spring Boot (development framework)
No data plane restart during data plane policy upgrade
60% better performance and 50% less resource usage when tens and
thousands of pods are managed by a mesh
1 Fixed the out of memory (OOM) issue in large-scale scenarios (50,000
2 Istio O&M Service Traffic Outlier pods)
Rate limiting
Governance routing detection Envoy performance optimization: 70% higher QPS for the same number of
connections and 50% lower request latency for more threads
Pilot Protocol extension
3
Service discovery and
traffic management 6 gRPC: cross-language and high-performance RPC framework
Grayscale Canary Blue-green
A/B testing RESTful: front-end API management
Release release deployment Extended governance capability for services using the Dubbo protocol
Citadel
ID management Solution of integrating with microservice SDKs such as Spring Cloud
Galley 5 Hybrid deployment
Traffic Tracing Traffic Exception 4
Configuration
verification/ monitoring topology analysis warning Unified governance of hybrid deployment of VM applications and
management/delivery containerized applications

5 Observability
7
4 CCE cluster VM cluster BMS cluster End-to-end intelligent monitoring, logs, topologies, and tracing

6 Grayscale release
Node 1 Node N Node 1...N Node 1...N
Diverse forms of built-in grayscale release, such as canary
Pod 1 . Pod 1...N release, blue-green deployment, and A/B testing; traffic policies
VM BMS based on traffic ratio, request content (cookie, OS, and browser),
. VM VM
Container
Container Container
Container VM VM and source IP address
Container . Container
7 Cross-cluster governance
All-domain, unified service governance in multi-/hybrid
16 clouds
 UCS: Extends Cloud Native to Wherever Needed
Government Large enterprises Internet Finance
Seamless cross-cloud and cross-region
Smart Smart Smart Smart Live Internet DR and Asset
Security IoV Gaming Education
transportation community healthcare stores streaming finance backup management

Multi-cluster & multi-cloud management

Ubiquitous Cloud Native Service
(UCS) Public cloud | Third-party cluster | Self-built IDC | Edge node |
...

Intelligent traffic distribution

Unified cloud native app center with global distribution
and cross-cloud governance
App & data
Compute supply Traffic governance 150+ applications
collaboration
Unified cluster management
Global, multi-dimensional analysis
Unified permissions
Cluster lifecycle Config policies O&M observability
management Industry-leading Container Intelligent
Analysis

Hours ->
Third-party cloud 10 times 30%
Minutes
Local IDC O&M
Smartcom travel App access
efficienc
upgrade latency
Central region y
Dedicated region Edge cloud

17
 Contents
1. Huawei Cloud Container Services

2. Container Migration

18
 Cloud Native Application Migration Process
Scenario Phase Description
1. Cluster and node configuration information is automatically
collected by using tools. 2. Information about networking
Cloud native application survey
planning, dependencies, and service requirements is obtained
through surveys.
1. Evaluate whether the migration is complete and migration
actions are properly executed based on the migration checklist.
1. Use the image migration tool to migrate images. 2. After the
Image migration enterprise edition repository is brought online, migrate images on
the enterprise edition repository page.
1. Create a CCE cluster. 2. Create nodes (including labels and
Cloud native application taints) in the target cluster. 3. Customize resources (such as
Preparations for a CCE cluster
migration external DNS server configuration, storage and scheduling add-ons,
and credentials for accessing OBS).
Cloud disk, OBS, and file storage migration 1. Use the corresponding storage service for migration. 2. Create
the corresponding PVC/PV in the target cluster.

1. Connect the source and target clusters to UCS using the console
Application migration (free of charge). 2. Migrate clusters using the console (the
source cluster version must be 1.15 or later).

1. Modify the load balancer ID after the ingress and load balancer
Adaptation and modification after migration
are migrated.
Joint commissioning, test, and verification Cooperate with the customer to verify services.

19
 Container Migration Process - Survey

【1】 Migration scope survey

【2】 Infrastructure survey

【3】 Technology stack survey (covering cicd, monitoring, configuration center, etc.)

【4】 Application dependency survey (covering services for external systems,

internal mutual access, and application access to databases)

20
 Output a Service Architecture Topology of
Containerized Applications
Containerized Application
Service Architecture Topology of
Cloud services
a Project
MySQL Redis Kafka NAS
CCE cluster
Public access CCI APM
CCI AOM OBS
Servers
Deployment SWR
ConfigMap
CDN WAF DNS Microservic Load PV/PVC Secret
e gateway balancing Pod Pod Pod
OpenResty
Self-built services on
Git ECS Nexus server
repository
Node Node Node
Self-built services on ECS
Development Testing Pre-release Production
Build and push an image ZooKeeper MQs Apollo Elasticsearch
to SWR. Create a
Deployment workload using
Jenkins- the image on CCE.
ECS

21
 Item Sub-item Option Migration Method Remarks
Compute VM ECS SMS + Redeployment
VPC VPC
Load balancer ELB
Elastic IP address EIP
SNAT NAT Gateway Redeployment
Networking
DANT NAT Gateway
Private line Direct Connect
Security group SEG
SSL-VPN N/A Self-built and re-deployed VMs Dial-up access to cloud resources in offices
Interconnection between self-built CI/CD and CCE and
K8s CCE Container image, redeployment
containerization are involved.
Containerize Image repository, Harbor SWR is not supported. Self-built server + SMS
d Self-built MySQL RDS + DDM
application DRS + Database and table re-sharding Database and table sharding on the source client
Service-oriented MySQL RDS
Adaptation between the original standalone CME and the two-replica
Document database DDS DRS
deployment
RabbitMQ RabbitMQ Migrating the production first
Message
middleware Kafka Kafka Migrating the production first

Cache Redis DCS Online migration using DCS

middleware/
database Aerospike N/A Redeployment + Data backup and restoration Main cache middleware in the advertising industry

Dubbo N/A Containerization and redeployment

Microservice
architecture Self-built VMs + Elasticsearch cluster
ZooKeeper N/A
migration
Monitoring and alarms Cloud Eye + AOM Reconfiguring the traffic policies
The version does not There are some problems in the adaptation between the self-built
Self-built Elasticsearch Logstash+SMS
adapt to the CSS. 2.x version and the 5.x version on the cloud.
Service-oriented
CSS Logstash
O&M Elasticsearch
Grafana N/A Self-built server + SMS
Solr N/A Self-built server + SMS
CAT N/A Cluster redeployment An open source monitoring application
DNS DNS is not supported. SMS + Redeployment DNS takes effect only in the VPC.
22
 Migrating Cloud Native Applications

The process of migrating cloud native Compute

applications from one Kubernetes cluster to

another
Details:
✓ Compute: Applications (Deployment, StatefulSet, Cloud native
Storage Networking
application
DaemonSet, ...) and images
✓ Networking: Container networking, Services, and
ingresses
✓ Storage: StorageClass, PersistentVolumeClaim, and
Other
PersistentVolume dependencies

✓ Other dependencies: Middleware, relational

database, ...

23
 Example of Collecting Cloud Native Application
Information
Kubernetes version and cluster
scale
Container engine: runC, Kata, and Podman
Whether the vendor or self-developed CRD or operator is
used.
Whether there are strong requirements for guest
OSs. Kubernetes-
related Performance requirements
Whether the vendor-developed Webhook and sidercar are information
used. Network connectivity. Pods can be accessed
Whether the vendor or self-developed scheduler is Container externally.
used. networking Security requirements: flat network and network
Data source of auto scaling rules policies
Whether a fixed IP address is
Whether heterogeneous resources, such as Arm and GPU, are required.
used.
Whether snapshot, encryption, backup, capacity Workload type
expansion, and block storage are required. Containerized
applications Whether to use Helm to manage applications. Helm
Whether file is used for
versions.
storage.
Information
Whether OBS is used. collection
Container storage
Service mesh Whether to use Istio. Istio version.
Whether local disk is used
Whether to interconnect with the DevOps pipeline and whether standard
Whether local PV is used. Kubernetes APIs or vendor's private APIs are used

Whether security tools such as image scanning, image runtime Whether to interconnect with IaaS to automatically manage and scale
detection, and image signature are used. nodes.
Securit Whether to connect to the user authentication system and the connection
Whether the vendor-provided key management system (KMS) is used or y Interconnection mode.
whether the encryption protocol is used. with the Whether to interconnect with the O&M system and whether to interconnect with the
customer system O&M system through private APIs or middleware such as Kafka and Elasticsearch.
Whether vendor-provided middleware services are used, such as Other Whether the cloud CMDB platform is interconnected and whether standard Kubernetes
databases, caches, and message queues. dependencies APIs or vendor's private APIs are used.
Whether to connect to the load balancer and whether to use the vendor-defined
ingress.

24
 Summary of Cloud Native Migration Services
Project Sub-project Migration Method Remarks
Kubernetes Scale: 50/200/1000/2000
cluster Type: General and Turbo cluster

Container image Use Docker pull/push commands or image-syncer to push images to SWR.

Container For details, see Huawei Cloud planning: VPC network, ENIs, and overlay
Container networking
networking network.
Service/Ingress After the migration, modify the load balancer configuration.
Kubernetes Source cluster: Velero tool
metadata Target cluster: e-backup add-on
Container
storage
Source cluster: Velero tool
PV Single PV < 500 GB
Target cluster: e-backup add-on
Mount the volume to VMs and migrate data to Huawei Cloud using private
Non PV
lines.
Dependency
Relational database Migrate data through DRS.
OBS Migrate data through OMS.
Redis Migrate data through DCS.
NAS Migrate data using the rclone tool.

25
 Creating a CCE Cluster and an SWR Repository

Creating a CCE Cluster

1. Create CCE clusters, nodes, and namespaces based on the LLD output in the survey table.
2. Configure the kubectl command on nodes.

Creating an SWR Repository

1. Create an SWR repository and set access permissions.
2. Obtain a long-term credential for logging in to the
repository.

26
 Migrating Images to Huawei Cloud SWR Using
image-syncer Migration
Application scenario: Migrating an open source or self-built principle
Migrating an image repository using the image synchronization
image repository to Huawei Cloud SWR function
Other cloud or
Huawei Cloud
self-built
repository
(SWR) Migration procedure
1. Create an SWR repository.
Source Kubernetes 2. Create a namespace, set the default repository type, and
platform
CCE or CCI
enable the function of automatically creating a
Repository Migration SWR
repository.
3. Configure access control for the public network.
4. Configure access credentials.
5. Use the password created in the access credential to
complete the synchronization configuration of image-
syncer.
Image migration 6. Run the tool.

1. Many-to-many image repository synchronization is supported.

2. Docker image repository services (such as Docker Hub, Quay, and Harbor) based on Docker Registry V2 are supported.
Features 3. Memory- and network-dependent synchronization is fast.
and 4. Flushing the Blob information of synchronized images avoids repetition.
Advantages 5. Concurrent synchronization can be achieved by adjusting the number of concurrent tasks in the configuration files.
6. Automatic retry of failed synchronization tasks can solve most network jitters during image synchronization.
7. Docker or other programs are not required.

Notice Open source tool needs to be built.

27
 Migrating Dependencies
Storage data migration

1. Migrate backend storage volumes NAS and OBS to Huawei Cloud using the rclone tool.

2. Connect to an existing SFS file system (OBS) to create a PV and PVC using kubectl.

Cloud service migration

Migrate the source cloud services to the corresponding Huawei Cloud services.

3. Self-built ECS migration

Migrate the self-built ECS and source services to Huawei Cloud using the SMS host or

build a Huawei Cloud ECS for the migrated services.

28
 Migrating Applications
Application scenario: Migrating self-built Kubernetes Migration
clusters and vendors' Kubernetes applications to Huawei principle
The Velero migration principle is to back up and restore the
Cloud CCE 2. Data migration YAML file and PV/PVC storage of Kubernetes clusters to the
target cloud container engine.
Database Database on 4. Cluster
6. Cluster
server ECS testing Migration procedure
releasing
Image SWR 1. Deploy and configure a Kubernetes cluster on Huawei Cloud.
2. Migrate files, PVs, and PVCs using the Velero tool.
3. Migrate data.
5. Service
Ingress Ingress 1) Database and the stored data can be migrated. Databases are
switchover
migrated using DRS, object storage is migrated using OMS, and
file storage is migrated using rclone.
Service Service 2) For container image migration, you can use image-syncer to
automatically implement large-scale image migration. If the
image scale is small, you can also run the pull/push command to
Deployment Application Deployment migrate images.
Application restore 4. After the migration, perform regression tests to check
Config map related services.
3. Application
5. Grayscale release is used to switch service traffic to the
Pod Pod Pod configuration Pod Pod Pod
migration Huawei Cloud CCE container cluster.
Back up data to OBS. 6. After services run stably in the new environment, gradually
Kubernetes cluster Huawei Cloud CCE
bring the old cluster offline and complete the migration.
Install the velero add- Kubernetes cluster migration
Install the velero
on. server.
Notice
Only full migration is supported.
Applications and configurations can be migrated through reconfiguration using the CCE console or a YAML file.

29
 Modifying Applications for Adaptation
⚫ After the migration, update and adapt the service system to Huawei Cloud as required
to secure that services can run properly. The adaptation content includes but is not
limited to the following:

Adaptation Content Key Point

1. Obtain the SWR image repository address.

Image
2. Update the image address in the application configuration as required.

1. Obtain the connection information of a load balancer.

Accessing a Service 2. Update the application configuration as required and change the access mode to load
balancer.

Change the store class type as required. Currently, Huawei Cloud supports csi-disk, csi-disk-
Storage class
topology, csi-nas, csi-obs, and csi-sfsturbo.

1. Obtain the connection mode of Huawei Cloud RDS (check and update for cloud native
Database databases).
2. Update the application configuration and connect to the target database.

30
 Verifying Services
Check the running status of the Deployment workload and
check whether pods are killed due to a health check
failure.
Check the container service logs on AOM.

Enable the APM probe and view the call chain on the APM console.

【4】 Perform the function test and CTPS pressure test based on the service.

31
 Thank You.
Copyright©2023 Huawei Technologies Co., Ltd. All Rights Reserved.
The information in this document may contain predictive statements including,
without limitation, statements regarding the future financial and operating
results, future product portfolio, new technology, etc. There are a number of
factors that could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements. Therefore, such
information is provided for reference purpose only and constitutes neither an
offer nor an acceptance. Huawei may change the information at any time
without notice.

32