| Cloud Roster™
Data Engineer
WHAT DOES A DATA ENGINEER DO?
Data Engineers are responsible for building data pipelines. They develop, construct, and
test the large databases and the scalable data processing systems that data scientists
need to do their jobs.
Day-to-Day Expectations and Responsibilities
Design, develop, and deploy Improve overall efficiency
scalable data pipelines by introducing new engineering
and data services processes and tools
Research opportunities Understand existing systems
for data acquisition and new uses and recommend ways to marry disparate
for existing data data sets using automation
Integrate new data Collaborate with data
management technologies scientists and researchers
and software engineering tools to develop algorithms including
into existing infrastructure prototypes, predictive models,
and proofs of concept
Promote best practices
that enable teams to accelerate their
consumption and understanding of data
Key Traits of a Data Engineer
Creative Thinker: Collaborative: Curious:
Data Engineers connect the Ability to collaborate with Data Engineers
dots and see the big picture both internal technology demonstrate a track record
to evaluate how data and business teams as well of learning and deep-diving
insights affect the business. as integration partners and into complex systems.
customers
| cloudacademy.com
Cloud Roster™
THE JOB ROLES MATRIX FROM CLOUD ACADEMY
DATA ENGINEER
Trending Technology Skills for Data Engineers Data Engineer Target Skill Set
Technical Skill Set
TECHNOLOGY DEMAND
Statistical analysis and modeling
0% 50% 100%
Database architectures
Python
Python, Java
SQL Big data processing
Spark & streaming analytics
Experience with machine
Hadoop learning algorithms
Java
Non-Technical Skill Set
AWS
Data-driven approach to
Scala problem solving
Kafka The ability to visualize and
communicate complex concepts
Linux Strong sense of ownership
Oracle
*Figures represent the share of all Data Engineer job postings analyzed that require
a given skill. Data as of October 1, 2018. View this week’s data here.
View the Full Index
Starter Training Plan for Data Engineers
The following Cloud Academy courses and training plans prepare professionals to become Data Engineers.
Learn More About Training Plans >
Amazon Web Services Microsoft Azure Google Cloud Platform
Introduction to Machine Big Data Analytics on Azure Data Engineer – Professional
Learning on AWS Introduction to Azure Machine Certification Preparation
Applying Machine Learning and Learning for Google
AI Services on AWS Machine Learning on Google
Big Data – Specialty Certification Cloud Platform
Preparation for AWS Google BigQuery
Industry Certification(s) for Data Engineers
After taking courses with Cloud Academy, Data Engineers can consider the following certifications.
Amazon Web Services Microsoft Azure Google Cloud Platform
AWS Certified Big Data 70-776 Exam: Engineering Data Google Certified Professional
- Specialty with Microsoft Cloud Service Data Engineer
| cloudacademy.com
