GDG Helwan Introduction to python

DEV SUMMIT 2017 EXTENDED
Introduction to Python 3
Mohamed Hegazy
R&D Engineer
.PY

What is python . WIKI
• Python is a widely used high-level programming language used for general-
purpose programming. An interpreted language, Python has a design philosophy
which emphasizes code readability (notably using whitespace indentation to
delimit code blocks rather than curly braces or keywords), and a syntax which
allows programmers to express concepts in fewer lines of code than possible in
languages such as C++ or Java.The language provides constructs intended to
enable writing clear programs on both a small and large scale.
• A scripting language is a subset of programming language that is used to produce
scripts, which are sets of instructions that automate tasks that would otherwise
be performed manually by a human. Of course, these "tasks" are essentially a
human giving instructions to a machine, as described above, so you could say that
a script instructs a machine on what instructions to give itself that a human
would otherwise give.
2

Agenda
1. Python Installation
2. Syntax
3. Objects
4. Conditions and Loops
5. Classes and Functions
6. Error Handling
7. Modules
8. Working with Files
9. Working With database
3

Python Installation
• Perquisites on windows.
• Download python for windows
https://www.python.org/downloads/windows/
• For this session we will use 3.6.0
• Download python for mac (for latest Update)
https://www.python.org/downloads/mac-osx/
• In Mac and Lunix its installed by default
• To run python use CMD -> Python
• Editors such as Pycharm , eclipse + pydev, Sublime, Aptana
4

Everything is an Object
• Id, type, value
• Id is unique identifier for the object
• Type is the class it is initialized from
• Value is the content
5

Syntax
• Python doesn’t use “int, double, string, var, etc…” as variable type.
Instead, python creates everything as an object and based on
assignment it calls the class and initialize
• Python doesn’t use “;”. It depends on whitespaces and indentation.
Wrong indentation means buggy code. P.S: you either use
whitespaces or tab.
• Example
• A = 0
• a,b,c = 0,1,”Hello”
6

Syntax Mutable vs immutable
• Number , Strings, Tuples are immutable.
• List , Dictionaries , other objects depending upon implementation.
7

Syntax
• When changing a value of an Immutable object. the reference of
value changes.
• Example
x=2
Id(x)
>>> 1231312
x=3
Id(x)
>>> 2349233
x=2
Id(x)
>>> 1231312
8

Syntax
• The variables created are stored in ram for later garbage collection to
pick it up. However there is a “faster” way using “del” keyword to
delete that variable
9

Syntax
• However, we can figure out the type of object with 2 ways.
1- the value
2- type(x) <- this will print out <class ‘str’>
a. it can be used to if want to make sure that the values are of certain type.
Note that print by default adds new line, it can be manipulated using
print (‘----’, end=‘’) <- end here will add zero space instead of newline
10

Syntax Dealing with Strings
• Strings can be used for chars and strings.
• Using single-quoted or double quoted is the same
• Since there is no char support, the work around is substring.
var1 = ‘hello world’
var2 = ‘python programming’
print(var1[0]) -> h
print(var2[8:]) -> programming
0 1 2 3 4
H E L L O
-5 -4 -3 -2 -1
11

Syntax Dealing with Strings
• r’string writenn here’ will result in string writenn here
• ‘String writenn here’ ->
• ‘My name is {} ‘. Format(“Mohamed hegazy”)
• My name is Mohamed hegazy
string write
n here
12
var = "hello World"
print(var[0]) # Single Char
print(var[5:]) # Range
print(var[:6]) # Range
print(var[:-1]) # Range
print("W" in var)
print('helntnn')
print(r'helntnn')

Syntax (*args and **kwargs)
• *Args prints data as tuple (remember tuple is immutable)
• **Kwargs prints and saves data as dictionary (remember dictionary is
mutable)
13

Conditions
• If condition doesn’t have “()”
Ex. If a<b:
print(‘a is bigger than b’)
• Python doesn’t support the switch statement, however there is a work
around.
If v == ‘----’:
Print(“this is first one”)
elif v == ‘qwdqwdqq’:
print(“this is the second one”)
else:
print(“this is not in the switch case”)
and, or, not
14

Loops
• While loop.
• For loop
For i = 0, i<x , i++ -> it doesn’t exist
15
while i < 10:
print(i)
i += i
for line in cursor.fetchall():
print(line, end=" ")

Defining Classes and functions
• Class is how to create your own object
• Class is a blueprint for an object
• Object is instance of a class
• Classes can be used in inheritance
16

Defining Classes
17

Defining Classes
18
if __name__ == '__main__':main()

Error Handling
• Python error handling, Mitigating the app from crashing on error.
• Raising Exception
• Raise is used to create custom exceptions
• Ex. Have a .docx file and .txt file. Mainly, my data is stored in .txt file,
therefore I created a custom exception to notify the user to only use. Txt file
19

Error Handling
20

Power of Yield
• Yield is like the return . However, yield can be used to return the
argument and continue the loop
• def __iter__(self):
i = self.start
while(i <= self.stop):
yield i
i += self.step
21

modules
• Modules are used to add “extension/inheritance” .
• Basic Python support limited functions comparing to modules.
• Using PIP and Easy_install
• Ex. Adding web functionality to python .
• Import Django
• Import sqlite3
Ex.
Import Django .
Django.Version
pinrt(Django.get_version())
22

File I/O
• Read from file
• Write to file
• Read from file binary
• Write to file binary
• Buffer read , Buffer write
• appending
23

File I/O
24

Database
• CRUD Operation IN Python combining it with try/catch , class,
functions.
25

Database
26

Database
27

Database
28

Concepts of BigData
Mohamed Hegazy

Agenda
• Introduction
• What is Big Data
• Concepts of Big Data
• Google Big Data Tools
• BigQuery
• Cloud Dataflow
• Cloud Dataproc
• Cloud Datalab
• Cloud Dataprep
• Cloud Pub/Sub
• Genomics
• Google Data Studio

What is Big Data
• extremely large data sets that may be analyzed computationally to
reveal patterns, trends, and associations, especially relating to human
behavior and interactions.
• 90% of World Data Generated in the last 2 Years (May 2013)
• Huge Volume
• Super speed Velocity
• Uncountable Variety

What is Big Data
• Facebook Handles 40 Billion Photos from its users
• Facebook Handles 500 TB of new data every day
• Walmart Handles 1 Billion Transactions per hour
• IOT
• etc

What is Big Data

Concepts of Big Data
• Volume. Organizations collect data from a variety of sources,
including business transactions, social media and information from
sensor or machine-to-machine data. In the past, storing it would’ve
been a problem – but new technologies (such as Hadoop) have eased
the burden.
• Velocity. Data streams in at an unprecedented speed and must be
dealt with in a timely manner. RFID tags, sensors and smart metering
are driving the need to deal with torrents of data in near-real time.
• Variety. Data comes in all types of formats – from structured, numeric
data in traditional databases to unstructured text documents, email,
video, audio, stock ticker data and financial transactions.

• Size of the internet (estimate) 1061677 Petabytes Doubles Every 2
years – excluding the Deep Web
• Data could be structured (SQL)
• Data could be semi- structured (CSV, NoSQL)
• Data could be un-structured ( e-mail messages, word processing
documents, videos, photos, audio files, presentations, webpages and
many other )

• Machine Generated unstructured data:
• Satellite images
• Scientific data
• Photos and videos
• Radars and sensors
• Human Generated unstructured data:
• Text internal to your company
• Social media data
• Mobile data
• Website content

Big Query
• BigQuery is Google's fully managed, petabyte scale, low cost
enterprise data warehouse for analytics. BigQuery
is serverless. There is no infrastructure to manage and you don't need
a database administrator, so you can focus on analyzing data to find
meaningful insights using familiar SQL. BigQuery is a powerful Big
Data analytics platform used by all types of organizations, from
startups to Fortune 500 companies.

Big Query

Cloud Dataflow
• Dataflow is a unified programming model and a managed service for
developing and executing a wide range of data processing patterns
including ETL, batch computation, and continuous computation.
Cloud Dataflow frees you from operational tasks like resource
management and performance optimization.
•

Cloud Dataflow

Cloud Dataproc
• Use Google Cloud Dataproc, an Apache Hadoop, Apache
Spark, Apache Pig, andApache Hive service, to easily process big
datasets at low cost. Control your costs by quickly creating managed
clusters of any size and turning them off when you're done. Cloud
Dataproc integrates across Google Cloud Platform products, giving
you a powerful and complete data processing platform.
•

Cloud Dataproc

Cloud Dataprep
• Google Cloud Dataprep is an intelligent data service for visually
exploring, cleaning, and preparing structured and unstructured data
for analysis. Cloud Dataprep is serverless and works at any scale.
There is no infrastructure to deploy or manage. Easy data preparation
with clicks and no code.

Cloud Dataprep

Cloud Pub/Sub
• Reliable, async, topic-based message service
• Many to many
• a scalable messaging middleware service in the cloud that allows
developers to quickly pass information between applications, no
matter where they’re hosted.

Cloud Pub/Sub

Cloud Datalab
• Cloud Datalab is a powerful interactive tool created to explore,
analyze, transform and visualize data and build machine learning
models on Google Cloud Platform. It runs on Google Compute
Engine and connects to multiple cloud services easily so you can focus
on your data science tasks.
• Go from data to deployed machine-learning (ML) models ready for
prediction. Explore data, build, evaluate and optimize Machine
Learning models using TensorFlow orCloud Machine Learning Engine.

Cloud Datalab

Genomics
• Google Genomics helps the life science community organize the
world’s genomic information and make it accessible and useful. Big
genomic data is here today, with petabytes rapidly growing toward
exabytes. Through our extensions to Google Cloud Platform, you can
apply the same technologies that power Google Search and Maps to
securely store, process, explore, and share large, complex datasets.

GOOGLE Data Studio (beta)
• Google Data Studio turns your data into informative dashboards and
reports that are easy to read, easy to share, and fully customizable.
Dashboarding allows you to tell great data stories to support better
business decisions.
• https://datastudio.google.com/u/0/org//reporting/0B_U5RNpwhcE6
TmpwV2hBOGdKYWM/page/qlD

Refrence
• https://blogs.sap.com/2015/01/20/why-airlines-need-to-keep-
planes-in-the-air/
• https://www.slideshare.net/nasrinhussain1/big-data-ppt-31616290
• https://jeremyronk.wordpress.com/2014/09/01/structured-semi-
structured-and-unstructured-data/
• https://cloud.google.com/data-studio/

Thank You For Attending
Any Questions ?
53

GDG Helwan Introduction to python

More Related Content

What's hot

Similar to GDG Helwan Introduction to python

Recently uploaded

GDG Helwan Introduction to python