KEMBAR78
Consequences of an Insightful Algorithm | PDF
@ C C Z O N A
C O N S E Q U E N C E S O F A N
I N S I G H T F U L A L G O R I T H M
C A R I N A C . Z O N A
@ C C Z O N A
@ C C Z O N A
infertility
sex shaming
miscarriage
bullying
stalking

grief
PTSD
segregation
racial profiling
white supremacy
holocaust
CONTENT WARNING
@ C C Z O N A
ALGORITHMS 

IMPOSE
CONSEQUENCES 

ON PEOPLE 

ALL THE TIME
@ C C Z O N A
PATTERNS OF
INSTRUCTIONS,
ARTICULATED IN
CODE OR FORMULAS
A L G O R I T H M S O F C O M P U T E R S C I E N C E & M AT H E M AT I C S
@ C C Z O N A
@ C C Z O N A
STEP-BY-STEP SET OF
OPERATIONS FOR 

PREDICTABLY ARRIVING
AT AN OUTCOME
A L G O R I T H M
@ C C Z O N A
PATTERNS OF
INSTRUCTIONS,
ARTICULATED IN
WAYS SUCH AS…
A L G O R I T H M S I N O R D I N A RY L I F E
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
Ch 12, sl st to join. Rnd 1: ch 3, 17 dc. Rnd 2: ch 3, 1 dc, ch 4,
skip 1 dc, *2 dc, ch 4, skip 1 dc*, repeat 5 times, sl st. Rnd 3:
ch 3, 2 dc, ch 5, skip 2 ch and 1 dc, *3 dc, ch 5, skip 2 ch and
1 dc*, repeat 5 times, sl st. Rnd 4: ch 3, 3 dc, ch 5, skip 3 ch
and 1 dc, *4 dc, ch 5, skip 3 ch and 1 dc*, repeat 5 times, sl
st. Rnd 5: ch 3, 4 dc, ch 6, skip 3 ch and 1 dc, *5 dc, ch 6, skip
3 ch and 1 dc*, repeat 5 times, sl st. Rnd 6: ch 3, 6 dc, ch 6,
skip 3 ch and 1 dc, *7 dc, ch 6, skip 3 ch and 1 dc*, repeat 5
times, sl st. Rnd 7: ch 3, 8 dc, ch 6, skip 3 ch and 1 dc, *9 dc,
ch 6, skip 3 ch and 1 dc*, repeat 5 times, sl st. Rnd 8: ch 3, 10
dc, ch 6, skip 3 ch and 1 dc, *11 dc, ch 6, skip 3 ch and 1 dc*,
repeat 5 times, sl st .Rnd 9: ch 3, 12 dc, ch 6, skip 3 ch and 1
dc, *13 dc, ch 6, skip 3 ch and 1 dc*, repeat 5 times, sl st.
Rnd 10: ch 3, 14 dc, ch 7, skip 3 ch and 1 dc, *15 dc, ch 7,
skip 3 ch and 1 dc*, repeat 5 times, sl st. Rnd 11: ch 3, 16 dc,
ch 7, skip 4 ch and 1 dc, *17 dc, ch 7, skip 4 ch and 1 dc*,
repeat 5 times, sl st. Rnd 12: ch 3, 18 dc, ch 8, skip 4 ch and 1
dc, *19 dc, ch 8, skip 4 ch and 1 dc*, repeat 5 times, sl st.
Rnd 13: ch 3, 20 dc, ch 8, skip 5 ch and 1 dc, *21 dc, ch 8,
skip 5 ch and 1 dc*, repeat 5 times, sl st. Rnd 14: ch 3, 16 dc,
ch 8, skip 3 dc and 2 ch, 1 dc, ch 8, skip 1 dc, *17 dc, ch 8,
skip 3 dc and 2 ch, 1 dc, ch 8, skip 1 dc,* repeat 5 times, sl st.
@ C C Z O N A
The exhaustive rendering of our
conscious and unconscious patterns
into data sets and algorithms.
P R E D I C T I V E A N A LY T I C S
—Charles Duhigg
@ C C Z O N A
ALGORITHMS FOR
FAST, TRAINABLE,
ARTIFICIAL NEURAL
NETWORKS
D E E P L E A R N I N G
@ C C Z O N A
INPUT
Words, images, sounds,
objects, or abstract concepts
EXECUTION
Run a series of functions
repeatedly in a black box
OUTPUT
Prediction of properties useful
for drawing intuitions about
similar future inputs
@ C C Z O N A
ARTIFICIAL NEURAL NETWORK'S
AUTOMATED DISCOVERY
OF PATTERNS WITHIN 

A TRAINING DATASET
APPLIES DISCOVERIES 

TO DRAW INTUITIONS 

ABOUT FUTURE DATA
D E E P L E A R N I N G
@ C C Z O N A
– P e t e Wa rd e n
" T H I N K O F T H E M A S 

D E C I S I O N - M A K I N G 

B L A C K B O X E S . "
@ C C Z O N A
Medical Diagnosing
Pharmaceutical Development
Emotion Detection
Predictive Text
Face Identification
Voice-Activated Commands
Fraud Detection
Sentiment Analysis
Translating Text
Video Content Recognition
Self-Driving Cars
@ C C Z O N A
Ad Targeting
Behavioral Prediction
Recommendation Systems
Image Classification
Face Recognition
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
#DataMiningFail
Algorithmic
Profiling
De-
Anonymization
Black Box
Disparate
Impact
Consent

Issues
Replicates

Bias
Personally
Identifiable Info
Human
Complexity Fail
Unproven
Methods
Inadvertent
Algorithmic
Cruelty
Uncritical
Assumptions
False

Neutrality
High Risk
Consequences
Deception Filtering
Moved Fast, 

Broke Things
Invasion Of
Privacy
No Recourse
Creepy

Stalkery
Messing With
Heads
Not How Valid
Research Works
Data

Insecurity
Accurate, But 

Not Right
Shaming
Diversity

Fail
#DataMiningFail
🔵 🔵
🔵
🔵
🔵
🔵
🔵 🔵
🔵🔵
🔵
🔵
🔵Algorithmic
Profiling
De-
Anonymization
Black Box
Disparate
Impact
Consent

Issues
Replicates

Bias
Personally
Identifiable Info
Human
Complexity Fail
Unproven
Methods
Inadvertent
Algorithmic
Cruelty
Uncritical
Assumptions
False

Neutrality
High Risk
Consequences
Deception Filtering
Moved Fast, 

Broke Things
Invasion Of
Privacy
No Recourse
Creepy

Stalkery
Messing With
Heads
Not How Valid
Research Works
Data

Insecurity
Accurate, But 

Not Right
Shaming
Diversity

Fail
TA R G E T
@ C C Z O N A
“IF WE WANTED TO FIGURE OUT

IF A CUSTOMER IS PREGNANT,
EVEN IF SHE
DIDN’T WANT US TO KNOW, 

CAN YOU DO THAT? ”
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
‟As long as a pregnant
woman thinks she hasn’t
been spied on… 

as long as we don’t
spook her, it works.”
🔵 🔵 🔵 🔵
🔵
🔵
🔵 🔵
🔵
🔵
🔵
🔵
Algorithmic
Profiling
De-
Anonymization
Black Box
Disparate
Impact
Consent

Issues
Replicates

Bias
Personally
Identifiable Info
Human
Complexity Fail
Unproven
Methods
Inadvertent
Algorithmic
Cruelty
Uncritical
Assumptions
False

Neutrality
High Risk
Consequences
Deception Filtering
Moved Fast, 

Broke Things
Invasion Of
Privacy
No Recourse
Creepy

Stalkery
Messing With
Heads
Not How Valid
Research Works
Data

Insecurity
Accurate, But 

Not Right
Shaming
Diversity

Fail
S H U T T E R F LY
@ C C Z O N A
@ C C Z O N A
‟Thanks, Shutterfly, for 

the congratulations on my
'new bundle of joy'. 

I'm horribly infertile,
but hey,
I'm adopting 

a kitten,

so...”
@ C C Z O N A
‟ I l o s t a b a b y i n
N o v e m b e r w h o
w o u l d h a v e b e e n
d u e t h i s w e e k .
I t w a s l i k e     

h i t t i n g a w a l l 

a l l o v e r
a g a i n … ˮ
@ C C Z O N A
‟ T H E I N T E N T O F T H E 

E M A I L WA S T O TA R G E T
C U S T O M E R S W H O
H AV E R E C E N T LY
H A D A B A B Y "
@ C C Z O N A
‟You start imagining who 

they'll become and dreaming 

of hopes for their future.
You start making plans.
And then they're gone.
It's a lonely experience."
🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵
Algorithmic
Profiling
De-
Anonymization
Black Box
Disparate
Impact
Consent

Issues
Replicates

Bias
Personally
Identifiable Info
Human
Complexity Fail
Unproven
Methods
Inadvertent
Algorithmic
Cruelty
Uncritical
Assumptions
False

Neutrality
High Risk
Consequences
Deception Filtering
Moved Fast, 

Broke Things
Invasion Of
Privacy
No Recourse
Creepy

Stalkery
Messing With
Heads
Not How Valid
Research Works
Data

Insecurity
Accurate, But 

Not Right
Shaming
Diversity

Fail
FA C E B O O K 

Y E A R I N R E V I E W
@ C C Z O N A
The result of code that
works in the overwhelming
majority of cases but
doesn’t take other use
cases into account.
I N A D V E R T E N T A L G O R I T H M I C C R U E LT Y
—Eric Meyer
@ C C Z O N A
@ C C Z O N A
‟ I N C R E A S E A WA R E N E S S O F 

A N D C O N S I D E R AT I O N F O R 

T H E FA I L U R E M O D E S 

T H E E D G E C A S E S 

T H E W O R S T- C A S E
S C E N A R I O S
– E r i c M e y e r
@ C C Z O N A
BEHUMBLE.WECANNOTINTUIT

INNERSTATE,EMOTIONS,

PRIVATESUBJECTIVITY
— C a r i n a C . Z o n a
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵
Algorithmic
Profiling
De-
Anonymization
Black Box
Disparate
Impact
Consent

Issues
Replicates

Bias
Personally
Identifiable Info
Human
Complexity Fail
Unproven
Methods
Inadvertent
Algorithmic
Cruelty
Uncritical
Assumptions
False

Neutrality
High Risk
Consequences
Deception Filtering
Moved Fast, 

Broke Things
Invasion Of
Privacy
No Recourse
Creepy

Stalkery
Messing With
Heads
Not How Valid
Research Works
Data

Insecurity
Accurate, But 

Not Right
Shaming
Diversity

Fail
F I T B I T
@ C C Z O N A
@ C C Z O N A
🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵
Algorithmic
Profiling
De-
Anonymization
Black Box
Disparate
Impact
Consent

Issues
Replicates

Bias
Personally
Identifiable Info
Human
Complexity Fail
Unproven
Methods
Inadvertent
Algorithmic
Cruelty
Uncritical
Assumptions
False

Neutrality
High Risk
Consequences
Deception Filtering
Moved Fast, 

Broke Things
Invasion Of
Privacy
No Recourse
Creepy

Stalkery
Messing With
Heads
Not How Valid
Research Works
Data

Insecurity
Accurate, But 

Not Right
Shaming
Diversity

Fail
U B E R
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
🔵
🔵
🔵
🔵
🔵
🔵🔵
🔵
🔵
Algorithmic
Profiling
De-
Anonymization
Black Box
Disparate
Impact
Consent

Issues
Replicates

Bias
Personally
Identifiable Info
Human
Complexity Fail
Unproven
Methods
Inadvertent
Algorithmic
Cruelty
Uncritical
Assumptions
False

Neutrality
High Risk
Consequences
Deception Filtering
Moved Fast, 

Broke Things
Invasion Of
Privacy
No Recourse
Creepy

Stalkery
Messing With
Heads
Not How Valid
Research Works
Data

Insecurity
Accurate, But 

Not Right
Shaming
Diversity

Fail
G O O G L E
A D W O R D S
@ C C Z O N A
Amy
Claire
Emma
Katelyn
Katie
Madeline
Molly
Aaliyah
Diamond
Ebony
Imani
Latanya
Nia
Shanice
Cody
Connor
Dustin
Jack
Luke
Tanner
Wyatt
Darnell
DeShawn
Malik
Marquis
Terrell
Trevon
Tyrone
G O O G L E A D W O R D S
PLACEHOLDER@ C C Z O N A
PLACEHOLDER
PLACEHOLDERA BLACK IDENTIFYING NAME WAS 

25% MORE LIKELY

TO RESULT IN AN AD THAT I M P L I E D A N 

ARREST RECORD
G O O G L E A D W O R D S
@ C C Z O N A
G O O G L E A D W O R D S
🔵 🔵
🔵
🔵
🔵
🔵
🔵
🔵Algorithmic
Profiling
De-
Anonymization
Black Box
Disparate
Impact
Consent

Issues
Replicates

Bias
Personally
Identifiable Info
Human
Complexity Fail
Unproven
Methods
Inadvertent
Algorithmic
Cruelty
Uncritical
Assumptions
False

Neutrality
High Risk
Consequences
Deception Filtering
Moved Fast, 

Broke Things
Invasion Of
Privacy
No Recourse
Creepy

Stalkery
Messing With
Heads
Not How Valid
Research Works
Data

Insecurity
Accurate, But 

Not Right
Shaming
Diversity

Fail
@ C C Z O N A
Classifying people as “similar”,
where careless prompts create
scenarios harder to control 

and prepare for.
A C C I D E N TA L A L G O R I T H M I C R U N - I N S
—adapted from Joanne McNeil
@ C C Z O N A
A L G O R I T H M I C H U B R I S
@ C C Z O N A
"If you are stalked by a former
coworker, Twitter may reinforce
this connection, algorithmically
boxing you into a past while
you are trying to move on. 

Your affinity score with your
harasser will grow higher with
every person who follows this
person at Twitter’s
recommendation."
T W I T T E R - W H O T O F O L L O W
🔵🔵
🔵
🔵
🔵
🔵
🔵 🔵
🔵 🔵
🔵
Algorithmic
Profiling
De-
Anonymization
Black Box
Disparate
Impact
Consent

Issues
Replicates

Bias
Personally
Identifiable Info
Human
Complexity Fail
Unproven
Methods
Inadvertent
Algorithmic
Cruelty
Uncritical
Assumptions
False

Neutrality
High Risk
Consequences
Deception Filtering
Moved Fast, 

Broke Things
Invasion Of
Privacy
No Recourse
Creepy

Stalkery
Messing With
Heads
Not How Valid
Research Works
Data

Insecurity
Accurate, But 

Not Right
Shaming
Diversity

Fail
@ C C Z O N A
I P H O T O FA C E D E T E C T I O N
@ C C Z O N A
M I C R O S O F T H O W - O L D . N E T
@ C C Z O N A
F L I C K R M A G I C V I E W A U T O - TA G G I N G
@ C C Z O N A
F L I C K R M A G I C V I E W A U T O - TA G G I N G
@ C C Z O N A
G O O G L E P H O T O S A U T O - C AT E G O R I Z I N G
@ C C Z O N A
S H I R L E Y C A R D S
@ C C Z O N A
The tools used to make film,
b y M o n t ré A z a M i s s o u r i
@ C C Z O N A
not
are
the science of it,
racially
neutral
S H I R L E Y C A R D S
🔵
🔵🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵
🔵 🔵
Algorithmic
Profiling
De-
Anonymization
Black Box
Disparate
Impact
Consent

Issues
Replicates

Bias
Personally
Identifiable Info
Human
Complexity Fail
Unproven
Methods
Inadvertent
Algorithmic
Cruelty
Uncritical
Assumptions
False

Neutrality
High Risk
Consequences
Deception Filtering
Moved Fast, 

Broke Things
Invasion Of
Privacy
No Recourse
Creepy

Stalkery
Messing With
Heads
Not How Valid
Research Works
Data

Insecurity
Accurate, But 

Not Right
Shaming
Diversity

Fail
@ C C Z O N A
A L G O R I T H M S A R E G AT E K E E P E R S
Credit Housing Jobs
@ C C Z O N A
@ C C Z O N A
IF A MACHINE IS EXPECTED TO BE
INFALLIBLE
IT CANNOT ALSO BE
INTELLIGENT— A l a n Tu r i n g
@ C C Z O N A
!=I R L F R I E N D S FA C E B O O K F R I E N D S
@ C C Z O N A
F R I E N D S
F I N A N C E S
— i l l u s t r a t i o n b y S u s i e C a g l e
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
UNDERLYING
ASSUMPTIONS
INFLUENCE
OUTCOMES
AND
CONSEQUENCES @ C C Z O N A
@ C C Z O N A
— H e n r y D a v i d T h o re a u
“ T H E U N I V E R S E I S W I D E R
T H A N O U R V I E W S O F I T ”
@ C C Z O N A
@ C C Z O N A
OUR INDUSTRY IS 

IN AN ARMS RACE
@ C C Z O N A
Amazon
Apple
AT&T
Baidu
DARPA
Dropbox
eBay
Facebook
Flickr
Google
IBM
Microsoft
Netflix
Pandora
PayPal
Pinterest
Skype
Snapchat
Spotify
Tesla
Twitter
Yahoo
Uber
@ C C Z O N A
MOREPRECISE

INCORRECTNESS
MORE

DAMAGING

INWRONGNESS
@ C C Z O N A
FLIPPING THE
PARADIGM
@ C C Z O N A
CONSIDER
DECISIONS'
POTENTIAL
IMPACTS
F L I P P I N G T H E PA R A D I G M
A C M C o d e o f E t h i c s
1
@ C C Z O N A
•PROJECT THE LIKELIHOOD OF
CONSEQUENCES
•MINIMIZE NEGATIVE
CONSEQUENCES
E VA L U AT E P O T E N T I A L I M PA C T S
@ C C Z O N A
FIRST DO NO
HARM
@ C C Z O N A
BE HONEST
BE TRUSTWORTHY
F L I P P I N G T H E PA R A D I G M
A C M C o d e o f E t h i c s
2
@ C C Z O N A
BUILD IN
RECOURSE
F L I P P I N G T H E PA R A D I G M
A C M C o d e o f E t h i c s
@ C C Z O N A
FULLY DISCLOSE
LIMITATIONS.
CALL ATTENTION TO
SIGNS OF RISK OF HARM.
F L I P P I N G T H E PA R A D I G M
3
@ C C Z O N A
@ C C Z O N A
BE VISIONARY
ABOUT
COUNTERING BIAS
F L I P P I N G T H E PA R A D I G M
4
@ C C Z O N A
ANTICIPATE
DIVERSE WAYS
TO SCREW UP
E M PAT H E T I C C O D E R S
5
@ C C Z O N A
W E M U S T H A V E
D E C I S I O N - M A K I N G
A U T H O R I T Y 

I N T H E H A N D S O F
H I G H LY 

D I V E R S E 

T E A M S
@ C C Z O N A
The point of culture fit is to
avoid disruption of groupthink
@ C C Z O N A
Unidimensional variety

is not diversity
@ C C Z O N A
@ C C Z O N A
AUDIT
OUTCOMES

CONSTANTLY
F L I P P I N G T H E PA R A D I G M
6
@ C C Z O N A
@ C C Z O N A
AUDIT OUTCOMES CONSTANTLY
@ C C Z O N A
AUDIT OUTCOMES CONSTANTLY
@ C C Z O N A
AUDIT OUTCOMES
@ C C Z O N A
1,852318
" C A R E F U L LY C H O S E N T R A I L S A C R O S S T H E W E B 

I N S U C H A WAY T H AT A N A D - TA R G E T I N G N E T W O R K
W I L L I N F E R C E RTA I N I N T E R E S T S O R A C T I V I T I E S . "
A D F I S H E R
@ C C Z O N A
C R O W D K N O W L E D G E > = B L A C K B O X I N T U I T I O N S
7C R O W D S O U R C E A L L T H E
T H I N G S
F L I P P I N G T H E PA R A D I G M
@ C C Z O N A
Party.
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
@ C C Z O N A
CULTIVATE
INFORMED
CONSENT
F L I P P I N G T H E PA R A D I G M
8
@ C C Z O N A
@ C C Z O N A
"LIST THINGS YOU 

DON'T WANT 

TO THINK ABOUT"
@ C C Z O N A
Opt-Out Opt-In
Block Follow
Ignore View
Filter Select
Disable Enable
S T O P T H E A L G O R I T H M B E T H E A L G O R I T H M
@ C C Z O N A
"Why not let Twitter users
select the people they
recommend you follow?
We already have Follow
Friday." —adapted from Joanne McNeil & Melissa Gira Grant
@ C C Z O N A
Would you like
special coupons for
pregnancy & infant
care?
@ C C Z O N A
COMMIT TO DATA
TRANSPARENCY.
COMMIT TO ALGORITHMIC
TRANSPARENCY.
F L I P P I N G T H E PA R A D I G M
9
@ C C Z O N A
@ C C Z O N A
We DO NOT BUILD here 

without understanding
CONSEQUENCES
TO OTHERS
@ C C Z O N A
PROFESSIONALS

APPLY EXPERTISE &
JUDGEMENT
ABOUT 

HOW TO 

SOLVE PROBLEMS
Algorithmic
Profiling
De-
Anonymization
Black Box
Disparate
Impact
Consent

Issues
Replicates

Bias
Personally
Identifiable Info
Human
Complexity Fail
Unproven
Methods
Inadvertent
Algorithmic
Cruelty
Uncritical
Assumptions
False

Neutrality
High Risk
Consequences
Deception Filtering
Moved Fast, 

Broke Things
Invasion Of
Privacy
No Recourse
Creepy

Stalkery
Messing With
Heads
Not How Valid
Research Works
Data

Insecurity
Accurate, But 

Not Right
Shaming
Diversity

Fail
❌ ❌ ❌ ❌ ❌
❌
❌
❌ ❌ ❌
❌
❌
❌❌
❌
❌
❌ ❌❌ ❌
❌ ❌ ❌ ❌ ❌
@ C C Z O N A
R E F U S E T O 

P L AY A L O N G .
@ C C Z O N A
T H A N K Y O U .
@ C C Z O N A
Code Newbies
http://www.codenewbie.org/podcast/algorithms
“CONSEQUENCES OF AN INSIGHTFUL ALGORITHM”
CONTINUES AT:
Ruby Rogues
https://devchat.tv/ruby-rogues/278-rr-consequences-of-an-
insightful-algorithm-with-carina-c-zona
@ C C Z O N A
CASE STUDIES
FitBit: http://thenextweb.com/insider/2011/07/03/fitbit-users-are-inadvertently-
sharing-details-of-their-sex-lives-with-the-world/
Uber: http://www.forbes.com/sites/kashmirhill/2014/10/03/god-view-uber-
allegedly-stalked-users-for-party-goers-viewing-pleasure/ & http://
valleywag.gawker.com/uber-allegedly-used-god-view-to-stalk-vip-users-as-
a-1642197313 & http://www.forbes.com/sites/chanellebessette/2014/11/25/does-
uber-even-deserve-our-trust/ & http://www.wired.com/2011/04/app-stars-uber/ &
http://motherboard.vice.com/read/ubers-god-view-was-once-available-to-drivers
& http://motherboard.vice.com/read/ubers-god-view-was-once-available-to-
drivers
OkCupid: http://blog.okcupid.com/index.php/the-best-questions-for-first-dates/
Affirm, et al: http://recode.net/2015/05/06/max-levchins-affirm-raises-275-
million-to-make-loans/ & http://time.com/3430817/paypal-levchin-affirm-lending/
& http://www.nytimes.com/2015/01/19/technology/banking-start-ups-adopt-
new-tools-for-lending.html & http://www.spiegel.de/international/germany/
critique-of-german-credit-agency-plan-to-mine-facebook-for-data-a-837713.html
& http://www.motherjones.com/politics/2013/09/lenders-vet-borrowers-social-
media-facebook
Shutterfly: http://jezebel.com/shutterfly-thinks-you-just-had-a-baby-1576261631
& http://www.adweek.com/adfreak/shutterfly-congratulates-thousands-women-
babies-they-didnt-have-157675
Zuckerberg on miscarriage: https://www.facebook.com/photo.php?
fbid=10102276573729791
Target: http://www.nytimes.com/2012/02/19/magazine/shopping-habits.html
Facebook Year in Review: http://meyerweb.com/eric/thoughts/2014/12/24/
inadvertent-algorithmic-cruelty/ & http://www.theguardian.com/technology/
2014/dec/29/facebook-apologises-over-cruel-year-in-review-clips
Google AdWords: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2208240
& http://freakonomics.com/2013/04/08/how-much-does-your-name-matter-full-
transcript/
Flickr: http://www.theguardian.com/technology/2015/may/20/flickr-complaints-
offensive-auto-tagging-photos
Google Photos: https://twitter.com/jackyalcine/status/615329515909156865 &
https://www.yahoo.com/tech/google-photos-mislabels-two-black-americans-
as-122793782784.html
Twitter Who To Follow & Last.fm Similar Listeners: https://medium.com/
message/harassed-by-algorithms-f2b8229488df
Chicken Breast: http://metro.co.uk/2016/01/27/this-chicken-breast-has-a-
surprisingly-healthy-heart-rate-considering-its-dead-5647836/ & https://
www.youtube.com/watch?v=7rO5knyjDR0
LinkedIn http://www.slate.com/articles/technology/technology/2016/05/
linkedin_called_me_a_white_supremacist.html
Shirley Cards, Race, & Film Emulsion: http://priceonomics.com/how-
photography-was-optimized-for-white-skin/ & http://www.buzzfeed.com/
syreetamcfadden/teaching-the-camera-to-see-my-skin#.wkv3Jmd8O
@ C C Z O N A
RESOURCES
• Association for Computing Machinery Code
of Ethics http://www.acm.org/about/code-
of-ethics
• The 10 Commandments of Egoless
Programming http://www.techrepublic.com/
article/the-ten-commandments-of-egoless-
programming-6353837/
• Disparate Impact Analysis is Key to Ensuring
Fairness in the Age of the Algorithm http://
www.datainnovation.org/2015/01/disparate-
impact-analysis-is-key-to-ensuring-fairness-
in-the-age-of-the-algorithm/
• On Algorithmic Fairness, Discrimination and
Disparate impact. http://
fairness.haverford.edu/
• Big Data's Disparate Impact http://
papers.ssrn.com/sol3/papers.cfm?
abstract_id=2477899
• Algorithmic Accountability & Transparency
http://www.nickdiakopoulos.com/projects/
algorithmic-accountability-reporting/
• What is Deep Learning and why should you
care? http://radar.oreilly.com/2014/07/what-
is-deep-learning-and-why-should-you-
care.html
• Facebook Research | Machine Learning
https://research.facebook.com/
machinelearning
@ C C Z O N A
LAW, GOVERNANCE, & POLICYMAKING
• AI is Setting Up the Internet for a Huge
Class with Europe (2016) https://
www.wired.com/2016/07/artificial-
intelligence-setting-internet-huge-clash-
europe/
• California Law Review (2016): Big Data's
Disparate Impact http://papers.ssrn.com/
sol3/papers.cfm?abstract_id=2477899
• United States Federal Trade Commission
Report (2016) Big Data: A Tool for Inclusion
or Exclusion? https://www.ftc.gov/system/
files/documents/reports/big-data-tool-
inclusion-or-exclusion-understanding-
issues/160106big-data-rpt.pdf
• White House Big Data Privacy Report
(2014) https://www.whitehouse.gov/sites/
default/files/docs/
big_data_privacy_report_may_1_2014.pdf
• Algorithmic Transparency and Platform
Loyalty or Fairness in the French Digital
Republic Bill (2016) http://blogs.lse.ac.uk/
mediapolicyproject/2016/04/22/algorithmic-
transparency-and-platform-loyalty-or-
fairness-in-the-french-digital-republic-bill/
• European Data Protection Supervisor
Opinion (2015): Meeting the Challenges of
Big Data https://secure.edps.europa.eu/
EDPSWEB/webdav/site/mySite/shared/
Documents/Consultation/Opinions/
2015/15-11-19_Big_Data_EN.pdf
@ C C Z O N A
FLICKR CREATIVE COMMONS
Cookie face: https://www.flickr.com/photos/amayu/4462907505/in/
pool-977532@N24/
Eye shadows: https://www.flickr.com/photos/niallb/5300259686/
Colored pencils: https://www.flickr.com/photos/jenson-lee/
6315443914/
Audit keyboard: https://www.flickr.com/photos/jakerust/16811751576/
Crystal balls: https://www.flickr.com/photos/cmogle/3348401259/
Eye: https://www.flickr.com/photos/weirdcolor/3312989961/
Sea Wall: https://www.flickr.com/photos/christing/1447039348/
Door: https://www.flickr.com/photos/chrisschoenbohm/14181173109/
WTF Was This WTF Was That by Birger King: https://www.flickr.com/
photos/birgerking/7080728907/
Tasveer ByTaru: https://www.flickr.com/photos/tasveerbytaru/
19547308031/sizes/h/
Boxing day: https://www.flickr.com/photos/erix/6306715646/
Printing machine by Teddy Kwok: https://www.flickr.com/photos/
bblkwok/8247948713/
Caduceus: https://www.flickr.com/photos/spirit-fire/4832779325/
wocintech (microsoft) - 69 by #wocintech: https://www.flickr.com/
photos/wocintechchat/25926648871/in/album-72157664006621903/
ねこ -ちゃーちゃん- by atacamaki: https://www.flickr.com/photos/
mikey-a-tucker/14792307898/
Driving home by Christian Weidinger: https://www.flickr.com/photos/
ch-weidinger/13888995404/
Server graphs by Aaron Parecki: https://www.flickr.com/photos/
aaronpk/5967579738/
On the Banks of Loch Linnhe by Henry Hemming: https://
www.flickr.com/photos/henry_hemming/8280971407/
Scaffolding by ajjyt: https://www.flickr.com/photos/
54588550@N08/9387947434/
Monarchial Scrutiny by Joel Penner: https://www.flickr.com/photos/
featheredtar/2949748121/
Canadian Money $100 Dollar Bill Macro of Robert Borden by Wilson
Hui: https://www.flickr.com/photos/wilsonhui/6604606217/
The grindstone by Kathryn Decker: https://www.flickr.com/photos/
waponigirl/5621810815/
"img049" by Steve Walker Photography: https://www.flickr.com/
photos/stover98074/11361095594/
Street Passion by Johnny Silvercloud: https://www.flickr.com/photos/
johnnysilvercloud/15718876258/
Buttons: https://www.flickr.com/photos/48462557@N00/3616793654/
@ C C Z O N A
NOUN PROJECT CREATIVE COMMONS
Businesspeople by Honnos Bondor

https://thenounproject.com/term/businesspeople/17542/
Send by David Papworth

https://thenounproject.com/term/send/135469/
Users by Lorena Salagre

https://thenounproject.com/term/users/32253/
Friend by Megan Mitchel

https://thenounproject.com/term/friend/6808/
Woman (1) by Thomas Helbig

https://thenounproject.com/term/woman/120381/
Woman (2) by Thomas Helbig

https://thenounproject.com/term/woman/120380/
Couple by Shankar Narayan

https://thenounproject.com/term/couple/1014/
Couple by Pasquale Cavorsi

https://thenounproject.com/term/couple/33095/
Check mark by Kris Brauer

https://thenounproject.com/term/check-mark/192830/
Job Seeker by Stijn Elskens

https://thenounproject.com/term/job-seeker/226871/
Credit Card by Oliviu Stoian

https://thenounproject.com/term/credit-card/269411/
Single House by Aaron K. Kim

https://thenounproject.com/term/single-house/123924/
Downloads by Icons8

https://thenounproject.com/term/downloads/61761/
Id Card (1) by Boudewijn Mijnlieff

https://thenounproject.com/term/id-card/203566/
Id Card (2) by Boudewijn Mijnlieff

https://thenounproject.com/term/id-card/203578/
Cancel by Kris Brauer

https://thenounproject.com/term/cancel/182501/
@ C C Z O N A
ADDITIONAL IMAGES
Recipe cards: Olivia Juice & Co. http://
olivejuiceco.typepad.com/my_weblog/2006/04/
easter_recap_an.html
Target circulars: http://flyers.smartcanucks.ca/uploads/
pages/26431/target-canada-weekly-flyer-july-11-
to-1713.jpg
Crochet pattern & shawl: http://www.abc-knitting-
patterns.com/1129.html
Space Invaders: http://www.caffination.com/
backchannel/shooting-for-the-high-score-3942/
Facebook b/w logo sketch: http://connect-
communicate-change.com/wp-content/uploads/
2011/11/facebook-logo-square-webtreatsetc.png
Stephen Hawking by CERN/Lawrent Egli: https://
www.facebook.com/stephenhawking/photos/
710802829006817
Jonathon Arrears by Susie Cagle: https://twitter.com/
susie_c/status/631619118865604610/photo/1
Twitter keyboard: http://www.npr.org/sections/
itsallpolitics/2012/05/04/152028258/navigating-politics-
on-twitter-some-npr-followfriday-suggestions
Tesla's Autopilot System MobilEye http://
wccftech.com/tesla-autopilot-story-in-depth-
technology/
Uber home screen https://www.supermoney.com/
2016/05/5-reasons-uber-can-risky-choice-drivers-
passengers/
Walden Pond by Walden Pond State Reservation
https://waldenpondstatereservation.wordpress.com/
The Force is Strong With This One by Mark Zuckerberg
https://www.facebook.com/zuck/posts/
10102531565693851
Bubble-sort with Hungarian ("Csángó") folk dance
https://www.youtube.com/watch?v=lyZQPjUT5B4
@ C C Z O N A
T H E E T H I C S &
R E S P O N S I B I L I T I E S O F
O P E N S O U R C E I O T
https://www.youtube.com/
watch?v=7rO5knyjDR0
Emily Gorcenski
@emilygorcenski
@ C C Z O N A
I M A G E U N D E R S TA N D I N G : 

D E E P L E A R N I N G W I T H
C O N V O L U T I O N A L N E U R A L N E T S
http://www.slideshare.net/
roelofp/python-for-image-
understanding-deep-learning-
with-convolutional-neural-nets
Roelof Pieters
@graphific
@ C C Z O N A
T H E E T H I C S O F B E I N G
A P R O G R A M M E R
https://youtu.be/
DB7ei5W1eRQ
Kate Heddleston
@heddle317
@ C C Z O N A
A H I P P O C R AT I C O AT H
F O R D ATA S C I E N C E
http://www.slideshare.net/
roelofp/a-hippocratic-oath-for-
data-science
Roelof Pieters
@graphific
@ C C Z O N A
D E E P L E A R N I N G :
I N T E L L I G E N C E F R O M 

B I G D ATA
https://www.youtube.com/
watch?v=czLI3oLDe8M
Adam Berenzweig
@madadam
@ C C Z O N A
A L G O R I T H M I C
A C C O U N TA B I L I T Y
W O R K S H O P
https://vimeo.com/125622175
Columbia Journalism School
@ C C Z O N A
M A R I / O
https://youtu.be/
qv6UVOQ0F44
Seth Bling
@sethbling
@ C C Z O N A
Noah Kantrowitz
Heidi Waterhouse
VM Brasseur
Yoz Grahame
Mike Foley

Estelle Weyl
Chris Hausler
Scott Triglia
Russell Keith-Magee
Tim Bell
We So Crafty
THANK YOU ♥
@ C C Z O N A
BONUS!
“Eleven”
by Burnistoun (Robert Florence & Iain Connell)
https://www.youtube.com/watch?v=NMS2VnDveP8
@ C C Z O N A

Consequences of an Insightful Algorithm

  • 1.
    @ C CZ O N A C O N S E Q U E N C E S O F A N I N S I G H T F U L A L G O R I T H M C A R I N A C . Z O N A @ C C Z O N A
  • 2.
    @ C CZ O N A infertility sex shaming miscarriage bullying stalking
 grief PTSD segregation racial profiling white supremacy holocaust CONTENT WARNING
  • 3.
    @ C CZ O N A ALGORITHMS 
 IMPOSE CONSEQUENCES 
 ON PEOPLE 
 ALL THE TIME
  • 4.
    @ C CZ O N A PATTERNS OF INSTRUCTIONS, ARTICULATED IN CODE OR FORMULAS A L G O R I T H M S O F C O M P U T E R S C I E N C E & M AT H E M AT I C S
  • 5.
    @ C CZ O N A
  • 6.
    @ C CZ O N A STEP-BY-STEP SET OF OPERATIONS FOR 
 PREDICTABLY ARRIVING AT AN OUTCOME A L G O R I T H M
  • 7.
    @ C CZ O N A PATTERNS OF INSTRUCTIONS, ARTICULATED IN WAYS SUCH AS… A L G O R I T H M S I N O R D I N A RY L I F E
  • 8.
    @ C CZ O N A
  • 9.
    @ C CZ O N A
  • 10.
    @ C CZ O N A Ch 12, sl st to join. Rnd 1: ch 3, 17 dc. Rnd 2: ch 3, 1 dc, ch 4, skip 1 dc, *2 dc, ch 4, skip 1 dc*, repeat 5 times, sl st. Rnd 3: ch 3, 2 dc, ch 5, skip 2 ch and 1 dc, *3 dc, ch 5, skip 2 ch and 1 dc*, repeat 5 times, sl st. Rnd 4: ch 3, 3 dc, ch 5, skip 3 ch and 1 dc, *4 dc, ch 5, skip 3 ch and 1 dc*, repeat 5 times, sl st. Rnd 5: ch 3, 4 dc, ch 6, skip 3 ch and 1 dc, *5 dc, ch 6, skip 3 ch and 1 dc*, repeat 5 times, sl st. Rnd 6: ch 3, 6 dc, ch 6, skip 3 ch and 1 dc, *7 dc, ch 6, skip 3 ch and 1 dc*, repeat 5 times, sl st. Rnd 7: ch 3, 8 dc, ch 6, skip 3 ch and 1 dc, *9 dc, ch 6, skip 3 ch and 1 dc*, repeat 5 times, sl st. Rnd 8: ch 3, 10 dc, ch 6, skip 3 ch and 1 dc, *11 dc, ch 6, skip 3 ch and 1 dc*, repeat 5 times, sl st .Rnd 9: ch 3, 12 dc, ch 6, skip 3 ch and 1 dc, *13 dc, ch 6, skip 3 ch and 1 dc*, repeat 5 times, sl st. Rnd 10: ch 3, 14 dc, ch 7, skip 3 ch and 1 dc, *15 dc, ch 7, skip 3 ch and 1 dc*, repeat 5 times, sl st. Rnd 11: ch 3, 16 dc, ch 7, skip 4 ch and 1 dc, *17 dc, ch 7, skip 4 ch and 1 dc*, repeat 5 times, sl st. Rnd 12: ch 3, 18 dc, ch 8, skip 4 ch and 1 dc, *19 dc, ch 8, skip 4 ch and 1 dc*, repeat 5 times, sl st. Rnd 13: ch 3, 20 dc, ch 8, skip 5 ch and 1 dc, *21 dc, ch 8, skip 5 ch and 1 dc*, repeat 5 times, sl st. Rnd 14: ch 3, 16 dc, ch 8, skip 3 dc and 2 ch, 1 dc, ch 8, skip 1 dc, *17 dc, ch 8, skip 3 dc and 2 ch, 1 dc, ch 8, skip 1 dc,* repeat 5 times, sl st.
  • 11.
    @ C CZ O N A The exhaustive rendering of our conscious and unconscious patterns into data sets and algorithms. P R E D I C T I V E A N A LY T I C S —Charles Duhigg
  • 12.
    @ C CZ O N A ALGORITHMS FOR FAST, TRAINABLE, ARTIFICIAL NEURAL NETWORKS D E E P L E A R N I N G
  • 13.
    @ C CZ O N A INPUT Words, images, sounds, objects, or abstract concepts EXECUTION Run a series of functions repeatedly in a black box OUTPUT Prediction of properties useful for drawing intuitions about similar future inputs
  • 14.
    @ C CZ O N A ARTIFICIAL NEURAL NETWORK'S AUTOMATED DISCOVERY OF PATTERNS WITHIN 
 A TRAINING DATASET APPLIES DISCOVERIES 
 TO DRAW INTUITIONS 
 ABOUT FUTURE DATA D E E P L E A R N I N G
  • 15.
    @ C CZ O N A – P e t e Wa rd e n " T H I N K O F T H E M A S 
 D E C I S I O N - M A K I N G 
 B L A C K B O X E S . "
  • 16.
    @ C CZ O N A Medical Diagnosing Pharmaceutical Development Emotion Detection Predictive Text Face Identification Voice-Activated Commands Fraud Detection Sentiment Analysis Translating Text Video Content Recognition Self-Driving Cars
  • 17.
    @ C CZ O N A Ad Targeting Behavioral Prediction Recommendation Systems Image Classification Face Recognition
  • 18.
    @ C CZ O N A
  • 19.
    @ C CZ O N A
  • 20.
    @ C CZ O N A #DataMiningFail
  • 21.
    Algorithmic Profiling De- Anonymization Black Box Disparate Impact Consent
 Issues Replicates
 Bias Personally Identifiable Info Human ComplexityFail Unproven Methods Inadvertent Algorithmic Cruelty Uncritical Assumptions False
 Neutrality High Risk Consequences Deception Filtering Moved Fast, 
 Broke Things Invasion Of Privacy No Recourse Creepy
 Stalkery Messing With Heads Not How Valid Research Works Data
 Insecurity Accurate, But 
 Not Right Shaming Diversity
 Fail #DataMiningFail
  • 22.
    🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵🔵 🔵 🔵 🔵Algorithmic Profiling De- Anonymization BlackBox Disparate Impact Consent
 Issues Replicates
 Bias Personally Identifiable Info Human Complexity Fail Unproven Methods Inadvertent Algorithmic Cruelty Uncritical Assumptions False
 Neutrality High Risk Consequences Deception Filtering Moved Fast, 
 Broke Things Invasion Of Privacy No Recourse Creepy
 Stalkery Messing With Heads Not How Valid Research Works Data
 Insecurity Accurate, But 
 Not Right Shaming Diversity
 Fail TA R G E T
  • 23.
    @ C CZ O N A “IF WE WANTED TO FIGURE OUT
 IF A CUSTOMER IS PREGNANT, EVEN IF SHE DIDN’T WANT US TO KNOW, 
 CAN YOU DO THAT? ”
  • 24.
    @ C CZ O N A
  • 25.
    @ C CZ O N A
  • 26.
    @ C CZ O N A ‟As long as a pregnant woman thinks she hasn’t been spied on… 
 as long as we don’t spook her, it works.”
  • 27.
    🔵 🔵 🔵🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 Algorithmic Profiling De- Anonymization Black Box Disparate Impact Consent
 Issues Replicates
 Bias Personally Identifiable Info Human Complexity Fail Unproven Methods Inadvertent Algorithmic Cruelty Uncritical Assumptions False
 Neutrality High Risk Consequences Deception Filtering Moved Fast, 
 Broke Things Invasion Of Privacy No Recourse Creepy
 Stalkery Messing With Heads Not How Valid Research Works Data
 Insecurity Accurate, But 
 Not Right Shaming Diversity
 Fail S H U T T E R F LY
  • 28.
    @ C CZ O N A
  • 29.
    @ C CZ O N A ‟Thanks, Shutterfly, for 
 the congratulations on my 'new bundle of joy'. 
 I'm horribly infertile, but hey, I'm adopting 
 a kitten,
 so...”
  • 30.
    @ C CZ O N A ‟ I l o s t a b a b y i n N o v e m b e r w h o w o u l d h a v e b e e n d u e t h i s w e e k . I t w a s l i k e     
 h i t t i n g a w a l l 
 a l l o v e r a g a i n … ˮ
  • 31.
    @ C CZ O N A ‟ T H E I N T E N T O F T H E 
 E M A I L WA S T O TA R G E T C U S T O M E R S W H O H AV E R E C E N T LY H A D A B A B Y "
  • 32.
    @ C CZ O N A ‟You start imagining who 
 they'll become and dreaming 
 of hopes for their future. You start making plans. And then they're gone. It's a lonely experience."
  • 33.
    🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 Algorithmic Profiling De- Anonymization Black Box Disparate Impact Consent
 Issues Replicates
 Bias Personally Identifiable Info Human ComplexityFail Unproven Methods Inadvertent Algorithmic Cruelty Uncritical Assumptions False
 Neutrality High Risk Consequences Deception Filtering Moved Fast, 
 Broke Things Invasion Of Privacy No Recourse Creepy
 Stalkery Messing With Heads Not How Valid Research Works Data
 Insecurity Accurate, But 
 Not Right Shaming Diversity
 Fail FA C E B O O K 
 Y E A R I N R E V I E W
  • 34.
    @ C CZ O N A The result of code that works in the overwhelming majority of cases but doesn’t take other use cases into account. I N A D V E R T E N T A L G O R I T H M I C C R U E LT Y —Eric Meyer
  • 35.
    @ C CZ O N A
  • 36.
    @ C CZ O N A ‟ I N C R E A S E A WA R E N E S S O F 
 A N D C O N S I D E R AT I O N F O R 
 T H E FA I L U R E M O D E S 
 T H E E D G E C A S E S 
 T H E W O R S T- C A S E S C E N A R I O S – E r i c M e y e r
  • 37.
    @ C CZ O N A BEHUMBLE.WECANNOTINTUIT
 INNERSTATE,EMOTIONS,
 PRIVATESUBJECTIVITY — C a r i n a C . Z o n a
  • 38.
    @ C CZ O N A
  • 39.
    @ C CZ O N A
  • 40.
    @ C CZ O N A
  • 41.
    @ C CZ O N A
  • 42.
    @ C CZ O N A
  • 43.
    @ C CZ O N A
  • 44.
    🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 Algorithmic Profiling De- Anonymization Black Box Disparate Impact Consent
 Issues Replicates
 Bias Personally Identifiable Info Human ComplexityFail Unproven Methods Inadvertent Algorithmic Cruelty Uncritical Assumptions False
 Neutrality High Risk Consequences Deception Filtering Moved Fast, 
 Broke Things Invasion Of Privacy No Recourse Creepy
 Stalkery Messing With Heads Not How Valid Research Works Data
 Insecurity Accurate, But 
 Not Right Shaming Diversity
 Fail F I T B I T
  • 45.
    @ C CZ O N A
  • 46.
    @ C CZ O N A
  • 47.
    🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 Algorithmic Profiling De- Anonymization Black Box Disparate Impact Consent
 Issues Replicates
 Bias Personally Identifiable Info Human ComplexityFail Unproven Methods Inadvertent Algorithmic Cruelty Uncritical Assumptions False
 Neutrality High Risk Consequences Deception Filtering Moved Fast, 
 Broke Things Invasion Of Privacy No Recourse Creepy
 Stalkery Messing With Heads Not How Valid Research Works Data
 Insecurity Accurate, But 
 Not Right Shaming Diversity
 Fail U B E R
  • 48.
    @ C CZ O N A
  • 49.
    @ C CZ O N A
  • 50.
    @ C CZ O N A
  • 51.
    @ C CZ O N A
  • 52.
    @ C CZ O N A
  • 53.
    🔵 🔵 🔵 🔵 🔵 🔵🔵 🔵 🔵 Algorithmic Profiling De- Anonymization Black Box Disparate Impact Consent
 Issues Replicates
 Bias Personally Identifiable Info Human ComplexityFail Unproven Methods Inadvertent Algorithmic Cruelty Uncritical Assumptions False
 Neutrality High Risk Consequences Deception Filtering Moved Fast, 
 Broke Things Invasion Of Privacy No Recourse Creepy
 Stalkery Messing With Heads Not How Valid Research Works Data
 Insecurity Accurate, But 
 Not Right Shaming Diversity
 Fail G O O G L E A D W O R D S
  • 54.
    @ C CZ O N A Amy Claire Emma Katelyn Katie Madeline Molly Aaliyah Diamond Ebony Imani Latanya Nia Shanice Cody Connor Dustin Jack Luke Tanner Wyatt Darnell DeShawn Malik Marquis Terrell Trevon Tyrone G O O G L E A D W O R D S
  • 55.
    PLACEHOLDER@ C CZ O N A PLACEHOLDER PLACEHOLDERA BLACK IDENTIFYING NAME WAS 
 25% MORE LIKELY
 TO RESULT IN AN AD THAT I M P L I E D A N 
 ARREST RECORD G O O G L E A D W O R D S
  • 56.
    @ C CZ O N A G O O G L E A D W O R D S
  • 57.
    🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵Algorithmic Profiling De- Anonymization Black Box Disparate Impact Consent
 Issues Replicates
 Bias Personally IdentifiableInfo Human Complexity Fail Unproven Methods Inadvertent Algorithmic Cruelty Uncritical Assumptions False
 Neutrality High Risk Consequences Deception Filtering Moved Fast, 
 Broke Things Invasion Of Privacy No Recourse Creepy
 Stalkery Messing With Heads Not How Valid Research Works Data
 Insecurity Accurate, But 
 Not Right Shaming Diversity
 Fail
  • 58.
    @ C CZ O N A Classifying people as “similar”, where careless prompts create scenarios harder to control 
 and prepare for. A C C I D E N TA L A L G O R I T H M I C R U N - I N S —adapted from Joanne McNeil
  • 59.
    @ C CZ O N A A L G O R I T H M I C H U B R I S
  • 60.
    @ C CZ O N A "If you are stalked by a former coworker, Twitter may reinforce this connection, algorithmically boxing you into a past while you are trying to move on. 
 Your affinity score with your harasser will grow higher with every person who follows this person at Twitter’s recommendation." T W I T T E R - W H O T O F O L L O W
  • 61.
    🔵🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 Algorithmic Profiling De- Anonymization BlackBox Disparate Impact Consent
 Issues Replicates
 Bias Personally Identifiable Info Human Complexity Fail Unproven Methods Inadvertent Algorithmic Cruelty Uncritical Assumptions False
 Neutrality High Risk Consequences Deception Filtering Moved Fast, 
 Broke Things Invasion Of Privacy No Recourse Creepy
 Stalkery Messing With Heads Not How Valid Research Works Data
 Insecurity Accurate, But 
 Not Right Shaming Diversity
 Fail
  • 62.
    @ C CZ O N A I P H O T O FA C E D E T E C T I O N
  • 63.
    @ C CZ O N A M I C R O S O F T H O W - O L D . N E T
  • 64.
    @ C CZ O N A F L I C K R M A G I C V I E W A U T O - TA G G I N G
  • 65.
    @ C CZ O N A F L I C K R M A G I C V I E W A U T O - TA G G I N G
  • 66.
    @ C CZ O N A G O O G L E P H O T O S A U T O - C AT E G O R I Z I N G
  • 67.
    @ C CZ O N A S H I R L E Y C A R D S
  • 68.
    @ C CZ O N A The tools used to make film, b y M o n t ré A z a M i s s o u r i @ C C Z O N A not are the science of it, racially neutral S H I R L E Y C A R D S
  • 69.
    🔵 🔵🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 🔵 Algorithmic Profiling De- Anonymization Black Box Disparate Impact Consent
 Issues Replicates
 Bias Personally IdentifiableInfo Human Complexity Fail Unproven Methods Inadvertent Algorithmic Cruelty Uncritical Assumptions False
 Neutrality High Risk Consequences Deception Filtering Moved Fast, 
 Broke Things Invasion Of Privacy No Recourse Creepy
 Stalkery Messing With Heads Not How Valid Research Works Data
 Insecurity Accurate, But 
 Not Right Shaming Diversity
 Fail
  • 70.
    @ C CZ O N A A L G O R I T H M S A R E G AT E K E E P E R S Credit Housing Jobs
  • 71.
    @ C CZ O N A
  • 72.
    @ C CZ O N A IF A MACHINE IS EXPECTED TO BE INFALLIBLE IT CANNOT ALSO BE INTELLIGENT— A l a n Tu r i n g
  • 73.
    @ C CZ O N A !=I R L F R I E N D S FA C E B O O K F R I E N D S
  • 74.
    @ C CZ O N A F R I E N D S F I N A N C E S — i l l u s t r a t i o n b y S u s i e C a g l e
  • 75.
    @ C CZ O N A
  • 76.
    @ C CZ O N A
  • 77.
    @ C CZ O N A
  • 78.
    @ C CZ O N A UNDERLYING ASSUMPTIONS INFLUENCE OUTCOMES AND CONSEQUENCES @ C C Z O N A
  • 79.
    @ C CZ O N A — H e n r y D a v i d T h o re a u “ T H E U N I V E R S E I S W I D E R T H A N O U R V I E W S O F I T ” @ C C Z O N A
  • 80.
    @ C CZ O N A OUR INDUSTRY IS 
 IN AN ARMS RACE
  • 81.
    @ C CZ O N A Amazon Apple AT&T Baidu DARPA Dropbox eBay Facebook Flickr Google IBM Microsoft Netflix Pandora PayPal Pinterest Skype Snapchat Spotify Tesla Twitter Yahoo Uber
  • 82.
    @ C CZ O N A MOREPRECISE
 INCORRECTNESS MORE
 DAMAGING
 INWRONGNESS
  • 83.
    @ C CZ O N A FLIPPING THE PARADIGM
  • 84.
    @ C CZ O N A CONSIDER DECISIONS' POTENTIAL IMPACTS F L I P P I N G T H E PA R A D I G M A C M C o d e o f E t h i c s 1
  • 85.
    @ C CZ O N A •PROJECT THE LIKELIHOOD OF CONSEQUENCES •MINIMIZE NEGATIVE CONSEQUENCES E VA L U AT E P O T E N T I A L I M PA C T S
  • 86.
    @ C CZ O N A FIRST DO NO HARM
  • 87.
    @ C CZ O N A BE HONEST BE TRUSTWORTHY F L I P P I N G T H E PA R A D I G M A C M C o d e o f E t h i c s 2
  • 88.
    @ C CZ O N A BUILD IN RECOURSE F L I P P I N G T H E PA R A D I G M A C M C o d e o f E t h i c s
  • 89.
    @ C CZ O N A FULLY DISCLOSE LIMITATIONS. CALL ATTENTION TO SIGNS OF RISK OF HARM. F L I P P I N G T H E PA R A D I G M 3
  • 90.
    @ C CZ O N A
  • 91.
    @ C CZ O N A BE VISIONARY ABOUT COUNTERING BIAS F L I P P I N G T H E PA R A D I G M 4
  • 92.
    @ C CZ O N A ANTICIPATE DIVERSE WAYS TO SCREW UP E M PAT H E T I C C O D E R S 5
  • 93.
    @ C CZ O N A W E M U S T H A V E D E C I S I O N - M A K I N G A U T H O R I T Y 
 I N T H E H A N D S O F H I G H LY 
 D I V E R S E 
 T E A M S
  • 94.
    @ C CZ O N A The point of culture fit is to avoid disruption of groupthink
  • 95.
    @ C CZ O N A Unidimensional variety
 is not diversity
  • 96.
    @ C CZ O N A
  • 97.
    @ C CZ O N A AUDIT OUTCOMES
 CONSTANTLY F L I P P I N G T H E PA R A D I G M 6
  • 98.
    @ C CZ O N A
  • 99.
    @ C CZ O N A AUDIT OUTCOMES CONSTANTLY
  • 100.
    @ C CZ O N A AUDIT OUTCOMES CONSTANTLY
  • 101.
    @ C CZ O N A AUDIT OUTCOMES
  • 102.
    @ C CZ O N A 1,852318 " C A R E F U L LY C H O S E N T R A I L S A C R O S S T H E W E B 
 I N S U C H A WAY T H AT A N A D - TA R G E T I N G N E T W O R K W I L L I N F E R C E RTA I N I N T E R E S T S O R A C T I V I T I E S . " A D F I S H E R
  • 103.
    @ C CZ O N A C R O W D K N O W L E D G E > = B L A C K B O X I N T U I T I O N S 7C R O W D S O U R C E A L L T H E T H I N G S F L I P P I N G T H E PA R A D I G M
  • 104.
    @ C CZ O N A Party.
  • 105.
    @ C CZ O N A
  • 106.
    @ C CZ O N A
  • 107.
    @ C CZ O N A
  • 108.
    @ C CZ O N A
  • 109.
    @ C CZ O N A
  • 110.
    @ C CZ O N A
  • 111.
    @ C CZ O N A
  • 112.
    @ C CZ O N A CULTIVATE INFORMED CONSENT F L I P P I N G T H E PA R A D I G M 8
  • 113.
    @ C CZ O N A
  • 114.
    @ C CZ O N A "LIST THINGS YOU 
 DON'T WANT 
 TO THINK ABOUT"
  • 115.
    @ C CZ O N A Opt-Out Opt-In Block Follow Ignore View Filter Select Disable Enable S T O P T H E A L G O R I T H M B E T H E A L G O R I T H M
  • 116.
    @ C CZ O N A "Why not let Twitter users select the people they recommend you follow? We already have Follow Friday." —adapted from Joanne McNeil & Melissa Gira Grant
  • 117.
    @ C CZ O N A Would you like special coupons for pregnancy & infant care?
  • 118.
    @ C CZ O N A COMMIT TO DATA TRANSPARENCY. COMMIT TO ALGORITHMIC TRANSPARENCY. F L I P P I N G T H E PA R A D I G M 9
  • 119.
    @ C CZ O N A
  • 120.
    @ C CZ O N A We DO NOT BUILD here 
 without understanding CONSEQUENCES TO OTHERS
  • 121.
    @ C CZ O N A PROFESSIONALS
 APPLY EXPERTISE & JUDGEMENT ABOUT 
 HOW TO 
 SOLVE PROBLEMS
  • 122.
    Algorithmic Profiling De- Anonymization Black Box Disparate Impact Consent
 Issues Replicates
 Bias Personally Identifiable Info Human ComplexityFail Unproven Methods Inadvertent Algorithmic Cruelty Uncritical Assumptions False
 Neutrality High Risk Consequences Deception Filtering Moved Fast, 
 Broke Things Invasion Of Privacy No Recourse Creepy
 Stalkery Messing With Heads Not How Valid Research Works Data
 Insecurity Accurate, But 
 Not Right Shaming Diversity
 Fail ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌❌ ❌ ❌ ❌ ❌❌ ❌ ❌ ❌ ❌ ❌ ❌
  • 123.
    @ C CZ O N A R E F U S E T O 
 P L AY A L O N G .
  • 124.
    @ C CZ O N A T H A N K Y O U .
  • 125.
    @ C CZ O N A Code Newbies http://www.codenewbie.org/podcast/algorithms “CONSEQUENCES OF AN INSIGHTFUL ALGORITHM” CONTINUES AT: Ruby Rogues https://devchat.tv/ruby-rogues/278-rr-consequences-of-an- insightful-algorithm-with-carina-c-zona
  • 126.
    @ C CZ O N A CASE STUDIES FitBit: http://thenextweb.com/insider/2011/07/03/fitbit-users-are-inadvertently- sharing-details-of-their-sex-lives-with-the-world/ Uber: http://www.forbes.com/sites/kashmirhill/2014/10/03/god-view-uber- allegedly-stalked-users-for-party-goers-viewing-pleasure/ & http:// valleywag.gawker.com/uber-allegedly-used-god-view-to-stalk-vip-users-as- a-1642197313 & http://www.forbes.com/sites/chanellebessette/2014/11/25/does- uber-even-deserve-our-trust/ & http://www.wired.com/2011/04/app-stars-uber/ & http://motherboard.vice.com/read/ubers-god-view-was-once-available-to-drivers & http://motherboard.vice.com/read/ubers-god-view-was-once-available-to- drivers OkCupid: http://blog.okcupid.com/index.php/the-best-questions-for-first-dates/ Affirm, et al: http://recode.net/2015/05/06/max-levchins-affirm-raises-275- million-to-make-loans/ & http://time.com/3430817/paypal-levchin-affirm-lending/ & http://www.nytimes.com/2015/01/19/technology/banking-start-ups-adopt- new-tools-for-lending.html & http://www.spiegel.de/international/germany/ critique-of-german-credit-agency-plan-to-mine-facebook-for-data-a-837713.html & http://www.motherjones.com/politics/2013/09/lenders-vet-borrowers-social- media-facebook Shutterfly: http://jezebel.com/shutterfly-thinks-you-just-had-a-baby-1576261631 & http://www.adweek.com/adfreak/shutterfly-congratulates-thousands-women- babies-they-didnt-have-157675 Zuckerberg on miscarriage: https://www.facebook.com/photo.php? fbid=10102276573729791 Target: http://www.nytimes.com/2012/02/19/magazine/shopping-habits.html Facebook Year in Review: http://meyerweb.com/eric/thoughts/2014/12/24/ inadvertent-algorithmic-cruelty/ & http://www.theguardian.com/technology/ 2014/dec/29/facebook-apologises-over-cruel-year-in-review-clips Google AdWords: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2208240 & http://freakonomics.com/2013/04/08/how-much-does-your-name-matter-full- transcript/ Flickr: http://www.theguardian.com/technology/2015/may/20/flickr-complaints- offensive-auto-tagging-photos Google Photos: https://twitter.com/jackyalcine/status/615329515909156865 & https://www.yahoo.com/tech/google-photos-mislabels-two-black-americans- as-122793782784.html Twitter Who To Follow & Last.fm Similar Listeners: https://medium.com/ message/harassed-by-algorithms-f2b8229488df Chicken Breast: http://metro.co.uk/2016/01/27/this-chicken-breast-has-a- surprisingly-healthy-heart-rate-considering-its-dead-5647836/ & https:// www.youtube.com/watch?v=7rO5knyjDR0 LinkedIn http://www.slate.com/articles/technology/technology/2016/05/ linkedin_called_me_a_white_supremacist.html Shirley Cards, Race, & Film Emulsion: http://priceonomics.com/how- photography-was-optimized-for-white-skin/ & http://www.buzzfeed.com/ syreetamcfadden/teaching-the-camera-to-see-my-skin#.wkv3Jmd8O
  • 127.
    @ C CZ O N A RESOURCES • Association for Computing Machinery Code of Ethics http://www.acm.org/about/code- of-ethics • The 10 Commandments of Egoless Programming http://www.techrepublic.com/ article/the-ten-commandments-of-egoless- programming-6353837/ • Disparate Impact Analysis is Key to Ensuring Fairness in the Age of the Algorithm http:// www.datainnovation.org/2015/01/disparate- impact-analysis-is-key-to-ensuring-fairness- in-the-age-of-the-algorithm/ • On Algorithmic Fairness, Discrimination and Disparate impact. http:// fairness.haverford.edu/ • Big Data's Disparate Impact http:// papers.ssrn.com/sol3/papers.cfm? abstract_id=2477899 • Algorithmic Accountability & Transparency http://www.nickdiakopoulos.com/projects/ algorithmic-accountability-reporting/ • What is Deep Learning and why should you care? http://radar.oreilly.com/2014/07/what- is-deep-learning-and-why-should-you- care.html • Facebook Research | Machine Learning https://research.facebook.com/ machinelearning
  • 128.
    @ C CZ O N A LAW, GOVERNANCE, & POLICYMAKING • AI is Setting Up the Internet for a Huge Class with Europe (2016) https:// www.wired.com/2016/07/artificial- intelligence-setting-internet-huge-clash- europe/ • California Law Review (2016): Big Data's Disparate Impact http://papers.ssrn.com/ sol3/papers.cfm?abstract_id=2477899 • United States Federal Trade Commission Report (2016) Big Data: A Tool for Inclusion or Exclusion? https://www.ftc.gov/system/ files/documents/reports/big-data-tool- inclusion-or-exclusion-understanding- issues/160106big-data-rpt.pdf • White House Big Data Privacy Report (2014) https://www.whitehouse.gov/sites/ default/files/docs/ big_data_privacy_report_may_1_2014.pdf • Algorithmic Transparency and Platform Loyalty or Fairness in the French Digital Republic Bill (2016) http://blogs.lse.ac.uk/ mediapolicyproject/2016/04/22/algorithmic- transparency-and-platform-loyalty-or- fairness-in-the-french-digital-republic-bill/ • European Data Protection Supervisor Opinion (2015): Meeting the Challenges of Big Data https://secure.edps.europa.eu/ EDPSWEB/webdav/site/mySite/shared/ Documents/Consultation/Opinions/ 2015/15-11-19_Big_Data_EN.pdf
  • 129.
    @ C CZ O N A FLICKR CREATIVE COMMONS Cookie face: https://www.flickr.com/photos/amayu/4462907505/in/ pool-977532@N24/ Eye shadows: https://www.flickr.com/photos/niallb/5300259686/ Colored pencils: https://www.flickr.com/photos/jenson-lee/ 6315443914/ Audit keyboard: https://www.flickr.com/photos/jakerust/16811751576/ Crystal balls: https://www.flickr.com/photos/cmogle/3348401259/ Eye: https://www.flickr.com/photos/weirdcolor/3312989961/ Sea Wall: https://www.flickr.com/photos/christing/1447039348/ Door: https://www.flickr.com/photos/chrisschoenbohm/14181173109/ WTF Was This WTF Was That by Birger King: https://www.flickr.com/ photos/birgerking/7080728907/ Tasveer ByTaru: https://www.flickr.com/photos/tasveerbytaru/ 19547308031/sizes/h/ Boxing day: https://www.flickr.com/photos/erix/6306715646/ Printing machine by Teddy Kwok: https://www.flickr.com/photos/ bblkwok/8247948713/ Caduceus: https://www.flickr.com/photos/spirit-fire/4832779325/ wocintech (microsoft) - 69 by #wocintech: https://www.flickr.com/ photos/wocintechchat/25926648871/in/album-72157664006621903/ ねこ -ちゃーちゃん- by atacamaki: https://www.flickr.com/photos/ mikey-a-tucker/14792307898/ Driving home by Christian Weidinger: https://www.flickr.com/photos/ ch-weidinger/13888995404/ Server graphs by Aaron Parecki: https://www.flickr.com/photos/ aaronpk/5967579738/ On the Banks of Loch Linnhe by Henry Hemming: https:// www.flickr.com/photos/henry_hemming/8280971407/ Scaffolding by ajjyt: https://www.flickr.com/photos/ 54588550@N08/9387947434/ Monarchial Scrutiny by Joel Penner: https://www.flickr.com/photos/ featheredtar/2949748121/ Canadian Money $100 Dollar Bill Macro of Robert Borden by Wilson Hui: https://www.flickr.com/photos/wilsonhui/6604606217/ The grindstone by Kathryn Decker: https://www.flickr.com/photos/ waponigirl/5621810815/ "img049" by Steve Walker Photography: https://www.flickr.com/ photos/stover98074/11361095594/ Street Passion by Johnny Silvercloud: https://www.flickr.com/photos/ johnnysilvercloud/15718876258/ Buttons: https://www.flickr.com/photos/48462557@N00/3616793654/
  • 130.
    @ C CZ O N A NOUN PROJECT CREATIVE COMMONS Businesspeople by Honnos Bondor
 https://thenounproject.com/term/businesspeople/17542/ Send by David Papworth
 https://thenounproject.com/term/send/135469/ Users by Lorena Salagre
 https://thenounproject.com/term/users/32253/ Friend by Megan Mitchel
 https://thenounproject.com/term/friend/6808/ Woman (1) by Thomas Helbig
 https://thenounproject.com/term/woman/120381/ Woman (2) by Thomas Helbig
 https://thenounproject.com/term/woman/120380/ Couple by Shankar Narayan
 https://thenounproject.com/term/couple/1014/ Couple by Pasquale Cavorsi
 https://thenounproject.com/term/couple/33095/ Check mark by Kris Brauer
 https://thenounproject.com/term/check-mark/192830/ Job Seeker by Stijn Elskens
 https://thenounproject.com/term/job-seeker/226871/ Credit Card by Oliviu Stoian
 https://thenounproject.com/term/credit-card/269411/ Single House by Aaron K. Kim
 https://thenounproject.com/term/single-house/123924/ Downloads by Icons8
 https://thenounproject.com/term/downloads/61761/ Id Card (1) by Boudewijn Mijnlieff
 https://thenounproject.com/term/id-card/203566/ Id Card (2) by Boudewijn Mijnlieff
 https://thenounproject.com/term/id-card/203578/ Cancel by Kris Brauer
 https://thenounproject.com/term/cancel/182501/
  • 131.
    @ C CZ O N A ADDITIONAL IMAGES Recipe cards: Olivia Juice & Co. http:// olivejuiceco.typepad.com/my_weblog/2006/04/ easter_recap_an.html Target circulars: http://flyers.smartcanucks.ca/uploads/ pages/26431/target-canada-weekly-flyer-july-11- to-1713.jpg Crochet pattern & shawl: http://www.abc-knitting- patterns.com/1129.html Space Invaders: http://www.caffination.com/ backchannel/shooting-for-the-high-score-3942/ Facebook b/w logo sketch: http://connect- communicate-change.com/wp-content/uploads/ 2011/11/facebook-logo-square-webtreatsetc.png Stephen Hawking by CERN/Lawrent Egli: https:// www.facebook.com/stephenhawking/photos/ 710802829006817 Jonathon Arrears by Susie Cagle: https://twitter.com/ susie_c/status/631619118865604610/photo/1 Twitter keyboard: http://www.npr.org/sections/ itsallpolitics/2012/05/04/152028258/navigating-politics- on-twitter-some-npr-followfriday-suggestions Tesla's Autopilot System MobilEye http:// wccftech.com/tesla-autopilot-story-in-depth- technology/ Uber home screen https://www.supermoney.com/ 2016/05/5-reasons-uber-can-risky-choice-drivers- passengers/ Walden Pond by Walden Pond State Reservation https://waldenpondstatereservation.wordpress.com/ The Force is Strong With This One by Mark Zuckerberg https://www.facebook.com/zuck/posts/ 10102531565693851 Bubble-sort with Hungarian ("Csángó") folk dance https://www.youtube.com/watch?v=lyZQPjUT5B4
  • 132.
    @ C CZ O N A T H E E T H I C S & R E S P O N S I B I L I T I E S O F O P E N S O U R C E I O T https://www.youtube.com/ watch?v=7rO5knyjDR0 Emily Gorcenski @emilygorcenski
  • 133.
    @ C CZ O N A I M A G E U N D E R S TA N D I N G : 
 D E E P L E A R N I N G W I T H C O N V O L U T I O N A L N E U R A L N E T S http://www.slideshare.net/ roelofp/python-for-image- understanding-deep-learning- with-convolutional-neural-nets Roelof Pieters @graphific
  • 134.
    @ C CZ O N A T H E E T H I C S O F B E I N G A P R O G R A M M E R https://youtu.be/ DB7ei5W1eRQ Kate Heddleston @heddle317
  • 135.
    @ C CZ O N A A H I P P O C R AT I C O AT H F O R D ATA S C I E N C E http://www.slideshare.net/ roelofp/a-hippocratic-oath-for- data-science Roelof Pieters @graphific
  • 136.
    @ C CZ O N A D E E P L E A R N I N G : I N T E L L I G E N C E F R O M 
 B I G D ATA https://www.youtube.com/ watch?v=czLI3oLDe8M Adam Berenzweig @madadam
  • 137.
    @ C CZ O N A A L G O R I T H M I C A C C O U N TA B I L I T Y W O R K S H O P https://vimeo.com/125622175 Columbia Journalism School
  • 138.
    @ C CZ O N A M A R I / O https://youtu.be/ qv6UVOQ0F44 Seth Bling @sethbling
  • 139.
    @ C CZ O N A Noah Kantrowitz Heidi Waterhouse VM Brasseur Yoz Grahame Mike Foley
 Estelle Weyl Chris Hausler Scott Triglia Russell Keith-Magee Tim Bell We So Crafty THANK YOU ♥
  • 140.
    @ C CZ O N A BONUS! “Eleven” by Burnistoun (Robert Florence & Iain Connell) https://www.youtube.com/watch?v=NMS2VnDveP8
  • 141.
    @ C CZ O N A