Data and Algorithmic Bias in the Web

7/20/16
1
Data and Algorithmic Bias
in the Web
Ricardo Baeza-Yates
California (NTENT), Catalonia (UPF), Chile (UChile)
WebVisions Barcelona, July 2016
Steve Jobs and Bias
Pervasive Optimistic Bias (Kahneman)
Reality Distortion Field

7/20/16
2
Every Website is an
Information Market
Good Design
Good Interaction
Right Incentives

7/20/16
3
All Data has Bias
§  Gender
§  Racial
§  Sexual
§  Religious
§  Social
§  Linguistic
§  Geographic
§  Political
§  Educational
§  Economic
§  Technological
§  from Noise or Spam
§  Validity (e.g. temporal)
§  Completeness
§  Gathering process
§  ….
However many people extrapolate
results to the whole population
(e.g., social media analysis)
In addition there is bias when
measuring bias as well as bias
towards measuring it!
Yes, We Live in a (Very) Biased World!

7/20/16
4
A Non-Technical Question
Algorithm
Biased
Data
Neutral?
Same
Bias
Not
Always!
Unbias the data
Tune the algorithm
Unbias the output
Bias awareness!
Big Data and Bias
15
§  The quality of any algorithm is bounded by the
quality of the data that uses
§  Data bias awareness
§  Algorithmic fairness
§  Key issues for machine learning
§  Uniformity of data properties
§  In the Web, distributions resemble a power law
§  Uniformity of error
§  Data sample methodology
§  E.g., sample size to see infrequent events or
sampling bias issues

7/20/16
5
Data bias
Activity bias
Selection bias
Sampling bias and size
Algorithmic bias
Interface
(Self) selection bias
Second order bias
Sparsity
Privacy
Algorithm

7/20/16
6
Quantity
Quality
User-
generated
Traditional
publishing
What is in the Web? How Much Data?
How Good is it? 168 million active web servers,
1083 million hostnames and
infinitely many pages!
26
What else is in the Web?

7/20/16
7
Noise and Spam
27
§  Noise may come from many places:
§  Instruments that measure (e.g., IoT)
§  How we interpret the data
§  Spam is everywhere
§  Fight both with the wisdom of the crowds
Data Bias and Redundancy
38
§  There is any dependency in the data?
§  There is any duplication?
§  Lexical duplication in the Web is around 25%
§  Semantic duplication is larger (more later)
§  Any other biases? Many!
§  Web structure (economic, cultural)
§  Web content (linguistic, geography, gender)

7/20/16
8
39
[Baeza-Yates, Castillo & López. Characteristics of the Web of Spain.
The Information Professional (Spanish), 2006, vol. 15, n. 1, pp. 6-17]
Economic Bias in Links
Number of linked domains
Exports(thousandsofUS$)
40
Baeza-Yates & Castillo, WWW2006
Exports/Imports vs. Domain Links

7/20/16
9
41
[Baeza-Yates, Castillo, Efthimiadis, TOIT 2007]
Website Structure
Minimal effortShame
42
Linguistic Bias

7/20/16
10
Geographical Bias
[E. Graells-Garrido and M. Lalmas,
“Balancing diversity to counter-measure
geographical centralization in microblogging
platforms”, ACM Hypertext’14]
Gender Bias
[E. Graells-Garrido et al,. “First Women, Second Sex: Gender Bias in Wikipedia”, ACM Hypertext’15]
Systemic bias?
Equal opportunity?

7/20/16
11
•  The Web already is influenced by small groups
•  "0.05% of the user population, attract almost
50% of all attention within Twitter" (50K users)
[Wu, Hofman, Mason & Watts, WWW 2011]
•  We explored this issue further with four different datasets:
1.  a large one from Twitter (2011),
2.  a small one from Facebook (2009),
3. Amazon reviews (2013), and
4.  Wikipedia editors (2015).
•  Digital desert: the content that is never seen
Activity Bias: Wisdom of a Few?
[Baeza-Yates & Saez-Trumper, ACM Hypertext 2015]
Examples
[Baeza-Yates & Saez-Trumper, ACM Hypertext 2015]

7/20/16
12
October 2015
Quality of Content?
51 Yahoo Confidential & Proprietary
•  Adding content implies adding wisdom?
•  We use Amazon’s reviews helpfulness
•  We computed the text entropy
•  Content-based-wise users
•  How many of those users are being paid?

7/20/16
13
Digital Desert
52 Yahoo Confidential & Proprietary
Weblands
of Wisdom

7/20/16
14
Bias in the Interface
Position bias
Ranking bias
Presentation bias
Social bias
Interaction bias
Presentation Bias
§  Interaction data will be biased to what is shown
§  In recommender systems, items recommended will get
more clicks than items not recommended
§  In search systems top ranked results will get more clicks
than other results
›  Ranking bias
›  Interaction bias
CTR
(log)
1 11 21 Rank
[Dupret & Piwowarski, SIGIR 2008]
[Chapelle & Zhang, WWW 2009]bias

7/20/16
15
[WHY AMAZON’S RATINGS MIGHT MISLEAD YOU; The Story of Herding Effects
Ting Wang and Dashun Wang, Big Data, 2014]
Social Bias
Extreme Algorithmic Bias

7/20/16
16
Second Order Bias in Web Content
[Baeza-Yates, Pereira & Ziviani,
Geneological Trees in the Web, WWW 2008]
Person
Web content is redundant
Clicks in results are biased to
the ranking and the interaction
Query
Ranking bias
Redundancy grows (35%)
Search results
New
Most measures in the Web follow a power law
The Long Tail: Sparsity
[Anatomy of the long tail: Ordinary People with Extraordinary Tastes,
Goel, Broder, Gabrilovich, Pang; WSDM 2010]
§  Why there is a long tail?
§  Sampling in the tail
§  When the crowd dominates
§  Empowering the tail

7/20/16
17
When the Crowd
Dominates
Kills the long tail
80
Personalization “facets”:
•  Language (not always)
•  Location
•  Semantic facets per user
•  Query intent prediction in search
Empowering the Tail
The Filter “Bubble”, Eli Pariser
•  Avoid the Poor get Poorer Syndrome
•  Avoid the Echo Chamber
•  How to expose opposite views?
81
Cold start problem solution:
Explore & Exploit
Solutions:
•  Diversity
•  Novelty
•  Serendipity

7/20/16
18
A Data Portrait is a visual
context where users can
explore how the system
understand their interests.
This context is used to
embed content-based
recommendations, displayed
visually to facilitate
exploration and user
engagement.
To combat homophily,
recommendations are
generated having political
diversity in mind.
Does it work? Yes, by using
intermediary topics that are shared!
But only when users are interested
in politics.
Demo at http://auroratwittera.cl/perfil/YahooLabs
[E. Graells-Garrido, M. Lalmas and R. Baeza-Yates, ACM UAI 2016]
•  Exploit the context (and deep learning!)
91% accuracy to predict the next app you will use
[Baeza-Yates et al, WSDM 2015]
•  Personalization vs. Contextualization
Recall that user interaction is another long tail
People
Interests
Aggregating in theTail

7/20/16
19
[De Choudhury et al, ACM HT 2010]
87[Quercia et al, ACM HT 2014]
Crowdsourcing Data: Good Paths

7/20/16
20
Regions from Pictures
[Thomee et al, Demo at CHI 2014]
AOL Query Logs Release Incident
§  No. 4417749 conducted hundreds of searches over
a three-month period on topics ranging from “numb
fingers” to “60 single men”.
§  Other queries: “landscapers in Lilburn, Ga,” several
people with the last name Arnold and “homes sold in
shadow lake subdivision gwinnett county georgia.”
§  Data trail led to Thelma Arnold, a 62-year-old widow
who lives in Lilburn, Ga., frequently researches her
friends’ medical ailments and loves her three dogs.
A Face Is Exposed for AOL Searcher No. 4417749,
By MICHAEL BARBARO and TOM ZELLER Jr,
The New York Times, Aug 9 2006
90

7/20/16
21
91
Risks of Privacy in Query Logs
§  Profile [Jones, Kumar, Pang, Tompkins, CIKM 2007]
•  Gender: 84%
•  Age (±10): 79%
•  Location (ZIP3): 35%
§  Vanity Queries [Jones et al, CIKM 2008]
•  Partial name: 8.9%
•  Complete: 1.2%
§  More information:
•  A Survey of query log privacy-enhancing techniques
from a policy perspective [Cooper, ACM TWEB 2008]
§  A good anonymization technique is still an open problem

7/20/16
22
Privacy Awareness
§ How our privacy changes when we change our social network?
§ Information gain to predict a private attribute based on public data
§ Each user may have a promiscuity score
§ Example: new friendship request
Promiscuity( me ) > Promiscuity( new)
Promiscuity( me ) ≥ Promiscuity( new ) + max-gain-I-allow
Promiscuity( me ) < Promiscuity( new ) + max-gain-I-allow
Related work by [Estivill-Castro & Nettleton; Singh, ASONAM 2015]
The Web Works Thanks to Bias!
§ Web traffic
›  Local caching
›  Proxy/Akamai caching
§ Search engines
›  Answer caching
›  Essential web pages
•  25% queries can be answered with less than 1% of the URLs!
[Baeza-Yates, Boldi, Chierichetti, WWW 2015]
§ E-Commerce
›  Large fraction of revenue comes from few popular items
Activity bias
(Self) selection bias

7/20/16
23
Web Data
§  A mirror of ourselves, the good, the bad and the ugly
§  The web amplifies everything, good or bad, but always
leaves traces
§  We have to be aware of the biases and contrarrest them
§  We have to be aware of our privacy
Big Data of People is huge…..
….. but is tiny compared to the future
Big Data of the Internet of Things (IoT)
It’s Hard to Get Data to Tell the Truth
§  The blindness of the averages
§  Look at distributions
§  Absolute vs. relative
§  Income per capita vs. Inequality
§  Local vs. global optimization
§  Teams competing without knowing, uncorrelated criteria
§  You can always see/torture data as you wish
›  61 analysts, 29 teams: 20 yes and 9 no (Univ. of Virginia, COS)

7/20/16
24
Contact: rbaeza@acm.org
www.baeza.cl
@polarbearby
ASIST 2012
Book of the
Year Award
Questions?
Biased Questions?

Data and Algorithmic Bias in the Web

More Related Content

What's hot

Viewers also liked

Similar to Data and Algorithmic Bias in the Web

More from WebVisions

Recently uploaded

Data and Algorithmic Bias in the Web