KEMBAR78
Sparse matrix computations in MapReduce | PDF
DG
Uploaded byDavid Gleich
PDF, PPTX12,443 views

Sparse matrix computations in MapReduce

The document announces an ICME MapReduce workshop from April 29 to May 1, 2013. The workshop goals are to learn the basics of MapReduce and Hadoop and be able to process large volumes of scientific data. The workshop overview includes presentations on sparse matrix computations in MapReduce, extending MapReduce for scientific computing, and evaluating MapReduce for science. The document also discusses how to program data computers using MapReduce and Hadoop and provides examples of matrix operations like matrix-vector multiplication in MapReduce.

Download as PDF, PPTX
ICME MapReduce Workshop! April 29 – May 1, 2013! ! David F. Gleich! Computer Science! Purdue University David Gleich · Purdue 1 ! Website www.stanford.edu/~paulcon/icme-mapreduce-2013 Paul G. Constantine! Center for Turbulence Research! Stanford University MRWorkshop
Goals Learn the basics of MapReduce & Hadoop Be able to process large volumes of data from science and engineering applications … help enable you to explore on your own! David Gleich · Purdue 2 MRWorkshop
Workshop overview Monday! Me! Sparse matrix computations in MapReduce! Austin Benson Tall-and-skinny matrix computations in MapReduce Tuesday! Joe Buck Extending MapReduce for scientific computing! Chunsheng Feng Large scale video analytics on pivotal Hadoop Wednesday! Joe Nichols Post-processing CFD dynamics data in MapReduce ! Lavanya Ramakrishnan Evaluating MapReduce and Hadoop for science David Gleich · Purdue 3 MRWorkshop
Sparse matrix computations in MapReduce! David F. Gleich! Computer Science! Purdue University David Gleich · Purdue 4 Slides online soon! Code https://github.com/dgleich/mapreduce-matrix-tutorial MRWorkshop
How to compute with big matrix data ! A tale of two computers 224k Cores 10 PB drive 1.7 Pflops 7 MW Custom ! interconnect! $104 M 80k cores! 50 PB drive ? Pflops ? MW GB ethernet $?? M 625 GB/core! High disk to CPU 45 GB/core High CPU to disk 5 ORNL 2010 Supercomputer! Google’s 2010? ! Data computer! David Gleich · Purdue MRWorkshop
My data computers 6 Nebula Cluster @ Sandia CA! 2TB/core storage, 64 nodes, 256 cores, GB ethernet Cost $150k These systems are good for working with enormous matrix data! ICME Hadoop @ Stanford! 3TB/core storage, 11 nodes, 44 cores, GB ethernet Cost $30k David Gleich · Purdue MRWorkshop
My data computers 7 Nebula Cluster @ Sandia CA! 2TB/core storage, 64 nodes, 256 cores, GB ethernet Cost $150k These systems are good for working with enormous matrix data! ICME Hadoop @ Stanford! 3TB/core storage, 11 nodes, 44 cores, GB ethernet Cost $30k ^ but not great, David Gleich · Purdue some ^ MRWorkshop
By 2013(?) all Fortune 500 companies will have a data computer David Gleich · Purdue 8 MRWorkshop
How do you program them? 9 David Gleich · Purdue MRWorkshop
MapReduce and! Hadoop overview 10 David Gleich · Purdue MRWorkshop
MapReduce is designed to solve a different set of problems from standard parallel libraries 11 David Gleich · Purdue MRWorkshop
The MapReduce programming model Input a list of (key, value) pairs Map apply a function f to all pairs Reduce apply a function g to ! all values with key k (for all k) Output a list of (key, value) pairs 12 David Gleich · Purdue MRWorkshop
Computing a histogram ! A simple MapReduce example 13 Input! ! Key ImageId Value Pixels Map(ImageId, Pixels) for each pixel emit" Key = (r,g,b)" Value = 1 Reduce(Color, Values) emit" Key = Color Value = sum(Values) Output! ! Key Color Value ! # of pixels David Gleich · Purdue 5 15 10 9 3 17 5 10 1 1 1 1 Map Reduce 1 1 1 1 1 1 1 1 1 1 1 1 shuffle MRWorkshop
Many matrix computations are possible in MapReduce Column sums are easy ! Input Key (i,j) Value Aij Other basic methods ! can use common parallel/out-of-core algs! Sparse matrix-vector products y = Ax Sparse matrix-matrix products C = AB 14 Reduce(j,Values) emit Key = j, Value = sum(Values) David Gleich · Purdue Map((i,j), val) emit" Key = j, Value = val A11 A12 A13 A14 A21 A22 A23 A24 A31 A32 A33 A34 A41 A42 A43 A44 (3,4) -> 5 (1,2) -> -6.0 (2,3) -> -1.2 (1,1) -> 3.14 … “Coordinate storage” MRWorkshop
Many matrix computations are possible in MapReduce Column sums are easy ! Input Key (i,j) Value Aij Other basic methods ! can use common parallel/out-of-core algs! Sparse matrix-vector products y = Ax Sparse matrix-matrix products C = AB 15 Reduce(j,Values) emit Key = j, Value = sum(Values) David Gleich · Purdue Map((i,j), val) emit" Key = j, Value = val A11 A12 A13 A14 A21 A22 A23 A24 A31 A32 A33 A34 A41 A42 A43 A44 (3,4) -> 5 (1,2) -> -6.0 (2,3) -> -1.2 (1,1) -> 3.14 … “Coordinate storage” Beware of un-thoughtful ideas MRWorkshop
Why so many limitations? 16 David Gleich · Purdue MRWorkshop
The MapReduce programming model Input a list of (key, value) pairs Map apply a function f to all pairs Reduce apply a function g to ! all values with key k (for all k) Output a list of (key, value) pairs Map function f must be side-effect free! All map functions run in parallel Reduce function g must be side-effect free! All reduce functions run in parallel 17 David Gleich · Purdue MRWorkshop
A graphical view of the MapReduce programming model David Gleich · Purdue 18 data Map data Map data Map data Map key value key value key value key value key value key value () Shuffle key value value dataReduce key value value value dataReduce key value dataReduce MRWorkshop
Data scalability The idea ! Bring the computations to the data MR can schedule map functions without moving data. 1 M M R R M M M Maps Reduce Shuffle 2 3 4 5 1 2 M M 3 4 M M 5 M 19 David Gleich · Purdue MRWorkshop
After waiting in the queue for a month and ! after 24 hours of finding eigenvalues, one node randomly hiccups. heartbreak on node rs252 David Gleich · Purdue 20 MRWorkshop
Fault tolerant Redundant input helps make maps data-local Just one type of communication: shuffle M M R R M M Input stored in triplicate Map output! persisted to disk! before shuffle Reduce input/! output on disk David Gleich · Purdue 21 MRWorkshop
Fault injection 10 100 1000 1/Prob(failure) – mean number of success per failure Timetocompletion(sec) 200 100 No faults (200M by 200) Faults (800M by 10) Faults (200M by 200) No faults ! (800M by 10) With 1/5 tasks failing, the job only takes twice as long. David Gleich · Purdue 22 MRWorkshop
Data scalability The idea ! Bring the computations to the data MR can schedule map functions without moving data. 1 M M R R M M M Maps Reduce Shuffle 2 3 4 5 1 2 M M 3 4 M M 5 M 23 David Gleich · Purdue MRWorkshop
Computing a histogram ! A simple MapReduce example 24 Input! ! Key ImageId Value Pixels Map(ImageId, Pixels) for each pixel emit" Key = (r,g,b)" Value = 1 Reduce(Color, Values) emit" Key = Color Value = sum(Values) Output! ! Key Color Value ! # of pixels David Gleich · Purdue 5 15 10 9 3 17 5 10 1 1 1 1 Map Reduce 1 1 1 1 1 1 1 1 1 1 1 1 shuffle The entire dataset is “transposed” from images to pixels. This moves the data to the computation! (Using a combiner helps to reduce the data moved, but it cannot always be used) MRWorkshop
Hadoop and MapReduce are bad systems for some matrix computations. David Gleich · Purdue 25 MRWorkshop
How should you evaluate a MapReduce algorithm? Build a performance model! Measure the worst mapper Usually not too bad Measure the data moved Could be very bad Measure the worst reducer Could be very bad David Gleich · Purdue 26 MRWorkshop
Tools I like hadoop streaming dumbo mrjob hadoopy C++ David Gleich · Purdue 27 MRWorkshop
Tools I don’t use but other people seem to like … pig java hbase mahout Eclipse Cassandra David Gleich · Purdue 28 MRWorkshop
hadoop streaming the map function is a program! (key,value) pairs are sent via stdin! output (key,value) pairs goes to stdout the reduce function is a program! (key,value) pairs are sent via stdin! keys are grouped! output (key,value) pairs goes to stdout David Gleich · Purdue 29 MRWorkshop
mrjob from a wrapper around hadoop streaming for map and reduce functions in python class MRWordFreqCount(MRJob): def mapper(self, _, line): for word in line.split(): yield (word.lower(), 1) def reducer(self, word, counts): yield (word, sum(counts)) if __name__ == '__main__': MRWordFreqCount.run() David Gleich · Purdue 30 MRWorkshop
How can Hadoop streaming possibly be fast? Iter 1 QR (secs.) Iter 1 Total (secs.) Iter 2 Total (secs.) Overall Total (secs.) Dumbo 67725 960 217 1177 Hadoopy 70909 612 118 730 C++ 15809 350 37 387 Java 436 66 502 Synthetic data test 100,000,000-by-500 matrix (~500GB) Codes implemented in MapReduce streaming Matrix stored as TypedBytes lists of doubles Python frameworks use Numpy+Atlas Custom C++ TypedBytes reader/writer with Atlas New non-streaming Java implementation too David Gleich (Sandia) All timing results from the Hadoop job tracker C++ in streaming beats a native Java implementation. 16/22MapReduce 2011 David Gleich · Purdue 31 Example available from github.com/dgleich/mrtsqr! for verification mrjob could be faster if it used typedbytes for intermediate storage see https://github.com/Yelp/mrjob/pull/447 MRWorkshop
Code samples and short tutorials at github.com/dgleich/mrmatrix github.com/dgleich/mapreduce-matrix-tutorial David Gleich · Purdue 32 MRWorkshop
Matrix-vector product David Gleich · Purdue 33 Ax = y yi = X k Aik xk A x Follow along! mapreduce-matrix-tutorial! /codes/smatvec.py! MRWorkshop
Matrix-vector product David Gleich · Purdue 34 Ax = y yi = X k Aik xk A x A is stored by row $ head samples/smat_5_5.txt ! 0 0 0.125 3 1.024 4 0.121! 1 0 0.597! 2 2 1.247! 3 4 -1.45! 4 2 0.061! x is stored entry-wise ! $ head samples/vec_5.txt! 0 0.241! 1 -0.98! 2 0.237! 3 -0.32! 4 0.080! Follow along! mapreduce-matrix-tutorial! /codes/smatvec.py! MRWorkshop
Matrix-vector product! (in pictures) David Gleich · Purdue 35 Ax = y yi = X k Aik xk A x A x Input Map 1! Align on columns! Reduce 1! Output Aik xk! keyed on row i A x Reduce 2! Output sum(Aik xk)! y MRWorkshop
Matrix-vector product! (in pictures) David Gleich · Purdue 36 Ax = y yi = X k Aik xk A x A x Input Map 1! Align on columns! def joinmap(self, key, line):! vals = line.split()! if len(vals) == 2:! # the vector! yield (vals[0], # row! (float(vals[1]),)) # xi! else:! # the matrix! row = vals[0]! for i in xrange(1,len(vals),2):! yield (vals[i], # column! (row, # i,Aij! float(vals[i+1])))! MRWorkshop
Matrix-vector product! (in pictures) David Gleich · Purdue 37 Ax = y yi = X k Aik xk A x A x Input Map 1! Align on columns! Reduce 1! Output Aik xk! keyed on row i A x def joinred(self, key, vals):! vecval = 0. ! matvals = []! for val in vals:! if len(val) == 1:! vecval += val[0]! else:! matvals.append(val) ! for val in matvals:! yield (val[0], val[1]*vecval)! Note that you should use a secondary sort to avoid reading both in memory MRWorkshop
Matrix-vector product! (in pictures) David Gleich · Purdue 38 Ax = y yi = X k Aik xk A x A x Input Map 1! Align on columns! Reduce 1! Output Aik xk! keyed on row i A x Reduce 2! Output sum(Aik xk)! y def sumred(self, key, vals):! yield (key, sum(vals))! MRWorkshop
Move the computations to the data? Not really! David Gleich · Purdue 39 A x A x Input Map 1! Align on columns! Reduce 1! Output Aik xk! keyed on row i A x Reduce 2! Output sum(Aik xk)! y Copy data once, now aligned on column Copy data again, align on row MRWorkshop
Matrix-matrix product David Gleich · Purdue 40 A B AB = C Cij = X k Aik Bkj Follow along! mapreduce-matrix-tutorial! /codes/matmat.py! MRWorkshop
Matrix-matrix product David Gleich · Purdue 41 A B AB = C Cij = X k Aik Bkj A is stored by row $ head samples/smat_10_5_A.txt ! 0 0 0.599 4 -1.53! 1! 2 2 0.260! 3! 4 0 0.267 1 0.839 B is stored by row $ head samples/smat_5_5.txt ! 0 0 0.125 3 1.024 4 0.121! 1 0 0.597! 2 2 1.247! Follow along! mapreduce-matrix-tutorial! /codes/matmat.py! MRWorkshop
Matrix-matrix product ! (in pictures) David Gleich · Purdue 42 A B AB = C Cij = X k Aik Bkj A Map 1! Align on columns! B Reduce 1! Output Aik Bkj! keyed on (i,j) A B Reduce 2! Output sum(Aik Bkj)! C MRWorkshop
Matrix-matrix product ! (in code) David Gleich · Purdue 43 A B AB = C Cij = X k Aik Bkj A Map 1! Align on columns! B def joinmap(self, key, line):! mtype = self.parsemat()! vals = line.split()! row = vals[0]! rowvals = ! [(vals[i],float(vals[i+1])) ! for i in xrange(1,len(vals),2)]! if mtype==1:! # matrix A, output by col! for val in rowvals:! yield (val[0], (row, val[1]))! else:! yield (row, (rowvals,))! MRWorkshop
Matrix-matrix product ! (in code) David Gleich · Purdue 44 A B AB = C Cij = X k Aik Bkj A Map 1! Align on columns! B Reduce 1! Output Aik Bkj! keyed on (i,j) A B def joinred(self, key, line):! # load the data into memory ! brow = []! acol = []! for val in vals:! if len(val) == 1:! brow.extend(val[0])! else:! acol.append(val)! ! for (bcol,bval) in brow:! for (arow,aval) in acol:! yield ((arow,bcol),aval*bval)! MRWorkshop
Matrix-matrix product ! (in pictures) David Gleich · Purdue 45 A B AB = C Cij = X k Aik Bkj A Map 1! Align on columns! B Reduce 1! Output Aik Bkj! keyed on (i,j) A B Reduce 2! Output sum(Aik Bkj)! C def sumred(self, key, vals):! yield (key, sum(vals))! MRWorkshop
Why is MapReduce so popular? if (root) {! PetscInt cur_nz=0;! unsigned char* root_nz_buf;! unsigned int *root_nz_buf_i,*root_nz_buf_j;! double *root_nz_buf_v;! PetscMalloc((sizeof(unsigned int)*2+sizeof(double))*root_nz_bufsize,root_nz_buf);! PetscMalloc(sizeof(unsigned int)*root_nz_bufsize,root_nz_buf_i);! PetscMalloc(sizeof(unsigned int)*root_nz_bufsize,root_nz_buf_j);! PetscMalloc(sizeof(double)*root_nz_bufsize,root_nz_buf_v);! ! unsigned long long int nzs_to_read = total_nz;! ! while (send_rounds 0) {! // check if we are near the end of the file! // and just read that amount! size_t cur_nz_read = root_nz_bufsize;! if (cur_nz_read nzs_to_read) {! cur_nz_read = nzs_to_read;! }! PetscInfo2(PETSC_NULL, reading %i non-zeros of %llin, cur_nz_read, nzs_to_read);! 600 lines of gross code in order to load a sparse matrix into memory, streaming from one processor. MapReduce offers a better alternative David Gleich · Purdue 46 MRWorkshop
Thoughts on a better system Default quadruple precision Matrix computations without indexing Easy setup of MPI data jobs David Gleich · Purdue 47                                        Initial data load of any MPI job Compute task MRWorkshop
Double-precision floating point was designed for the era where “big” was 1000-10000 David Gleich · Purdue 48 MRWorkshop
Error analysis of summation s = 0; for i=1 to n: s = s + x[i] A simple summation formula has ! error that is not always small if n is a billion David Gleich · Purdue 49 fl(x + y) = (x + y)(1 + ) fl( X i xi ) X i xi  nµ X i |xi | µ ⇡ 10 16 MRWorkshop
If your application matters then watch out for this issue. Use quad-precision arithmetic or compensated summation instead. David Gleich · Purdue 50 MRWorkshop
Compensated Summation “Kahan summation algorithm” on Wikipedia s = 0.; c = 0.; for i=1 to n: y = x[i] – c t = s + y c = (t – s) – y s = t David Gleich · Purdue 51 Mathematically, c is always zero. On a computer, c can be non-zero The parentheses matter! fl(csum(x)) X i xi  (µ + nµ2 ) X i |xi | µ ⇡ 10 16 MRWorkshop
Summary MapReduce is a powerful but limited tool that has a role in the future of computational math. … but it should be used carefully! See Austin’s talk next! David Gleich · Purdue 52 MRWorkshop Code samples and short tutorials at github.com/dgleich/mrmatrix github.com/dgleich/mapreduce-matrix-tutorial
David Gleich · Purdue 53 MRWorkshop

More Related Content

PPT
Hashing
byAbbas Ali
 
PDF
Harris corner detector and face recognition
byShih Wei Huang
 
PPTX
Binomial Heaps and Fibonacci Heaps
byAmrinder Arora
 
PPTX
CCS334 BIG DATA ANALYTICS UNIT 5 PPT ELECTIVE PAPER
byKrishnaVeni451953
 
PPTX
Apache Spark GraphX & GraphFrame Synthetic ID Fraud Use Case
byMo Patel
 
KEY
Python for cloud computing
byPython Ireland
 
PDF
module4_dynamic programming_2022.pdf
byShiwani Gupta
 
PPTX
Hadoop and Big Data
byHarshdeep Kaur
 
Hashing
byAbbas Ali
 
Harris corner detector and face recognition
byShih Wei Huang
 
Binomial Heaps and Fibonacci Heaps
byAmrinder Arora
 
CCS334 BIG DATA ANALYTICS UNIT 5 PPT ELECTIVE PAPER
byKrishnaVeni451953
 
Apache Spark GraphX & GraphFrame Synthetic ID Fraud Use Case
byMo Patel
 
Python for cloud computing
byPython Ireland
 
module4_dynamic programming_2022.pdf
byShiwani Gupta
 
Hadoop and Big Data
byHarshdeep Kaur
 

What's hot

PPTX
Unit 1 ppt
byGRajendra
 
PDF
Neo4j - 5 cool graph examples
byPeter Neubauer
 
PPTX
Depth-First Search
byDakshitha Dissanayaka
 
PPT
Hadoop HDFS.ppt
by6535ANURAGANURAG
 
PPT
Branch and bound
byDr Shashikant Athawale
 
PPT
Cure, Clustering Algorithm
byLino Possamai
 
PPT
Chapter 12 ds
byHanif Durad
 
PPTX
MapReduce
byTilani Gunawardena PhD(UNIBAS), BSc(Pera), FHEA(UK), CEng, MIESL
 
PDF
HDFS Selective Wire Encryption
byKonstantin V. Shvachko
 
PPTX
Dynamic filtering for presto join optimisation
byOri Reshef
 
PDF
SQL Performance Improvements at a Glance in Apache Spark 3.0
byDatabricks
 
PDF
Linear time sorting algorithms
byDr Sandeep Kumar Poonia
 
PPTX
Process Synchronization - Monitors
byDr.M.Karthika parthasarathy
 
PPT
TabuSearch FINAL
byMustafa RASHID
 
PDF
Buffer cache unix ppt Mrs.Sowmya Jyothi
bySowmya Jyothi
 
PDF
Hadoop 2.0 Architecture | HDFS Federation | NameNode High Availability |
byEdureka!
 
PDF
Spark graphx
byCarol McDonald
 
PDF
Chapter 7: Matrix Multiplication
byHeman Pathak
 
PPTX
Introduction to Apache Spark
byRahul Jain
 
PPT
Hadoop Map Reduce
byVNIT-ACM Student Chapter
 
Unit 1 ppt
byGRajendra
 
Neo4j - 5 cool graph examples
byPeter Neubauer
 
Depth-First Search
byDakshitha Dissanayaka
 
Hadoop HDFS.ppt
by6535ANURAGANURAG
 
Branch and bound
byDr Shashikant Athawale
 
Cure, Clustering Algorithm
byLino Possamai
 
Chapter 12 ds
byHanif Durad
 
MapReduce
byTilani Gunawardena PhD(UNIBAS), BSc(Pera), FHEA(UK), CEng, MIESL
 
HDFS Selective Wire Encryption
byKonstantin V. Shvachko
 
Dynamic filtering for presto join optimisation
byOri Reshef
 
SQL Performance Improvements at a Glance in Apache Spark 3.0
byDatabricks
 
Linear time sorting algorithms
byDr Sandeep Kumar Poonia
 
Process Synchronization - Monitors
byDr.M.Karthika parthasarathy
 
TabuSearch FINAL
byMustafa RASHID
 
Buffer cache unix ppt Mrs.Sowmya Jyothi
bySowmya Jyothi
 
Hadoop 2.0 Architecture | HDFS Federation | NameNode High Availability |
byEdureka!
 
Spark graphx
byCarol McDonald
 
Chapter 7: Matrix Multiplication
byHeman Pathak
 
Introduction to Apache Spark
byRahul Jain
 
Hadoop Map Reduce
byVNIT-ACM Student Chapter
 

Viewers also liked

PDF
Matrix methods for Hadoop
byDavid Gleich
 
PDF
How does Google Google: A journey into the wondrous mathematics behind your f...
byDavid Gleich
 
PPTX
What is sparse matrix
bySaddam Hussain
 
PPT
Sparse Matrix and Polynomial
byAroosa Rajput
 
PDF
Higher-order organization of complex networks
byDavid Gleich
 
PDF
Spacey random walks and higher order Markov chains
byDavid Gleich
 
PDF
Using Local Spectral Methods to Robustify Graph-Based Learning
byDavid Gleich
 
PDF
SPARSE STORAGE RECOMMENDATION SYSTEM FOR SPARSE MATRIX VECTOR MULTIPLICATION ...
byIAEME Publication
 
PPTX
Implementing page rank algorithm using hadoop map reduce
byFarzan Hajian
 
PPTX
Face recognition and math
byKejti Cela
 
PPT
Behm Shah Pagerank
bygothicane
 
PPTX
Sparse matrices
byJonghoon Park
 
PPTX
LINEAR ALGEBRA BEHIND GOOGLE SEARCH
byDivyansh Verma
 
PDF
HADOOP + R
byCdiscount
 
PDF
Direct tall-and-skinny QR factorizations in MapReduce architectures
byDavid Gleich
 
PDF
Gaps between the theory and practice of large-scale matrix-based network comp...
byDavid Gleich
 
PDF
A multithreaded method for network alignment
byDavid Gleich
 
PDF
Tall and Skinny QRs in MapReduce
byDavid Gleich
 
PDF
Iterative methods for network alignment
byDavid Gleich
 
PDF
A history of PageRank from the numerical computing perspective
byDavid Gleich
 
Matrix methods for Hadoop
byDavid Gleich
 
How does Google Google: A journey into the wondrous mathematics behind your f...
byDavid Gleich
 
What is sparse matrix
bySaddam Hussain
 
Sparse Matrix and Polynomial
byAroosa Rajput
 
Higher-order organization of complex networks
byDavid Gleich
 
Spacey random walks and higher order Markov chains
byDavid Gleich
 
Using Local Spectral Methods to Robustify Graph-Based Learning
byDavid Gleich
 
SPARSE STORAGE RECOMMENDATION SYSTEM FOR SPARSE MATRIX VECTOR MULTIPLICATION ...
byIAEME Publication
 
Implementing page rank algorithm using hadoop map reduce
byFarzan Hajian
 
Face recognition and math
byKejti Cela
 
Behm Shah Pagerank
bygothicane
 
Sparse matrices
byJonghoon Park
 
LINEAR ALGEBRA BEHIND GOOGLE SEARCH
byDivyansh Verma
 
HADOOP + R
byCdiscount
 
Direct tall-and-skinny QR factorizations in MapReduce architectures
byDavid Gleich
 
Gaps between the theory and practice of large-scale matrix-based network comp...
byDavid Gleich
 
A multithreaded method for network alignment
byDavid Gleich
 
Tall and Skinny QRs in MapReduce
byDavid Gleich
 
Iterative methods for network alignment
byDavid Gleich
 
A history of PageRank from the numerical computing perspective
byDavid Gleich
 

Similar to Sparse matrix computations in MapReduce

PDF
MapReduce for scientific simulation analysis
byDavid Gleich
 
PDF
Introduction to map reduce
byBhupesh Chawda
 
PPTX
Streaming Python on Hadoop
byVivian S. Zhang
 
PPT
Hadoop classes in mumbai
byVibrant Technologies & Computers
 
PDF
MapReduce and Hadoop
byNicola Cadenelli
 
PPTX
This gives a brief detail about big data
bychinky1118
 
PPTX
Hadoop and MapReduce
byAbhishek Dey
 
PDF
MapReduce Algorithm Design
byGabriela Agustini
 
PPTX
Introduction to Map Reduce
byApache Apex
 
PPT
Seminar Presentation Hadoop
byVarun Narang
 
PPTX
Enar short course
byDeepak Agarwal
 
PDF
Map Reducec and Spark big data visualization and analytics
byitesm
 
PDF
Seminar_Report_hadoop
byVarun Narang
 
PDF
MapReduce basics
byHarisankar H
 
PPTX
Hadoop and Mapreduce for .NET User Group
byCsaba Toth
 
PDF
Distributed Computing with Apache Hadoop. Introduction to MapReduce.
byKonstantin V. Shvachko
 
PPTX
introduction to Complete Map and Reduce Framework
byharikumar288574
 
PDF
Big Data, a space adventure - Mario Cartia - Codemotion Rome 2015
byCodemotion
 
PPTX
COMPLETE MAP AND REDUCE FRAMEWORK INTRODUCTION
byharikumar288574
 
PDF
2 mapreduce-model-principles
byGenoveva Vargas-Solar
 
MapReduce for scientific simulation analysis
byDavid Gleich
 
Introduction to map reduce
byBhupesh Chawda
 
Streaming Python on Hadoop
byVivian S. Zhang
 
Hadoop classes in mumbai
byVibrant Technologies & Computers
 
MapReduce and Hadoop
byNicola Cadenelli
 
This gives a brief detail about big data
bychinky1118
 
Hadoop and MapReduce
byAbhishek Dey
 
MapReduce Algorithm Design
byGabriela Agustini
 
Introduction to Map Reduce
byApache Apex
 
Seminar Presentation Hadoop
byVarun Narang
 
Enar short course
byDeepak Agarwal
 
Map Reducec and Spark big data visualization and analytics
byitesm
 
Seminar_Report_hadoop
byVarun Narang
 
MapReduce basics
byHarisankar H
 
Hadoop and Mapreduce for .NET User Group
byCsaba Toth
 
Distributed Computing with Apache Hadoop. Introduction to MapReduce.
byKonstantin V. Shvachko
 
introduction to Complete Map and Reduce Framework
byharikumar288574
 
Big Data, a space adventure - Mario Cartia - Codemotion Rome 2015
byCodemotion
 
COMPLETE MAP AND REDUCE FRAMEWORK INTRODUCTION
byharikumar288574
 
2 mapreduce-model-principles
byGenoveva Vargas-Solar
 

More from David Gleich

PDF
Engineering Data Science Objectives for Social Network Analysis
byDavid Gleich
 
PDF
Correlation clustering and community detection in graphs and networks
byDavid Gleich
 
PDF
Spectral clustering with motifs and higher-order structures
byDavid Gleich
 
PDF
Spacey random walks and higher-order data analysis
byDavid Gleich
 
PDF
Non-exhaustive, Overlapping K-means
byDavid Gleich
 
PDF
Localized methods in graph mining
byDavid Gleich
 
PDF
PageRank Centrality of dynamic graph structures
byDavid Gleich
 
PDF
Iterative methods with special structures
byDavid Gleich
 
PDF
Big data matrix factorizations and Overlapping community detection in graphs
byDavid Gleich
 
PDF
Anti-differentiating approximation algorithms: A case study with min-cuts, sp...
byDavid Gleich
 
PDF
Localized methods for diffusions in large graphs
byDavid Gleich
 
PDF
Anti-differentiating Approximation Algorithms: PageRank and MinCut
byDavid Gleich
 
PDF
Fast relaxation methods for the matrix exponential
byDavid Gleich
 
PDF
Fast matrix primitives for ranking, link-prediction and more
byDavid Gleich
 
PDF
MapReduce Tall-and-skinny QR and applications
byDavid Gleich
 
PDF
Recommendation and graph algorithms in Hadoop and SQL
byDavid Gleich
 
PDF
Relaxation methods for the matrix exponential on large networks
byDavid Gleich
 
PDF
Personalized PageRank based community detection
byDavid Gleich
 
PDF
Vertex neighborhoods, low conductance cuts, and good seeds for local communit...
byDavid Gleich
 
PDF
A dynamical system for PageRank with time-dependent teleportation
byDavid Gleich
 
Engineering Data Science Objectives for Social Network Analysis
byDavid Gleich
 
Correlation clustering and community detection in graphs and networks
byDavid Gleich
 
Spectral clustering with motifs and higher-order structures
byDavid Gleich
 
Spacey random walks and higher-order data analysis
byDavid Gleich
 
Non-exhaustive, Overlapping K-means
byDavid Gleich
 
Localized methods in graph mining
byDavid Gleich
 
PageRank Centrality of dynamic graph structures
byDavid Gleich
 
Iterative methods with special structures
byDavid Gleich
 
Big data matrix factorizations and Overlapping community detection in graphs
byDavid Gleich
 
Anti-differentiating approximation algorithms: A case study with min-cuts, sp...
byDavid Gleich
 
Localized methods for diffusions in large graphs
byDavid Gleich
 
Anti-differentiating Approximation Algorithms: PageRank and MinCut
byDavid Gleich
 
Fast relaxation methods for the matrix exponential
byDavid Gleich
 
Fast matrix primitives for ranking, link-prediction and more
byDavid Gleich
 
MapReduce Tall-and-skinny QR and applications
byDavid Gleich
 
Recommendation and graph algorithms in Hadoop and SQL
byDavid Gleich
 
Relaxation methods for the matrix exponential on large networks
byDavid Gleich
 
Personalized PageRank based community detection
byDavid Gleich
 
Vertex neighborhoods, low conductance cuts, and good seeds for local communit...
byDavid Gleich
 
A dynamical system for PageRank with time-dependent teleportation
byDavid Gleich
 

Recently uploaded

PPTX
Classification and Applied Aspects of Joints.pptx
byMathew Joseph
 
PPTX
THERAPEUTIC ENVIORNMENT.............pptx
byAneetaSharma15
 
PPTX
Hudson Vitale "AI Essentials: From Tools to Strategies: A 2025 NISO Training ...
byNational Information Standards Organization (NISO)
 
PDF
Slit lamp parts 2/ppt/notes/download.pdf
byAnmol Singh
 
PPTX
🧪“Blood Chemistry Tests in Pharmacy Practice: Clinical Laboratory Guide for P...
byBanny S.V
 
PDF
Nernst Distribution Law and factors affecting distribution constant
byMithil Fal Desai
 
PDF
BD1TL - TNTEU - Teaching and Learning - Unit - 2.pdf
byDr Raja Mohammed T
 
PPTX
ILA 2025 Prague CLS Presentation of Leadership journal
byjohanalvehus
 
PPTX
DO 34 s 2025 - ammendment of DO 24 s 2025.pptx
byJeanalynEstrellado3
 
PDF
History of Department of AIHC and Archaeology_BHU
byBanaras Hindu University
 
PPTX
How to Create a Manifest File in Odoo 18
byCeline George
 
PPTX
Safety Needs and Prevention of environmental hazards.pptx
byAneetaSharma15
 
PPTX
DBP - BITA Introduction Slides Oct 202526
byGreat Files
 
PPTX
GAS chromatography ppt (Presentation) by Jatin Chauhan
byjatinchauhan1618
 
PPTX
Agriculture Lesson Note for Grade 11 Chapter 3 & 4.pptx
byNajibMuhidin
 
DOCX
Extension Education: From theory to practice by Dr Subin K Mohan.docx
byDrSubinKMohan
 
PDF
BUSINESS ETHICS – UNIT V: Ethical Insights from Thirukkural and Modern Thought
byD NANEE
 
PDF
BUSINESS ETHICS – UNIT I: Introduction to Business Ethics
byD NANEE
 
PDF
The purpose of Ethics and Values in a project context webinar, 14 October 202...
byAssociation for Project Management
 
PDF
Yaksha Prashna | General Quiz at RLAC | Amlan Sarkar | Full Set
byAmlan Sarkar
 
Classification and Applied Aspects of Joints.pptx
byMathew Joseph
 
THERAPEUTIC ENVIORNMENT.............pptx
byAneetaSharma15
 
Hudson Vitale "AI Essentials: From Tools to Strategies: A 2025 NISO Training ...
byNational Information Standards Organization (NISO)
 
Slit lamp parts 2/ppt/notes/download.pdf
byAnmol Singh
 
🧪“Blood Chemistry Tests in Pharmacy Practice: Clinical Laboratory Guide for P...
byBanny S.V
 
Nernst Distribution Law and factors affecting distribution constant
byMithil Fal Desai
 
BD1TL - TNTEU - Teaching and Learning - Unit - 2.pdf
byDr Raja Mohammed T
 
ILA 2025 Prague CLS Presentation of Leadership journal
byjohanalvehus
 
DO 34 s 2025 - ammendment of DO 24 s 2025.pptx
byJeanalynEstrellado3
 
History of Department of AIHC and Archaeology_BHU
byBanaras Hindu University
 
How to Create a Manifest File in Odoo 18
byCeline George
 
Safety Needs and Prevention of environmental hazards.pptx
byAneetaSharma15
 
DBP - BITA Introduction Slides Oct 202526
byGreat Files
 
GAS chromatography ppt (Presentation) by Jatin Chauhan
byjatinchauhan1618
 
Agriculture Lesson Note for Grade 11 Chapter 3 & 4.pptx
byNajibMuhidin
 
Extension Education: From theory to practice by Dr Subin K Mohan.docx
byDrSubinKMohan
 
BUSINESS ETHICS – UNIT V: Ethical Insights from Thirukkural and Modern Thought
byD NANEE
 
BUSINESS ETHICS – UNIT I: Introduction to Business Ethics
byD NANEE
 
The purpose of Ethics and Values in a project context webinar, 14 October 202...
byAssociation for Project Management
 
Yaksha Prashna | General Quiz at RLAC | Amlan Sarkar | Full Set
byAmlan Sarkar
 
In this document
Powered by AI

Overview of the ICME MapReduce Workshop, its goals, and the schedule of topics to be covered.

Techniques for handling large matrix data and processing capabilities of different supercomputers.

Introduction to MapReduce concepts. The programming model involves input pairs and functions for processing.

Examples of matrix computations using MapReduce, including sparse matrix-vector and matrix-matrix products.

Discusses the limitations of MapReduce, data scalability, fault tolerance, and the efficiency in computation.

Practical examples of coding in MapReduce and performance considerations using different programming tools.

Evaluating MapReduce algorithms by building models and measuring performance metrics for effectiveness.

Detailed methodology for performing matrix-vector and matrix-matrix products in MapReduce.

Error analysis and solutions in arithmetic computations for precision in data processing.

Final thoughts on the utility of MapReduce in computational math, emphasizing care in its application.

Sparse matrix computations in MapReduce

  • 1.
    ICME MapReduce Workshop! April29 – May 1, 2013! ! David F. Gleich! Computer Science! Purdue University David Gleich · Purdue 1 ! Website www.stanford.edu/~paulcon/icme-mapreduce-2013 Paul G. Constantine! Center for Turbulence Research! Stanford University MRWorkshop
  • 2.
    Goals Learn the basicsof MapReduce & Hadoop Be able to process large volumes of data from science and engineering applications … help enable you to explore on your own! David Gleich · Purdue 2 MRWorkshop
  • 3.
    Workshop overview Monday! Me! Sparsematrix computations in MapReduce! Austin Benson Tall-and-skinny matrix computations in MapReduce Tuesday! Joe Buck Extending MapReduce for scientific computing! Chunsheng Feng Large scale video analytics on pivotal Hadoop Wednesday! Joe Nichols Post-processing CFD dynamics data in MapReduce ! Lavanya Ramakrishnan Evaluating MapReduce and Hadoop for science David Gleich · Purdue 3 MRWorkshop
  • 4.
    Sparse matrix computations inMapReduce! David F. Gleich! Computer Science! Purdue University David Gleich · Purdue 4 Slides online soon! Code https://github.com/dgleich/mapreduce-matrix-tutorial MRWorkshop
  • 5.
    How to computewith big matrix data ! A tale of two computers 224k Cores 10 PB drive 1.7 Pflops 7 MW Custom ! interconnect! $104 M 80k cores! 50 PB drive ? Pflops ? MW GB ethernet $?? M 625 GB/core! High disk to CPU 45 GB/core High CPU to disk 5 ORNL 2010 Supercomputer! Google’s 2010? ! Data computer! David Gleich · Purdue MRWorkshop
  • 6.
    My data computers 6 Nebula Cluster @ Sandia CA! 2TB/core storage, 64 nodes, 256 cores, GB ethernet Cost $150k These systems are good for working with enormous matrix data! ICME Hadoop @ Stanford! 3TB/core storage, 11 nodes, 44 cores, GB ethernet Cost $30k David Gleich · Purdue MRWorkshop
  • 7.
    My data computers 7 Nebula Cluster @ Sandia CA! 2TB/core storage, 64 nodes, 256 cores, GB ethernet Cost $150k These systems are good for working with enormous matrix data! ICME Hadoop @ Stanford! 3TB/core storage, 11 nodes, 44 cores, GB ethernet Cost $30k ^ but not great, David Gleich · Purdue some ^ MRWorkshop
  • 8.
    By 2013(?) allFortune 500 companies will have a data computer David Gleich · Purdue 8 MRWorkshop
  • 9.
    How do youprogram them? 9 David Gleich · Purdue MRWorkshop
  • 10.
    MapReduce and! Hadoop overview 10 DavidGleich · Purdue MRWorkshop
  • 11.
    MapReduce is designedto solve a different set of problems from standard parallel libraries 11 David Gleich · Purdue MRWorkshop
  • 12.
    The MapReduce programming model Inputa list of (key, value) pairs Map apply a function f to all pairs Reduce apply a function g to ! all values with key k (for all k) Output a list of (key, value) pairs 12 David Gleich · Purdue MRWorkshop
  • 13.
    Computing a histogram! A simple MapReduce example 13 Input! ! Key ImageId Value Pixels Map(ImageId, Pixels) for each pixel emit" Key = (r,g,b)" Value = 1 Reduce(Color, Values) emit" Key = Color Value = sum(Values) Output! ! Key Color Value ! # of pixels David Gleich · Purdue 5 15 10 9 3 17 5 10 1 1 1 1 Map Reduce 1 1 1 1 1 1 1 1 1 1 1 1 shuffle MRWorkshop
  • 14.
    Many matrix computations arepossible in MapReduce Column sums are easy ! Input Key (i,j) Value Aij Other basic methods ! can use common parallel/out-of-core algs! Sparse matrix-vector products y = Ax Sparse matrix-matrix products C = AB 14 Reduce(j,Values) emit Key = j, Value = sum(Values) David Gleich · Purdue Map((i,j), val) emit" Key = j, Value = val A11 A12 A13 A14 A21 A22 A23 A24 A31 A32 A33 A34 A41 A42 A43 A44 (3,4) -> 5 (1,2) -> -6.0 (2,3) -> -1.2 (1,1) -> 3.14 … “Coordinate storage” MRWorkshop
  • 15.
    Many matrix computations arepossible in MapReduce Column sums are easy ! Input Key (i,j) Value Aij Other basic methods ! can use common parallel/out-of-core algs! Sparse matrix-vector products y = Ax Sparse matrix-matrix products C = AB 15 Reduce(j,Values) emit Key = j, Value = sum(Values) David Gleich · Purdue Map((i,j), val) emit" Key = j, Value = val A11 A12 A13 A14 A21 A22 A23 A24 A31 A32 A33 A34 A41 A42 A43 A44 (3,4) -> 5 (1,2) -> -6.0 (2,3) -> -1.2 (1,1) -> 3.14 … “Coordinate storage” Beware of un-thoughtful ideas MRWorkshop
  • 16.
    Why so manylimitations? 16 David Gleich · Purdue MRWorkshop
  • 17.
    The MapReduce programming model Inputa list of (key, value) pairs Map apply a function f to all pairs Reduce apply a function g to ! all values with key k (for all k) Output a list of (key, value) pairs Map function f must be side-effect free! All map functions run in parallel Reduce function g must be side-effect free! All reduce functions run in parallel 17 David Gleich · Purdue MRWorkshop
  • 18.
    A graphical viewof the MapReduce programming model David Gleich · Purdue 18 data Map data Map data Map data Map key value key value key value key value key value key value () Shuffle key value value dataReduce key value value value dataReduce key value dataReduce MRWorkshop
  • 19.
    Data scalability The idea! Bring the computations to the data MR can schedule map functions without moving data. 1 M M R R M M M Maps Reduce Shuffle 2 3 4 5 1 2 M M 3 4 M M 5 M 19 David Gleich · Purdue MRWorkshop
  • 20.
    After waiting inthe queue for a month and ! after 24 hours of finding eigenvalues, one node randomly hiccups. heartbreak on node rs252 David Gleich · Purdue 20 MRWorkshop
  • 21.
    Fault tolerant Redundant inputhelps make maps data-local Just one type of communication: shuffle M M R R M M Input stored in triplicate Map output! persisted to disk! before shuffle Reduce input/! output on disk David Gleich · Purdue 21 MRWorkshop
  • 22.
    Fault injection 10 100 1000 1/Prob(failure) – mean number of success per failure Timetocompletion(sec) 200 100 No faults (200M by 200) Faults (800M by 10) Faults (200M by 200) No faults ! (800M by 10) With 1/5 tasks failing, the job only takes twice as long. David Gleich · Purdue 22 MRWorkshop
  • 23.
    Data scalability The idea! Bring the computations to the data MR can schedule map functions without moving data. 1 M M R R M M M Maps Reduce Shuffle 2 3 4 5 1 2 M M 3 4 M M 5 M 23 David Gleich · Purdue MRWorkshop
  • 24.
    Computing a histogram! A simple MapReduce example 24 Input! ! Key ImageId Value Pixels Map(ImageId, Pixels) for each pixel emit" Key = (r,g,b)" Value = 1 Reduce(Color, Values) emit" Key = Color Value = sum(Values) Output! ! Key Color Value ! # of pixels David Gleich · Purdue 5 15 10 9 3 17 5 10 1 1 1 1 Map Reduce 1 1 1 1 1 1 1 1 1 1 1 1 shuffle The entire dataset is “transposed” from images to pixels. This moves the data to the computation! (Using a combiner helps to reduce the data moved, but it cannot always be used) MRWorkshop
  • 25.
    Hadoop and MapReduceare bad systems for some matrix computations. David Gleich · Purdue 25 MRWorkshop
  • 26.
    How should youevaluate a MapReduce algorithm? Build a performance model! Measure the worst mapper Usually not too bad Measure the data moved Could be very bad Measure the worst reducer Could be very bad David Gleich · Purdue 26 MRWorkshop
  • 27.
    Tools I like hadoopstreaming dumbo mrjob hadoopy C++ David Gleich · Purdue 27 MRWorkshop
  • 28.
    Tools I don’tuse but other people seem to like … pig java hbase mahout Eclipse Cassandra David Gleich · Purdue 28 MRWorkshop
  • 29.
    hadoop streaming the mapfunction is a program! (key,value) pairs are sent via stdin! output (key,value) pairs goes to stdout the reduce function is a program! (key,value) pairs are sent via stdin! keys are grouped! output (key,value) pairs goes to stdout David Gleich · Purdue 29 MRWorkshop
  • 30.
    mrjob from awrapper around hadoop streaming for map and reduce functions in python class MRWordFreqCount(MRJob): def mapper(self, _, line): for word in line.split(): yield (word.lower(), 1) def reducer(self, word, counts): yield (word, sum(counts)) if __name__ == '__main__': MRWordFreqCount.run() David Gleich · Purdue 30 MRWorkshop
  • 31.
    How can Hadoopstreaming possibly be fast? Iter 1 QR (secs.) Iter 1 Total (secs.) Iter 2 Total (secs.) Overall Total (secs.) Dumbo 67725 960 217 1177 Hadoopy 70909 612 118 730 C++ 15809 350 37 387 Java 436 66 502 Synthetic data test 100,000,000-by-500 matrix (~500GB) Codes implemented in MapReduce streaming Matrix stored as TypedBytes lists of doubles Python frameworks use Numpy+Atlas Custom C++ TypedBytes reader/writer with Atlas New non-streaming Java implementation too David Gleich (Sandia) All timing results from the Hadoop job tracker C++ in streaming beats a native Java implementation. 16/22MapReduce 2011 David Gleich · Purdue 31 Example available from github.com/dgleich/mrtsqr! for verification mrjob could be faster if it used typedbytes for intermediate storage see https://github.com/Yelp/mrjob/pull/447 MRWorkshop
  • 32.
    Code samples andshort tutorials at github.com/dgleich/mrmatrix github.com/dgleich/mapreduce-matrix-tutorial David Gleich · Purdue 32 MRWorkshop
  • 33.
    Matrix-vector product David Gleich· Purdue 33 Ax = y yi = X k Aik xk A x Follow along! mapreduce-matrix-tutorial! /codes/smatvec.py! MRWorkshop
  • 34.
    Matrix-vector product David Gleich· Purdue 34 Ax = y yi = X k Aik xk A x A is stored by row $ head samples/smat_5_5.txt ! 0 0 0.125 3 1.024 4 0.121! 1 0 0.597! 2 2 1.247! 3 4 -1.45! 4 2 0.061! x is stored entry-wise ! $ head samples/vec_5.txt! 0 0.241! 1 -0.98! 2 0.237! 3 -0.32! 4 0.080! Follow along! mapreduce-matrix-tutorial! /codes/smatvec.py! MRWorkshop
  • 35.
    Matrix-vector product! (in pictures) DavidGleich · Purdue 35 Ax = y yi = X k Aik xk A x A x Input Map 1! Align on columns! Reduce 1! Output Aik xk! keyed on row i A x Reduce 2! Output sum(Aik xk)! y MRWorkshop
  • 36.
    Matrix-vector product! (in pictures) DavidGleich · Purdue 36 Ax = y yi = X k Aik xk A x A x Input Map 1! Align on columns! def joinmap(self, key, line):! vals = line.split()! if len(vals) == 2:! # the vector! yield (vals[0], # row! (float(vals[1]),)) # xi! else:! # the matrix! row = vals[0]! for i in xrange(1,len(vals),2):! yield (vals[i], # column! (row, # i,Aij! float(vals[i+1])))! MRWorkshop
  • 37.
    Matrix-vector product! (in pictures) DavidGleich · Purdue 37 Ax = y yi = X k Aik xk A x A x Input Map 1! Align on columns! Reduce 1! Output Aik xk! keyed on row i A x def joinred(self, key, vals):! vecval = 0. ! matvals = []! for val in vals:! if len(val) == 1:! vecval += val[0]! else:! matvals.append(val) ! for val in matvals:! yield (val[0], val[1]*vecval)! Note that you should use a secondary sort to avoid reading both in memory MRWorkshop
  • 38.
    Matrix-vector product! (in pictures) DavidGleich · Purdue 38 Ax = y yi = X k Aik xk A x A x Input Map 1! Align on columns! Reduce 1! Output Aik xk! keyed on row i A x Reduce 2! Output sum(Aik xk)! y def sumred(self, key, vals):! yield (key, sum(vals))! MRWorkshop
  • 39.
    Move the computationsto the data? Not really! David Gleich · Purdue 39 A x A x Input Map 1! Align on columns! Reduce 1! Output Aik xk! keyed on row i A x Reduce 2! Output sum(Aik xk)! y Copy data once, now aligned on column Copy data again, align on row MRWorkshop
  • 40.
    Matrix-matrix product David Gleich· Purdue 40 A B AB = C Cij = X k Aik Bkj Follow along! mapreduce-matrix-tutorial! /codes/matmat.py! MRWorkshop
  • 41.
    Matrix-matrix product David Gleich· Purdue 41 A B AB = C Cij = X k Aik Bkj A is stored by row $ head samples/smat_10_5_A.txt ! 0 0 0.599 4 -1.53! 1! 2 2 0.260! 3! 4 0 0.267 1 0.839 B is stored by row $ head samples/smat_5_5.txt ! 0 0 0.125 3 1.024 4 0.121! 1 0 0.597! 2 2 1.247! Follow along! mapreduce-matrix-tutorial! /codes/matmat.py! MRWorkshop
  • 42.
    Matrix-matrix product ! (inpictures) David Gleich · Purdue 42 A B AB = C Cij = X k Aik Bkj A Map 1! Align on columns! B Reduce 1! Output Aik Bkj! keyed on (i,j) A B Reduce 2! Output sum(Aik Bkj)! C MRWorkshop
  • 43.
    Matrix-matrix product ! (incode) David Gleich · Purdue 43 A B AB = C Cij = X k Aik Bkj A Map 1! Align on columns! B def joinmap(self, key, line):! mtype = self.parsemat()! vals = line.split()! row = vals[0]! rowvals = ! [(vals[i],float(vals[i+1])) ! for i in xrange(1,len(vals),2)]! if mtype==1:! # matrix A, output by col! for val in rowvals:! yield (val[0], (row, val[1]))! else:! yield (row, (rowvals,))! MRWorkshop
  • 44.
    Matrix-matrix product ! (incode) David Gleich · Purdue 44 A B AB = C Cij = X k Aik Bkj A Map 1! Align on columns! B Reduce 1! Output Aik Bkj! keyed on (i,j) A B def joinred(self, key, line):! # load the data into memory ! brow = []! acol = []! for val in vals:! if len(val) == 1:! brow.extend(val[0])! else:! acol.append(val)! ! for (bcol,bval) in brow:! for (arow,aval) in acol:! yield ((arow,bcol),aval*bval)! MRWorkshop
  • 45.
    Matrix-matrix product ! (inpictures) David Gleich · Purdue 45 A B AB = C Cij = X k Aik Bkj A Map 1! Align on columns! B Reduce 1! Output Aik Bkj! keyed on (i,j) A B Reduce 2! Output sum(Aik Bkj)! C def sumred(self, key, vals):! yield (key, sum(vals))! MRWorkshop
  • 46.
    Why is MapReduceso popular? if (root) {! PetscInt cur_nz=0;! unsigned char* root_nz_buf;! unsigned int *root_nz_buf_i,*root_nz_buf_j;! double *root_nz_buf_v;! PetscMalloc((sizeof(unsigned int)*2+sizeof(double))*root_nz_bufsize,root_nz_buf);! PetscMalloc(sizeof(unsigned int)*root_nz_bufsize,root_nz_buf_i);! PetscMalloc(sizeof(unsigned int)*root_nz_bufsize,root_nz_buf_j);! PetscMalloc(sizeof(double)*root_nz_bufsize,root_nz_buf_v);! ! unsigned long long int nzs_to_read = total_nz;! ! while (send_rounds 0) {! // check if we are near the end of the file! // and just read that amount! size_t cur_nz_read = root_nz_bufsize;! if (cur_nz_read nzs_to_read) {! cur_nz_read = nzs_to_read;! }! PetscInfo2(PETSC_NULL, reading %i non-zeros of %llin, cur_nz_read, nzs_to_read);! 600 lines of gross code in order to load a sparse matrix into memory, streaming from one processor. MapReduce offers a better alternative David Gleich · Purdue 46 MRWorkshop
  • 47.
    Thoughts on abetter system Default quadruple precision Matrix computations without indexing Easy setup of MPI data jobs David Gleich · Purdue 47                                        Initial data load of any MPI job Compute task MRWorkshop
  • 48.
    Double-precision floating point wasdesigned for the era where “big” was 1000-10000 David Gleich · Purdue 48 MRWorkshop
  • 49.
    Error analysis ofsummation s = 0; for i=1 to n: s = s + x[i] A simple summation formula has ! error that is not always small if n is a billion David Gleich · Purdue 49 fl(x + y) = (x + y)(1 + ) fl( X i xi ) X i xi  nµ X i |xi | µ ⇡ 10 16 MRWorkshop
  • 50.
    If your applicationmatters then watch out for this issue. Use quad-precision arithmetic or compensated summation instead. David Gleich · Purdue 50 MRWorkshop
  • 51.
    Compensated Summation “Kahan summationalgorithm” on Wikipedia s = 0.; c = 0.; for i=1 to n: y = x[i] – c t = s + y c = (t – s) – y s = t David Gleich · Purdue 51 Mathematically, c is always zero. On a computer, c can be non-zero The parentheses matter! fl(csum(x)) X i xi  (µ + nµ2 ) X i |xi | µ ⇡ 10 16 MRWorkshop
  • 52.
    Summary MapReduce is apowerful but limited tool that has a role in the future of computational math. … but it should be used carefully! See Austin’s talk next! David Gleich · Purdue 52 MRWorkshop Code samples and short tutorials at github.com/dgleich/mrmatrix github.com/dgleich/mapreduce-matrix-tutorial
  • 53.
    David Gleich ·Purdue 53 MRWorkshop