Downloaded 11 times
Uploaded byDing Li
Machine learning with graph
This document provides an overview of machine learning with graphs. It discusses graph neural networks and deep learning in graphs. It covers representing graphs using adjacency matrices and lists. It also discusses node and graph level features, as well as node embeddings using random walks. Finally, it summarizes several graph neural network models like GCN and GraphSAGE and their applications to citation networks, social networks, and knowledge graphs.
More Related Content
Similar to Machine learning with graph
Machine learning with graph
- 1. Machine Learning withGraph Graph Neural Networks (GNN) Ding Li 2021.10
- 2. 2 CS224W: Machine Learningwith Graphs Prof. Jure Leskovec @ Stanford
- 3. 3 Deep Learning inGraph
- 4.
- 5.
- 6. 6 Representing Graph 1. AdjacencyMatrix 2. Edge List 3. Adjacency List
- 7.
- 8.
- 9. 9 Node Features: ClusteringCoefficient & Graphlets Clustering Coefficient
- 10.
- 11.
- 12.
- 13. 13 Weisfeiler-Lehman Kernel (ColorRefinement, computationally efficient)
- 14. 14 Node Embeddings: Encoderand Decoder
- 15. 15 Random Walk forNode Embeddings
- 16. 16 Random Walk Optimization& Stochastic Gradient Descent Stochastic Gradient Descent Perozzi 2014
- 17. 17 Node2vec: Biased Walks Grover2016 Breadth First Search Depth First Search
- 18. 18 Embedding Entire Graphs Approach3: Anonymous Walk Embeddings
- 19.
- 20.
- 21.
- 22.
- 23.
- 24. 24 Network Embedding asMatrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec Qiu 2018
- 25.
- 26.
- 27.
- 28.
- 29. 29 A: adjacency matrix Modelcan be of arbitrary depth
- 30.
- 31.
- 32. 32 Graphic Convolutional Networks(GCN) Kipf 2017
- 33. 33 Kipf 2017 Adjacency matrix Degreematrix Input-to-hidden weight matrix with H feature hidden-to-output weight matrix with F channels Citation networks: Citeseer, Cora and Pubmed Features: bag-of-words feature vectors Label: each document has a label, total 20 labels NELL: a dataset extracted from a knowledge graph Separated relation nodes were assigned for each entity (e1,r,e2) →(e1, r1), (e2,r2) Features: bag-of-words feature vectors Label: node label Propagation of feature information from neighboring nodes in every layer improves classification performance in comparison to previous methods, where only label information is aggregated. Results for all other baseline methods are taken from the Planetoid paper (Yang et al., 2016). Planetoid* denotes the best model for the respective dataset out of the variants presented in their paper.
- 34. 34 Hamilton 2018 Citation data:predicting paper subject categories (6) on a large citation dataset Features: i node degrees; ii word embedding on abstract Reddit data: predicting which community different Reddit posts belong to Posts were connected if the same user comments on both Features: i embedding of title; ii embedding on comments; iii post’s score; iv number of comments Protein-protein interactions: predicting the protein roles (121) in terms of their cellular functions Features: i positional gene sets; ii motif gene sets; iii immunological signatures
- 35. 35 Ying 2018 I (2billion pins) ↔ C (1 billion board) Method Hit-rate Visual embeddings (4,096 dimensions, from CNN) 17% Annotation embeddings (256 dim, title & description -> Word2Vec) 14% Combined embeddings (2-layer MLP on visual and annotation embeddings) 27% Pixie (random-walk-based, closeness only from graph structure) - PinSage (graph convolution with visual and annotation features) 67% Hit-rate: probability that positive samples were ranked among the top 500 among the 5M negative samples Importance pooling: based upon random walk similarity to choose positive sampling, leading to a 46% performance gain in offline evaluation metrics. Curriculum training: the algorithm is fed harder-and-harder examples (from PageRank score) during training, resulting in a 12% performance gain. Pinterest A/B tests show 30% to 100% improvements in user engagement across various settings after deploying PinSage
- 36. 36 Velickovic 2018 K multi-headattention Concatenation Averaging (final layer) GitHub
- 37. 37 For the inductivetask, two unseen test graphs were tested.
- 38.
- 39.
- 40.
- 41.
- 42. 42 DeepSNAP provides coremodules for this pipeline GraphGym further implements the full pipeline to facilitate GNN design
- 43.
- 44.
- 45.
- 46.
- 47.
- 48.
- 49.
- 50.
- 51.
- 52. 52 Block Diagonal Matrices BasisLearning
- 53.
- 54.
- 55.
- 56.
- 57.
- 58.
- 59.
- 60.
- 61.
- 62. 62 Lin 2015 KG Embeddingsin Practice
- 63.
- 64.
- 65.
- 66.
- 67.
- 68.
- 69.
- 70. 70 z-score: classify subgraph“significance” • Negative: under-representation • Positive: over-representation
- 71.
- 72.
- 73.
- 74.
- 75.
- 76.
- 77.
- 78.
- 79.
- 80.
- 81.
- 82.
- 83. 83 EPINIONS: who-trusts-whom socialnetwork Leskovec 2010
- 84.
- 85.
- 86.
- 87.
- 88.
- 89.
- 90.
- 91.
- 92.
- 93.
- 94. 94 Wu 2019 The accuracyof Simple Graph Convolution (SGC) is similar as GCN’s
- 95.
- 96.
- 97.
- 98. 98 CS224W Colab • Colab0: Networkx, PyTorch Geometric, & GCN • Colab 1: Node Embeddings • Colab 2: GCN Implementation • Colab 3: GraphSAGE Implementation