17.5 Using the Unsupervised EdgeWise Algorithm

UnsupervisedEdgeWise is an inductive edge representation learning algorithm which is able to leverage vertex and edge feature information. It can be applied to a wide variety of tasks, including unsupervised learning edge embeddings for edge classification.

Unsupervised EdgeWise is based on top of the GraphWise model, leveraging the source vertex embedding and the destination vertex embedding generated by the GraphWise model to generate inductive edge embeddings.

The training is based on Deep Graph Infomax (DGI) by Velickovic et al.

Model Structure

An UnsupervisedEdgeWise model consists of graph convolutional layers followed by an embedding layer which defaults to a DGI layer.

First, the source and destination vertices of the target edge are processed through the convolutional layers. The forward pass through a convolutional layer for a vertex proceeds as follows:

1. A set of neighbors of the vertex is sampled.
2. The previous layer representations of the neighbors are mean-aggregated, and the aggregated features are concatenated with the previous layer representation of the vertex.
3. This concatenated vector is multiplied with weights, and a bias vector is added.
4. The result is normalized such that the layer output has unit norm.

The edge embedding layer concatenates the source vertex embedding, the edge features and the destination vertex embedding, and then forwards it through a linear layer to get the edge embedding.

The DGI Layer consists of three parts enabling unsupervised learning using embeddings produced by the convolution layers.

1. Corruption function: Shuffles the node features while preserving the graph structure to produce negative embedding samples using the convolution layers.
2. Readout function: Sigmoid activated mean of embeddings, used as summary of a graph.
3. Discriminator: Measures the similarity of positive (unshuffled) embeddings with the summary as well as the similarity of negative samples with the summary from which the loss function is computed.

Since none of these contains mutable hyperparameters, the default DGI layer is always used and cannot be adjusted.

The second embedding layer available is the Dominant Layer, based on Deep Anomaly Detection on Attributed Networks (Dominant) by Ding, Kaize, et al.

Dominant is a model that detects anomalies based on the features and the neighbors' structure. Using GCNs to reconstruct the features in an autoencoder's settings, and the mask with the dot products of the embeddings.

The loss function is computed from the feature reconstruction loss and the structure reconstruction loss. The importance given to features or to the structure can be tuned with the alpha hyperparameter.

The following describes the usage of the main functionalities of UnsupervisedEdgeWise in PGX using the Movielens graph as an example.

• Loading a Graph
  
• Building a Minimal Unsupervised EdgeWise Model
  
• Advanced Hyperparameter Customization
  
• Supported Property Types for Unsupervised EdgeWise Model
  
• Applying Unsupervised EdgeWise for Partitioned Graphs
  
• Setting the Edge Combination Production Method
  
• Training the Unsupervised EdgeWise Model
  
• Getting the Loss Value for an Unsupervised EdgeWise Model
  
• Inferring Embeddings for an Unsupervised EdgeWise Model
  
• Storing an Unsupervised EdgeWise Model
  
• Loading a Pre-Trained Unsupervised EdgeWise Model
  
• Destroying an Unsupervised Anomaly Detection GraphWise Model
  
• Example: Computing Edge Embeddings on the Movielens Dataset