Temporal Graph Networks

Description

The temporal_graph_networks (TGNs) are a type of graph neural network (GNN) for dynamic graphs. In recent years, GNNs have become very popular due to their ability to perform a wide variety of machine learning tasks on graphs, such as link prediction, node classification, and so on. This rise started with Graph convolutional networks (GCN) introduced by Kipf et al., followed by GraphSAGE introduced by Hamilton et al., and recently a new method was presented which introduces attention mechanism to graphs, known as Graph attention networks (GAT), by Veličković et al. The last two methods offer a great possibility for inductive learning. But they haven't been specifically developed to handle different events occurring on graphs, such as node features updates, node deletion, edge deletion and so on. These events happen regularly in real-world examples such as the Twitter network, where users update their profile, delete their profile or just unfollow another user.

In their work, Rossi et al. introduce Temporal graph networks, an architecture for machine learning on streamed graphs, a rapidly-growing ML use case.

There exist two main modalities of dynamic graphs. Discrete-time dynamic graphs (DTDG) represent snapshots of static graphs taken at intervals in time. On the other hand, continuous-time dynamic graphs (CTDG) are more general and can be represented as timestamped lists of events which may include edge addition or deletion, node addition or deletion, and node or edge feature transformations.

Temporal graph networks is a modality of CTDG that consists of temporal (timestamped) events. An interaction event represents a temporal edge creation, and a nodewise event represents node creation or feature update.

Materials

• Firstly, read the original paper.
• Afterward, you can also watch an explanation of temporal graph networks.
• Be sure to check our explanation of node embeddings if you still have a problem with understanding what node embeddings are.

And finally, we prepared an example of link prediction on the Amazon dataset.

Implementation

Temporal graph networks are implemented as part of the MAGE project. Make sure to check it out in the link above. ☝️

Use cases

Temporal graph networks can be used to find communities in social networks like Twitter, Facebook, and so on. To find communities, we can use semi-supervised learning, and from graph structure and available labels, TGN will try to predict new labels and find communities that way.

We can use temporal graph networks in the case of link prediction. In the link prediction task, TGN needs to correctly predict which edges won't appear - positive samples and which edges will not appear - negative samples. We use the edges that arrive in batches of a stream as a positive sample, and we artificially create negative samples. The Negative samples consist of nodes that are not linked. Now, all that is left is to fire up TGN in self_supervised mode and it will do the work for you.

Furthermore, we can use temporal graph networks for the task of node classification. Again, with edge features, node features, and graph structure, we will try to predict labels. In social networks, labels may indicate interests, beliefs, or demographics.