1 Mar 2019

• Applied GCN autoencoder matrix completion algorithm to this SNAP facebook dataset
  • Best results found in literature do well with the problem framed as a network completion problem. Network completion problem: dataset split into observed and unobserved nodes.
  • Good results obtained when framing the problem as a link prediction problem. However no literature to compare results with for link prediction setting.
  • Yet to be investigated: Removing the feature data and relying only on graph structure to predict edges.
• Normalisation option added, but produces worse results, so it is not applied.

15 Feb 2019

To do:

• Find another domain (e.g. biomedical) to apply this framework of graph autoencoder matrix factorisation and completion. Possible domains:
  • Predict friendships in social media graphs. Currently found 2-3 research papers to compare results with.
  • Predict disease classes in incomplete datasets, as in this paper. The ADNI dataset consists of biomarkers for patients and has incomplete features. The aim is to predict the conversion of mild cognitive impairment to Alzheimer’s Disease.
• Add normalisation on edges.

22 Jan 2019

• Updated model that includes side features, such that it can now use multiple graph convolution layers.
• Working on interim report.

22 Dec 2018

• RGGCN doesn’t perform very well using 3 or more layers. Best results so far using 1 or 2 layers. Testing done on Douban dataset so far.

26 Nov 2018

Progress:

• Corrected a bug in the code that caused some wrong labels to be fed to the model while training and testing. With this fix, RGGCN performs much better. Experimental results to come.

To do:

• Perform hyperparameter search to optimise RGGCN architecture for the various datasets.

12 Nov 2018

Progress:

• Experimented with simple feed forward layers as user and item embedding extractors - results currently beat RGGCN though still worse than GCMC.

To do:

• Find out about edge gating from Prof
• Experiment with various basic algorithms using surprise package, e.g. use collaborative filtering, matrix factorization. These results could be used as a benchmark.

29 Oct 2018

Progress:

• Tested results using amended code for spatial graph convnets
• Experimented with using more RGGCN layers in the GCMC recommender system encoder-decoder framework
• Using the same technique as spatial graph convnets of decaying the learning rate when training loss does not improve, a better test RMSE is obtained.

Obstacles:

• Tried to test GCMC with more layers but code fails at sparse_tensor operations for support and support_t. GCMC paper only uses one message passing layer.
• Experimenting with more RGGCN layers but still overfitting with poorer test accuracy than state-of-the-art.

Thoughts:

• Could a different architecture work better? E.g. edge gates are a major component of the RGGCN architecture but they may not be necessary in a recommender system task - all edges should probably be important in a recommender systems
• Is it better to predict a continuous value for ratings, rather than a softmax probability over the discrete rating types? Although having a softmax probability will allow the model to reflect that a user might either love or hate some item.

15 Oct 2018

Progress:

• Set up Google Cloud Platform GPU instance, installing CUDA 9.0 and CUDNN so that TensorFlow 1.10 runs with GPU acceleration.
• Translated PyTorch code from spatial graph convnets repo to TensorFlow code so that it runs within the framework by Berg et al. in GCMC
• Initial results here. The models where RGGCN is used instead of the original message passing method in GCMC are severely overfitting. Train RMSE is very much lower than test RMSE. However, the train RMSE is lower than that of the original model for most datasets.
• Number of parameters much more using RGGCN. 16052760 parameters compared to 3048760 using GCMC. (more than 5x more)

Questions:

• Possible typo in spatial graph convnets code
  • In line x = Uix + torch.mm(E_end.t(), x1*x2) + self.bu1, should it be Ujx instead of Uix?

17 Sep 2018

Progress:

• Read and understood code (TO DO)
  • MGCNN code
  • GCMC code
  • RGGCN code
• Ran code for MGCNN and GCMC on local machine
  • Reproduction results here: google doc link
• Preliminary modifying of code to replace message passing encoder step with RGGCN (TO DO)

Questions:

• MGCNN: why is parameters 1 for user+item graphs (Table 1) and m for user graph only (Table 2)?
• MGCNN code: how to tell if model is using user or user+item graphs?
• MGCNN: any reason why RMSE is higher when user+item graphs are used compared to when user graph is used for Flixster dataset?
• MGCNN code: movielens model runs out of CUDA memory at about 1120 iterations
• GCMC code: what’s polyak RMSE?

Observations:

Both MGCNN and GCMC were trained on my personal computer with its GTX 1050 GPU. GCMC trains much faster than MGCNN. For example, GCMC takes about a minute to train on the Flixster dataset while MGCNN takes over an hour.

3 Sep 2018

Progress:

• Read the four papers listed in 16 Aug update
• Project plan completed

Some insights:

• Multigraphs: Graphs that are permitted to have multiple/parallel edges; where two vertices can be connected by more than one edge. However multi-graph in Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks (Monti, 2017) probably refers to how spectral convolution is applied to row and column graphs instead of just a single graph.

Thoughts on applying residual gated graph convnets to recommender systems:

• Building on encoder-decoder framework in GCMC:
  • Use RGGCN layer instead of message passing method in the encoder network to produce node embeddings
  • Use RGGCN layer on user and item graphs to produce additional feature information for encoder-decoder network
• Building on framework in MGCNN:
  • Use RGCNN layer to produce W and H as in a separable MGCNN?

Some questions:

• Sanity check: what does fixed/variable length graph mean? That number of vertices in a graph can change over time?
• RGGCN is applicable for graphs with variable length. GCMC looks like it can handle variable length? But not MGCNN?
• What does the test set look like for a recommender system matrix completion problem?

16 Aug 2018

Brief outline of project:

Investigate different graph CNN approaches as recommender systems. The 3 methods to be compared are as follows.

1. Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks (Monti, 2017) (Abbreviated as MGCNN in this project to avoid confusion with RGGCN)
2. Graph Convolutional Matrix Completion (Berg, 2017) (GCMC)
3. Residual Gated Graph ConvNets (Bresson, 2018) (RGGCN)

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering (Defferrard, 2017) is an earlier paper that provides some theoretical background to the matrix completion method used in MGCNN by Monti et. al., 2017.

The third approach (Bresson, 2018) proposes a novel architecture for graph learning tasks, incorporating gated edges and residuality with graph ConvNets. The challenge will be to apply this architecture to recommender system tasks.