This paper aims to address the out-of-distribution (OOD) problem in current evaluation metrics, which arises because these metrics directly feed the explanatory subgraph into the model. To tackle this issue, the authors propose the OOD-resistant Adversarial Robustness (OAR).
This paper is inspired by adversarial robustness. The core idea of OAR is to compute an OOD score, which measures the extent to which the generated graph deviates from the data manifold.
In this paper, they adopt the idea of anomaly detection to measure OOD score. Specifically, they train a variational graph autoencoder (VGAE) on the dataset and use the reciprocal of the reconstruction loss as the OOD score.
Finally the OOD score is used to weight each prediction when calculating the expection of the generated graph’s prediction.
To expedite computation and simplify OAR, they propose a simplified version called SimOAR. The key idea is that the degree of distribution shift is roughly proportional to the number of perturbation operations.
They further define a gold evaluation metric, and the consistency of this metric across different evalaution methods is used to assess the reliability of these methods.
References
NeurIPS 2023 Evaluating post-hoc explanations for graph neural networks via robustness analysis