I ran across GLNBench today and found their benchmarking results on Graph Neural Networks (GNNs) under label noise incredibly refreshing.
Usually, GNN papers claim amazing robustness on tiny, clean citation datasets using basic backbones. The authors of this benchmark actually stress tested many robustness methods across multiple datasets and multiple backbones, and some of their findings are brutal;
1. Almost every state-of-the-art robust method beats a standard baseline on simple, narrow GNN architectures. But when upgraded to a strong, wide backbone, the gap compresses to nothing or the methods fail entirely.
2. Geometry based methods fall straight to chance accuracy on wider backbones because their embedding rank collapses to 1.
3. Some of these safety methods are insanely expensive. GNNGuard, for example, recomputes cosine attention every epoch, making it slower than a standard baseline.
It's a really well designed site with an interactive experiment builder, and they track everything down to carbon emissions and oversmoothing metrics (like NumRank and MAD) to actually show why these models fail when they do.
I ran across GLNBench today and found their benchmarking results on Graph Neural Networks (GNNs) under label noise incredibly refreshing.
Usually, GNN papers claim amazing robustness on tiny, clean citation datasets using basic backbones. The authors of this benchmark actually stress tested many robustness methods across multiple datasets and multiple backbones, and some of their findings are brutal;
1. Almost every state-of-the-art robust method beats a standard baseline on simple, narrow GNN architectures. But when upgraded to a strong, wide backbone, the gap compresses to nothing or the methods fail entirely.
2. Geometry based methods fall straight to chance accuracy on wider backbones because their embedding rank collapses to 1.
3. Some of these safety methods are insanely expensive. GNNGuard, for example, recomputes cosine attention every epoch, making it slower than a standard baseline.
It's a really well designed site with an interactive experiment builder, and they track everything down to carbon emissions and oversmoothing metrics (like NumRank and MAD) to actually show why these models fail when they do.