There are a lot of papers using GNNs for physics simulations (e.g. computational fluid dynamics) because the unstructured meshes used to discretize the problem domain for such applications map very neatly to a graph structure.<p>In practice, every such mesh/graph is used once to solve a particular problem. Hence it makes little sense to train a GNN for a specific graph. However, that's exactly what most papers did because no one found a way to make a GNN that can adjust well to a different mesh/graph and different simulation parameters. I wonder if there's a breakthrough waiting just around the corner to make such a generalization possible.
Very high quality work, its a pity that distill.pub did not find a sustainable way forward [1].<p>On GNN's, the lack of datasets [2] might be a reason they are not as talked about. This is something that has affected also the semantic web domain.<p>[1] <a href="https://distill.pub/2021/distill-hiatus/" rel="nofollow">https://distill.pub/2021/distill-hiatus/</a><p>[2] <a href="https://huggingface.co/datasets?task_categories=task_categories:graph-ml&sort=trending" rel="nofollow">https://huggingface.co/datasets?task_categories=task_categor...</a>
GNNs have been a bit of a disappointment to me. I've tried to apply them a couple times to my research but it has never worked out.<p>For a long time GNNs were pitched as a generalization of CNNs. But CNNs are more powerful because the "adjacency weights" (so to speak) are more meaningful: they learn relative positional relationships. GNNs usually resort to pooling, like described here. And you can output an image with a CNN. Good luck getting a GNN to output a graph. Topology still has to be decided up front, sometimes even during training. And the nail in the coffin is performance. It is incredible how slow GNNs are compared to CNNs.<p>These days I feel like attention has kinda eclipsed GNNs for a lot of those reasons. You can make GNNs that use attention instead of pooling, but there isn't much point. The graph is usually only traversed in order to create the mask matrix (ie attend between nth neighbors) and otherwise you are using a regular old transformer. Often you don't even need the graph adjacencies because some kind of distance metric is already available.<p>I'm sure GNNs are extremely useful to someone somewhere but my experience has been a hammer looking for a nail.
It seems GNNs operate on a fixed topology. What if I want to approximate some transformation of the topology of the graph? For example learning how to layout a graph, or converting program abstract syntax trees to data flow graphs.
I feel very dumb. There is an example on that page with 4 nodes (a,b,c,d) and it shows a total of 24 possible combinations.<p>What is the generalised formula for calculating this, given the number of nodes but also edges need to be considered.<p>It doesn't appear to be explained in the article. I think it may be a factorial?
Is there consensus about whether gnn architectures are better than transformer based ones at this point? I am aware that transformers can be viewed as a gnn too.