Hello,
I read about Graph Neural Networks, and understood that it is very similar to CNN, only that is on graphs and without pooling for downsizing. Each iteration the "convolution" is of different distant neighbors of a given node, so it's the same as different size convolution filters (again without pooling).
Isn't it hierarchical? Where I say hierarchical, I see it as "features of features".
How is DIFFPOOL make it hierarchical just by adding hierarchical clustering and pooling?
Also, if it's not hierarchical as CNNs, how is classification tasks performed on top of it, the GNN?
Thanks
Hello Shm,
As you underlined, the parallelism between Graph Neural Networks (GNNs) and Convolutional Neural Networks (CNNs) is not easy to define, and probably it does not even have a clear meaning. If the objective of the two models is similar (to process structured data without loss of information), the original paradigms are different and, even more importantly, the type of structure they work on is different. Images are regular grids of pixels, while graphs structures have variable connectivity. Graphs can even be "directed", meaning that node N1 can be the neighbor of N2 without N2 being the neighbor of N1.
In GNNs, each node has a feature vector which is updated iteratively with the contributions of its neighbor nodes. These contributions can be averaged, summed up, or aggregated in other ways, including convolutions (Graph Convolution Networks) and even LSTMs on random permutations of the neighborhood (see https://cs.stanford.edu/people/jure/pubs/graphsage-nips17.pdf).
Various pooling mechanisms can be applied. Clustering nodes and/or features is possible for certain models, but its efficacy really depends on the nature of the problem.
In GNNs, there are many types of classification problems:
1) Node classification happens on each node, or on a subset of the nodes. An output is produced for each node that must be classified.
2) Edge classification works the same, but the outputs are produced on edges.
3) Graph classification happens on the whole graph. Node features can be pooled up to the point in which only one feature vector is obtained, and this is classified with a neural network. A more simple method consists in producing an output at each node and then averaging over the whole graph.
You are welcome.
I do not know if I have got the question right, but I believe any latent hierarchy of nodes/features can be managed with pooling/clustering techniques.
Talking about hierarchical graph representations, instead, like graphs of graphs, or graphs which are split into subgraphs for any reason, I would rather use a hierarchical model. For instance, a network can produce an embedding of each subgraph. Then this embedding is set as the feature vector of the node which corresponds to that subgraph in the upper level of the hierarchy, which is processed by another network, and so on.
Have a look at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.133.1405&rep=rep1&type=pdf
http://scholar.google.com.au/scholar_url?url=https://ageconsearch.umn.edu/record/116024/files/sjart_st0028.pdf&hl=en&sa=X&scisig=AAGBfm1h1gsC8LpAIAbwvjuPXalNAqOPCw&nossl=1&oi=scholarr
When clustering, the hierarchy may be broken, because the space is non-Euclidean and sometimes the triangle rule is not fulfilled in it.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Graphs