Hello, everyone, now I embark on an new research about the relationship on Twitter. However, I was confused how can I map a large amount of links on one map quickly. I have a 250000*250000 0 or 1sparse matrixI which represents the relationship on Twitter. I tried some software like Gephi, but it is too slow to draw the picture on it and it often fail to work. I tried package igraph on R and it runs a whole night and end up with a picture. However, it is very ugly... Can you give me some advice to deal with it with igraph? or can you recommand some powerful software for me? Thank you very much!
I might suggest using "Island Method" to cut off some of less important edges, so that core nodes and corresponding edges could be drawn on your computer.
You'll find how to conduct Island Method in this chapter:
https://www.safaribooksonline.com/library/view/social-network-analysis/9781449311377/ch04.html
I might suggest using "Island Method" to cut off some of less important edges, so that core nodes and corresponding edges could be drawn on your computer.
You'll find how to conduct Island Method in this chapter:
https://www.safaribooksonline.com/library/view/social-network-analysis/9781449311377/ch04.html
With such a large network, you cannot see all nodes and how they are organized with a single picture, this is just too messy.
I would say that the method you should use to visualize your network depends on what you want to see. If you're interested in seeing some influential nodes for instance, why not filter out of your network some of the less influential nodes ? Because the graph is probably a small world type, the large majority of your nodes have probably only a few edges. You could remove them, your graph becomes much simpler but you keep all the big players nonetheless.
On the contrary, if you're interested in the organization of the network, why not use some methods such as community detection ? Using a method such as Louvain or Infohiermap, you'll obtain a hierarchical decomposition of your network, and you can display much more easily these "bags of nodes" as a network, that is, the network of the relation between the well defined clusters of nodes in your network.
I am using Gephi when ever it is possible due to its power and appealing visualization. I have not used networks larger than 500-600 nodes. But I have seen people displaying networks (much larger, but not sure about number of nodes and edges). Since Gephi use JVM you may check memory allocation for java in your machine. If you are using 32bit version max is 4096MB. But if you are using 64bit java version you may increase the memory for java according to your physical memory.
Maybe have a try on Mage (although I never tested it with such a huge data set):
http://kinemage.biochem.duke.edu/software/mage.php
Regards, Volker:
You could try Pajek, a visualization tool designed for large networks. http://mrvar.fdv.uni-lj.si/pajek/
One of it's input formats is the matrix[1], so it might be easy to get your data into Pajek.
[1] Mrvar A. Pajek. http://mrvar.fdv.uni-lj.si/sola/info4/andrej/pajek_short.pdf
Maybe you can try CytoScape (cytoscape.org). It was designed to work with biomedical data and should be able to cope with large datasets. You can choose between a desktop version or a Javascript library.
Best,
Antoon
Visualizations of very large networks is often a futile effort. Do you have interesting "sub-networks" within your data? Something like this?
http://www.thenetworkthinkers.com/2013/03/big-data.html
Can you eliminate all of the empty cells and just send the link data like this? A simple text (CSV) file.
PersonA,PersonB,Frequency,ConnectionType
Mary,John,5,RT
Use the head/tail breaks to filter out big data based on the fact that the head is self-similar to the whole:
Jiang B. (2013), Head/tail breaks: A new classification scheme for data with a heavy-tailed distribution, The Professional Geographer, 65 (3), 482 – 494.
Ma D., Sandberg M., and Jiang B. (2015), Characterizing the heterogeneity of the OpenStreetMap data and community, ISPRS International Journal of Geo-Information, 4(2), 535-550.
I would use the Netdraw, but after assembly of the network, would select the most central actors to allow a better view. If you are interested in any peripheral region, I recommend using the Zoom tool.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Analysis
- Data Analysis
- Dataset
- Big Data