I'm not much of a programmer, but I can modify python code to fit some of my needs. My current need is to figure out how to take a whole bunch of CSV files in a folder (Raman Shift, Intensity) and plot in 3D using python (or R). Ideally I would like to have a contour plot. I mainly need a nice way to plot the data, but eventually I'll want to do some more sophisticated data analysis. I can provide more details if needed.
I'm sure someone has done this before, but I just can't find it. Any help would be greatly appreciated!
These links contain some useful hints for you.
http://stackoverflow.com/questions/13991905/how-to-plot-a-multicolumn-csv-file
http://stackoverflow.com/questions/13545388/plot-data-from-csv-file-with-matplotlib
http://stackoverflow.com/questions/11157378/python-how-to-plot-from-a-csv-file-plot-surface
Thanks Pavel! Those are very helpful links. You are probably right, this question might be better asked on stackoverflow. I spent this whole past weekend looking over stackoverflow for ideas, and there was a bunch of stuff that was pretty close to what I needed, including one of your links, but nothing that was quite right :)
I've managed to grab all my CSV files into a pandas with column format of:
X, Y1, Y2, ..., Y159
There are 3771 rows and 160 columns
I did plot it with 3D axes, but it was incredible slow using an iteration using below:
In [1]: for i in Z: # where Z contains the column header labels
In [2]: ax.plot(X, Y[i], Z[i]) # X is the first column, Y is the next column, and Z is the counter
I imagine that's why it's so slow. The code is slowly going through the pandas plotting X vs Y1 vs counter# then X vs Y2 vs. counter#+1 etc. etc.
You should be using a technique called "blitting". Essentially when you call ax.plot(), it has to everything in the plot window is redrawn, this is a slow process and likely is a big bottleneck for your code.
The following stackoverflow link explains things better than I could:
http://stackoverflow.com/questions/8955869/why-is-plotting-with-matplotlib-so-slow
(note: this is using matplotlib, which pandas uses in the background for its plotting)
You should check out the 2d/3d plots from our beta library, scikit-spectra.
http://hugadams.github.io/scikit-spectra/tutorials.html#ipython-notebooks
If these have what you need, and I can help you either install scikit-spectra, or reproduce them using our plotting functions. If you are interested, just message me directly. Good luck with your search.
Thank you to everyone. From everyones comments, I managed a 3D line plot with all that data OK, but it's still not great as the plot interface is still slow and python (or my 2GB-Radeon graphics card) lags when I rotate the figure. The best solution so far is a contour plot, which is what I really wanted anyway. I will check out the scikit-spectra that Adam mentioned above, that GUI and those graphs are pretty cool!
Thanks to Megat, I did use numpys to make the contour plots.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages
- Python