This question may help a lot of people to be able to decide on what to use, when, and why.
Also what I mean by the nature of data is beyond the type of the dataset itself ( Time series, images, speech etc ...)
I'll make sure to synthesize all the answers in one place.