This is an important question.

Datasets for visual odometry of UAVs are part of the research introduced in

https://www.researchgate.net/publication/224252287_Visual_odometry_learning_for_unmanned_aerial_vehicles

https://www.researchgate.net/publication/282937160_Efficient_visual_odometry_and_mapping_for_Unmanned_Aerial_Vehicle_using_ARM-based_stereo_vision_pre-processing_system

More to the point, see the following very interesting RG research projects:

https://www.researchgate.net/topic/UAV

Conference Paper Visual odometry learning for unmanned aerial vehicles

Article Efficient visual odometry and mapping for Unmanned Aerial Ve...

Thank you for answering. I think that the manuscripts you recommended are significantly considerable.

I examined your recommendations, but I cannot find any mention about public dataset collected from UAV...

@Sang-Il Oh: ....public dataset collected from UAV...

Here is a followup to what I wrote earlier.   See reference [33] for a public data set:

Wavelet-Based Visible and Infrared Image Fusion: A Comparative Study

http://www.mdpi.com/1424-8220/16/6/861/htm

More to the point, see

http://vigir.missouri.edu/~gdesouza/Research/Conference_CDs/IEEE_IROS_2013/media/IROS13_ContentListMedia_3.html

and

http://gtr.rcuk.ac.uk/project/750AB476-668B-4720-B267-5E5C610BDC4A?pn=3&fetchSize=50&selectedSortableField=firstAuthorName&selectedSortOrder=ASC

@James F Peters

I owe you for your services.

May be useful

www.aiia.csd.auth.gr/LAB_PROJECTS/MULTIDRONE/AUTH_MULTIDRONE_Dataset.html

In case this question is still current, I would like to add that we recently released a dataset that provides synchronized visual and inertial data records from quadcopter flights: Conference Paper Mid-Air: A Multi-Modal Dataset for Extremely Low Altitude Dr...

The visual data consists, among others, in RGB-D and stereo (with gound truth disparity) video records. The inertial data consists in accelerometer, gyroscope and GPS measurements. The dataset also provides the ground truth inertial state of the UAV for each inertial measurement.