To be clear: the images are basically App (e.g., Facebook, LinkedIn, Pinterest etc.) screenshots. The purpose is to classify the images (the ground-truth is the App name) and see the accuracy. In this case, I need to manually collect images from each App (around 1,000 Apps with 100 images in each App).

Usually in our research, the data or the images are already there, and we only need to focus on the features extraction and the optimisation of classifiers etc.

But this time when it comes to build training and testing images from scratch, I suddenly feel a bit lost ...

Please help!

I don't think I got the question well enough. By building training and test, do you mean according to which criterion train/test splits should be built or which features to extract from images?

Hi Melih,

Thanks for the quick respond! 

I do not mean the splits or features in my case. I mean collecting original images for training and testing. 

I need to know a bit more about the context. What's the application, what's the purpose, etc.?

In general, you want to have a training data to be rich enough in the feature space. For example, you don't want to train on an image of the cloudless sky if you want to use edge features. For uncontrolled environments, you also want to have spatio-temporal elements to be captured. For temporal effects, you can place a stationary camera and take time-lapse images (that should cover both light angle and intensity changes). If you move the camera in the PT directions in a predetermined way, you can capture the spatial effects.

The images are basically App (e.g., Facebook, LinkedIn, Pinterest etc.) screenshots. The purpose is to classify the images (the ground-truth is the App name) and see the accuracy. In this case, I need to manually collect images from each App (around 1,000 Apps with 100 images in each App). 

Hi! If your question is about a recommended METHOD for generating the images, I would use selenium with Python. It is extremely easy to use, and you can automate it to capture screenshots at different moments and urls previously defined by you.

from selenium import webdriver

browser = webdriver.Firefox()

browser.get('http://www.yourDesiredApp.com/')

browser.save_screenshot('yourScreenshotName.png')

browser.quit()

Fro image process automatically, you still need to extract features through conventional techs, no matter the images are from apps or something else. Then you need to do the classification or data mining and so on.

SInce you are going to be creating your own dataset, i suggest you should use the opportunity to narrow the scope of your experiment. I once generated my image dataset for satellite images by manually taking google earth screenshots. The process is tedious especially when you need 1000 or more images, but it produced a reliable dataset.

Thanks Tarun.

But I only mean how to create a scientifically good data set, or to create a representative enough data set.

Nobody has answered,therefore, you can understand that no one may be knowing   the answer.It is a very important question.

Preparation of training and testing data of images for classification is an important  task.