I am trying to append .h5 files saved in a directory for my training set using Python, but I face memory error as the Data set is very large. The following code is what I used for appending:
import h5py
Data_dir= ".../training"
Data = []
for sample in Data_dir:
img_path = Data_dir + sample
file = h5py.File(img_path)
Image= file['Image'][()]
Data.append(Image)
I was wondering if there is any way that I can use the directory as training set directly without making a list of arrays from them?
I have found the method of " flow_from_directory " from Keras ( https://keras.io/preprocessing/image/ ), but as I know it does not work for .h5 data. My problem is how to flow the data from directory for .h5 files for training a neural network.
Thanks.
Thanks for your reply and for sharing these two helpful links. I checked them. The tutorial in one of them is similar to what I am looking for (that one works for jpg, and mine is h5), so I asked my question there as well. Thanks again.
First of all check your equations & codes, e.g., you may just check the dimensions, sizes of your matrices etc. Check its compatibility to multiplication/addition. Also check the sparsity of your matrices.
- Badges
- Science topic
- Similar topics
- Filing