Generally, we prefer images with equal H and W. Many works always reshape the rectangular image firstly.
I wonder that does the rectangular image as the input (kernel size sets the same value for convience) affect the performance of the CNN-based models.
The issue with squared vs. rectangular images is important only in the classification tasks. In classification CNNs there is always a final fully-connected (FC) layer that performs the classification. It has the output size equal to the number of classes, and the size of its input is fixed (e.g. input = 512, output is 5 classes). So, the whole CNN part that is placed before the FC layer is tuned to produce an output of this specific size. In this way, the CNN must deal with an image of a predefined size, e.g. it's 221x221 in some popular CNNs, or it can be 256x256 in order to be more aligned with some processor architectures. So, if you want to classify an image of a different size, then feeding it through the CNN will produce a different input to the FC layer (e.g., not 512 but 499) and the FC layer will not be able to process this input (since the action of an FC layer is basically a matrix-vector multiplication).
Therefore, we usually either crop the part from the image that has the size that suits the particular CNN architecture, or we resize this image. Resizing is not always good since it kills the aspect ratio which may harm classification accuracy.
This requirement for a fixed input size for classification can be removed if we use tricks like Global average pooling, so that the input to the FC is always the same size (equal to the number of feature maps it the final convolutional layer that goes right before the FC), no matter which size in the input image.
So, for classification, there is no requirement for the image to be square. It can be rectangular as well, but it should have the same fixed size.
If the CNN doesn't contain an FC layer - e.g., it's fully-convolutional as in many classification tasks and almost all modern segmentation CNNs - then the shape of the input doesn't matter.
In general, if you resize an image, it will affect the performance of the machine learning algorithms such as CNN. Because extraction of features in an image utilizes pixels and resizing will remove or add pixels from the image consequently features after and before resizing may not be the same. Most of the software packages support rectangle images and the width and height may need not be the same. Though large size images take longer time to process and may cause an out-of-memory error if the data size is big.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Probability
- Probability Theory
- Reliability