In my opinion, the strength and beauty of Matrix theory do not stem as much from the arrangement structure (rectangular array) than the ingenious rules developed by mathematicians over centuries to manipulate them, transforming them into one another while preserving one or more attributes of the array, for example, the trace and determinant preservation under similarity transformation, inertia under congruence transformation and the more generally, the Equivalence transformations which only preserve the rank of the matrix.
an array is just a way of visualizing a set of numbers, it comes to life through the rules of composition/manipulation defined on them, that generates the Array Algebra.
In my opinion, the strength and beauty of Matrix theory do not stem as much from the arrangement structure (rectangular array) than the ingenious rules developed by mathematicians over centuries to manipulate them, transforming them into one another while preserving one or more attributes of the array, for example, the trace and determinant preservation under similarity transformation, inertia under congruence transformation and the more generally, the Equivalence transformations which only preserve the rank of the matrix.
an array is just a way of visualizing a set of numbers, it comes to life through the rules of composition/manipulation defined on them, that generates the Array Algebra.
The useful product for matrices is defined as row by column product, because it corresponds to composition of linear maps, so the rectangular array is simply the most convenient.
You could certainly do that if you have a good reason. In computer science, they use lists to store sparse matrices (with lots of zero), which is kind of irregular if you want to shape these lists.