This is quite a good question.   

One of the best sources of an answer to this question comes from

Weisstein, Eric W. "Dimension." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/Dimension.html

who writes: The dimension of an object is a topological measure of the size of its covering properties. Roughly speaking, it is the number of coordinates needed to specify a point on the object. For example, a rectangle is two-dimensional, while a cube is three-dimensional. The dimension of an object is sometimes also called its "dimensionality."

Other forms of dimension are

fractal dimension:  A fractal dimension is an index for characterizing fractal patterns or sets by quantifying their complexity as a ratio of the change in detail to the change in scale.   For more about this, see

https://en.wikipedia.org/wiki/Fractal_dimension

The fractal dimension is called the capacity dimension:

http://mathworld.wolfram.com/CapacityDimension.html

Hausdorff dimension: self-similar objects with parameters N and s are described by a power law N = s^d.    For more about this, see

http://mathworld.wolfram.com/HausdorffDimension.html

In addition:

From the algebraic point of view it relates to independence relative available operations.

The exchange property of closure operators specifies it (of matroid theory)

In the simplest sense every "dimension" is basically a control. Any control can be considered as a dimension, as is evident from Einstein's consideration of time as the fourth dimension, which revolutionized the world. In fact this provided an answer to the n-dimensional geometry of Riemann, which was ridiculed during his life time leading to his frustration.

The notion of dimension has expanded considerably to have different meanings based on the different mathematical structures it refers to. Dimension in simplest terms, the  one that refers to Euclidean spaces or general abstract manifolds which are locally Euclidean, is the number of linearly independent vectors that generate a given vector space. Dimensions which are described by dear James are also other types that are specific to the geometric nature of the space.

The complexity of a mathematical structure however is not merely from the dimension but from the foundations and higher levels of abstractions and descriptions in which the knowledge of higher order properties is represented and theoretical results are established. A mathematical structure of one dimension can be more complex than two dimensional when it is made to be so, but always increase in dimensions increases complexity of structures.