This for general is not correct. Tensile testing is used for some metals and polymers, because with plastically deformable materials or very elastic materials, tensile testing provides more information than pressure testing. For brittle materials it is the other way round: pressure tests (for example for concrete samples) and bending tests (brittle metals and ceramics) are used.
As vadim said it depends on what you want to characterise. For instance usually for tensile test we can use shouldered test bars allowing to have a homogeneous zone in the center of the sample (avoiding problems due to the edge and clipping of the sample).
With compression the behaviour can be more different and it is harder to measure modulus accurately but usefull for mechanical behavior.
The material behave in different manner if compared between the tensile and compression test. Therefore the type of test will be chosen according the application.
You have raised a good basic question!. This is because most of the failure phenomena that are associated with materials are due to tensite stresses,i.e., metal fatique,overloading, and so on, or mutually involved with tensite stresses,i.e., stress corrosion cracking, corrosion fatique, and so on.
In contrast, vew phenomena of materials failure are know to fail by compression stresses,i.e., wear, abrasion, and so on.
I think the compresive strength of a material is many times higher to its tensile strength,hence to get a minimum value of load the material can withstand.Moreover it depends on what type of load will the material have i.e compressive or tensile.