In my opinion:

In a composite you have homogeneously distributed particles, fibers or parts throughout the 3D structure (i.e. cement+stone, Titanium + carbon fibers). Distributed species do occupy a larger volume in the space compared to those in the alloys, separately. In alloys, generally ,distribution is at the atomic level ( sub-nanometric) and we can talk about diffusion when we talk about alloys.

However, there are some cases which would be a bit confusing.

You may ask yourself: " is the pearlite structure of Fe-C system a composite or an alloy in this case?" It involves diffusion process and it also does involves a composite structure with alternating phases. For an image of pearlite http://hsc.csu.edu.au/engineering_studies/application/civil/1-1/pearlite.jpg

Pearlite: A lamellar mixture of ferrite and carbide formed by decomposing austenite of eutectoid composition.

Ref: http://www.sv.vt.edu/classes/MSE2094_NoteBook/96ClassProj/examples/kimttt.html

As previously written, in alloys the mixing between phases is at the atomic level or of subnanometer clusters.

When it comes to composites, these are formed by two or more components regardless of stoichiometry. You can prepare composites by mixing solids, or solids with liquid organic compounds. The liquids can be trapped within pores or become strongly adsorbed on the solid surfaces, thus becoming immobilized on a solid surface. In composites, one or all components have from a few nanometers to several microns in size, whereas the other can be in the molecular scale and form thin films onto a large surface. In many composites, you may induce partial reactions between phases. But, in the latter case, you would then have a new phase introduced to your composite, and this is my opinion. If one component reacts entirely with another, then it's a new compound material.

An important parameter to classify a composite is the final property of the material, in that it differs from that of the individual components. That's not always the case of alloys, where a given property (catalytic, luminescent, etc.) may be affected but not completely changed. I could use the example of gold alloys with other metals for catalytic reactions, i.e. CO oxidation, or ruthenium added to platinum for hydrogen fuel cells. The metals other than gold or platinum in these alloys, are used to lower the cost of the catalyst, improve stability of nanoclusters, favor a given crystallographic orientation, etc. Although the alloy may have lower or higher activity, these remain as oxidation catalysts.

Now, pearlite is the microstructure of a mixture of two interpenetrating alloys, cementite and alfa-ferrite and it's not a phase. It's cementite platelets dispersed in a ferrite matrix. When it comes to materials property, pearlite, is one of the most resistant ductile materials known today. Alpha-ferrite, as in pure iron, is soft, whereas cementite is brittle.