Yes, the structure also changes. In general, first, dehydration, then, dehydroxilation occur. If the clay is, for example, kaolin, an amorphous phase named metakaolin, is obtained. For smectites, the structure collapses. At higher temperaturas, there are more changes. But the color changes would be associated to transition cations, the most usual, iron. Of course, other changes would be associated to impurities, for example, if the sample contains organic matter...
One reason that zeolites change color (usually they become yellowish at high temperatures) when you anneal them is because the furnace is not clean and at high temperatures organic molecules that are absorbed on the walls desorb and absorbed in the posrous structure of the zeolite It might be your case.Usually you should increase the temperature of the furnace/oven maybe 100oC higher and let it for some hours under inert atmosphere if possible and then cool down and use it.
During thermal treatment of the clay color can be changed by oxidation of iron or you have organic substances in your clay samples. You need to do an XRD and you will know what has happened :)
That is for organic matter oxidation mainly. Black (and gray) are indicious of carbon oxidation, when you work in vaccum, the is no sufficient oxygen to degradate all the carbon, so, coke deposit are formed ( black color), in oxygen atmosphere, all carbon is oxidate to carbon dioxide. You can do TGA to determinate the organic matter quantities.