do I understand correctly that you question is whether it is possible to transfer multivalued symbols by modulating an electromagnetic beam?
If so: I think this certainly possible, it's debatable which modulating technique would be best. My guesses are: Amplitude modulation would be error-prone; you mentioned phase modulation, I'm not sure about that; my favourite would be frequency modulation because the detection does not rely on any kind of synchronisation, and if the alphabet consists of, say, 30 characters, it's certainly possible to have a receiver with 30 separate frequency channels.
However, it seems doubtful to me whether the advantages outweigh the drawbacks as compared to transmission of binary signals.
Re the initial question: obviously yes - advanced modulation schemes like (Q)PSK, QAM and alike are ubiquitous where efficient usage of the (limited) bandwidth available is mandatory.
One should note however that in typical situation of additive white noise non-binary, complex modulation schemes are less energetically efficient than binary. The complicated modulation schemes are typically used when signal/noise is very high but the bandwidth is limited (e.g. by FCC regulation).