I recommend you to look beyond conventional massive MIMO, because a lot of things have already been done in these fields. Some open issues are discussed in these papers:
Emil Björnson, Luca Sanguinetti, Henk Wymeersch, Jakob Hoydis, Thomas L. Marzetta, “Massive MIMO is a Reality – What is Next? Five Promising Research Directions for Antenna Arrays,” Digital Signal Processing, Submitted for review. https://arxiv.org/pdf/1902.07678
Giovanni Interdonato, Emil Björnson, Hien Quoc Ngo, Pål Frenger, Erik G. Larsson, “Ubiquitous Cell-Free Massive MIMO Communications,” Submitted for review. https://arxiv.org/pdf/1804.03421
I recommend you to look beyond conventional massive MIMO, because a lot of things have already been done in these fields. Some open issues are discussed in these papers:
Emil Björnson, Luca Sanguinetti, Henk Wymeersch, Jakob Hoydis, Thomas L. Marzetta, “Massive MIMO is a Reality – What is Next? Five Promising Research Directions for Antenna Arrays,” Digital Signal Processing, Submitted for review. https://arxiv.org/pdf/1902.07678
Giovanni Interdonato, Emil Björnson, Hien Quoc Ngo, Pål Frenger, Erik G. Larsson, “Ubiquitous Cell-Free Massive MIMO Communications,” Submitted for review. https://arxiv.org/pdf/1804.03421
Traditionally, you can increase SNR in order to increase bit rate, why you don't transmit QAM codes and receive PSK code as available communication system, theoretically it can improve bit error rate hundred times for example a 13 dB bit error rate improvement was reached for Rayleigh channel, but you need a microwave devices to implement such system.
If you would like to work on massive MIMO, it is good to start studying and understanding Space Modulation Techniques, and their features, pros and cons as compared to conventional MIMO systems. As well as hybrid beamforming, digital and analog beamforming, channel estimation, antenna and user selection, and then machine learning or compressed sensing algorithms for these techniques. All are relevant areas with huge challenges along the way.