For beam alignment (802.11.ad) in a mmWave MIMO system, the transmitter starts with a quasi-ominidirectional beam. My question is how to design the hybrid precoder for a quasi-ominidirectional beam? Any comments would be appreciated!
As the 1st step your antenna elements should be have a quasi-ominidirectional pattern. On the other hand you should be use a different phase combination for all antennas elements of antenna array, which dependent from the number of channels.
If your MIMO array has omnidirectional antenna elements, it is enough to select one element. The excitation vector is just a unit vector with one nonzero component. Arrays where spacing is significant will always exhibit directional beams and/or nulls - after all, the purpose of an antenna array is to direct a narrow beam in the desired region of space.
Quasi-omnidirectional antenna pattern is nothing but a beam with wide beamwidth (180 degrees). A half-dipole or an inverted-F antenna can create a quasi-omni pattern in 2-D. But creating that pattern using an array antenna, which is your question, is somewhat complicated.
802.11ad simply introduces the MAC algorithm for beamforming without giving any details on the implementation. With hybrid beamforming, one way to create a quasi-omni pattern with an array antenna (assuming each element has omni-directional pattern) is to divide the array into 'subarrays' where each subarray with fewer elements now can create wider beams with lower gain. If those beams are aligned side-by-side such that they cancel others' sidelobes and increase the beamwidth; thereby can create quasi-omni pattern.
Taking your question one step further, the challenge is to use all antenna elements (and phase shifters), receiver structure (RF chains) with digital precoder at baseband and still to create both wide beam and fine-tuned narrow beams. To address that question, I published the following paper recently.
Varying the Beamwidth of Hybrid Beamforming for mmWave Communications