This depends hugely on your budget. if you're talking in the range of $4-5000 for the whole system then you'd be looking at consumer level GPUs like the RTX3080 or 90. The prices of these cards will probably drop considerably in the next few months. You'd also want to have a reasonable amount of RAM, say 64GB min, and also a good CPU. If you're talking $15-20000 you could get a nice server setup with multiple enterprise level GPUs. What framework would you be using?

Transformer and BERT