I think the model itself doesn’t directly determine the quantity of fine-tuning data needed. It’s more about the model’s generalization ability from pretraining. Larger LLMs like GPT-4o can often perform well with little or no fine-tuning (just prompting), but that doesn’t mean they need less fine-tuning data if you actually want to specialize them further. From my experience, in specific tasks (like geolocation identification) NLP models can sometimes match GPT4 or even outperform GPT-3.5.

Yes, compared to GPT-1 and GPT-2, modern large language models (LLMs) like GPT-3+ or domain-specific fine-tuned variants often require less training data for specific tasks due to pretraining on vast corpora, transfer learning capabilities, and better parameter scaling, enabling efficient adaptation with minimal supervision.