I believe this is an effect that can be found for reading text out-loud vs reciting text out loud as well, but I'm having a hard time finding references for instrumental performance.
This study shows that for piano performance "the subjects motions were less efficient in sight-reading tasks than in repertoire tasks", but does not explicitly say that less efficient = lower quality.
Wristen, B., Evans, S. and Stergiou, N., 2006. Sight-reading versus repertoire performance on the piano. Medical Problems of Performing Artists, 21(1), pp.10-16.
Article Sight-Reading Versus Repertoire Performance on the Piano: A ...
I guess it's not so much a question of high vs. low quality but of the expressive quality of the performance.
This is one reason that singers say they prefer to memorize the music, so that they can focus more on things like the expression of emotion for example.
look at research by Emery Schubert - it might be helpful
My research is about efficient practice and I have compared sight reading, practice and performance (so not relevant so much for you)
Emery Schubert and Dorrothya Fabian have written around on this topic and have several papers and a great book
and check out Daniel Bangart
Jane Ginsborg has written much with others on memorization process and that might also include material on emotion- she and Chaffin talk about the cues used for memory - and some of these could be looked at as "emotional" responses to the music - if my memory serves me correctly.