This isn't a complete answer, but might help you think where to look next or stimulate some other comment.

Have a look at memory limits in R:

https://stat.ethz.ch/R-manual/R-devel/library/base/html/Memory-limits.html

which might cause problems for you even if you think that you should have enough memory. At the very end of the webpage see the command

object.size(a) for the (approximate) size of R object a

which should give you an idea of the size of the problem you are dealing with, and you can compare that against what R can handle.

Have you used any special package?

http://tm.r-forge.r-project.org/

https://cran.r-project.org/web/packages/tm/tm.pdf

A good example here:

https://deltadna.com/blog/text-mining-in-r-for-term-frequency/

Thanks Demetris..Yes I am using tm and some packages.

 Thanks a lot Jason   for very useful information.

Probably you have to check the memory status during execution of command as.matrix(), given your operational system (for windows leave open the task manager). What is your RAM in GB? Are you using 32 or 64 bit computer?