1.) Assuming that you can't use arrays, then environments may provide a good alternative to lists. The main practical advantage to using an environment rather than a list is that R passes environments by reference and lists by value. Of course, this is only an 'advantage' in some contexts. In particular passing an object by value to a function requires making a new copy of the object every time. Depending on what you're doing, you can end up with a program that spends most of it's time allocating memory for copying arrays.

You may also want to look at data.table, which I think is meant to improve the performance of large data frames.

2.) I can't provide much insight without a reproducible example, preferably a minimal reproducible example. Maybe

output[[ as.numeric(exp.num) ]]

Shane - thank you, right on the mark. I think the issue with passing an object by value is what has been bogging me down. The day will be spent looking more closely at environments and also data.tables (the output has multiple objects, so I will have to see if this structure will suit my needs). Thanks again!

Better go for "data.table"

Thanks for your help! I think the data.table package is what I need. Now to undo the damage I did when organizing the output for lists...

OK - making progress with data.tables. However, I'm still having a problem where R is crashing as I'm running the model - I believe the data file is getting too large for the environment (and it has to access it on every permutation to add data). Is there a way to write a GROWING data.table straight to disk to keep it out of RAM?? Many thanks!

You can write things straight to disk using structured text files by adding the append=TRUE argument. I do not know how, or even if you can, write binary objects to disk in R in that way. In any case, that many disk operations would probably be the limiting factor determining the speed of your code, and would probably be a very low limit.

I'm skeptical that your problems are arising because of allocating too much memory. I've never crashed R by allocating too much space. I've tried, for fun. R happily allocates more than the available physical ram. After you start reading and writing large amounts of data to the SWAP partition in Linux (or the equivalent under Windows, virtual memory I think) everything runs very slow. But I can't get R to crash like this. In any case, several hundred MB should not be large enough to give you problems under pretty much any reasonable conditions I can imagine.

So it seems to me that there must be something specific to the model you are running. For what it's worth, the only times I've ever personally "successfully" crashed R were running subroutines in C or Fortran that either contained some incorrect pointer calculations or were compiled incorrectly. 

Thanks Shane - I've gotten R to stop crashing by using data.tables. I think when I was using lists and data.frames, R was copying the data each time I appended a new model run... The data.table solution, because it passes by reference, seemed to correct the problem. However, I think I'm going to start appending a text output, as you mentioned, and not store a large data file in the RAM. I think this will speed everything up. Thanks!

Thanks for the update, glad to hear data.tables are working out. Reading and writing to the hard disk can be slow. Reading from disk is always slow, the effect of writing to the disk can be more variable. It may be worthwhile to try writing the results in blocks of 10 or 100.