I think you'll find that there is substantial evidence of an association between spatial (or visuospatial) abilities and mathematical competence. However, most studies will focus on schoolchildren and most of the rest on adults in school (e.g., college students) or those with acquired brain lesions.
Psychological research on healthy, non-literate adults is quite scarce. To begin with, such people are difficult to find and recruit in the developed world - where virtually all psychologists are located.
I am attaching one possibly relevant document. The author, Samar Zabar, is now (I think) at the American University in Beirut. Try [email protected] to contact him - but I'm not sure he's still working on this topic.
You also might want to check out the literature on developmental economics. I know, for example, that Chris Udry at Yale had planned to include a couple of cognitive tasks in his research in Ghana (probably a partial Raven and a digit span test, which would be pretty good for your purposes). You can find some of his working papers and data at the attached site; it will also connect you to other developmental economics sites.
If you're serious about this, it could be an excellent idea for a research program! Because very little previous work bears on the specific issue, yet the broader context is well established.
Thanks for the excellent references, Steven! Yes, I am serious (I've been working on this for years, but not with academic credentials) and have published several papers on my own tests which are available here. If you are interested in collaborating on this program, I am certainly looking for such.
Full studies on illiterate or even underliterate adults are rare, and tend to focus on reading skills, but there are some interesting specialized areas that are related to your question:
One line of inquiry could be contact with isolated tribes. There is a famous study from about 10 years ago with the Pirahã in Brazil (Frank, Everett, Federenko, & Gibson, 2008). Their conclusion is around linguistic determinism, but it has some interesting connections about number cognition.
Another is the area of comparitive psychology. Furlong and Opfer (2007) found that chimps behave a lot like children on some number tasks.
The bulk of existing research on the positive correlation between spatial training, spatial reasoning and mathematical competence has been on the rise involving different age groups and contexts.It is assumed that the relationship between space and math may be based on common underlying processes which suggest a promising avenue for mathematical improvement. Notably, spatial reasoning is a malleable skill that can be improved with training. As you have rightly observed, math is a symbolic language which enables humans to think about the quantity of matters while relating thoughts and the relations between quantities to each other. For more information, I refer you to the following links which I hope are beneficial.