I am a little out of my depth here, since I don't use Boost and have limited experience with parallel programming, but I don't understand why you need a parallel version of memcpy ... to my knowledge no such thing exists. Can you not use the typical trick of protecting the memcpy calls using semaphors (i.e. a mutex block)? That may slow your code down a bit I suppose, but at least you will get the benefit of using the appropriate architecture-tuned version of memcpy.

I and a friend have designed and implemented exactly such a thing, with a queue of blocks to copy implemented using only atomic operations.

As you may be aware, 1 hardware thread is not actually sufficient to fully exploit memory bandwidth on a modern multicore CPU, and so memcpy between threads runs faster if they all participate.

I'll see if we can't release this code somewhere.

Some years ago I worked with the "Intel integrated performance primitives" library (commercial). Originally, it was targeted at image and signal processing. However, it should offer a parallelized version of memcpy as e.g. ippiCopy_XYZ. This family of functions should perform quite well, especially on Intel hardware. Maybe the calling arguments suit your actual application. It also offers some control on how many threads to use.

David, the point is like Phil said, a single hardware thread will not necessarily be able to saturate memory bandwidth. In many cases, it will, but not all. In an ideal NUMA setting, you should certainly be able to exhaust more bandwidth from the memory subsystem by engaging all memory controllers, from their local cores, rather than doing multi-hop accesses. I don't want to explore the details (within this project), but I figured that this could have been done by someone else.

I'll check out the current Intel offerings, like Christian suggested. I do know that we have institution-wide licenses on some of the Intel performance products, so maybe I can at least test if there is an improvement.

> [...] our current design has a master thread copying that data to newly allocated internal data structures [...]

On a side note, have you considered experimenting with multi-threading-friendly allocation strategies? For instance, TCMalloc (Thread-Caching Malloc) from Google Performance Tools or one of the allocator templates from Intel TBB:

http://goog-perftools.sourceforge.net/

http://goog-perftools.sourceforge.net/doc/tcmalloc.html

http://software.intel.com/sites/products/documentation/doclib/tbb_sa/help/tbb_userguide/Memory_Allocation.htm

Intel compilers attempt to choose a suitable version of memcpy() automatically, but not normally with automatic threading.

For fans of the Intel IPP library (which doesn't support all Intel platforms) the obscurely named optimized headers option of Intel C++ engages IPP where it is considered useful.

It may be counter-productive to concentrate on certain platform-dependent details when you aren't willing to give attention to others. For example, current linux glibc memcpy, as well as Intel ones, has built-in algorithms to deal not only with variations on alignments but to choose between cached and non-cached implementation according to data block size.

You didn't even say whether your target platform has multiple memory controllers, which stand to produce some gain by engaging multiple threads when accessing memory.

Any parallelism hidden inside a function has to observe whatever threading models may be used elsewhere in the application, e.g. OpenMP or Cilk(tm) Plus, with the underlying implementation e.g. Microsoft threads or pthreads chosen according to the usual conventions of the operating system. So there is a lot to deal with if you are going to second guess your compiler and the vendors of your libraries.

Your remarks about a master thread imply that you have chosen a threading model. Why not use that model explicitly to distribute your memcpy work among a few threads?