I am working on a research project in which we are doing a comparative analysis of reinforcement learning (RL) with evolutionary algorithms in solving a nonconvex and nondifferentiable optimization problem with respect to solution quality and computation time.
We are using python implementations, but one difficulty is that, although we can use GPUs for the execution of reinforcement learning algorithm, there is not much support for using GPUs with evolutionary algorithms in Python.
On the other hand, if we want to compare the algorithms with respect to computation time, we have to execute them on the same hardware (parallel computing system).
However, we cannot run RL algorithm on CPU based parallel system because of our resource constraints.
Can anyone tell us how to establish an equivalent parallel computing systems, one based on CPUs & GPUs (for RL algorithms), and the other based on CPUs only (for evolutionary algorithms), so that we can compare them with respect to computation time.
Thanks in advance,
Best Regards
Are you limited to only one computer? Are there other computers available on a network? Even with old scrap machines and networking equipment, it is possible to build a Beowulf Cluster (https://www.linux.com/training-tutorials/building-beowulf-cluster-just-13-steps). A supporting package is MPI-Python (https://mpi4py.readthedocs.io/en/stable). This enables relatively easy message passing between networked computers so that independent processes that do not share memory can be coordinated. There is no reason such processes cannot use GPUs within the node hosting a process.
Beware of multiple processes, shared and non-shared memory, running on the same machine. The OS has to manage those and there has to be cores and memory to support them. With multiple machines (nodes) on a network, message traffic ultimately can overcome any gains made. Computation time per message interval needs to be much large enough to adequately limit message traffic.
Peter Raeth thanks for the response. Although you are right, we do not have any hardware with which we might be able to make our own cluster.
By the way, I am thinking that is there a way to determine the maximum number of CPUs (nodes) to maximally exploit the parallism in a particular evoulutionary computing algorithm like biased rendom key genetic algorithm (BRKGA) or ant colony optimzation (ACO) algorithm so as to achieve maximum speed up.
We are using Pymoo implementation of BRKGA and Pygmo implementation of ACO in Python.
Wish I could help but I am not familiar with Pymoo or Pygmo, or ACO for that matter. As for predicting the number of CPUs that would be optimal before an experiment, that, in my personal opinion, is very complex to pre-determine in any realistic scenario. My own approach with a particular system is empirical, adjusting the number as experience dictates.
It seems you are using a single computer with multiple cores, and processes that share memory. The operating system has to manage everything. That alone introduces overhead.
Are you part of a larger organization? I know in Ethiopia at University of Gondar, their IT Directorate had a stack of junk they were getting ready to bring to a landfill. We used that to build UoG's first Beowulf Cluster and to teach others how to program it, and its value to research. Perhaps searching about would give you the resources you need. There is a lot that can be accomplished, even with outdated equipment.
Peter, although you are right that predicting the optimal number of CPUs for running algorithm is tough, but I am currently building my knowledge base in parallel computing, algorithms and advanced computer architecture.
In regards to compute resources, actually I am running my machine learning and other optimization algorithms on google colab and as you probably know, we have only one gup or tpu with dual core CPU here.
I am graduate student at UET, Lahore (which is a big organization), and as it is a government university in a third world country, I am not able to get HPC resources here. By the way amaizing story at University of Gondar, I will try my best to hunt down these kind of compute resources!!
However, I will join LUMS as a research associate (for AI research) very soon, and here, I probably get resources I want.
- Badges
- Science topic
- Similar topics
- Linguistics
- Composition Studies
- Writing