There are new discovered devices which are nonvolatile. They motivate us to think about brain modeling. Is it possible? Are there any other newly coming architectures?
Interestingly,by the discovery of Memristor in 2008 in HP lab many former precious stagnated conventions like Von Neumann architecture may substitute by new Memristor-Based idea. As a simple example , in Von Neumann architecture the memory unit (RAM) and processing unit (CPU) is considered as separate parts and perform their tasks as two divided units while through memristor-based crossbar memory unit we can have a memory unit which can process simultaneously. Therefore the space and the power dissipation , variability in future computers will dramatically diminished.
The architecture that will prevail in the future is a multi-core architecture (32, 64, 128, … cores). The single-core architectures possibilities are now exhausted. New multi-core architecture will be based on a model that will facilitate the development of distributed systems using exclusively messaging communication (as communication using shared memory model is already dead). See functional language Erlang for concurrent and distributed systems.
As of now Von Neumann seems to be having an edge over Harward Architecture
The following Course Material brief
Microprocessor Systems Week 2 Unit 3: Computer Architecture Overview
discusses advantages and disadvantages of Harvard and von Neumann architectures. Link:
http://eecs.wsu.edu/~aofallon/ee234/lectures/ComputerArchitectureOverview.pdf
The Presentation by SGI Innovations projects "The von Neumann Architecture and Alternatives " for better results, discusses application specific accelerators and announces that it launched in 2008 Accelerator enabling Program. It suggests
• is your application / workflow , because it has to be parallelized
• and while you’re doing this, why not look at alternatives to get
– higher performance
– better silicon utilization
– less power consumption
– better price/performance
Link:
http://wwz.ifremer.fr/pcim/content/download/29481/407627/file/mfl.pdf
The leader in cell phones Architecture ARM also makes use of one of the two Architectures
We may have to wait for development of a Superior Architecture in future.
Probably AI may lead to a clue like "Summly " Program which summarizes pages of news into a single page. An Optimization Program with weights for different constraints regarding Fetch Time, Execution Time,Memory and their Types, number of Processors required etc...
Dr.P.S.
Fon Neumann architecture was built in times when both hardware and memory resources were too expensive. Now things are exchanged dramatically. The only thing which stops the Neumann architecture disappear is a huge socium of programmers who use this architecture to program.
I think that the future architecture will be probably the Java parallel machine.
The real question is not "if", but "when". While new technologies and models can out perform older architectures -- the older architectures still work, are proven, and are cost-effective. There are still plenty of applications and uses for 32 bit VN-based architectures, and at the same time, more complex and probably un-thought of problems will be solved by new memory/processing architectures, especially as expertise emerges (see prior comment by Sergiyenko).
Yes and No. Yes for numbers. No due to the latest technologies that replaces the old.
In my view all CPUs are still von Neumann architectures. Of course, some improvements have been made like Harvard (have 2 buses) and all kind of micro-level tricks (pipelining, branch predictions, VLIW, ...) that allow to have a higher peak performance, but fundamentally there all just more complicated von Neumann architectures.
What we need is an architecture that helps to translate specifications and (real-world) models into a processing/simulation engine. To start with the real-world is more or less concurrent/parallel by design. Except maybe the INMOS trarnsputer, no processor has good support for that (which means very cheap processes and communication). Even then the transputer was still a (sequential) von Neumann architecture at its core. It only had put some typical OS services in hardware.
The only deviating architectures I see are FPGAs. They have a dataflow based architecture and have parallelism in the space domain, whereas a von Neumann architecture can only have parallelism in the time domain (hence it emulates the parallelism that we model).
For the future, quantum computing might be the way to go. Not sure however how practical that will be. After all, whatever someone invents, it need to be general purpose enough. Same applies to e.g. more associative memory architectures (e.g. neural nets). In the end the real issue is that processing is I/O an storage bound. This must became part of the architecture.
As Prof.Eric Verhuist rightly pointed out, finally it is I /O and storage that govern any architecture or Processing time and capacity.
That is why we go for multiple Instruction Registers or Multiple ALUs or Multi-Core Architecture / Parallel Processing, Bus Multiplexing etc.
P.S.
Good question. I am sure that all bus oriented architectures, even multi-core, will be replaced by FPGA. Emulating parallel computing will be replaced by the real thing. Instead of intelligent systems we will have INTELLIGENT CIRCUITS.
Thanks so much. I got so much knowledge through your answers.
As a summary, there is a list of architectures and based technologies used for microprocessors
1- Multi-Core architecture with shared or separated memory
2- Parallel computing
3- Harvard architecture
4- Java parallel machine
There are also some hidden technologies used in design like pipeline, VLIW, branch predictions, etc.
Upon my experience, I am thinking about using another new architecture which is more compatible to artificial intelligent algorithms. Artificial neural networks and Fuzzy logics and others are used as techniques to solve some problems that are difficult or impossible to be solved with algebraic mathematics. These techniques uses some blocks that are trained first to be used later for new prediction. Therefore nowadays it is quite possible to build these blocks with new devices that can memorize last operations "as for training" to act for new predictions. So we need a new number system and new architectures to facilitate using these techniques that do not separate memory and processing units.
I think it will be so powerful technology as well as brain computing.
This article gives more description: "Memristive devices for computing"
http://www.nature.com/nnano/journal/v8/n1/full/nnano.2012.240.html
Good Article.
As Mohammed Mohammed pointed out Memristive Devices like Resistors,Capacitors that can remember previous Assignments, can be used for use of Learning / Adaptive Optimization Techniques as pointed out earlier by me that recourse to Optimization may lead to solution.
Can they be integrated with Multi -Core Processors as of now since Memristive Devices are of Nano Materials?
P.S.
Prof. Marius M. Balas idea of Intelligent Circuits is an interesting concept.
In a sense we can say that even our Logic Gates are intelligent except that they can act to only one condition. Logic Circuits can act to multiple IF Conditions.Only thing is that they cannot adapt and act to new situations that arise due to their one time action.
We can think of integrating them with with different layers of Neural Networks etc. in Optimization Process.
P.S.
Prof. Pisupati Subramanyam
I was thinking about the same question. I do not know if we can integrate a core based on memristive devices building Artificial Neural Network that can be used for recognition and prediction problems.
The current path seems to be branching out. DSPs are generally Harvard architecture, control CPUs are all Von Neumann, The future looks to be branching into two directions.
One is the Adapteva which will be soon selling a $99 credit card size 5 Watt supercomputer called the Parallella which will be a dual-core ARM A9 connected to 8x8 array of RISC floating-point processors.
Another path, followed by D-Wave is the D-Wave-1 which is a 128 qubit quantum computer. The front-end to the quantum computer is an off-the-shelf computer system to configure the problem for the quantum computer. They claim they will have a 512 qubit machine in the near future. For this particular quantum computer all problems have to be recast as minimization problems then presented to the quantum computer. The quantum computer will never be a commodity due to expensive refrigeration and shielding.
Regarding quantum computers becoming a commodity I should never say never although I think my comment is safe for a decade or two.
No, because the limit of CMOS technology is almost reached. Moreover, in the short term (say 100 years) the nanotechnology (as we see in the movies) will not be possible due to the enormous issues that must be addressed - one of that, is a self-assembling process, e.g., while with CMOS the individual blocks can be defined, in nano-world this is not possible without a significant number of hazards; therefore, self-assembling is required (as it happens with the ice crystals: beautifull and fascinating, but uneuseful).
J. P. Carmo, The limits of CMOS technology have not been reached. This technology has been basically 2D but 3D is here on a very limited scale with much more to come. Memory stacks easily due to their low power consumption. Eventually processors will be stacked as well. Wafers can be thinner to facilitate this as well.
Architectures condition the way people is writing the software. Rather than forcing the software to follow an architecture, in the TERAFLUX project ( http://teraflux.eu ) we try to propose a minimalistic variation to existing many-core/multi-core systems (that are therefore the classical Von-Neumann cores) so that people in the software could start producing software that is more "Von-Neumann indipendent".
Of course, in TERAFLUX we advocate the "Dataflow paradigm", i.e. as Jack Dennis, the father of Dataflow has defined it: "A Scheme of Computation in which an activity is initiated by presence of the data it needs to perform its function".
This is a very powerful concept, that will hopefully stimulate to think at current key problems like: programmability, resiliency/reliability, complexity/architecture in a radically different way.
Many, e.g. Pawlosky of Intel at a recent Exascale Seminar pointed what the end of the road for scalability as the systems will be so unreliable that we should chekpoint them too often. The fact that progamming multi-cores and many-cores is way too complex for normal humans is under the eye of everybody. While leveraging on current Von-Neumann cores we can still explore lot of interesting opportunities to tackle these problems.
Companies like Maxeler http://www.maxeler.com are commercializing "dataflow computers" since a few years. Apparently, most scientists and engineers recognize the importance of using dataflow paradigms. However, I see that few people is really capturing the potentials (and limitations, i.e. where and how to use dataflow paradigms).
Happy to investigate more this topic with those who are interested...
Still hoping that the software people will take a new approach!
As pointed out by Roberto Dataflow Paradigm is one of the good approaches for Parallel Programming different from Von Neumann Architecture.
There are different types of Dataflow again like synchronous Flow etc.
After all, the Multi-core, Parellal Programming are all needed for Processing huge amount of data as in Super Computers,which are used for weather prediction,Political analysis,Military Strategy,Space Programs etc.. They can be used for Knowledge based A.I. Programs also.
In this context I may point out that I am a bit fascinated by Dataflow and actors approach. Each actor can lay seize to one of the Processors / or more,if need be,for his role with the corresponding full or part of total Data or related processed data like in Pattern Recognition for Data Mining etc Of-course,the time taken by each actor may vary.There can be a Director to decide the amount and type of data to be allotted to an actor and he can also play the role of an actor for allocation of Data to different actors(like symptoms for different diseases and a particular actor to prescribe medicines for those particular symptoms)
Thus the Graph search approach used in Dataflow approach with Actors' role
instead of tree search followed by Von Neumann approach is drawing my attention
http://www.drdobbs.com/database/dataflow-programming-handling-huge-data/231400148?pgno=2
Dataflow Programming: Handling Huge Data Loads Without Adding Complexity
By Jim Falgout, September 19, 2011
Dataflow and Actors:
The actor model has been popularized by languages such as Erlang and Scala. In these models, independent actors communicate using messages. When an actor receives a message, it acts upon it as defined by functions inside the actor. Actors define communication endpoints that allow other actors to find them and send them messages. In some implementations, messages are free form and actors use regular expression forms to distinguish messages and how they are handled.
The following LINKS may also be of some interest.
http://www.ni.com/white-paper/6098/en
Notice that these the above example does not include code for explicit thread management. The LabVIEW dataflow programming paradigm ...
Why Dataflow Programming Languages are Ideal for Programming Parallel Hardware
http://www.visual-paradigm.com/highlight/highlightdfd.jsp
Data Flow Diagram
Data flow diagram is a well known approach to visualize the data processing in business analysis field. A data flow diagram is strong in illustrating the relationship of processes, data stores and external entities in business information system.
P.S.
Joe Armstrong, inventor & developer of the Erlang language, said in the Foreward for the book “Handbook of Neuroevolution Through Erlang”:
“Erlang was designed to scale. So today we can run a few million processes per node, and a few dozen nodes per chip. Computer architectures have changed from the single core Von-Neumann machine, to the multicore processor, and the architectures will change again. Nobody actually knows how they are going to change, but my bet is that the change will be towards network-on-chip architectures.
We’re already talking about putting a few hundred to a thousand cores on a single chip, but for this to happen we have to move to network on chip architectures. We can imagine large regular matrices of CPUs connected into a regular switching infrastructure. So we’ll soon end up with thousands of cores, each capable of running millions of Erlang processes.
What will we do with such a monster computer and how are we going to program it? Well I suspect Gene has an answer to this question; he’ll want to build a brain.”
Since the 60's dataflow architecture pops up over and over again. I believe the dataflow hardware "bottle neck" problem hasn't been solved to everyone's satisfaction.
Mammal brain organization, I think gives us a clue where we are likely to end up. Here we have tens of thousands of specialized processes with a global associative mechanism (the cerebral cortex) that we are just beginning to understand. After we understand how it actually functions we may be using millions of Von Neumann, Harvard or a combined architectures or still another yet to be determined architecture to perform the same function. There will be many ways to perform these same functions and many approaches will be used at first. After, solutions will likely become more application specific with architectures chosen that are best for those applications.
Dataflow programming languages have some features in common with functional languages. Erlang is a functional language.
...I defend dataflow here.
Instead of discarding possible solutions, I'd rather suggest to look at the main problems now: are we able to easily program many-core based Von-Neumann style architectures? Anything that solves this (and it proves it now) will be worth consideration.
Dataflow paradigm, often misunderstood, is and will always be the most efficient way to manage data, since it is based on the super-solid foundations recalled by Jack Dennis. Like it or not, all superscalar processor of nowdays -- the one in your computer, smartphone, tv-set, ... -- are based on a dataflow engine, which is what makes them working so well ! What slows down and impedes further developments is what is around such dataflow engine...
While we require determinism there will always be a place for Von Neumann style systems. The multi-core, dataflows , SIMD etc are all based on deterministic - program load, data process style. The associative, state-based style organic systems (eg us) which is inherently non-deterministic and can access "potential responses" (states - an equivalent to data) in such a manner that they are perceived simultaneously is currently undesirable. Determinism and speed is what so good about von-N /CPUs but with the downside of not being so suitable for adaptive responses (intelligence). Association shifts the paradigm away from serial processing (or multi-serial = parallel ) to response-lookup in the context of continuous consistent feedback
Prof.John Sander's description, " multi-serial = parallel " is interesting.
Von Neumann Archtecture will not die,but it may become an appendage to other Parallel Architectures.
P.S.
In my opinion, even it may seem far, I think the future architecture will analyze the application and they will adapt in order to achieve the best performance. E.g. FPGA based architectures or GPU based applications are developed for high-throughput digital communications, real time signal processing etc. A significant breakthrough would be an architecture which analyze the application regarding computational complexity, and using parallel computing resources an optimal architecture is generated which achieves the best performances regarding reliability and speed. Various application specific hardware architectures for image processing are proposed in "FPGA based system for automatic cDNA microarray image processing", Computerized Medical Imaging and Graphics, vol. 36, no. 5, 2012.
I agree with Subramanyam, Von Neumann architecture will not die. We will see in the recent future new Heterogeneous machines, with regular processors, FPGAs, GPUs and ASICs integrated in one unique system. For this reason, I also wouldn´t bet on a unique programming language. We will have a better language for each kind of application: maybe C/C++ for systems that will how on the processors, VHDL or Verilog for FPGA, CUDA for GPUs and maybe a Actor-Based language for the application in a higher-level of abstraction.
My view is that the heart of the problem with von Neumann is data is stored in addressable memory. Addressable memory has to be managed. An alternative approach to storing values in an address is required if we are to get a significantly different type of architecture. The focus should be on how to process variables rather than processing instructions. This can be achieved if a variable contains semantic information as well as the value. The semantic information needs to both uniquely identifies the information being processed and details of how to process it. Such a model allows variables to be stored on a queue getting away from storing values in addressable memory. The biggest problem as well as the biggest potential benefit is how to synchronise the values involved in a binary operation. If the problem can be solved parallelism is at the finest grain. I am working on developing this approach which is showing promise. The major problem is to establish a proof of the concept with few resources.
Conrad Mueller's idea of a variable containing semantic information as well as the value and placing them in a queue or Stack or list may work for computation work.Even the object oriented concepts like classes, inheritance etc. for such variables may be tried.But even in such cases I feel that Address Location of group of such Variables cannot be avoided altogether(like Header Address) though it can be minimized.
For other types of operations further detailed investigation will be needed.
P.S.
You could find the answer on https://www.researchgate.net/post/What_happened_with_The_Fifth_Generation_Computer_Systems_project_FGCS
The probable future architecture will be the multithreaded architecture. That means that thousands of threads are running on hundreds of PUs, controlling each other by signals. The main entity will be not an instruction but a single process. This reminiscents the forgotten transputer system.
It may depend on the definition of Von Neumann Architecture: if you refer to a single bus, almost no architecture today follows Von Neumann. At list, from the CPU, exit two buses, One for I-Cache, the other for D-cache. After, if there is a unified LLC, the system become again a Van Neumann ones.
In the field of DPSs, noone in van neumann, as they need of more buses to put data inside (for instance, to perform MAC in a single cycle). They have also more than one address space.
More advanced/specialized architectures, as they all need to provide data fast to the core, can not follow the van neumann paradigm.
As this is also the main trouble of multicore, it is possible to argue that the future of the van neumann, in such field is a bit dark.
Anyway, in the field of low performance microcontroller (that represents a big fraction of the CPU market), van neumann is still one of the most valid solution.
Hope it may help,
Piero
The alternative to Von Neumann Architecture,which may be compared to Serve-Client Architecture is to search in Peer-Peer Architecture,with no or minimal server control.
P.S.
von Neumann means instruction-stream-based, independent of the number of buses and the number of processors. it is by several orders of magnitude less efficient than a data-stream-based machine paradigm. next generation supercomputers are unaffordable if von-Neumann-based only - bacause of the unaffordable electricity bill. That's why supercomputing is heading toward heterogeneous. However, von Neumann will survive because of the enormous amount of legacy software throughout the world. It will take decades ro re-implement all this for going data-stream-based. Also see
http://helios.informatik.uni-kl.de/staff/hartenstein/Reiner-ASAP-2013.pdf
http://helios.informatik.uni-kl.de/staff/hartenstein/Reiner-ASAP-2013.pdf
Tiny cores of processors can be used with Harvard architecture along with pipes.
The von Neuman architecture could only be replaced by a Quantum Computer.
@Breuer that is a CDN, or follow-the-sun type deployments.
To go back to the original question: will the von Neumann architecture see a diminished use in the next couple of years? Answer is a resounding no, we will see a significant growth of the use of these machines as the Internet of Things will proliferate computational devices far and wide and the bulk of those will be small von Neumann machines. In addition to von Neumann Architectures, we'll see an increased use of dedicated computational pipelines that take advantage of the attributes of the workload as that is the key to functional efficiency and consequently to low-power consumption.
Can anyone define what makes an architecture Von Neuman (other than a stored program) ?
Wikipedia has a rich description:
http://en.wikipedia.org/wiki/Von_Neumann_architecture
in essence it 'stored program". von Neumann was the one that observed that program is 'data' in its essence and thus could be stored along side data data in the 'core' (what we now call memory).
The Von Neumann cycle:
- fetch instruction
- decode it
- execute it
- increment program counter
Flavors with more details exist.
A quantum computer version is underway.
http://phys.org/news/2011-09-physicists-quantum-von-neumann-architecture.html
http://arxiv.org/abs/1109.3743
Regards,
Joachim
When you ask if the " Von Neumann Architecture will be diminished ?" you are really referencing the CPU, memory (RAM) and the medium where the program is stored (flash, HDD etc ...) . Architectures do not just diminish. There are thousands of programmers that are thought to think Von Neumann-like. I teach a C programming course. Once I teach C to a student, (s)he is "programmed" that way forever (almost). It would be very hard for INTEL to try to get 100000's of programmers to start thinking in a different way. All the way up to Core i7, Xeon, they all work the Von Neumann-way. There have been very very cool attempts in the past: THINKING MACHINES introduced a hypercube machine in the early 90's, which was one of the starting points of the GPUs (SIMD machines). I actually programmed in it. I had to spend an entire semester to learn the PARIS language (PARallel Instruction Set). THINKING MACHINES was later sold to Sun Microsystems. It was an amazing machine, but, nothing more than a SPECIALTY machine.
I saw an answer "Quantum Computer." This is a very good answer, but, the Quantum computer won't be anything more than a specialty machine. Cray has been making super-computers for decades. They are all specialty machines. Quantum Computers (which I actually have a great interest in) are optical and they take up a lot of space. Before they remotely catch up to CMOS based computers, CMOS computers will be at 11nm technology working happily with their trigate transistors (if not 6nm !) . Including myself, we have been saying that "CMOS will be dead in the next few years." Ok, it has been 20 years. It's not dead ! Something like CMOS and Von Neumann just do not die. When there are such prominent architectures at the time when people need it, 10x billions of dollars of investments are made based around them (including training programmers, building fabs for chips, documentations, training programs, training to fabricate the chips). I would say that, something has to be 100x (if not 1000x) more advantageous to wipe out the predecessor (and be a disruptive technology).
This is coming from a guy that actually teaches GPU classes. I teach GPU programming with great enthusiasm since they are fundamentally different than CPUs. Each core follows a Von Neumann architecture, but, the GPU as a whole is fundamentally different. And , they ARE 100x faster if the application is right. So, why didn't GPUs wipe out CPUs ? Well, where do I start ?
1) Training : It is very difficult to understand how to write high performance GPU programs. Once your brain is wired for CPUs, we have to break it down and build it back up ! So, you can learn how to program GPUs. If there is any fancy architecture to wipe out the Von Neumann, its training will possibly be a nightmare ! unless this is the first programming language you learned !
2) They are specialty hardware. Not every application benefits from it.
3) If you want to add two 32-bit numbers, a, and b , it is a single instruction in a CPU, and it is a meaningless operation in a GPU. You can solve 95% of the problems SERIALLY. You don't need the massive parallelism of a GPU. This is why the new CPU designs are being very careful about SINGLE THREAD PERFORMANCE (i.e., serial execution performance). INTEL made the grave mistake of improving the WRONG performance before (for example, Pentium PRO improved 32 bit performance at a time when everybody had 16 bit machines, and people thought Pentium Pro was SLOWER).
4) To contradict (1) and (2), they actually ARE seeping into the CPUs and becoming
the best roommate of the CPUs and showing up in architectures like AMD's Fusion APUs.
Just to summarize your question: NO ! Old habits die slow ! There will be pretty cool attempts to kill the Von Neumann. But, he won't die !!!
I made a comment earlier regarding the D-Wave quantum computer. The D-Wave 512 qubit quantum computer has been delivered to NASA Ames Research. Their roadmap has them quadrupling qubits every two years. The real holdback for quantum machines is that they need an RF and magnetically shielded enclosure and refrigeration to 0.1 degrees Kelvin. This shielded refrigeration system comes at a price that doesn't follow Moore's Law thus limiting commoditization.
Tolga Soyta above spotlights another trend using GPUs for large scale computing like NVidia's Tesla K20 that has dynamic parallelism making it easier to write compilers for parallel computation. This and the commoditization of the Parallella "credit card" computer will possibly usher in a new age of high performance computing.
I am answering John Sanders' question: Can anyone define what makes an architecture Von Neuman (other than a stored program) ?
*** Von Neumann is, in its simplest form is composed of four parts: a) Input data, b) program, c) execution engine, d) output data. Notice that, eliminating part (a) won't remove the von-Neumann feature. So, (a) is optional.
*** a computer that calculates and outputs all prime numbers and requires no input, however, does it based on a PROGRAM is considered von-Neumann (I will call it VN for short). Also, notice that, the ability to CHANGE THE STORED PROGRAM any time we want is a requirement for VN. IF I can't change the program, then, I really don't have Part B. So, it is dedicated hardware, so, it is not VN.
*** A dataflow machine is not VN, since it is missing Part b. This is the essential part. In a dataflow machine, what is done is always fixed, so, no reason for having a stored program. In a sense Part (b) and (c) are merged into a single dedicated hardware.
*** An FPGA is tricky, since it is a reconfigurable architecture and can be MADE to be VN, with, say, soft cores like Xilinx MicroBlaze. Also, it can be MADE to be a dataflow engine, hence a non-VN. So, it is not the FPGA that matters, but, rather, how I RE-configure it.
*** A Quantum computer is defining itself these days, with Qubits being used as registers, etc , so, a stored program is too difficult to achieve due to the difficulty in reading that program ultra-fast. However, if a program is stored to execute a sequence of instructions and IS CHANGEABLE ANYTIME, it is a VN. Here, in a Quantum computer, the (a) input data is supplied by a user (or an array of sensors) , (b) a stored program is stored in a medium that can be accessed via fiber optic lines that read the program at speeds unheard of for a CPU and can be changed anytime by uploading a new program, (c) execution engine if the optical Josephson junctions, and all the optical gear to COMPUTE, and, finally, (d) the output is displayed to the user through a high speed output interface. So, it IS von Neumann !
*** On the contrary, a Quantum computer can be made to execute the same thing without requiring a program. Here you go, NON-VN.
*** An ASIC (Application Specific Integrated Circuit) is a VN architecture if it is designed to accept a program from a storage medium, say, RAM, or flash. On the contrary, it IS NOT VN if it is designed to accept an array of data, do the same thing to it, and output it.
To my knowledge, it is possible that Von Neumann architecture may diminish significantly as a new technology of CPU, RAM and computing methods will evolve over a period of time in the future. I think that, if you consider Von Neumann as a overall architecture of the computer you are sitting next to, then according to this definition, the Von Neumann architecture is already diminished. The motherboard has more than one CPU because it has GPU and lots of microcontrollers and FPGA's which all together form a network of processors at work. I see them as complex neurons linked together performing multitasking which is fantastic. In the future, we are going to see more parallel processors, evolution chips that mimics genetic code, and of course quantum computers that may not necessarily mimic Von Neumann architecture. We already have alternative architectures such as Modified Harvard architecture in pic's such as Z8Encore!, routers and modems.you can make your own simple alternative architecture by downloading OS on SD in Raspberry Pi and making SD write protect using a physical switch on SD. It runs without modifying data or program on SD. I have tested it. In my opinion, that is pretty much an alternative to Von Neumann architecture.
The FPGA is not a language based system and its purpose is defined by the model downloaded onto the internal programmable circuit-layout. It processes data streams such as those found in sensor based systems and it well established in the Real-time / sensor domain. This is an architecture, but not for general purpose processing but for customised hardware. If the original question was looking for new architectures in the computer-base command and control / sensor / engineering domain then this is here now (and you can now skirt round the complex models and simulation (Quartus (Altera) and ISE? for Xilinx from example , instead you can just about access through a C-like kernel using OpenCL (see Altera website). This belongs to a soft-configurable hardware initiative (OpenCL - Heterogeneous Parallel Computing - its started with Multi-core and GPUs and now has extended to FPGAs. (see Kronos website for details. FPGA are software configurable hardware devices - no stored program just a circuit. Non von Neumann but speacialised to a degree.
Hi John,
I actually listed FPGAs as a discussion point . If you assume that, we are building dedicated hardware with an FPGA, then, T.J.'s points are completely accurate. But, sure, since I can build anything with an FPGA, I can go ahead and build a Von-Neumann architecture machine. So, both of you are right !
At present the FPGA systems are on the main for small embedded devices, but if scaled up you could in principle compile anything into an FPGA. Already there are high level tools such as Matlab and LabView which allow you to convert an algorithm into an FPGA circuit on a target device. Scale this up and you have a compiler which converts any software to hardware - and it's concurrent too.
Maybe the architecture of von neuman could dissapear why?
Because all incomes and the tech in the future could be that the memory of instructions could be in the " cloud "
Hmmm. Carlos, I think this is a very important answer ! Somebody had to bring cloud computing into this discussion. Thank you for that. With the cloud computing becoming so pervasive, this is the only thing that could really threaten the von Neumann (VN) architecture in a major way at the "Macro" scale. However, each datacenter will still need a processor ... So, the trend seems to be VN at the "micro level" and non-VN at the "macro level" with the MACRO architecture shifting away from VN.
Now, let's even challenge this a little bit. I remember an article in the MICRO magazine (top microarchitecture conference) a couple months ago questioning the "role of the CPU." Actually, forget the VN architecture, will the CPU even survive ??? With the cloud computing becoming so widespread, the kind of CPUs that can efficiently process the information at the datacenters are more like THROUGHPUT engines, not necessarily like traditional CPUs, which try to process a SINGLE JOB efficiently. The best paper award at the ASPLOS 2012 conference (one of the top Systems conferences) went to a research group that had an article : CLEARING THE CLOUDS. The point of the paper was that: the kind of CPUs that can efficiently process the information that comes into a datacenter computer are significantly different than the Core i5, Core i7 you and I use at home. They should be throughput engines, with 8, or , 16 threads in each core with 8, 16 cores total (i.e., 256 threads in the CPU), where each thread is not necessarily that powerful, and each core with not necessarily a huge amount of cache (e.g., 8KB as opposed to the 64KB of the core i7). The reason for the need for less cache memory is, since, the data re-use is very minimal due to the huge THROUGHPUT.
So, to re-phrase the question : will the CPU even be as important (or, to exaggerate, even relevant) in the future ? Will the network equipment surpass the CPU in importance ?
In my view, the CPUs will remain Von Neumann with increasing support for *innovative* multi-threaded multi-core implementations. The dim silicon problem will turn into ever smaller and more numerous cores, with a one-thread one-core phylosophy as the easiest and therefore more power efficient solution. This will turn into dim silicon management support, native concurrent process management and communication support (which does not mean that shared memory is excluded as extremely flexible underlying mechanism), possibly approximate computing instructions/procedures to trade off power and accuracy.
The only alternative to the von Neumann model I see at present time, if we stick to digital world, is the "CLB" ( Configurable Logic Block) used in FPGAs "every CLB consists of a configurable switch matrix with 4 or 6 inputs, some selection circuitry (MUX, etc), and flip-flops." (from Xilinx site) every CLB is strongly connected with many other CLBs. Here the model is pretty different from the VN machine: not a fixed unit accessing a variable program on a fixed memory, but many much simpler elements that are programmed by means of interconnection, internal and external. Must be noted anyway that the CLB is not the product of a theoretical modeling effort but the product of a technological evolution, therefore, as long as I know, it was never used to make ipothesis on the capabilty of a programming machine or like the Touring machine to solve problems in computability and solvability. Usually a model is used for understanding and development and, seen by this perspective, the CLB can be not very useful.
Good question.
1. CPU single-core, CPU multi-core, vector processors (machines), GPU, and MIC (Xeon Phi, Intel device), and ARM processors are examples of Von Neumann architecture. FPGA is a device able to do computation into a non Von Neumann architecture, and even trans-Turim computation (beyond Turim's machine).
2. We are still embedded into electronic world (circuits). However, we are moving to a new world: photonic computers (linked to optical computing). The first photonic processor is FPGA type. Additional information (and controversy) see: http://en.wikipedia.org/wiki/Optical_computing
If we are to make an order of improvement on the von Neumann instruction/address based model, there needs to be a shift away from addressing. Not only does it cause the data bottlenecks identified by Backus, but results in undesirable coupling. Worse of all that the order of execution needs to be defined.
The model I am working on processes elements that are made up of semantic information and a value. Processing an element results in zero or more elements being created. The semantic information is unique and part of it determines how the element is processed. There is no restriction on the order or how many elements get processed at once. The most important aspect is that they do not need to be stored in addressable memory.
While I have emulated a cut down version of the model in C, there is a long way before I will be able to implement a realistic implementation with limited resources.
If we use electronics the VN Architecture will be the choosen one. The architecture is well understood and the effort to change is huge and complicated.
Defining a new Architecture depends on the used technology. If you use quantum computer that can leed to a different architecture. But nowadays the quantum computer device will be interfaced by classicla electronics. So you have a more or less parallel working computer but the data will be read and write sequentially.
An different approach can be done by biological machines. Self replicating and organized "bacteria" which full fils a higher purpose like bilduing structures. Thus is different from the calculation machines which we use today but it can leed to a diiferent architecture.
How long it will take to create those machines id dont know. It depends on the available technology the understanding and at least of the application. If you can earn enough mony with those new machines then they will be build.
Dear Tolga,
I think there is a multilayer architecture for biological machines. First you need a layer for building and destrcution. Name it recycling. Nature is full of such processes. Than you need the layer of control to keep the process in line. And you need a meta layer to pursuite a goal. That are the necessary mechanism. If you have different of those "machines" wich are "cooperating" or "fighting for ressources" than building a living planet or structures with a higher purpose is possible.
I think that Neural Networks may increase the computational power. Our brain work on this model, with slow clock speeds and more processing power.
The architecture is the computational model where the algorithm is running.
PC has the architecture which implements millions different algorithms.
Neural network implements a single algorithm.
Therefore, at least 20 next years none architecture will substitute the von Neyman architecture and its derivatives.
Marcelo, I'm not so sure about neural networks. Much of their pattern matching behavior can be matched with the analogue of using hidden Markov processes on a highly parallel machine built with tens of thousands of VN or Harvard MIMD RISC cores or an SIMD Harvard machine that lacks some of the flexibility of the former. Graphics boards have been doing both for years and there is no end in sight. It's like gallium arsenide competing with silicon. It's a matter of where the money is concentrated.
There is nothing new under the sun. VN, at least interpreted broadly, is absolutely fundamental if you want a general-purpose computer. The volatility of your memory is not architecturally relevant: whether it stores values is all that matters, since VN uses it for code and data, and both need to change. Parallelism is also not really contradictory - it just means you have lots of VN machines - and this is the path we've been following for decades. In the future, the particular arrangement of VN cores will certainly change: almost any system today has multiple cores, usually mutating the same shared memory.
Personally, I expect that cores will start to be packaged *with* memory, so that when you buy a system, it is mainly a fabric connecting dimm-like memory-compute modules (with, of course, some IO devices). (NV circuit elements doesn't really change this picture, though it would certainly change power requirements and would permit some interesting new system/software designs.)
I don't think the idea of an FPGA based web-server will catch on anytime soon. The reason being that the configuration would have to depend on the web address asked for whereas what is required to exploit the parallelism is a configuration dependent on the number of simultaneous accesses. Any problem if sufficiently recursive will reduce to a Turing machine which conveniently is implemented as a VN architecture. The one compelling reason for FPGA is massively parallel multipliers which are rarely used on the web, otherwise price/performance is much worse than a hard processor
For general purpose and near future (next 10 years), von Neumann architectures follow to be the leader. For some intense processing applications, FPGA (a trans-Turim machine, and they are not a von Neumann architecture type) enhanced its use. FPGA is more "green computing" driving than CPU. Other issue: photonic computers will go to compete with electronic computers (before quantum computers)
Perhaps, reconfiguration CPU Architecture ( P.L, Selective No. of cores, reconfiguration of both fine grained and coarse grained PE, nano or quantum computing). From another perspective, merging CPU and GPU computing units.
Von Neumann architecture is a binary model for computation. i think as two programs in my book available on research gate indicate the binary model can be replaced by rnary model ( yes, no or can not predict) and further n valued logic very easily using ? switch combination using c language and using c as native machine language like lisp machines.
the other program which is palindrome in my book and there multiplication by zero is used to change a state permanently. this idea can be implemented in hardware.
both these ideas are simple like von neumann architecture and easily implementable in present sate of technology and logic gates could be manufactured..
also another improvement that is possible is implementing relational algenra ( basically set theory ) in hardawre. since relational algenra has same power as simple predicate calculus, skolemization can bring some inherent intelligence to machines.
The reason computers are binary is purely technology driven. n-ary logic assumes n voltage levels and that's much harder (without errors) than binary logic whereby the transistors are either switched off or driven into saturation. Even then, the signals don't look like nice squares. So, n-ary logic could work at higher voltages (more margin) and lower frequencies. So we would have less dense chips and higher power consumption. Neural net chips (analog) come in that neighbourhood, but these are statistical classifiers, not really useful for precise calculations (even if they can do better than binary implementations). Maybe the future will be hybrid and heterogeneous. Chips will have several on-chip domains with architectures adapted to the task. Is more or less already done in the digital domain. For integrating analog circuits, unfortunately we still need wider silicon features. Although some RF chips already exist in CMOS operating at 70 GHz.
The other aspect is that it very much depends on what chip design tools allow. For example asynchronous logic has many advantages, but most people (designers) are afraid because they don't know it and the tools are lagging behind. And there are projects underway whereby chips with feature size accuracy of 5 nm will be printed. This could be a revolution as well, as it means that - given adequate design software - anyone can produce his own application specific chips, even in low volumes (like 3D printing does now). This also opens the door for innovative people to come up with innovative architectures (which are not affordable with the current batch based processing).
Three level(Tri- state) are used in Buffers in Micro Processors.Is there any Digital Device available with more Levels?
P.S.
Mr, Eric thank s for your answer. but i suggested very different. there are lisp machnies available.. binary chip design is economical. what i suggest is two programs in my book available on reserchnet. rots of a quadratic here switch is used to simulate three states. and palindromes here multiplication by zero is used so that once flag is ed( zero) it can not become green ( nonzero) again to vaoid use of one more iof. these ideas can be used in design of chips. and machines can be bulit with a subsetof c as native lenaguage like lisp machines.
also relational agebra is used in all databsses and thus an archtechture which uses the relational algebra( basically boolean algebra ) can be moee efficient. in fact if one studies logic programing a substantial pat of it then could be implemented and we can have more intelligent machines if hardawre and spftware engineers work in tandem on this aspect.
My answer for Von Neumann machine be alive or not, is yes or no. Our Brains consist with huge neural nets and somethings called mind. The formers are like hardware circuits, and latter is seemed like software. But, no one cold not find Von Neumann like CPU in the Brain. This means different type processing architecture, more efficient processing one is found by the Brain. The distributed and parallel processing modules, those are connected in data flow manner, are controlled under OS like software so called self- consciousness. To make this machine, we have to find design methodology for brain, different from Von Neumann machine, However, I imagine that the rational Brain is trying to work such a logic machine as Von Neumann machine, because the brain does not work logically, Then, Next machine will be hardware and software complex system with Von Newuman CPU with hardware circuits built in many FPGA chips.
I agree with Masatoshi Sekine that the next really fundamental computation paradigm will require an understanding of the computational methods found in biological systems. Whether these systems can be replicated in silicon, or whether they fundamentally require wet-ware to operate will need to be determined by experiment rather than philosophy. There is much work to be done and we are still very much in the dark regarding how single neurons work and especially on how a unified consciousness arises from a large mass of independently operating neuronal cells. None of the existing theories or books purporting to 'explain consciousness' ring true, and we might still be centuries away from a proper theory of mind.
It looks like that in the near decade or two, the general purpose computers will stuck to the von Neumann architecture. It will require significant, or better say - revolutionary technological advances and also financial investment (hence - time) to switch to other model. No doubt that recent advances in brain research and brain simulation will support a possible change in the model. One must also take into account the emergence of Internet of Everything (or of Things), where distributed processing (even though "things" will most likely follow the traditional until now architecture) will became dominant, as well as the growing volume of digital stored information. Of course, one can think of the quantum computing model and maybe some biological models but that does not appear to be round the corner.
I don't think that the von Neumann architecture will go away anytime soon. The current development has some really interesting modifications to the von Neumann architecture. CPU and GPU cores get integrated onto the same chip. CPU memory becomes accessible by the GPU as well. And with the new AMD chips and OpenCL both CPU and GPU can issue new computing tasks that can either be handled by the CPU or GPU. This gives a lot of opportunities for faster computations during the next couple of years or even decades. As long as computing power further increases there is no real need to replace the von Neumann architecture.
The real reason why I think the von Neumann architecture will not be replaced anytime soon is because of compatibility. With a new architecture there will be no tools, no OS and no libraries. We would need to start at zero. Besides the cost for new technology there is also a very high cost for software. Due to the lack of an OS and corresponding applications a new technology will not make it to the consumer market. But the money that comes from the consumer market drives a lot of the investments. So, the main reason against adaption of a new technology is the cost.
Still, quantum computing might have a chance. But not because it is a new architecture, but because it is a new computation model that allows to solve different problems in an acceptable time. Similarly, DNA computing and other biological computing methods might have a chance for scientific applications because the algorithm is the computing hardware itself.
The von Neumann architecture hasn't "failed" as such, but Moore's Law flat-lined back in 2003, and we are now reliant upon having multiple cores in parallel to speed up any parallel elements of our our computational tasks.
http://www.extremetech.com/wp-content/uploads/2012/02/CPU-Scaling.jpg
There is a limit to how long this can continue for, because the algorithms to be calculated will only have so much parallelism that can be exploited, and when the number of cores exceeds that number then no more speed-up can be obtained.
I work with field programmable logic arrays (FPGAs), the biggest ones money can buy. (>3 Million equivalent ASIC gates).
FPGAs do not generally use von Neumann architecture processing and do not have the same bottle neck limitations.
They do not have instructions stored in memory which must be retrieved, taken to a CPU and executed, and then the next instruction fetched.
Rather, they have distributed memory, and distributed state machines, operating in a parallel and pipelined way.
I am currently designing an FPGA to process 100Gbps Ethernet data in real-time, and an FPGA which eschews the von Neumann architecture is the only way to do this. DSPs and CPUs can't even get close to the required speed.
FPGAs are currently a pain to program, but so were the first patch-panel programmed computers, and the programming tools and compilers will get better over time.
The Von Neumann architecture will not be diminished briefly, but unconventional architectures like IPNoSys (dx.doi.org/10.1049/iet-cdt.2008.0071) or others reconfigurable architectures, may occupy an important space in the next few years.
TriBA (Triplet-Based Architecture) is a Object oriented CMP architecture, hope will be at crest in future.
"Feng, S.; Weixing, J., Baojun, Q., Bin, L., Haroon, .R.,:A Triplet Based Computer Architecture Supporting Parallel Object Computing. Proceedings of the Eighteenth International Conference on ASAP, pp. 192-197, July 2007"
Look at the https://www.researchgate.net/post/What_happened_with_The_Fifth_Generation_Computer_Systems_project_FGCS
With the explosion of portable and embedded devices, energy efficiency is likely to become the biggest driving force between innovations in computer architecture. I think it will reopen the doors for the idea of stochastic computing. Why? One of the ways how to reduce the energy consumption of a CPU is to reduce its operating voltage. For conventional digital-logic based systems there is a strict limit of how low you can go while still getting reliable results; lowering the voltage further leads to unreliable operation, because it becomes harder to guarantee that all components of the system will operate correctly when voltage levels of digital zero and digital one are not apart far enough. So the idea is to counter this by building systems which internally use probabilities rather than digital logic, and can produce reliable results even when working on unreliable hardware.
Moving further along one of the dimensions of this approach, you get into the domain of analog computing, which is another area suitable for future exploration. I think that big advances are possible in these areas (stochastic and analog computing) because they are comparatively much worse understood than building digital-logic based systems. From what I have read, nearly fully automated design checks of digital systems are possible at the moment, while in analog systems you have to rely on experienced engineer's know-how.
We performed a project rp8601 being a complete new technology from chip cells to programming language. It was based on that execution was a see of computation. Everything was in parallel. Real time was side effect free!
http://www.carlstedt.se/Carlstedt/forlag/skrifter/91-89546-15-6%20rp8601.en.htm
The Brief Introduction on rp8601 is Quite Interesting. These Chips when available and used may mitigate the problem of slow execution faced in Von Neumann Architecture.
P.S.
Von Neumann Architecture is at the highest level of abstraction.
I feel still the computation model remains or will remain the same. Input-process-output.
Almost all the previous post were talking about specifics. One may add more sophistication to the architectural entities and see a leap in performance.
New architectures for near future could be the fusion of CPU and co-processors. Today the main co-processors under investigation are GPU and FPGA, and Intel has presented MIC (Many Integrated Cores = Xeon Phi chip). As I mentioned, there are initiatives combining CPU+GPU (the Fusion processor from the AMD), and CPU+FPGA (the Stellarton chip).
For CPU+GPU (in the same chip) see:
http://www.xbitlabs.com/news/cpu/display/20120202102405_AMD_Promises_Full_Fusion_of_CPU_and_GPU_in_2014.html
For CPU+FPGA (in the same chip) see:
http://www.eejournal.com/archives/articles/20101123-stellarton/
Xilinx Zinq7000 or Altera SOC has Arm cortex CPU and FPGA block in a chip. Linux OS is running in these CPU for controlling the circuits in the FPGA block. Thus, Application software running on Von Neumann Architecture machine controls the virtual circuits proceeding in the FPGA block, at the same time. The requirement to this direction, Von Neumann Architecture will be extended to handle concurrent controls for the circuits. Perhaps , multi-threads /multi-cores/multi-circuits Architecture will be proposed by some one. In the abstraction level, software in the memory space and virtual circuits in the circuit space will be recognized.
Dear Mohamed Gamal Ahmed Mohamed,
The concept of Von Neumann Architecture has not changed because of the developments. Parallel processing, multi (cores, thread, ..), are extension of the basic concept of Von Neumann Architecture. Further, the ASIC, SOC, PLDs, FPGA, .. are good support for extending the architecture.
Actually, I think that in the future, Harward architecture might prevail. It is applied in caches already, and with the falling prices of memory, and need for more security, there might be a good reason to switch to it some day...
My question is what really has to be addressed in future architectures? Surely it is to improve parallelism and reduce the memory wall. The instruction addressed based architectures are not promising candidates to make progress on these two areas for the following reasons.
In an instruction based model, the sequence/order of the instructions is important. Yes to some degree, as in data flow, parts of the sequence can be done in parallel but it is still expressed as sequential code. Parallelism can be achieved by having concurrent processes. The parallelism is at the coarse level of processes. Programming concurrent processes is complex and introduces overheads. The operating system has to manage these processes: allocating CPU time, access to shared memory or message passing, interrupts etc.
The way in which addressable memory is used is highly inefficient. Each instruction uses at most two values in memory: an instruction and an operand. In RISC, most instructions only require to load an instruction. There are large parts of memory, even in cache, that are: never accessed, accessed a limited number of times or accessed only for a short period of time. What is the probability that any random memory location will be accessed in the next 1000 instructions? My guess very close to zero. Large chucks of memory are moved in and out of different hierarchies of memory of which a minute faction are used while the block is in cache. This does not look like the correct model to address the memory wall. One requires a model that is more efficient where only the data about to be processed is in cache/memory: more of a just in time model.
Ideally the model should be able to break away from the concept of a sequential set of instructions and addressable memory. A program should rather be described as relations. By attaching semantic information to data, this information can be used to determine to which relation(s) the data applies. The results are the creation of zero or more new data elements which in turn can be processed. In this way the memory wall is overcome in that only data that is about to be processed is fed to processors. The model is inherently parallel in that any data elements ready to be processed can be processed in parallel at the finest grain. The model show promise but with the few resources I have only been able to emulate the model in C.
Two days ago i rememberd on asynchronos logic during my studies also i was confronted with reconfigable computer logic. And yesterday i've learnd that Niel Gerschenfield and his group from Bits and Atoms developed the (Reconfigurable Asynchronous Logic Automata) RALA approach (see http://dspace.mit.edu/handle/1721.1/72349)
In case it schould be possible to develop a minimal set of asynchron logic unit which are capable of calculate or store data. If it is possbile te establish a "learnig mechanism" to create computer shich ar capable to learn to perform a task, than we developed a new kind of computer architecture. Under the hood it is still the classic von Neuman because you have calculation, storage, control and in and output.
I agree with the general sense here that exploring concurrency (by any means) does not alter Von Newman's architecture premises, only enhances it. I would like to go beyond the immediate future here and explore a bit more Atis Elsts comments.
The rise of stochastic computing, especially in fields that normally already use some kind of Monte Carlo or optimization approach is an upcoming reality.
I would like to throw a completely different idea here on the manner of this computation. For those of us who have worked with protein folding, we have realized how lengthy is the computation of something that nature produces in a split second (BTW, stochastic in nature). Could the use of Bio-machines expedite this kind of computing?
Imagine for a moment that we are able to represent our problem into a sequence of proteins, we let it combine and collect the answer. interpreting the statistics out of the protein sequences.
I know it is very unconventional, and somewhat difficult to imagine, but is actually more realistic than quantum computing for practical purposes.
Just imagine.
"Why think out of the box? There is No box!"
see http://fpl.org/s/ We have to go heterogeneous, or, as I like to call it: "the twin paradigm approach" (cooperation between data streams and instruction streams), since we cannot get rid of the masses of legacy software as fast as would be desirable. With brain modeling I am not sufficiently familiar with, although I am a scholar of Karl Steinbuch, see:
http://xputers.informatik.uni-kl.de/papers/publications/karl-steinbuch_en.html
@Renato
Although I cannot cite any references, as far as I know what you call Bio-machines is already a reality. The problem is that there are only few people who know how to program protein sequences that can act as programs. I think this approach is nice for complex problems that would otherwise take a long time to compute on a Neumann architecture. But I don't see how this compute model would apply to desktop machines or smartphones. If a technology is not useful for the masses it probably cannot prevail. The consumer market usually drives development of new features because that's where the money is.
Dear Mohamed Gamal Ahmed Mohamed
DNA based computation, that might emerge from, or serve to motivate, the creation of a working Bio molecular Computer.
Von Neumann Architecture is the basic architecture, many developed architecture have been addressed such as Harvard architecture and their performance might be comparable with Von Neumann Architecture, however Von Neumann architecture will stay in parallel with all novel architectures.
In the future we have special architectures for audio signal processing, video signal processing and chemical structure analysis. There are the important human-machine-interfaces. In these interfaces is the Von Neumann Architecture only a slave to compute numerical values, sort and compare numeracy and combine Boolean value. An example for the special architectures of audio signal processing is the ADµP descripted in
https://www.researchgate.net/publication/235222986_Verfahren_zum_zeitnahen_Ermitteln_der_Kennwerte_Harmonischen_und_Nichtharmonischen_von_schnell_vernderlichen_Signalen?ev=prf_pub
Excuse all me, it's only patent description in German!
Patent Verfahren zum zeitnahen Ermitteln der Kennwerte, Harmonische...
- Badges
- Science topic
- Similar topics
- Statistics
- Survey