Excel can help you in solving differential equations numerically, specially if you know the programming language of Visual Basic for Applications under Excel, Which is so easy to be learnt. After writting VBA code that solve your problem and compile it to get output data, you can use Excel spread sheets to organize your data and graph it so easily with little effort using the facilities of Excel to graph so many types of charts.
GNU Octave is a high-level interpreted language, primarily intended for numerical computations. It provides capabilities for the numerical solution of linear and nonlinear problems, and for performing other numerical experiments. It also provides extensive graphics capabilities for data visualization and manipulation. Octave is normally used through its interactive command line interface, but it can also be used to write non-interactive programs. The Octave language is quite similar to Matlab so that most programs are easily portable.
Octave is distributed under the terms of the GNU General Public License
In Excel you can only solve differential equations numerically, but in Mathematica you can try to solve it analytically too. When you learn Mathematica or Matlab I think you will not need further assistance from Excel, because you can also plot your output data in Mathematica or Matlab. and if you want to link between Mathematica and Excel for example you can use the following tool powered by Wolfram company that produce Mathematica program, the following link has details: