It depends on what you are trying to do with the analysis.
For example, to start as a faculty/student of an educational institution, you have access to free software from Autodesk products from the following web site. Autodesk simulation (former Algor) will be available free to download and use.
Autodesk simulation is relatively easy and simple user interface to start with. I would suggest you to start with and then move on to more commercial one if your requirements are not met by Autodesk simulation.
You must understand the FE procedure thoroughly before you go into implementing this on GUI softwares. I suggest you start first with some hands-on experience of code writing using Fortran 90 or C++ or MATLAB C, then you can begin to use GUIs as suggested in previous answers.
I guess you want some material that would lead you gently into FEM programming and easy to follow. I would recommend this book 'Finite element method using MATLAB' by Kwon and Bang. CRC press published 1996. You should be able to follow the programming steps easily and plan to write your own in MATLAB C or even in C++ or Fortran. Then you graduate into pdetool which is the GUI for FE modeling in MATLAB. After you should be able to do all you want using other GUI based software like Abaqua, Ansys etc.
If you are interested in solving for systems of PDE's (like solid mechanics, heat conduction, chemistry and more) at the same time then you should take a look at MOOSE: http://mooseframework.org
It is an open source package that provides a high-level interface for defining your physics, boundary conditions, material properties, etc. and then can use massively parallel nonlinear solvers to get your answer.
PCFEAP is a small general purpose finite element analysis program which is designed for research and educational use. The program is described in the references: The Finite Element Method, 4th ed., Vols. 1 and 2, by O.C. Zienkiewicz and R.L. Taylor, McGraw-Hill, London, 1987 and 1991. The program has been modified to include elements from both volumes and to be compatible for compilation in both UNIX and PC environments.
It depends on what you want to do. For an in-house or academic code for cases without heavy number of degrees of freedom I will recommend MatLAb (or Octave or SciLab).
You may see the book fom Antonio Ferreira "MATLAB Codes for Finite Element Analysis- Solids and Structures" which present many codes.
I would like to mention that some tasks like assembling or imposing boundary conditions are well addressed in a paper from Sigmund (DTU) "A 99 line topology optimization code written in Matlab" (part of this is FE code) at e.g. http://www.ewp.rpi.edu/hartford/~ernesto/F2010/EP2/Materials4Students/Krawec/Sigmund2001.pdf (and also in Efficient topology optimization in MATLAB using 88 lines of code).