Here is my go at a layperson's answer!

Fisher scoring is a hill-climbing algorithm for getting results - it maximizes the likelihood by getting successively closer and closer to the maximum by taking another step ( an iteration).  It knows when it has reached the top of the hill in that taking another step does not increase the likelihood.  It is known to be an efficient procedure - not many steps are usually needed - and generally converges to an answer. When this is the case you do not need to be concerned about it - accept what  you have got. Changing metaphors,  you can drive the care without knowing how the internal combustion engine works.

Occasionally however the likelihood surface you are trying to climb is not a sharp peak but there are multiple peaks and the there is no much depth in the landscape. There are a series of bumps each giving potentially different answers in terms of the estimates but with pretty similar likelihoods.  When this is the case good software will warn you and you will see a large number of iterations and a failure to converge. When that is the case it is usually due to lack of information ( small sample size)  or failure to meet assumptions of the model  ( eg multicollinearity between predictors). When this happens  the solution is not usually a different algorithm but improved data and/or model. Some software allows you to profile the likelihood to see a map of the surface in which you are trying to find the peak.  

In lots  of software for the logistic model the Fisher scoring method (which is equivalent to iteratively reweighted least squares) is the default ; an  alternative is the Newton-Raphson algorithm . Both should give  same parameter estimates; but the standard errors may differ  slightly. [The Fisher scoring is based on the expected information matrix; Newton-Raphson method on the observed information matrix.] In fact in  a binary logit model, the observed and expected information matrices are identical, resulting in identical estimated standard errors.

If you have rare events ( eg many people but few of them have died - lots of zeroes , not many ones) then you may wish to use other procedures to get less biased results; such as the Firth method which is a penalized likelihood approach to reducing small-sample bias in maximum likelihood estimation; see

https://www3.nd.edu/~rwilliam/stats3/RareEvents.pdf

Here is my go at a layperson's answer!

Fisher scoring is a hill-climbing algorithm for getting results - it maximizes the likelihood by getting successively closer and closer to the maximum by taking another step ( an iteration).  It knows when it has reached the top of the hill in that taking another step does not increase the likelihood.  It is known to be an efficient procedure - not many steps are usually needed - and generally converges to an answer. When this is the case you do not need to be concerned about it - accept what  you have got. Changing metaphors,  you can drive the care without knowing how the internal combustion engine works.

Occasionally however the likelihood surface you are trying to climb is not a sharp peak but there are multiple peaks and the there is no much depth in the landscape. There are a series of bumps each giving potentially different answers in terms of the estimates but with pretty similar likelihoods.  When this is the case good software will warn you and you will see a large number of iterations and a failure to converge. When that is the case it is usually due to lack of information ( small sample size)  or failure to meet assumptions of the model  ( eg multicollinearity between predictors). When this happens  the solution is not usually a different algorithm but improved data and/or model. Some software allows you to profile the likelihood to see a map of the surface in which you are trying to find the peak.  

In lots  of software for the logistic model the Fisher scoring method (which is equivalent to iteratively reweighted least squares) is the default ; an  alternative is the Newton-Raphson algorithm . Both should give  same parameter estimates; but the standard errors may differ  slightly. [The Fisher scoring is based on the expected information matrix; Newton-Raphson method on the observed information matrix.] In fact in  a binary logit model, the observed and expected information matrices are identical, resulting in identical estimated standard errors.

If you have rare events ( eg many people but few of them have died - lots of zeroes , not many ones) then you may wish to use other procedures to get less biased results; such as the Firth method which is a penalized likelihood approach to reducing small-sample bias in maximum likelihood estimation; see

https://www3.nd.edu/~rwilliam/stats3/RareEvents.pdf

Thanks Kelvyn Jones for such a detailed explanation. this is for layman's understanding or in your words "layperson's answer". So this explains how many iterations there would be before the model reaches that maximum likelihood however does there a significance of when this number is either lets say 4 or 5 or for matter any other number. 

No it is not  an issue in general  between 4 and 5 but  between 400 and 500 would suggest that there is an issue.- most software would report an issue before 500 is reached!

It might be helpful to give an example of non-convergence by illustrating one of the major causes of this problem which is known as complete separation or its variant, quasi-complete separation. To take a simple case ,  take either of  two two-by-two table in  Allison P. D.  (2008) Convergence failures in logistic regression., SAS Global Forum: Statistics and Data Analysis, Paper 306-2008, 1–11. Available at http://www2.sas.com/proceedings/forum2008/360-2008.pdf.

just enter his data and run a model - it should not converge;  both these data patterns do not admit to a solution; Put simply a frequency of zero in any single cell of the 2 by 2 table implies quasi-separation; two diagonally opposed zeros in the table means that the condition is complete - the slope is infinite  - the maximum likelihood estimates simply do not exist. Good software will tell you there is a problem - less good software will continue churning out iterations.

Thanks again Kelvyn for your time 

For nominal Data, is it possible to use Fisher Score?

The models discussed in my earlier contributions eg the 2 by 2 table are nominal but a parametric logit model is being used in the analysis to estimate a function of the underlying probability of the outcome.