Put simply - polynomials are a very restrictive form of a curve - the GAM adapts the fitted curve to the data and is much more flexible than a polynomial.

These two papers are supposed to be short introductions

https://www.researchgate.net/publication/225303313_Moving_out_of_the_Linear_Rut_The_Possibilities_of_Generalized_Additive_Models

https://www.researchgate.net/publication/225303308_Generalized_Additive_Models_Graphical_Diagnostics_and_Logistic_Regression

The latter is more technical

Article “Moving Out of the Linear Rut: The Possibilities of Generali...

Article Generalized Additive Models, Graphical Diagnostics, and Logi...

Dear Hossny,

Generalized additive model is normally used when you are dealing with linear regression models which are class of additive model i.e when you assume a linear form of a given process by example with normal linear regression model, you can replace the linear regression form by  a sum of unspecified functions that you can estimate using several iterative Algorithm ( by example these unspecified function can be replaced by prediction models which can explain the present observations and predict the future from the past and so on)

Polynomial regression is just used for non-linear regression problem in which the relationship between dependent and independent variables is modeled by a polynomial of nth degree. Most of the time this model provide a better fit and difficult interpretation.

Put simply - polynomials are a very restrictive form of a curve - the GAM adapts the fitted curve to the data and is much more flexible than a polynomial.

These two papers are supposed to be short introductions

https://www.researchgate.net/publication/225303313_Moving_out_of_the_Linear_Rut_The_Possibilities_of_Generalized_Additive_Models

https://www.researchgate.net/publication/225303308_Generalized_Additive_Models_Graphical_Diagnostics_and_Logistic_Regression

The latter is more technical

Article “Moving Out of the Linear Rut: The Possibilities of Generali...

Article Generalized Additive Models, Graphical Diagnostics, and Logi...

The "generalized" bit of "generalized additive models" refers to the generalization of the distribution for the response variable and of the link function between the linear predictor and the expectation of the response. So "generalized additive model" is to "additive model" as "generalized linear model" is to "linear model".

Polynomial regression is one kind of linear model, and it too can be generalized by including polynomial terms in a generalized linear model. In fact, polynomial regression is an example of an additive model as well, with a particularly simple form of additive function of explanatory variables: i.e., a polynomial function. Other types of additive function include smoothing splines, regression splines and b-splines, and no doubt many others that I have not met yet, Many of these more complex additive models have a big advantage over the polynomial model in that they do not strongly prescribe the shape of the function. For example, a second-order polynomial is a quadratic curve, which is always symmetric about its maximum or minimum, whatever the shape suggested by the data. A smoothing spline, by contrast, even with only two degrees of freedom like the quadratic polynomial, can adapt to a much wider range of shapes suggested by the data.

To answer your second question, I recommend using a flexible additive model like a smoothing spline if you don't want to impose a particular shape on the relationship between an explanatory and a response variable. This is particularly useful if you want to take account of a clearly nonlinear effect of some quantitative explanatory variable, to allow you to concentrate on the effects of other explanatory variables which may be represented as simple factor or linear effects. One downside of using a complex additive function for an explanatory variable is that it is harder to investigate interactions between that variable and any others. Software should enable you to produce fitted values and curves whatever model you choose, but beware of extrapolation using any type of additive function: particularly using the polynomial function. 

My blog post http://learnerworld.tumblr.com/post/152330635640/enjoystatisticswithme

explains the concept of generalized additive model (GAM), generalized linear model (G LM) and general linear model with simple examples.

Now, polynomial regression is the extension of linear regression which includes non-linear relationships between response and predictors. One can use transformations to make the relationship linear.

Also, there are multilevel models. which generalize linear models by allowing error terms which are not independent.

There are many other extensions of these models.

http://learnerworld.tumblr.com/