There is one difference which I know is that City and Metropolitan areas areas are two different demarcatiion in developed world. For example New York city area is not the same as New York Metropolitan area. 

Thanks for your answer,

It is true, but why is that needed? metropolitan and city are acceptable. For example, it considers Block Group, Tract, TAZ. Why all of them are considered as one of them can do the same task? Or suppose country, place state. Why the smallest is not considered? Most of these units considered have overlap with each other, although one or two integer units can be considered.

The answer in my opinion is truly granularity and it goes back to databases.  We have to remember that GIS ultimately is a spatial database.

Let's say we normalize the whole thing completely so that you use only the smallest level things.  This is a smart and flexible approach.  From that, you can build everything else (county, congressional district, metropolitan, etc.) because the smallest level units do not cross over.  It is fully normalized.  Everything can be aggregated from there.

The problem with that is being able to use the features in specific applications in a time-effective manner.  SQL has a group by function that essentially gives us a way to aggregate data.  To do that in GIS and have the corresponding geometry, you have to use what is called the Dissolve operation.  That is enormously expensive and its complexity is tied to how many features are in the map.

Thus the census bureau has essentially employed a key database concept called denormalization to make their product more efficiently and effectively used by practitioners.  You can also imagine that this is an object-oriented process as well called aggregation.  This isn't the time for a database lecture, but it is helpful to see that database concepts drive this.

So to me, the answer to this question is about efficiency.  Sure the data is redundant, but when you get down to brass tacks and try to use it in an efficient manner, having the different formats from the census bureau is a huge benefit.