I would like to recommend my paper "Estimating the number of clusters in a numerical data set via quantization error modelling", Pattern Recognition, March 2015, where the problem of Number of Clusters is studied.

I would like to introduce Golay Code Clustering method, it works with fuzziness modelling and also it has approximation clustering method, that mean, it is included multi-cluster feature. or you can read some paper about Golay Code template, it works on template clustering for all possible data points. extraction of this model is used for FuzzyFind Dictionary. I will upload construction of this algorithm next two or three week after my paper is published.  

Hi

In my opinion most of the things in the world and the knowledge to make a theory about them are fuzzy as well. Therefore, humans clustering also according to the their knowledge varies from on person to another. Indeed in the computational programs depend on their prior knowledge clustering is done. Lets try our recently published article in this way, I believe that it will answer most of your questions.

http://www.sciencedirect.com/science/article/pii/S0924271614002056

In this article you can find that if you also have not necessary knowledge for clustering you can be delieverd relible outputs from the algorithm. 

I can only say that the answer to your question is: soft clustering and learning vector quantization algorithms. Such algorithms were developed exactly to be able to deal with the uncertainty that is necessarily involved in the clustering problem.

I think features representation learning capability of deep learning, which is believed to mimic how our brain works, is worth to explore. The following link hope can give more insights: 

https://www.reddit.com/r/MachineLearning/comments/1rsmlt/whats_wrong_with_kmeans_clustering_compared_to/

I think the algorithm depends on the data and the purpose. Clustering algorithms are essentially data compression algorithms; you are trying to represent items as belonging to a group in a way that they share characteristics. Thus in the stock market, humans might cluster by one set of metrics for a short term speculator and by another set of metrics for a long term pension manager. It all depends on the usage of whatever information you hope to extract from the groups you develop. I would work backward from what you hope the application will be.

For example, Amazon groups purchases by incident (people that bought this item also bought this item at the same time), and also groups purchases by customer (people that liked Book A also liked Book B).

I think what humans do is roughly similar to Bayesian, in the sense of identifying features that have a greater chance of predicting some particular outcome, we look for associations that "work" to provide us with new information that leads to effective action. For example, symptoms group to identify a disease or condition that in turn links us to a most likely course of treatment or therapy.