What has a p-value < .05? KS or Shapiro-Wilk test? Both are very sensitive in case of large sample sizes, so I would suggest also a visual inspection via scatterplot and QQ-plot to see how strong the deviation from normal is.
On the other hand, the choice of the "best" measure of the central tendency also depends on the context and what you are implying to say and the conclusions you want to draw from it. So, this might be the arithmetic mean, the median, mode or maybe trimmed mean or M-estimator.
Please have a look at "Wilcox - Fundamentals of modern statistical methods" as a primer.
You have "How can I represent a non-normal grain size distribution?" But populations do not usually follow a Gaussian distribution. I do not know your subject matter. Is there some reason that you expect such a distribution? The "normal" (Gaussian) distribution - misleadingly named - is important because of the Central Limit Theorem, but a population distribution can be very different. Lognormal is not the only option. The possibilities are endless, but there are some standard forms one may use which themselves differ greatly by changing parameter values.
Histograms can be helpful, but may mislead, depending upon your bin-size, so you could try different ones. You could also estimate the various moments - mean, variance, skewness, kurtosis - and look at various possibilities, such as gamma, whatever.
So if you are really looking at the population distribution, you may have to estimate various descriptive statistics from your data. You could remove a few data points to see how stable your results are. You may need to obtain additional high quality data.
By the way, a p-value is not only test dependent, it is dependent for estimation upon population standard deviation and sample size. It has taken a long time, but the research community is finally starting to accept that using a "standard," such as 0.05, results from a very bad misconception. (Try using that in "big data," to understand this.)
This might help understand the practical approach:
PS - You need to decide if you are looking at a population distribution, or are you looking at a sample mean distribution, or both, for your application.