First there is nothing written in stone to mandate the use of a neutral response; you may chose to use your scale to "force" the participant to make a choice towards agreement or disagreement. This is your choice to make as a researcher.
Second, be aware that surveys tend to have "heavy tails" in that most participants will select a strong opinion on either side of your scale, and this is especially true among your most passionate respondents. in fact, if you are more concerned with the opinions of those on the edges, you might choose to weight their responses.
Third, you should very seriously consider not using a mean value, but rather using a median and non-parametric analysis. Survey data rarely follows a normal distribution. Indeed, one of the most helpful ways of sharing survey data is a good table of descriptive statistics.
There is a lot of debate whether parametric tests or non-parametric tests can be used. But in summary if you are prepared to assume your data represents interval data then parametric tests are usually not inappropriate.
Some useful references in this article: http://eprints.uwe.ac.uk/30814/
With regards the neutral response, these references may be of interest:
Armstrong, R. L. (1987). The midpoint on a five-point Likert-type scale. Perceptual and Motor Skills, 64(2), 359-362.
Bishop, P. A., & Herron, R. L. (2015). Use and misuse of the Likert item responses and other ordinal measures. International journal of exercise science, 8(3), 297.
For most analyses, you should probably not collapse a 5-category Likert scale into just 2 categories (positive/negative). But if you need to do so (say, for reporting to an audience not prepared to understand complicated results), you should consider putting the neutral category into whichever end of your scales is usually the smaller end for the items you will be reporting. Example: if on most of the items the majority of respondents are in the Strongly Agree or Agree categories, you would collapse Neutral with Disagree and Strongly Disagree. If 60% are in SA or A, 10% in N, and 30% in D or SD, it would be more informative (and simpler) to report 60% agreement than to say 70% don't disagree. To claim that 70% agree would be misleading.