You can either go the gentle route or the nuclear route.

Gentle: email the author(s) and ask why they didn't use the methods you believe are correct. They may be amenable to a dialogue about this. This gives them the opportunity to self-correct.

Nuclear: get a copy of their data (it should be archived somewhere) and analyse it yourself. Write a "Response to: ...." paper and send it to the same journal.

If, after finding significant errors in a published article, the editor remains unresponsive to your initial contact, you should first send a polite but persistent follow-up email with detailed evidence of the methodological and analytical issues; if there is still no action, escalate the matter confidentially to the journal's publishing ethics committee or a similar governing body (like COPE for member journals), providing all necessary documentation for their investigation while maintaining a professional and objective tone throughout the process to ensure the scholarly record can be corrected appropriately.

Ray Butler, well, the idea is that I made my analysis and reported my concerns point by point. The editor sent the concerns to the author, and the author responded that the fact that I made an analysis that provides different results does not imply that his results are incorrect. I just used a different analytical method, so it is normal I got different results. The editor then sent me an email saying that he was confident this response would bring the matter to a close. Of course, I responded that the author did not answer to my claims.

Adnan Asef, this observation of yours is very interesting. I have contacted COPE indeed, and they responded to me that their services are only for members. In what sense do you suggest to involve COPE? Can one involve COPE or ask COPE to intervene.

Luca Danieli COPE will not intervene directly on your behalf, as they are an advisory body for their member journals, not an enforcement agency.

My suggestion to "involve COPE" is strategic: you should use their published guidelines as objective evidence to support your case. In your follow-up to the editor, you can politely state that the issues you've identified appear to be a breach of COPE's core practices (e.g: on data integrity, post-publication corrections).

You can phrase it like: "As a framework for resolving this, I would like to respectfully refer to COPE's guidelines on [specific topic], which recommend [specific action]...

This approach encourages the editor to view the situation through an established ethical lens, increasing the pressure for a formal response. It's not about asking COPE to act, but about using their authority to strengthen your argument.

Adnan Asef "I just used a different analytical method, so it is normal I got different results" seems like a cop-out by the author. The results should still be close. How different can they be before it's obvious that one party is doing something wrong? Maybe it depends on the field of research.

Does this study produce error bars? If two valid methods give very different results, with no overlap within their error estimates, the most likely thing is that one method was mis-applied; the other possibility is that the data is useless.

Ray Butler No, apart for the methodology, to describe an evident issue in the processing of data: the participants in the experiment were asked to listen to a musical piece and choose on their willing to place some markers on a timeline. So, they could place either 1, 10, or 20 markers along the timelines, all at different positions. Each marker had a strength ("less important = 1", "important = 2", "very important = 3"). Therefore, at one point there may be two participants marking a time with the resulting cluster ["very important", "very important"] = [3, 3] (Average: 3). After 4 minutes, four participants may mark the moment with the resulting cluster ["very important", "very important", "less important", "less important"] = [3, 3, 1, 1] (Average: 2). The author chose to analyze this by considering the average level at each time. Therefore, the second example would receive a lower average despite having more people highlighting that moment and having the same amount of "very important" scores. I made them notice that in this case "averages" are misleading and incorrectly represent the responses, as "averages" may be useful only when the number of participants is constant. Therefore, for example, the first example is not represented by the cluster [3, 3], but it should be represented by the cluster [3, 3, 0, 0] (Average: 1.5).

He rephrased my complex explanation with a sentence similar to this: "it is self-evident that averages become less useful as the smaller number of markers."

Ray Butler To resolve this, your strongest approach is to objectively compare the statistical uncertainty: if the error bars or confidence intervals from the two methods show no overlap, this is concrete evidence the discrepancy is significant and not due to chance, strongly suggesting a methodological error in one study. You should present this specific analysis to the editor, framing it not as a personal critique but as an objective statistical inconsistency that requires investigation according to the journal's own guidelines on data integrity, as this shifts the focus to upholding scientific standards rather than a debate over techniques.

Adnan Asef, thank you very much. How can I do it? Is there any link you can send me to to learn this methodology? I can use R, but I am not so advanced. So, any tip on how to compare statistical uncertainty in this context would be helpful.

Luca Danieli To compare the statistical uncertainty and determine if the difference between your results is significant, you can visually analyze the 95% confidence intervals; using R, you can create a plot that displays the point estimates (the results) for each method with error bars extending 1.96 times the standard error above and below each point if these intervals do not overlap, it provides strong evidence of a significant discrepancy. A clear, practical resource for implementing this in R is the chapter on confidence intervals from The R Handbook (https://rpsychologist.com/d3/ci/), which will guide you through both the underlying concept and the necessary code using common packages like ggplot2 to create a definitive visual comparison for your communication with the editor.

Adnan Asef thank you very much. I have the rudimentary skills because I did two courses on statistical analysis, but I guess in this case are really not sufficient enough. From what I understand, what happens is that I have two arrays. E.g., array one, that of the authors with the averages: [2.1, 3, 1.2, 1.3, 2.8], and array two, mine: [1.7, 2.8, 2.2, 2.7, 1.1]. You suggest that I have to how different they are? If yes, how can one do that? Is there a specific model that you can point me to?

Ah no, maybe I got it. You mean that I have to calculate the 95% confidence interval point by point of the two arrays, and the see if the confidence interval of each point overlaps?

Luca Danieli To clarify, you should not compare the arrays point-by-point but as two complete samples. The goal is to determine if the overall average (mean) of their results is statistically different from yours. To do this, calculate the mean and standard error for each entire dataset, then use these to compute the 95% confidence interval for each mean (Mean ± 1.96 * Standard Error). If the confidence interval around their mean does not overlap with the interval around your mean, it provides strong, objective evidence that the two methods produced significantly different results, which is powerful evidence to present to the editor. This is a standard statistical approach for comparing the central tendency of two groups.

Hi, if editor of journal ignores concerns about erroneous data analysis;

Direct Communication by send a polite but firm email to the editor, summarizing your previous attempts to communicate and explicitly stating the core issue. Include a clear, concise summary of the evidence. It's helpful to ​contact the Publisher or contact the journal's publisher. Major academic publishers have dedicated teams and established procedures for handling research integrity issues. You should formally report the case, providing all your documentation. The publisher will likely initiate an investigation based on the Committee on Publication Ethics guidelines, which are widely accepted in academic publishing.

​Reach Out to the Authors

​they may be unaware of the error or may be willing to issue a correction or retraction. Frame your communication as a collaborative effort to ensure the accuracy of the scientific record.

​Public and Institutional Action

​As a final resort, if all other options fail, you can consider more public or institutional actions.

​Social Media: Posting about the issue on platforms like (formerly Twitter)

​University or Funder contact the authors' home institution. These institutions have a vested interest in the integrity of the research they support and may investigate the matter independently.

Also, posting on platforms like PubPeer, where researchers can comment on and review published papers.

First, you should provides all documents (documenting all correspondence and evidence) which includes emails, the manuscript in question, your original critique, and any data or analyses.

I don't fully understand the design of the study, but it looks like you have multiple and unbalanced numbers of data points from each participant. Analysing the average has many potential problems including the ecological fallacy and issues with the error in the models. There are various ways to analyse such data but the main approach here these days is a multilevel model - ideally a multilevel ordinal model as the data are ordinal.

What happens if you don't model it correctly? It depends but you could potentially bias result, increase or decrease Type I error and so forth. If the authors used simple averages at each point the model is not a good one.

There's lots of articles and web pages on this. For example:

https://www.bristol.ac.uk/cmm/learning/multilevel-models/what-why.html

In terms of what to do. If you do the reanalysis then using that and publishing a reply might be sensible - if the journal accepts it. Alternatively maybe try and send the reanalysis to a different journal. The most laborious route is to replicate (and maybe extend) with new data and use the correct analysis in that.