Milena Cukic - I think that "novel methodology" is often journal-speak for "people in this field generally do not use it (even if other fields do)", which I suppose makes that the no.1 factor. Machine learning has been around for 30 years, too. It is coming into use more now mainly because of improvements in technology (and access to technology) rather than improvements in methodology. Perhaps people also sometimes concentrate too much on the most recent literature when researching a field! But then someone of my age would say that...

Best wishes,

John E

As John has suggested, the context within which a claim is made is important. Is the claim about a technolgoy or its application? To be novel something has to be new. So if there is a published literature on a technology or analysis approach, it cannot by definition be novel. But technology is constantly being repurposed. So even an established technology could be applied in a new way, leading to a claim of a novel methodology (ie tech + application). I also agree that proper lit searches have to be done. Claims for novelty were made about deep brain stimulation for treating Parkinson's when it was "rediscovered" in the 80's. But experiments had been done in the 50's which had been forgotten about. I suspect there are lots of examples of this kind of thing.

I agree that context is important as indeed is the application. Sometimes the technology is not novel but the way it is applied might be. It often seems to be a cliam made in articesl to try to catch the readers eye. Maybe as reviewers we need to challenge the term a little more.

I'm sure authors do claim novelty to attract attention. Of course if someone claims to be first, as a reviewer I only have to find one earlier example to torpedo their claim, and thus the authors' overall credibility. This is why we tell students not to make such claims.

Thank you for your replying. I agree that context is important. The thing is that I face difficulty when trying to explain that a multidisciplinary approach to decision making in psychiatry is promising. We performed several successful research on how physiological complexity and machine learning can contribute to better diagnosis in depression. Also, why several electromagnetic treatment could be efficient in pharmacoresistant depression. And then you realize that majority of practitioners prefer status quo, over an useful and evidence based methodology, which they call 'novel'. Also, the attitude of reviewers is sometimes very discouraging, simply because the prefer to show that what you try to explore is too different over usual practice...and therefore suppressed. I know that introducing something new (and helpful) in clinical practice is going to last for a long time, but I found that also unreasonable. Very often I ask myself shouldn't those medical practitioners think more about their patients and their suffering, instead of their position and the power to block something new?

The introduction of new technologies/treatments is always problematic. Where there are clinical (and cost) risks, there is rightly a degree of conservatism. The evidence required to overcome conservatism and professional inertia, scales with risk, benefits and costs. This is something that we have discussed in Campbell & Knox (2016) International Journal of Technology Assessment in Health Care, 32(3), 122-125. doi:10.1017/S0266462316000234.

This is a good discussion thank you. I can add a couple of things from my own experience. 1) Some of my past research has focussed on how healthcare practitioners reason about 'new technologies' Article Surgeon opinion on new technologies in orthopaedic surgery

and my experience of conducting this research was that the manner of elicitation is very important in understanding why professionals are (or could be) interested in a novel technique. In this project, focussed on image-guided/computer assisted surgery and eliciting opinion from orthopaedic surgeons, we first asked what do you think constitutes success? and what do patients want in your experience? and the use and merits of existing technologies before asking about the benefits of the technologies in question. 2) IP issues are sometimes relevant. I once reviewed the evidence and value proposition for Vacuum-assisted closure therapy for wound healing (which has an interesting history as seen in this paper) Article The History of Negative Pressure Wound Therapy (NPWT): From ...

where the question of novelty shifted from the primary mechanism to the wound dressing, gauze or foam used, which was of interest to the companies with competing products since this could potentially be protected as intellectual property, whereas NPWT was prior art.

3) Most recently in our European project https://www.radar-cns.org/ https://www.researchgate.net/project/RADAR-CNS-Remote-Assessment-of-Disease-and-Relapse-Central-Nervous-System-3

part of the work we are doing (in the Clinical Pathways workpackage I am involved with) is to examine the 'value proposition' for remote monitoring for depression, multiple sclerosis and epilepsy. Similar to 1) above we have been asking healthcare professionals (and also patients as experts-by--experience) to consider or provide 'use case scenarios' where they express the potential benefit of a new technology in the context of their daily practice or lived experience, or an imagined future of those. Again, this 'taking a step back' from the 'novelty' of the technology provides insights into its value at present or of its potential. As such it may not matter how old or new the technology is, but how much it provides (or might provide) an answer to unmet needs.

Michael, thank you for sharing this. I would like to know more about the response of clinical professionals about monitoring in case of depression. Could you tell me what measures you were monitoring in case of relapse of depression? I am very much interested in learning from your results, since we are trying to implement something very easy to use online for professionals. I would like to stay in touch about this topic. Thank you!

We have one paper presented at International Ergonomics Assoc. 2018 conference on 'Investigating user needs and integration of remote monitoring technology into clinical practice' which we need to upload to RG. I'll upload it very soon. Thanks again and yes let's keep in touch.

This work is really valuable, and provides an important part of the whole. But it's some way short of the critical evidence required to decide the issue of whether this application of new tech/approaches/treatment poduces an improved outcome. Such evidence is hard to generate, but critical.

Agreed. I think what is becoming clearer for technology for mental health and in general is that implementation issues need to be addressed earlier, aligned with the evidence generation. I've commented about this recently in the context of computerised and internet CBT: Article Meeting the challenge of digital mental health

Also considered by others in the mental health tech domain in papers such as:Article Strategies for mHealth Research: Lessons from 3 Mobile Inter...

In other domains a strong research focus on implementation in studies such as:

Article Evaluating the implementation of HeLP-Diabetes within NHS se...

is providing useful insights.

The RADAR-CNS programme I mentioned above includes clinical evidence generation in each of the 3 clinical areas, while our Clinical Pathways work with healthcare practitioners and commissioners, and a patient-focussed workpackage with Patient Advisory Panels for each disease area, go alongside those to hopefully ensure what is produced has the best alignment with those users' needs and aspirations. Even so this work is intended to prove the technology in an observational (non-outcomes based) set of studies so yes the outcomes based work will still need to be done following this.

I've got one new collaborative project which also aims to address the implementation issues early on, with a year's work to do before a multi-centre trial starts (to evaluate VR therapy with an avatar therapist) in this case alongside the development of the virtual environments, with Lived Experience Advisory Panel involvement, and with focus on the commercial/business case in addition to the health economic analysis which forms part of the trial protocol, but we hope to have a good framework for the business case even before the trial: https://gamechangevr.com/

I think this is closest to the ideal of an evidence generating study which aims to cover all the bases to produce a solution that is wanted, needed and implementable, one which is made possible here due to the level of funding.

Even without bigger funding there may be 'proportionate' evaluations that can help prove a technology with users at the early phase of design, something we've tried to apply to healthcare apps for mental wellbeing: Article Proportionate methods for evaluating a simple digital mental...

and here the harder outcomes evidence generation can arguably come after, once there is evidence that the technology is actually being used (as long as it has shown some promise of benefit and non-harm).

Some of the methodology I'd say that is useful in the above comes from a human factors systems approach as well as from implementation science (normalisation process theory being used in the gameChangeVR project).