Validating a scale/tool can be a scientific study in its own right, and so the request from the reviewer for you to validate the revised scale within your meta-analysis may be a little harsh and unlikely to be practical. There are many different types of validity and they are not always defined/described using the same terms in the literature. Personally I like the definitions provided here:

https://socialresearchmethods.net/kb/measval.php

Perhaps you can run your revised quality assessment scale/tool pass a couple of people with methodological expertise in systematic reviewing, who can confirm the face validity (if they think the revised scale is capable of assessing risk of bias). Your revised scale should have certain level of content validity if you have only modified some wording of the original items which still cover the same domains of risk/quality, or if you have added/removed some items you can provide your justifications. Then perhaps all you can do is to acknowledge the lack of full validation of the revised tool as a potential limitation of the study, and recommend this (e.g. comparing results from this revised scale to other validated tools) as an area for further research if warranted

Dear,

There are no golden standards for this issue. However, there are several approaches, and the best for you can be determined based on a range of circumstances.

As you stated, this is an already validated tool, and you amended items. Then you are simply talking about a modified tool. It was not clear whether you change the items, or added new ones. Anyhow, I would recommend to adopt the following steps. First, you need to establish reliability and internal consistency. You may be looking at .85 or more. Then run factor analysis which you need to make sure that there is a limited number of item correlations of .03 or more to avoid overlapping items. Make sure you choose a reasonably acceptable cut off value for accepting the item when running factor analysis, say .50, for instance.

Then you will need to run a retest within a period of three months or so. You will run factor analysis (i.e., exploratory or confirmatory) for the two sets of data. And finally, you will examine the correlation between the two findings, the original test and retest sets of data for the valid items post factor analysis. If the correlation is high, then you may say that the tool has been validated using factor analysis and was stable over time for your study population. And you may say that it is valid and reliable for the study population and recommend further retesting.

I hope I was helpful in my suggestion.

Greetings to all

My experience suggests that journal editorial staff usually sends a note while requesting revision that authors are required to justify a response to a question if finds that questions needs no action under manuscript's manifesto. One can also seek guidance from coordinating editor.