[Edited to include my clarifications, as given in the Q&A below]
This question is in the realm of the "demarcation problem" of natural sciences, often called "the sciences", which includes physics. Mathematics is a formal science and, like philosophy, not natural science. However, we hope that the answer we seek here will be informative to all science disciplines, including mathematics and philosophy.
We are approaching this question in a different way, learning from Internet technical evolution and hoping to arrive at a better answer. We are using a methodology that is similar to that used to develop Internet technical standards, taking into account everything that works and also tentative implementations, and looking for consensus. Our first requirement to develop this "standard" (a "new sufficiency criterion", defining what does it mean for a theory or experiment to be scientific) , should be to promote interoperation; i.e., we want the standard to work not only in all branches of natural science but also in their dialogue, the interdisciplinary areas. What the standard considers to be scientific in biology should also be scientific in physics, in biophysics, and any other natural science area.
We already know that making clear (both new and known) predictions is useful as part of the "new sufficiency criterion" but this does not work as the entire "sufficiency criterion." We need more, and it was suggested we should also stress the "new" part -- that every theory needs to make predictions and solve problems that the previous ones could not do.
We already know that one cannot add consistency (even locally, within its own domain) to reinforce the "sufficiency criterion" because it already is an antecedent. A theory must be consistent within its own domain, before it can be considered a theory.
Karl Popper's answer, the falsifiability criterion, was that a theory is scientific if and only if it makes clear predictions that can be unambiguously falsified.
However, suppose that using Popper's falsifiability criterion in physics and natural sciences, can:
a. produce false positives (calling "scientific" a theory that is not); and
b. produce false negatives (calling "non-scientific" a theory that is).
If (a) and (b) are true, as I suggest they are, then Popper's criterion may just be "too blunt" to reliably define what is scientific or not.
Another problem with Popper's view of how science works, which also adds to the "bluntness" argument above, is that Popper's "logic of scientific discovery" falls apart in practice; scientific theories are not discovered as Popper (who never made such a discovery) imagines -- scientific theories have been and can be inductively inferred from experience.
Popper's "logic of scientific discovery" is disemboweled in the history of physics, with many textbook examples showing that experience can determine theory (i.e., we can indeed argue or infer from observation to theory, as Newton discovered the gravitation law, as Max Planck found the quantum).
In the posted question, please also consider modern natural sciences, including physics, quantum information, quantum cosmology, and biology. In a new area of natural science, what does it mean for a theory or experiment to be scientific?
I am hoping that your experience and reasoning can help in discussing these aspects of the "demarcation problem", including what is scientific, and how does scientific discovery work?
My own interest (and bet) in this question should be clear from the arguments above. I see a need to (1) recall Popper's falsifiability criterion; and (2) reject Popper's "logic of scientific discovery".
To be discussed in a forthcoming paper, my suggestion is that a new "sufficiency criterion" should be introduced, including what is already known, excluding Popper's falsifiability criterion, and introducing what I tentatively call "refutability" (the undeniable possibility that a statement is false). This is different from Popper's falsifiability criterion, although the word "refutability" is sometimes used as a lay synonym for Popper's "falsifiability".