Dear Brad,

I'm not sure I completely understand why you're asking this question, but I found it interesting; so here are a few thoughts.

I do not usually ask my students to determine the degree of 'how empirical' a given perspective is, as long as it is (i) coherent, (ii) testable, and (iii) either based on a reasonable theoretical framework or suggested by previous empirical findings (ideally both). I do believe it's important to make sure students (and researchers/people in general) make sure that assumptions they derived from (and attribute to) other people's research are not based on misunderstandings etc. (so they have to read original articles rather then just review articles and may have to contact the authors to be sure they interpret them correctly). However, in a way I find it less important what originally motivated a research question (e.g., someone may simply wake up with an ingenious hypothesis), rather my focus would be on whether it is testable/falsifiable and promises to significantly advance our understanding of a phenomenon in relation to current views. In contrast to natural sciences (physics in particular), it seems that psychological research often takes place within frameworks that are only shared by a minority of researchers in the field, simply because there is no compelling framework that would serve as 'the' default. This is true even for areas such as cognitive neuroscience, where at least the methodology seems somewhat similar to that in natural sciences. But still, different camps of researchers with very different perspectives regarding even the most basic assumptions about how the mind/brain works continue to do research. Superficially, this research may look like non-sense when viewed from a different perspective or camp (unless you start reinterpreting everything relative to those assumptions and may find that the actual 'data' may still be of value). Typically, proponents of the different camps tend not to take seriously the assumptions or the 'empirical basis' of the respective other camp. Coming back to your question, I believe it is important that students (and others) have some awareness of the relativizability of a given perspective and understand some of the framework-immanent justifications, historic milestones, and central findings leading to its current version. For this, I find it often helpful to encourage students to work for a few weeks or months with researchers belonging to different camps/frameworks/perspectives. However, there is also a limit of what one can do and expect.

Given our brain's processing limitations and our limited life expectancy, it seems inappropriate to (re-)consider and justify the entire set of assumptions (and their empirical or theoretical basis) when doing research. Similarly, it is usually not feasible to consider all side effects that a certain empirical finding may have on a framework. I believe, as humans we simply don't have the time to continuously check the coherence of our belief systems all the time (either in research or in other aspects of our life), thus there will likely be incompatible parts in most belief systems, including scientific frameworks (perhaps especially in social sciences). [To be honest, sometimes I am shocked to see how little awareness some people have of the inconsistencies and contradictions in their belief systems, but at the same time I know that the problem with my own blind spots is that I typically don't even know about them.]

So, what can we do as professors/teachers/parents/peers ? I believe we can and should try to teach students to reflect on the bases of their/our assumptions and to develop a critical attitude towards any kind of framework/belief system and research finding (especially one's own). However, I don't think it's either feasible or desirable to expect students (or anyone else) to 'formulate comprehensive ways to judge the quality of the empirical foundation of any given approach'. Very rough approximations along the lines of Popper's ideas (e.g., 'falsifiability') and - perhaps - 'Occam's razor', as well as some awareness of how to distinguish good from bogus research may have to do the job.

I'm not sure this is the response you were looking for, but perhaps you may find some of it helpful.

Cheers,

Karsten

Dear Karsten Steinhauer

I agree with you (EXCEPT the "(or anyone else)" part) when you say: "However, I don't think it's either feasible or desirable to expect students (or anyone else) to 'formulate comprehensive ways to judge the quality of the empirical foundation of any given approach'."

But, that said, I believe the "formulation of comprehensive ways to judge ..." must be done and must be established clearly, or it should be clear there is a structure and mechanism working toward this (and HOW) -- at least by leading persons' in the field.

I was not expecting to leave this to individual students; that is why I said "define a standard for empiricism in a field (pretty much "as a whole", i.e. an over-arching view)" -- and there I was referring to perspectives offered (OR which POTENTIALLY MAY BE OFFERED) that can be seen as well-established in the field as a whole (doing much of such evaluations "for" students, and for each other too). THUS:

Good standards must be met. In particular: the field as a "community" must have established standards for what degree of empiricism is possible in different circumstances AND where the highest level of empiricism can, in fact, (at least potentially) be shown: SPECIFICALLY: a way behavioral changes CAN/or may-well-be-expected-to-be-shown clearly related to both necessary types of proximate causes (as required for psychology, THE SCIENCE -- AS DEFINED): (1) the overt, directly-observable behavior patterns AT THE ROOT of major developing behavior patterns and (2) the corresponding directly observable aspects of the environment. The best-possible relationship of behavior patterns OTHERWISE (in other circumstances (of study/research)) _TO_ such ultimately best-possible empirical definitions should be clear in a perspective/theory, along with its associated, justified and proven assumptions. Lacking any or all of this, the field lacks a KEY it cannot afford to lack. IN SHORT: We need standards for: " (i) coherent, (ii) testable, and (iii) ...reasonable theoretical framework " AND THAT, just described in 2 parts (directly above), must be its nature. (We do not have that now, the "long history" of psychology NOTWITHSTANDING : Psychology is an infant science (even when viewed as generously as is reasonably possible); I have well-argued this, in detail, in other essays (Questions and Answers) here on researchgate.)

Your statement about "... rather my focus would be on whether it is testable/falsifiable" really just BEGS THE QUESTION. Its answer is NOT self-evident (for anyone). We need (and are lacking), and MUST have, STANDARDS FOR "testable/falsifiable" : evidence from direct observation _AND_ (IN ADDITION), if the circumstances are not that, how the INDIRECT EVIDENCE _IS_ RELATED TO _THAT_. Otherwise the psychology HAS NO GOOD FOUNDATION at all -- and I do not believe it is possible to argue otherwise.

The idea of "camps of researchers with very different perspectives regarding even the most basic assumptions" is not acceptable. Unless assumptions are proven (or you are seriously working towards proving them) _AND_ the relationship of YOUR findings to ACTUAL empirical findings (i.e. the relationship to 'the highest level of empiricism') ARE understood (OR might well be, given your approach), THEN you are in trouble. And, specifically: _HOW_ your data from these other circumstances relate CLEARLY to the FOUNDATIONAL EMPIRICISM, though indirectly, must also be based on ESTABLISHED (WELL-FOUNDED) assumptions. We should not have a situation where " proponents of the different camps tend not to take seriously the assumptions or the 'empirical basis' of the respective other camp ". That really, simply (and for-sure) IS NOT OKAY.

The " Given our brain's processing limitations " consideration: you will find I always take this into account, both for the Subject (of study) and for the student/researcher/theorist; I have entire Questions and Answers (essays) addressing this. We MUST be realistic about our Subject and about ourselves. And, I believe currently WE ARE NOT: we do not "see"/interpret/hypothesize (any of those) CLEARLY IN ACCORD WITH ESTABLISHED PRINCIPLES OF MEMORY (findings about our types of Memory -- among the strongest and most consistent and well-established findings in ALL PSYCHOLOGY). Psychology practitioners ('scientists') contradict themselves, even by contradicting their best findings, by their own behavior (including their thought formulations) AND often in the basic view of their Subject (subjects).

Your re-statement (at the end of your essay) that " However, I don't think it's either feasible or desirable ... to 'formulate comprehensive ways to judge the quality of the empirical foundation of any given approach' ", after all else you say, I FIND TOTALLY UNACCEPTABLE. You want to accept defying empiricism and the very foundation of the science of psychology (and thus, the very DEFINITION). Though much work remains in establishing the foundation (as I described, above) BECAUSE THIS HAS NEVER BEEN WELL DONE, it still needs to be done. It may be possible to do it ONLY now, with new technology (eye-tracking technology and computer-assisted analysis). I have put forth a foundational proposal in my paper, "A Human Ethogram ..." -- a perspective which NOW, thanks to that new technology, has a basic sort of hypotheses that are NOW TESTABLE. See: Article A Human Ethogram: Its Scientific Acceptability and Importanc...

"I am sorry" (as they say) for/to Psychology that it may be ONLY NOW that is gets a true and good and real and best empirical foundation. But, if complying with the standards indicated above can ONLY BE DONE NOW, we must (in a real sense) accept that Psychology, itself, has only just begun.

[ Frankly your answer ended up being the accepted, though unreasonable gobbledegook , much of it often spouted off by rote in the field. Obviously, I cannot view that kindly. I know you meant well and were trying to help, but I, myself, may have more of an Answer here, than a Question. ]

P.S. Much more of the actual work in formulating a research project may well and justifiably be direct observation of the Subject, with a broad understanding of ontogeny -- yet as much as possible showing inter-rater reliabilities (SO IT IS STILL GOOD RESEARCH) -- RATHER THAN going through the deceptively apparently-acceptable procedures described by Karsten Steinhauer to generate a hypothesis (which are more based on traditions, than on good sense). Doing it properly (making it much of the nature I just described) then results in a hypothesis that is really just an aspect, perhaps a small aspect, of the actual overall research. The "procedures" outlined by Karsten Steinhauer will arguably lead to poor-quality hypotheses rather than better ones from a working with and in a TRUE CONTEXT (certainly a "true context" is not today's psychology theories or models).

The attitude I learned from the old-time ethologists was NOT to look for something to test, but LOOK and possibly find something to test. See what ALL you can see reliably "THERE", that is more of the research than ANY hypothetico-deductive "system" (ESPECIALLY better than any h-d system of "understanding" NOT generated by necessity through engaging in an observational process). And it is nice if, in the process, you find things consistent with BIOLOGICAL and necessarily applicable principles.

Dear Brad,

I must admit that I have not read your work (and may not have time for it in the near future either), but I am surprised that you seem convinced that eye-tracking and computer-assisted analysis (which have been around for quite a while) could be viewed as the 'technological solution' to the foundation of the 'science of psychology'. Despite important contributions of eye-tracking studies, and even if we add brain imaging techniques to the mix (the kind of research I do), your optimism is certainly not shared by many. In my opinion, three problems in social sciences including psychology (as compared to natural sciences) have to do with

(i) the complexity of psychological models and frameworks due to the complexity of the human brain and human behaviour,

(ii) limitations of methodological approaches, including brain imaging techniques and eye-tracking (all of which only provide quite crude data that are typically compatible with dozens of theoretical frameworks), along with the impossibility of controlling for the huge range of confounding factors in experimental/empirical research, including for ethical reasons, and

(iii) epistemological issues of various kinds, including the role of concepts of consciousness (and sub-consciousness), free will and the like that - at least partly - seem to rely on introspection rather than empirical research.

You seem to believe that certain data patterns have to converge on one theory, but I doubt this is correct. Neither is it typically the case that (small sets of) individual studies can falsify theories (unlike in physics). Due to the complexity of the human brain and the huge variability among humans, most neuro-physiological studies that go beyond simple perception phenomena (such as color vision) or simple motor behavior (such as reflexes) and instead focus on complex cognition, thinking, language, music etc. tend to result in complex patterns that very often appear contradictory and incompatible with other similar studies. At that point, the question is often: which experimental design was superior - the one that led to Result A or the one that led to Result B? Another consequence of the complexity of the human mind/brain is that hardly anyone is able to carry out (or understand) highly specialized research (as required by the huge amount of details) while also having a decent knowledge or related areas. Thus much research in sub-areas co-exists without strong exchange across those areas. Linguists, psychologists and neuroscientists may all work on language processing but may no be able to interact or collaborate because their terminology and perspectives are virtually incompatible. Even within disciplines such as neuroscience, language researchers may simply not be aware of what their colleagues in neurobiology may have found (and would potentially help reinterpret their data and rule out a certain model). Currently, frameworks such as psychoanalysis are still around, because their proponents simply do not accept that they have falsifield (or shown to be useless). You and I seem to agree that the postmodern (and trendy) approach of 'anything goes' and 'alternative facts' is not a good idea (at least in areas that allow for empirical research), and so I tend to agree that, overall, it'd be good (and thus desirable) to discuss and clarify the role empirical research can and should play in a field. But I do maintain that this is nothing that could be asked of or expected from any student or other individual (which I [mis-]interpreted as your original question). If anything, it does seem like a task for a whole field and would need to involve philosophers/epistemologists as well (who, I'm confident, have already figured out some answers, but without gaining much influence).

I am not very optimistic this will change soon, and I don't agree that the emergence of eye-tracking technology etc. has changed anything in a significant way.

Cheers,

Karsten

Dear Karsten Steinhauer

The eye-tracking patterns we could discover would likely be very good indicators of perceptual or perceptual/attentional-related behavior patterns and could only be seen with such technology. I propose that discovering such patterns would necessarily involve the correct understanding of the human developmental context and memory (as much as possible) -- and involve as much as possible real knowledge from real observations occurring over quite a significant period of study, "seeing" (also with eye-tracking) the quite likely SIMILAR, YET DIFFERENT, cognitive developments of younger children (during ontogeny) -- AND then richly seeing present phenomenology and with eye-tracking technology (computer-assisted analysis may also be of help here). One would not expect any "earth-shaking" things to just "pop up" and/or be obvious -- thus, your argument about eye-research so far is irrelevant.

Plus note: if the kind of behavior change instigating the inception of new cognitive abilities (new types of learning) could only be of the nature I indicated, then IF THERE IS ANY REAL EMPIRICAL grounding possible for COGNITIVE-DEVELOPMENTAL-BEHAVIORAL SCIENCE, I would suggest we look hard and look intelligently. (I doubt such phenomenon could be found "just looking/ just watching" but eye-tracking and related technologies would be involved.)

Nowadays the empirical basis for all (each) significant new type of cognition is often imagined "embodied" by 'theorists' using nothing but pure analogy to Piaget's sensori-motor Period (and they have yielded no significant meaningful research findings and seem to have no promise -- as reviewed and assessed by respectable peers; these 'theories' have only weak indirect evidence for them, and that situation will never improve). I offer the more consistent-with-psychology's-definition approach for understanding cognitive developments: look for the EMBEDDED-ness (in/with the environment) IN/with corresponding BEHAVIOR patterns. Given the much contextualization brought forward by the memories (memory capabilities), these could easily be (and likely ARE) as subtle as "perceptual shifts" or "perceptual/attentional shifts" and I think it is arguable that there is no other sort of thing which "it" could be. And, thus, failing to find these is in a real sense a failure to find behavioral/environmental referents for major cognitive [behavior pattern] changes -- an attitude amazingly many psychologists seem to, in effect, accept (though some go the goofy route of "embodiment" ). If you want to test if empiricism works (which of course it does, so of course you do), you basically MUST take my route. (You don't really think people conjure-up the raw basic ingredients of "abstractions" in their 'minds' 'purely' by manipulations of thought "itself" (an oxymoron), do you?)

For a person seemingly aware of "our brain's processing limitations", you most certainly, happily use the word "complex" more than enough. Truthfully, I find that more than suspicious when looking to understand or describe one's subject (doesn't use of the word "complex" arguably more often indicate a person's confusion?). Simple is all we understand. "Chunks", then better "chunks" and then still better "chunks".

Not only are subject matters described by you as complex (again and again), but the teams you indicate might have to be put together seems like it would necessarily generate complexities/confusion (and from my perspective the need for that really has not been indicated or certainly NOT for study in the psychology field, which is BY DEFINITION: behaviors and corresponding environmental aspects). (It is true, you do express much skepticism about such teams.)

About the one-theory issue. My Ethogram Theory is not a theory but simply an argued-for necessary approach (and just a starting approach, with a few necessarily applicable biological and empirical assumptions) -- not anything like what anyone would see as like other [full-blown] theories, and thus I guess it is NOT one (but sometimes you can go far with very little, by starting and continuing correctly). In the main paper I show the other classic perspectives as all poorly founded and using presumptive conclusions as explanations -- and more recently I have described their failings as related to likely-false, unfounded (certainly unproven) 'assumptions' and have detailed these and shown the more likely , more biologically consistent alternatives. (It is adherence to false assumptions and considering NO others that is behind the development of goofy 'theories' like "embodiment" 'theories' AND it convincingly explains the absolute inability to take other not unlikely perspectives.)

I would add that taking a perspective like mine and finding some of the empirical (shown/environmental), basic core, directly-observable phenomenon I hypothesize for cognitive development is not only necessary for a foundation of psychology, but it is also necessary if there is going to be any hope for true artificial intelligence. True AI Robots ("learning" and operating autonomously) must have as their impetus for change (for _ALL_ realistic change in their behavior patterns), clear concrete aspects of the environment -- and those processed correctly, yet allowing enough "openness" for the behavior of the AI machines not to be stilted: The robots have to be open to all possibly relevant experiences AND have the ability to represent, classify, etc, and discern causality by way of aspects of cognition which have developed -- all stemming from FROM concrete aspects of experience combining with that which had already been correctly, stored, related, classified and understood -- to THEN (at the appropriate time) correctly allow similar progression to the next level/"stage" of cognition (initially shown in aspects of their perception/attention), and to thus begin to show impressive new sorts LEARNINGS and "appreciations" of related-ness and classification, and that leading to the development of further understandings. This is exactly what I see the perceptual or perceptual/attentional "shifts" doing for the human -- and having the very same role there as they would in AI.

IN fact, because of the intransigent ways of psychology, I have "packaged" a good core of understanding of key capacities and key behavior patterns and what would be indications OF/for major core impetus-es for behavior development FOR ARTIFICIAL INTELLIGENCE people. (The latter part is simply, again, the "A Human Ethogram ... " paper, describing the key approach and what we must come to see and then understand -- perhaps in the human first, before used in AI). See: https://www.researchgate.net/project/Developing-a-Usable-Empirically-Based-Outline-of-Human-Behavior-for-FULL-Artificial-Intelligence-and-for-Psychology

Though the authors are not of a correct theoretical persuasion ** (in fact, some are "embodiment" thinkers, way "off-base" in their thinking and analysis), STILL the following study indicates the potential of eye-tracking technology to possibly DO WHAT I THINK NEEDS TO BE DONE to provide a empirical foundation for cognitive-development understandings:

Article Eye-Tracking Piaget: Capturing the Emergence of Attentional ...

and here is a link to the whole article (as a pdf): https://edrl.berkeley.edu/sites/default/files/Abrahamson.Shayan.Bakker.vdSchaaf.2016.HD_.pdf

AND:

Here is one having to do with art appreciation, which is certainly not directly of the same nature as the work I say is needs to be done, yet nonetheless shows the great potential of this technology (AND in it is NEW, at least the unobtrusive yet detail-providing tools developed quite recently): https://thepsychologist.bps.org.uk/volume-29/november-2016/window-soul-and-psyche

Here is another study that, while dealing only with infants, still shows the POTENTIAL OF this NEW TECHNOLOGY to track attention/gaze associated with cognitive development:

(use your pdf reader to see this): https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=13&cad=rja&uact=8&ved=0ahUKEwiIvdXv0vPWAhVI5mMKHebYAHE4ChAWCDEwAg&url=https%3A%2F%2Fwww.researchgate.net%2Ffile.PostFileLoader.html%3Fid%3D5882e6bd3d7f4b341669ba49%26assetKey%3DAS%253A452826913611778%25401484973757789&usg=AOvVaw10N5mbN_zdiJnHmZaE-MGc

** FOOTNOTE: This is a note I sent to Dor Abrahamson: (I do NOT support his approach):

Dear Dor Abrahamson ... As you may know I do greatly oppose what clearly seems to be your theoretical persuasion : the "enactive" and "embodied" 'stuff'. I do offer a potentially correct alternative (which is more consistent with necessarily applicable biological principles and which has better, more-likely and testable, assumptions) : it has the organism's basic new-learning-initiating processes (ultimately clearly related to qualitative shifts in thinking) EMBEDDED in/with the environment AS perceptual and/or perceptual attentional "shifts". Unlike your approach this approach is absolutely grounded in foundational empiricism: directly observable proximate causes (OVERT behavior patterns [(subtle)] AND corresponding environmental aspects). [ Your approach, in contrast, relies on weak, indirect evidence and, according to peer reviews, has not helped and shows no potential -- because of the total reliance on indirect evidence; YOU MUST FIND THE TRUE EMPIRICAL GROUNDING, even IF your theory TOO, along with my approach, may have some truth to it (there ARE other highly likely potentially detectable OVERT behavior patterns at the base of what my approach would find and that would also be a base for yours, to the extent it is helpful at all and true (which frankly, I doubt). ] This (my approach) requires the most modern eye-tracking technology, but still may offer the first real foundation for cognitive-developmental studies (/theory). It is possible psychology (as a real empirical science) can only NOW begin. Psychology may have to accept this.

[ For more on why I am dissatisfied with Abrahamson's view, see my Comment under the paper he has just provided the full-text for: Article Eye-Tracking Piaget: Capturing the Emergence of Attentional ...