Variables in a psychological study usually represent theoretical constructs, and the aim of replication usually is to get as close to representing the same original construct as possible. This means that a direct replication is far from a trivial undertaking. Identical operationalizations of a variable may prove useless, for example, if they do not represent the construct of interest any more - perhaps due to cultural or historical changes in the way that construct is expressed or should be measured.

An example from my own research: My student Lena Schlüter and I tried to replicate field experiments by Goldstein, Cialdini, and Griskevicius (2008) on descriptive norms as a means to increase hotel guests' towel reuse behavior. Goldstein and colleagues had provided US hotel guests with a majority norm suggesting that 75% of previous guests had reused their towels - this increased reuse rates compared to a control condition without the descriptive norm message. In our own replication study, which we conducted in Germany, we used the identical figure of 75%. Although we also found some interesting effects, it turned out that the US and German samples differed dramatically in their rates of towel reuse, which ranged roughly between 70 and 90 % in Germany but only between 35 and 50 % in the USA.

So the lesson here is that an absolutely identical operationalization (a 75% descriptive norm) may have been much less influential for German than for US respondents, due to cultural differences in the target behavior of interest.

Both the original hotel towel paper and our replication paper can be found on ResearchGate:

https://www.researchgate.net/publication/23547467_A_Room_with_a_Viewpoint_Using_Social_Norms_to_Motivate_Environmental_Conservation_in_Hotels?ev=pub_cit

and

https://www.researchgate.net/publication/250308847_A_Room_With_a_Viewpoint_Revisited_Descriptive_Norms_and_Hotel_Guests%27_Towel_Reuse_Behavior

Another paper that nicely demonstrates how identical operationalizations may take on different meanings for people in different cultures -- and how a good intercultural replication requires different, culture-specific operationalizations -- is Sedikides, Gaertner, and Toguchi (2003), also on ResearchGate:

https://www.researchgate.net/publication/10958513_Pancultural_self-enhancement?ev=prf_pub

Article A Room With a Viewpoint: Using Social Norms to Motivate Envi...

Article A room with a viewpoint revisited: Descriptive norms and hot...

Article Pancultural Self-Enhancement

'Many people prefer the term 'close' replication. Ideally, the original researcher has specified which factors and mechanisms underlie the observed effect. This would mean that, as long al you perfectly replicate these things, everything else is allowed to vary. That is not always true, but finding that out is a way to improve our understanding of the initial effect.

Variables in a psychological study usually represent theoretical constructs, and the aim of replication usually is to get as close to representing the same original construct as possible. This means that a direct replication is far from a trivial undertaking. Identical operationalizations of a variable may prove useless, for example, if they do not represent the construct of interest any more - perhaps due to cultural or historical changes in the way that construct is expressed or should be measured.

An example from my own research: My student Lena Schlüter and I tried to replicate field experiments by Goldstein, Cialdini, and Griskevicius (2008) on descriptive norms as a means to increase hotel guests' towel reuse behavior. Goldstein and colleagues had provided US hotel guests with a majority norm suggesting that 75% of previous guests had reused their towels - this increased reuse rates compared to a control condition without the descriptive norm message. In our own replication study, which we conducted in Germany, we used the identical figure of 75%. Although we also found some interesting effects, it turned out that the US and German samples differed dramatically in their rates of towel reuse, which ranged roughly between 70 and 90 % in Germany but only between 35 and 50 % in the USA.

So the lesson here is that an absolutely identical operationalization (a 75% descriptive norm) may have been much less influential for German than for US respondents, due to cultural differences in the target behavior of interest.

Both the original hotel towel paper and our replication paper can be found on ResearchGate:

https://www.researchgate.net/publication/23547467_A_Room_with_a_Viewpoint_Using_Social_Norms_to_Motivate_Environmental_Conservation_in_Hotels?ev=pub_cit

and

https://www.researchgate.net/publication/250308847_A_Room_With_a_Viewpoint_Revisited_Descriptive_Norms_and_Hotel_Guests%27_Towel_Reuse_Behavior

Another paper that nicely demonstrates how identical operationalizations may take on different meanings for people in different cultures -- and how a good intercultural replication requires different, culture-specific operationalizations -- is Sedikides, Gaertner, and Toguchi (2003), also on ResearchGate:

https://www.researchgate.net/publication/10958513_Pancultural_self-enhancement?ev=prf_pub

Article A Room With a Viewpoint: Using Social Norms to Motivate Envi...

Article A room with a viewpoint revisited: Descriptive norms and hot...

Article Pancultural Self-Enhancement

You might want to take a look at this: http://www.nature.com/news/psychologists-strike-a-blow-for-reproducibility-1.14232 . HTH, Bruno

Thanks all - very helpful. I've just written a draft of a paper on this topic and may solicit further feedback down the road!

Here's a useful discussion paper that is critical of the feasibility of direct replication (see attachment):

Stroebe, W., & Strack, F. (2014). The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science, 9(1), 59-71. doi:10.1177/1745691613514450

https://www.dropbox.com/s/g0llwc1s1lxvnrj/Schwarz_%26_Strack_Many_labs_commentary_pri.docx?dl=0