# Question
Title: Could Fitts Law be used to measure difficulty in platform genre games?
I've been learning about Fitts' law and am wondering if it's applicable to measuring the difficulty of platform type games where the challenge is to hit the platforms?

If so, this could be used to ensure there is a well designed difficulty curve.

I'm also wondering if anybody has heard if Fitts law was used to measure difficulty in a game.

# Answer
> 4 votes
Probably, though my brief literature search didn't turn up any examples of people actually measuring game difficulty using Fitts' law. One thing to keep in mind is that motor movements are only a portion of what might make a game physically or cognitively demanding, even in platform games which are typically quite simple. Users have to make additional decisions, such as which platform to jump to or where to attend (do I look at the platform I'm jumping to, or do I look ahead to anticipate the next jump?)

I did find one interesting paper by Looser, Cockburn, & Savage (2005), who use Fitts' law to predict movement time in first-person shooters. The authors found that the time taken to pan (using a mouse) to acquire a target could be accounted for by Fitts' law. So if the time to acquire a target (e.g. before an enemy starts shooting at you) is critical, it certainly suggests that your methodology could be used to moderate difficulty level, say, by placing enemies farthest away from the current position.

Again, this is mostly speculation as I don't think there are any actual studies that address your specific question. But your intuition seems quite reasonable.

> Looser, J., Cockburn, A., & Savage, J. (2005). On the validity of using First-Person Shooters for Fitts' law studies. Proceedings of. People and Computers XIX, 2, 33-36. PDF

---
Tags: cognitive-psychology, learning, video-games
---


# Question
Title: How can pain sensations appear to originate within the person's skull?
I've read that humans have some sort of a **kinesthetic model** of their body and muscles. This internal representation of the body is used to control and coordinate locomotion. I don't remember if feeling of pain is related to the same model.

I've checked the articles below in hopes of recalling what the correct term for the kinestetic model was, but was unable to find it. Sensory Homunculus concept seems close, but doesn't ring the bell. **Can someone suggest what the correct term for the internal representation used to coordinate motion is?**

Phantom Pain

Phantom Limb

**I'm interested in learning how people can feel various sensations that appear to originate within their skulls?** For example, sometimes I feel headache and can say that it is in the left side, or I feel something in the back of the head. From what I understand, the brain does not have pain receptors, and there are no voluntary muscles to contract within the brain. This makes it hard for me to imagine how such positioning system can develop or what its purpose is.

With this in mind, I'm puzzled by how the location of a sensation can be determined within the skull.

* Is this an extension of the kinestetic model of the body
* Or is it a side effect of other pain (ex: teeth, thyroid) being mis-mapped
* or just a side effect of a headache(ex: hallucination/synestesia)?

# Answer
This question might be better handled by biology.SE. I think the short answer is that there are many pain receptors around the skull.

From wikipedia:

> The brain tissue itself is not sensitive to pain because it lacks pain receptors. Rather, the pain is caused by disturbance of the pain-sensitive structures around the brain. Nine areas of the head and neck have these pain-sensitive structures, which are the cranium (the periosteum of the skull), muscles, nerves, arteries and veins, subcutaneous tissues, eyes, ears, sinuses and mucous membranes.
> 5 votes

---
Tags: neuroscience, perception, cognitive-modeling
---


# Question
Title: How does language change the understanding of a complex concept?
For example, does breaking down a complex concept, initially described with scientific and uncommon terms, into less complex language degrade the understanding of the concept or can it maintain all it's relations and objects? How does using simple language change the understanding or perception of a complex concept?

Another way to look at the question is can certain thoughts exist without specific words to wrap them in, or are some thoughts dependent on the vocabulary/language?

If so, does this imply that new words can lead to new concepts that have never been produced and shared by others? For example, in George Orwell's novel "***1984***" the government introduces a new language called "Newspeak" that must be spoken by all citizens. This new language does not have any words for the concept of freedom, liberty, etc - so that these ideas would be very difficult to expand into reliaty (keeping the government in control of course).

The complete Wikipedia article on language and specifically "Semantics" has a lot of insight but did not lead to any answers.

Thank You.

# Answer
This question is pretty broad, but perhaps these studies address your question.

In 2006, Daniel Oppenheimer won an Ig Nobel prize for his paper *Consequences of Erudite Vernacular Utilized Irrespective of Necessity: Problems with Using Long Words Needlessly*, which demonstrated that using overly-complex words when a simpler word would suffice resulted in lower intelligence ratings of the author. Additionally, this may lead readers to rate the content as less true (Reber & Schwarz, 1999). So to answer one of your questions, yes, the language one chooses to use can change a reader's perception of it.

More to the point, your question is related to the Sapir-Whorf hypothesis that language determines thought (or the weaker hypothesis, that language influences thought). The strong version of this hypothesis (language determines thought) has been pretty thoroughly debunked. However, there is some credence to the idea that language influences thought. Many studies have examined this hypothesis, but one example is that of Frank et al. (2008). In this article, Frank tests an Amazonian tribe (the Pirahã) who have no number words-- purportedly, not even a word for 'one' or 'two'. Frank tested them on a variety of counting tasks and found that overall, their performance suffered in comparison to nearby tribes who did have number words. His claim is that number words serve as a cognitive tool to perform basic counting and arithmetic, but the lack of those words *does not* affect their ability to *perceive* quantity.

So overall, there is very little evidence to support the strong hypothesis that not having a word for a particular set of semantic features affects our ability to *conceive* of those concepts. However, not having those words may have subsequent affects on other areas of our cognition, such as (in the case of the Pirahã) our ability to count.

> Oppenheimer, D. M. (2006). Consequences of erudite vernacular utilized irrespective of necessity: Problems with using long words needlessly. Applied Cognitive Psychology, 20(2), 139-156. PDF
>
> Reber, R., & Schwarz, N. (1999). Effects of perceptual ﬂuency on judgments of truth. Consciousness and Cognition, 8, 338–342. PDF
>
> Frank, M. C., Everett, D. L., Fedorenko, E., & Gibson, E. (2008). Number as a cognitive technology: Evidence from Pirahã language and cognition. Cognition, 108(3), 819-824. PDF
> 6 votes

---
Tags: language
---


# Question
Title: What contextual links would make full-text journal articles more useful?
After searching PubMed or Google Scholar researchers often arrive at a publisher's full-text article page. Assuming that you have access to the full-text access, **what contextual links should be included within the article?** i,e links to Brain Atlas, links to Gene Databases

# Answer
This is a subjective question and what is useful would depend on from which domain of cognitive science the article originated. In general, I prefer when the PDF looks like the journal article (i.e., no formatting to indicate links).

Here is a list of things that I find useful:

* Links from in text citations to the location of the full reference
* Links from references to the full-text resource
* Links between Table and figure references with the actual figure
* Table of contents tied to sections in the article
* URLs and emails that can be clicked on
* Tables with exportable data
* If the article includes supplementary material, links to that material that work and do not require password access.

A few extra things that might be nice perhaps in the HTML version:

* Ability to export both the article reference and the references in the article in various bibliographic formats (e.g., bibtex, Endnote, etc.)
* Page view, download, and citation statistics
* RSS feed options for the journal
> 1 votes

---
Tags: publication-process
---


# Question
Title: Is performance reducible to brain activity in an unambiguous way?
As a preface, let me assume that the entirety of mental states and their corresponding behaviors in a person are entirely reducible to the physical activity of neurons sending a variety of chemicals across synaptic clefts at appointed times. There's no extra-logical, metaphysical sense of being that guides our actions e.g. spirits, soul stuff, etc.

Beyond this provision, which is obvious, is the overarching question of how particular networks in the brain control various structural and behavioral faculties in the mind. For example, if we place a person in an MRI and ask them to count to three, we expect to find activation in a portion of the parietal lobe. **Just how fine-grained is this activation?** If we ask the same person to do the same counting at a different time, will a different network in the same area of the brain be used?

The first question has a corollary to it; a kind of role reversal: **given a sufficiently detailed picture of a person's brain, is it possible to more or less describe what the person is doing or thinking?**

# Answer
> 8 votes
If we stick them in an MRI, the activation will not be very fine-grained. There are roughly 630,000 neurons in a 3x3x3 mm voxel recorded during fMRI. However even this coarse detail is sufficient to perform many impressive mind-reading tasks (e.g. Mitchell et al., 2008)

But what if we were able to record every single neuron in the brain simultaneously, could we then find a unique 1-to-1 mapping between neuronal firing and behavior? Not really. Consider what happens when you count to three-- and let's ignore everything going on in your brain that's not contributing towards this task (e.g. autonomic functions, mind wandering, etc.) Do your tongue, jaw, and lips move in exactly the same way every time? Does the acoustic signal sound the same every time? No; there is some variability in neuronal firing that causes behavioral variability.

OK, well what if your muscles *did* move in *exactly* the same pattern. Then, would the neuronal signature be identical between trials? Not necessarily. Let's assume there are 100 billion neurons in the human brain. We can't observe that many degrees of freedom in our behavior. It's likely that if we removed a single neuron, our behavior might remain 'identical'. There's a massive amount of dimension reduction to go from an array from 100 billion neurons to human behavior. But overall, the answer to your question depends on how you characterize 'performance'. If you define performance to be sufficiently complex such that performance is *equivalent* to the pattern of neuronal firing, then the question becomes a tautology.

The answer to the reverse question is slightly different. It is likely that if we observe the exact same pattern of neuronal firing twice, then the behavior will be exactly the same. However, even then, there is the possibility of noise in the peripheral nervous system once signals leave our brain that leads to some variability.

In a broader sense, cognitive neuroscience is aimed at answering this exact question: How does the activity of the brain correspond with our cognition and behavior? I like to think we've made some decent progress in this area, but we're still nowhere near a complete answer to this question.

> Mitchell, T. M., Shinkareva, S. V., Carlson, A., Chang, K. M., Malave, V. L., Mason, R. A., & Just, M. A. (2008). Predicting human brain activity associated with the meanings of nouns. Science, 320(5880), 1191-1195. PDF

---
Tags: neuroscience, philosophy-of-mind
---


title: Cheapest way to measure arousal levels

text:
# Question
Title: Cheapest way to measure arousal levels
I'm interested in performing a psychological experiment from my own home (the research is controversial; not that it's unethical, but rather I won't be able to aquire funding for it) and in it I would like to measure the arousal level of my participants as accurately and - possibly even more important - as cheaply as possible. I have a regular computer with USB ports, and even though I prefer to work with OS X, I could use Linux, and, if I really was forced to, even Windows.

I'm thinking that galvanic skin response is the best measure because of the method's temporal responsiveness, but I guess heart rate could be used as well.

**Does anybody know of any good and cheap instruments that can a) measure arousal level and b) can be connected to a regular computer?**

# Answer
> 2 votes
**I've long been thinking that arousal levels can be measured by an actigraph**, although I have not seen scientific articles that deal with this topic. Nor did I conduct any kind of large scale test. A lot of actigraphy studies focus on sleep and distinguishing sleep from wakefulness. I hypothesize that actigraphy may say something about the level of arousal of the central nervous system.

But the good thing is that actigraphy that uses an iPhone as a sensor **is practically free**. I have just submitted an mood tracking app to the app store (still to be reviewed) that has two tools illustrated below. The app will be free, and I built it for experiments of my own. My previous incarnations of this app were able to provide information in 5 second epochs in comma separated format. That data could be analyzed with excel. The way I sent myself the data is by emailing myself the report.

An actigraph is a device (or an app) that analyzes human motion at rates of about 20 times per second. It looks at acceleration and rotational rate caused by voluntary and involuntary movement. Think about it as an evolution of a pedometer. The device is continuously compounding motion activity into bigger epochs. In the picture below, real time data is compounded into 5 second epochs that are presented by graphs that appear on the device with only 5 seconds of delay. The activity count varies between 10 (complete stillness) to ~300 (typing a question in a SE site) to 10000 (walking). Data can be compounded further, into 5 minute epochs, suitable for long-term monitoring.

The magic of actigraphy comes from the fact that activity **can be observed over an extended period of time**. For example, there are scoring rules that distinguish sleep from wakefulness based on 7 minute windows of data. I'd love to learn if there's something like this for the arousal in the waking world.

Alternatively, there are heart rate apps, like Azumio's Instant heart rate

The second tool is a self-reporting tool, based on Pick A Mood , where by breaking the "mood" into two components, it becomes more evident that a person is experiencing different levels of arousal.

# Answer
> 4 votes
I think you may have answered your own question.

You can build a GSR meter or purchase one online. You do not necessarily need to hook it up to your computer; if it has an LED you can simply record the results with pen and paper. If you want to measure heart rate, buy a stethoscope. Or if you want to be really cheap, check your pulse using two fingers; it's free.

Note that I have no experience with building my own GSR meter or using the GSR meter that I've linked to. I cannot vouch for the accuracy or reliability of these devices. The familiar adage is probably appropriate: *you get what you pay for*.

---
Tags: measurement, arousal
---


# Question
Title: Difference between voting patterns in men and women
I'm looking for recommended reading on difference in voting patterns between men and women - both empirical data and theoretical explanations. In particular, have any significant biases (in terms of political spectrum or specific issues) been confirmed when comparing male and female voting patterns?

# Answer
I imagine there would be a huge literature on this topic. I found one interesting article by Inglehart and Norris (2000). I recommend reading the article. The article reports empirical findings relating to gender differences in voting patterns across countries and over time. It also discusses some of the proposed reasons for these differences.

The authors note that gender is generally not as important as class, region, or religion in voting patterns and in the formation of political parties.

To quote a few particular passages:

> Studies carried out in many countries in previous decades found that women were more conservative than men and less likely to participate in politics.... The article draws on evidence from the World Values Surveys in the early 1980s, and the early and mid-1990s carried out in over sixty countries around the world. This study establishes that gender differences in electoral behavior have been realigning, with women moving toward the left of men throughout advanced industrial societies (though not in postcommunist societies or developing countries) ...

To explain this movement of women tending to vote for left wing candidates, they offer what they call the Developmental Theory of Gender Realignment:

> \[They\] argue that a developmental theory can be used to explain this phenomenon, in which the transformation of sex roles in postindustrial societies has influenced the process of value change. As women’s and men’s lifestyles and cultural attitudes have been altered by the process of societal modernization \[the authors\] expect this to have a major impact on their political preferences.

In short, the article is well worth reading particularly for the way they have summarised various empirical sources of information regarding gender differences.

### References

* Inglehart, R., & Norris, P. (2000). The developmental theory of the gender gap: Women’s and men’s voting behavior in global perspective. International Political Science Review, 21(4), 441-463. PDF
> 4 votes

---
Tags: reference-request, sex-differences, political-psychology
---


# Question
Title: Phenomenon causing people to change their opinion when they are asked to hypothetically defend an opposing viewpoint?
I remember reading a case study on this years ago and I am trying to track it down.

The study involved asking people who held strong opinions on varied subjects to defend the opposite opinion in a hypothetical debate. At the end of the debate the people had come to adopt the viewpoint they were defending, regardless of the hypothetical nature of the actual debate.

Does anyone know the name of this phenomenon, or better yet, have a link explaining it?

# Answer
> 3 votes
Fazio et al (1977) provide a review of some of the literature on attitude change that can be induced by getting participants to take on another perspecive. To quote the opening paragraph.

> If an individual freely chooses to perform a behavior which is discrepant from his attitude, he tends later to realign his attitude toward that behavior. For example, a subject who complies with an experimenter’s request to write an attitude-discrepant essay against the legalization of marijuana is typically found to favor such legalization to a lesser degree than previously. This now classic attitude change effect is readily explained by both **Festinger’s theory of cognitive dissonance** (1957) and **Bem’s self-perception theory** (1972). Dissonance theory, in general, concerns the relationship between various cognitions. The theory posits the existence of a drive-like motivation to maintain consistency among relevant cognitions. Self-perception theory, on the other hand, concerns the passive inference of attitudinal dispositions from behavior. According to Bem (1972), “Individuals come to ‘know’ their own attitudes, emotions, and other internal states partially by inferring them from observations of their own overt behavior and/or the circumstances in which this behavior occurs” (p. 2). Social psychologists vary in which theory they endorse.

### References

* Fazio, R. H., Zanna, M. P., & Cooper, J. (1977). Dissonance and self-perception: An integrative view of each theory's proper domain of application. Journal of Experimental Social Psychology, 13(5), 464-479. PDF

---
Tags: terminology, social-psychology
---


# Question
Title: Is there a database of emotional stimuli available?
For my research I need to use emotional stimuli to use for habituation to certain emotion.

Are there any databases available (preferably free or not expensive) of non-linguistic stimuli I could use? (example: music, pictures, videos)

# Answer
> 8 votes
Yes! The International Affective Picture System (IAPS) is widely used.

From the IAPS instruction manual:

> The International Affective Picture System (IAPS) is being developed to provide a set of normative emotional stimuli for experimental investigations of emotion and attention. The goal is to develop a large set of standardized, emotionally-evocative, internationally accessible, color photographs that includes contents across a wide range of semantic categories. The IAPS (pronounced EYE-APS), along with the International Affective Digitized Sound system (IADS), the Affective Lexicon of English Words (ANEW), as well as other collections of affective stimuli, are being developed and distributed by the NIMH Center for Emotion and Attention (CSEA) at the University of Florida in order to provide standardized materials that are available to researchers in the study of emotion and attention

The stimuli are completely free, but a request must be made by a faculty member (rather than a student). The reason they give for this is that "Making these materials familiar to the general public can seriously compromise their value as stimuli in many research projects."

You can request the stimuli here.

> Lang, P.J., Bradley, M.M., & Cuthbert, B.N. (2008). International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical Report A-8. University of Florida, Gainesville, FL.

---
Tags: measurement, experimental-psychology, emotion
---


# Question
Title: How much red/orange is needed to stimulate hunger?
A common claim thrown around is that red and orange are great for restaurants because it stimulates hunger. However, I've heard time and time again that this does not work in web design because you need a *whole room* to be red/orange to stimulate hunger; one such claim is here but I can never find an academic source to back this up.

How much of a color is necessary to actually provoke this reaction? Is it feasible for on-screen design to actually evoke hunger or is that a misapplication of this finding?

# Answer
I tried to find a source to back up the claim that 'red' stimulates hunger. I couldn't, and in fact I found a study that suggests the exact opposite: the color red *reduces* food intake, because it acts like a stop signal (Genschow et al., 2012).

This also reminded me of a study by Brian Wansink and colleagues at Cornell, who had study participants consume Pringles in which every 7th chip was dyed red. They found that these people actually consumed fewer chips than controls (where none of the chips were dyed red). This may be due to the color of the chip, but another likely hypothesis is that changing colors gave participants an implicit signal of how many chips were eaten. (i.e., if all chips were red, their intake would be the same as controls)

If red/orange *does* stimulate hunger, it is likely a very weak effect, and highly context-specific. Therefore, asking "how much red" might not be the appropriate question to ask.

> Genschow, O., Reutner, L., & Wänke, M. (2012). The color red reduces snack food and soft drink intake. Appetite. LINK
>
> Geier, A., Wansink, B., & Rozin, P. (2012). Red potato chips: Segmentation cues can substantially decrease food intake. Health Psychology, forthcoming. PDF
> 5 votes

---
Tags: perception, vision, color
---


# Question
Title: Are intelligence scores correlated with detection of second stimulus in an Attentional Blink test?
## **Definitions**

**Attentional Blink:** An attentional blink is a phenomenon where when presented with rapid visual stimuli if you are asked to track two particular stimuli you will fail to notice the second if it is preceded shortly after the first (Raymond, Shapiro, & Arnell, 1992).

**Intelligence:** A quantifiable score, or set of scores, aimed at measuring human cognitive capacities (such as the Raven's Progressive Matrices test, Wechsler Adult Intelligence Scale, etc.).

## Question:

The results of Attentional Blink Tests have led some to see attention as a commodity that can be allocated, by will or nature, but needs time to be restored before additional attentional resources can be spent on subsequent activities (Raymond, Shapiro, & Arnell, 1992). It has been noted that emotion plays a role in recognition of the second target – particularly, if there is an emotional investment or relevance in the second target it is more likely to be detected (Schwabe, 2010). This illustrates that emotion can mediate attention. My question is whether intelligence could also mediate attention.

* **Does intelligence increase a person’s attentional abilities?**
* **Does, or would, an intelligence score positively correlate with the detection of the second stimulus in an Attentional Blink Test?**

**References**

Raymond, J. E., Shapiro, K. L., & Arnell, K. M. (1992). Temporary suppression of visual processing in an RSVP task: An attentional blink? *Journal of Experimental Psychology: Human Perception and Performance*, 18(*3*), 849-860. doi:10.1037/0096-1523.18.3.849

Schwabe L, Wolf OT (April 2010). "Emotional modulation of the attentional blink: Is there an effect of stress?". *Emotion* 10(*2*): 283–288.

# Answer
> 2 votes
CHCH mentions in the comments that Attentional Blink has been shown to correlate with operation span, a component of working memory. (Colzato et al., 2007)

It has also been demonstrated that working memory correlates highly with general intelligence. (Colom et al., 2004)

So yes, **performance on an attentional blink task correlates with intelligence scores**.

> Colom, R., Rebollo, I., Palacios, A., Juan-Espinosa, M., & Kyllonen, P. C. (2004). Working memory is (almost) perfectly predicted by\< i\> g. Intelligence, 32(3), 277-296. PDF
>
> Colzato, L. S., Spapé, M. M., Pannebakker, M. M., & Hommel, B. (2007). Working memory and the attentional blink: Blink size is predicted by individual differences in operation span. Psychonomic Bulletin & Review, 14(6), 1051-1057. PDF

---
Tags: cognitive-psychology, intelligence, performance
---


# Question
Title: When faced with a choice and trying to make a quick decision, are 2 options better than 3 or more options?
Bastardi & Shafir (1998) show how too much data disabilitates your decision making. There is also the popular Paradox Of Choice argument.

However, I can't find a reference stating that, when faced with a decision, 2 options are better than 3 or many options. It makes sense--you either like one of the options less or more than the other, and can then come to a quicker conclusion.

Is my intuition correct? If so, Could someone please point me in the direction of a thesis or paper which shows that?

**UPDATE**

I just found this paper and will update my question (or answer it!) if I find an answer.

**UPDATE 2**

I realise that the converse can be true as well, especially in the scenario when obtaining something, like the latest tablet. For example, bias and culture could make it really easy for you to discount the vast majority of options (e.g. anything not made by Apple), but then you struggle to make the final decision (e.g. shall I get an iPad 64GB or an iPad 4G?).

# Answer
> 4 votes
As Piotr mentions in the comments, you must operationalize "better".

Total time spent making a decision is likely to increase as the number of decision alternatives increase. This is known as Hick's Law.

If each choice has some objective value, then the maximum possible value that could be attained is likely to increase as the number of decision alternatives increase. This is probability.

The truth is not so clear cut. Each choice has some subjective value that is governed not only by its own intrinsic properties, but by comparison to other choices. Barry Shwartz, author of The Paradox of Choice, will tell you that having other choices available to you can increase the experienced regret post-decision, which lowers the subjective value of the choices.

There's also a speed-accuracy trade-off. If a decision needs to be quick, as you say, there is some cost associated with time that needs to be subtracted from the value of the chosen option.

Overall, it would be too simplistic to say "2 choices are *always* better than one", or vice versa. The answer, as usual, is "it depends".

---
Tags: decision-making
---


# Question
Title: Accuracy of social judgements of observers watching business meetings
I'm very interested in how people size one another up in social situations and in particular what we key in on when we are making these judgements.

---

**Let's make an imaginary experiment.**

I make several video tapes of actual business meetings just prior to when they begin. I edit the videos so that all explicit characterizations of the people are cut out (e.g. you wouldn't here one person address another person as "sir" or "ma'am"). I also get several simultaneous camera angles on each meeting. So basically you're seeing the people freely interact.

Now, I want *you* as the subject to watch the videos and tell me what you think the people are like. Who's the leader? Who is smart? Who is ineffective? Who is conniving? Etc.

Next, I ask you *why* you made each of these judgements, and I attempt to elicit very explicit rationale - such as "people who slump in their seats seem ineffective to me" or "people who hold their chins higher seem like leaders to me".

---

**Can anyone point me to discussions of real experiments similar to this?**

In particular I'm interested in

* How accurate humans tend to be in quickly "sizing up" people.
* How aware people are of why they are making these judgements and aware of what they are cueing in on.
* What is it that they are actually cueing in on?

I'm more interested in all types of social appraisals, (intelligence, wealth, honesty, etc.), but I'm most interested in appraisals of leadership qualities.

# Answer
You may be interested in the research on "thin slices" of behavior, defined as a very short video clip of behavior, often without audio.

Abstract from Ambady & Rosenthal (1992):

> A meta-analysis was conducted on the accuracy of predictions of various objective outcomes in the areas of clinical and social psychology from short observations of expressive behavior (under 5 min). The overall effect size for the accuracy of predictions for 38 different results was .39. Studies using longer periods of behavioral observation did not yield greater predictive accuracy; predictions based on observations under 0.5 min in length did not differ significantly from predictions based on 4- and 5-min observations. The type of behavioral channel (such as the face, speech, the body, tone of voice) on which the ratings were based was not related to the accuracy of predictions. Accuracy did not vary significantly between behaviors manipulated in a laboratory and more naturally occurring behavior. Last, effect sizes did not differ significantly for predictions in the areas of clinical psychology, social psychology, and the accuracy of detecting deception.

This methodology has been used frequently since this publication. I do not know the literature well enough to cite specifics, but perhaps this will point you in the right direction.

> Ambady, N., & Rosenthal, R. (1992). Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. Psychological Bulletin; Psychological Bulletin, 111(2), 256. PDF
> 3 votes

---
Tags: reference-request, social-psychology, io-psychology, leadership, experimental-psychology
---


# Question
Title: Plotting publication-quality ball-and-stick models of brain connectivity in 3D
I'm working on brain imaging (fMRI) and **I'm looking for a way to plot brain effective connectivity (dynamic causal modeling) parameters between different brain regions in a 3D plot**. The plotting software should have the following input and output:

### Input:

The directed connections from each region to another are defined by a connectivity matrix (no-connections have a value of 0). The 3D coordinates of the brain regions are presented in another matrix, practically using the weighted mean MNI coordinates of the regions as their coordinates for visualization.

### Output:

The connection strength would be signaled by the color of the sticks, and/or numerically. However, a first step could be simple 3D ball-and-stick diagram without connection parameters. Output as 2D vector or bitmap image: .ps, .eps, .tex, .png, .jpg, etc. The output would be used in a LaTeX document but also in other formats.

---

In the field of molecular analysis (with which I'm completely unfamiliar with), I noticed a lot of free and open-source software for creating ball-and-stick plots of molecules (some with publication-quality ray-tracing). Before reinventing the wheel (programming some intermediate software to convert MNI or Talairach 3D coordinates of brain regions to 3D spatial coordinates of atoms and writing them in some molecule file format for molecule ball-and-stick visualization software eg. RasMol to use it for brain connectivity visualization), I want to ask:

**Is there a good (and preferibly free) software for brain connectivity visualization?**

I have checked the examples of MayaVi, R and TikZ but neither of them has anything related to plotting ball-and-stick visualizations.

# Answer
Have you tried:

Also Nico Dosenbach has some amazing picture of brain connectivity in this paper http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3135376/. The code is based on matlab and the plotting can be done in the caret software http://brainvis.wustl.edu/wiki/index.php/Main\_Page from the Van-Essen Lab.
> 5 votes

# Answer
Answering to my own question: Avogadro is a molecule editor, but for visualization purposes it does not check the validity of the molecule, thus permitting its usage in the creation of publication-quality 3D ball-and-stick models of any subject matter, such as brain connectivity networks. Avogadro reads several molecule file formats, for example `.cml` is an xml format and it's quite straight-forward to edit by hand or with almost any programming language. Avogadro can save the molecules in several molecule formats and it can export images as bitmap graphics (`.png`, `.jpg`, `.bmp`), as vector graphics (`.pdf`, `.svg`, `.eps`) or as POV-Ray file. You may want to check @Ubermensch's comments on Avogadro (and Blender) above.

For animated visualizations (video), Blender might be a good option.
> 2 votes

---
Tags: software, neuroimaging, publication-process, fmri
---


# Question
Title: A psychological theory that explains why people remember only the outcome?
I've read somewhere about a psychological theory that people often remember the end results or the outcome of a certain situation/discussion/conversation/etc, but not the details of it.

In other words, the theory postulates that people often forget about things like who invented the solution, or how this happened, or who should be doing this task. Rather, people tend to just remember what the solution was, what happened, and what task had to be done.

I don't remember exactly in what context I read about this theory. It could have been part of some other theory.

Does such a theory exist? If so, what is the name of that theory?

# Answer
> 2 votes
I'm not sure about the problem-solution context, but there is the fairly well-studied "peak-end" phenomenon in which people appear most sensitive to the strongest and/or the final levels of positive or negative experiences in an episode. Daniel Kahneman has done a lot of work on this phenomenon, you can find a list of his papers on his website. A fairly simple demonstration is in this article:

Kahneman, D., Fredrickson, D.L., Schreiber , C.A. , & Redelemeier, D.A. (1993). When more pain is preferred to less: Adding a better end. *Psychological Science, 4*(6), 401-405.

---
Tags: cognitive-psychology, terminology, memory
---


# Question
Title: Why do we prefer visually aligned objects?
We all know visual alignment is one of the foundations of design. Everything must be aligned with everything else.
We also know that when things are aligned it is easier to process information. **My question is *why*?**

Nothing in nature is "aligned". What's the anthropological/biology reason why humans prefer aligned information?

# Answer
One of the main reasons related to the **'Gestalt principles'**

* Law of Proximity Objects near each other tend to be grouped together.
* Law of Similarity Similar items tend to be grouped together.
* Law of Closure Objects grouped together are seen as a whole.
* Law of Continuity Lines are seen as following the smoothest path.
* Law of Common Region Items in similar areas tend to be grouped together.
* Law of Connectedness Items that are connected are seen as groups.

This is a nice summary from the article referenced below:

> **Coupling emotional and behavioral triggers in some form of cognitive grouping — whether by proximity, similarity or something else — strengthens the motivation to act on the behavioral cue.**

I'd add to this: We feel more comfortable when perceiving something expected. When something breaks this (such as misalignment), it causes stress because there is a gap between our expectations (as the principles outline) and our experience.

References
> 16 votes

# Answer
There are two related reasons, I believe for this:

1. Relationship and connection. When things are aligned, we see then as connected and related. Nature does give us the guidance for things that are related and connected in other ways, but often by a degree of alignment or similarity. In UX terms, we indicate the relationships between items by positioning and similarlity, including alignment.
2. Pattern matching. Our brains are awesome pattern matchers. However, if there is a misalignment, we try to see some meaning or pattern in the misalignment, when there probably isn't one. By aligning things, the pattern we see is one of order and similarity.

There is a degree of alignment in nature, and we do notice it. There is evidence that symmetrical faces are more attractive), because we like the alignments. It makes the UI look more attractive.
> 13 votes

# Answer
Jay's answer is correct, this occurs as a result of Gestalt processing. I will address your comment that:

> No response actually brings light on why we prefer things to be aligned, only the realization of such fact (for example, Gestalt principle).

But first I will say that I believe this assumption is wrong:

> Nothing in nature is "aligned".

Many things in nature are aligned. The sides of my body are roughly parallel. The same with the sides of my arm, a tree trunk, a river, etc. In fact many Gestalt properties occur so often in nature, that they have been deemed "non-accidental" properties (Lowe, 1987; Witkin & Tenenbaum, 1983). The idea is that the probability of seeing, e.g., two parallel lines by chance alone would be extremely small if lines in nature had random orientation. But it's not rare, and so it must have some significance (e.g. they are part of the same object).

Jepson and Richards (1992) formalize this argument in a Bayesian framework, and show that statistical regularities in the environment (such as Gestalt features) make good "features" precisely because they are "non-accidental", and thus probably carry informational content.

> Richards, W., & Jepson, A. (1992). What makes a good feature? (No. AIM-1356). MASSACHUSETTS INST OF TECH CAMBRIDGE ARTIFICIAL INTELLIGENCE LAB. PDF
> 6 votes

# Answer
Nothing in nature is "aligned" - not really.

One of nature's marveling fact is that there is unity everywhere starting with the way a water drop is forced to be spherical, to the ice flakes being symmetrical, to cells coming together to form organ systems etc. I could go on.

Perhaps its that 'come together' = 'grow in life' got us till here in evolution :)

In fact we sometimes end up grouping unrelated things together, if they are nearby (ah! 'Gestalt principles').

Also if I remember my algorithm basics correctly, its easier to sort items that are grouped. Because then you know they form a 'pattern' (say sorting numbers in descending manner before applying some algorithm).
> 2 votes

# Answer
Normally we design and design follows rules. If not we create art. Alignments follow a.) grids or patterns or b.) other objects (if we have not a grid) in X or Y (left, right, bottom, top, center).

And I would say everything in nature is aligned, but not in a symmetrical way. All life on earth is based in grids. Bubbles, cell structures, don´t forgot the the golden section, a relation which is very valuable. There are a lot of examples in bionics.

In UI we have to different reading directions from left to the right, or reversed, the same from the top to the left. The implication is, that we have to conduct the user with visual alignments.
> -4 votes

---
Tags: perception, vision, gestalt-psychology
---


# Question
Title: Psychological research on memorability of passwords?
There are lots of ad-hoc rules that people are happy to propose about passwords, such as:

* Pick a phrase you can remember easily, and then use the initial letters.
* A random short sequence of letters, numbers and symbols.
* A sequence of random common words (like XKCD's "battery correct horse staple").

There is also a pile of computer security research on what passwords are hard to crack.

It seems like a pretty simple empirical question to say what is the best password scheme for any given desired strength.

**Can anyone point me to any research about the ease of memorizing passwords generated according to various rules?**

# Answer
> 3 votes
It looks like there is some empirical evidence that passphrases are easier to remember than either random passwords (not surprising) or user-selected short passwords after standard advice to use mix of upper and lower and avoid words.

http://homepages.cs.ncl.ac.uk/jeff.yan/#exp

Thanks Jeff for the pointer in a comment (same Jeff?).

> J. Yan, A. Blackwell, R. Anderson and A. Grant. The memorability and security of passwords -- some empirical results. University of Cambridge, Computer Laboratory Technical Report No. 500, 2000. (see also Hungarian translation). PDF
>
> J. Yan, A. Blackwell, R. Anderson and A. Grant. Password Memorability and Security: Empirical Results. IEEE Security & Privacy, Vol. 2 No. 5, 2004. PDF

the above is reprinted as:

> J. Yan, A. Blackwell, R. Anderson and A. Grant. The Memorability and Security of Passwords. Refereed book chapter in Security and Usability: Designing Secure Systems that People Can Use (ed. by Lorrie Cranor and Simson Garfinkel), OReilly & Associates, USA, 2005.

---
Tags: cognitive-psychology, memory, hci
---


# Question
Title: Is there a random walk theory that can account for situations with more than two choices?
In the article "Two-stage Dynamic Signal Detection: A Theory of Choice, Decision Time, and Confidence" from 2010 by Pleskac and Busemeyer, a random walk model is presented for situations where a discrete choice is made (that is, signal present/not present, yes/no, 1/0 et cetera). Here, if there is no time pressure involved, a threshold value is set which has to be reached before an answer is given. Contrary, if there is a time pressure involved, the value at the last time point is used to decide which answer is given, even if the threshold point hasn't been reached at that time.

I wonder whether there are any extension of this model which can incorporate situations with *more* than two choices (from 3 alternatives and up to a fully continius choice space). I guess you just could incorporate more dimensions (the model referred to above is two-dimensional), but if the answers are non-independent (for example, when using a rating scale), I guess the random walk function would have to be fairly complicated. That is, if strong evidence is accumulated for number 3, how much evidence should be added to 4 and 2? Should evidence be subtracted from 1 and 5?

# Answer
Instead of having a single integrator with two bounds for two choices (symmetric random walk model), you can have many competing integrators each with a bound (race model). For example, see Fig 2. of Gold and Shadlen 2007 and references therein.

As for the continuous choices case, it is important to understand a limit of discrete choices can be very different from a continuous choice. For such a limit to make sense, there should be a notion of similarity between the choices, and I don't think those race models would generalize easily. A high dimensional continuous attractor dynamics is more likely to support such theory.
> 6 votes

# Answer
Diederich & Busemeyer (2003) presented a diffusion model for three choice alternatives (p. 314). The paper is a tutorial for calculating diffusion models with (discrete) matrix methods. The extension to three choice alternatives is reached by defining a two-dimensional diffusion process on a triangular plane (state space).

Recently, Wollschläger & Diederich (2012) presented the **2N-ary choice tree model** which models multi-alternative choices by random walks on decision trees.

**References**
Diederich, A. & Busemeyer J. R. (2003). Simple matrix methods for analyzing diffusion models of choice probability, choice response time, and simple response time. Journal of Mathematical Psychology, 47(3), 304-322. doi:10.1016/S0022-2496(03)00003-8 (preprint pdf)

Wollschläger LM and Diederich A (2012) The 2N-ary choice tree model for N-alternative preferential choice. Front. Psychology 3:189. doi: 10.3389/fpsyg.2012.00189
> 6 votes

# Answer
Bogacz et al. (2006) provide the most comprehensive overview of models in this domain. This includes comparisons of the Drift Diffusion Model (Ratcliff, 1978), Ornstein–Uhlenbeck (O-U) Model (e.g. Busemeyer & Townsend, 1993; "Decision Field Theory"), race models without inhibition (e.g. Vickers, 1970), and race models with inhibition (e.g. Usher & McClelland, 2001; "leaky competing accumulator model") and others.

While diffusion models are limited to 2 alternatives (e.g. DDM, DFT), race models are not. (Note that diffusion models are essentially the same as random walk models, except that random walks involve discrete steps, while diffusion models are continuous). The race models with inhibition reviewed by the authors are computationally reducible to DDM, while race models without inhibition are different, and generally not preferable.

Beck et al. (2008) also provide a model that accommodates continuous choice, rather than a limited number of alternatives. It's relationship to other models is unclear.

So yes, there are many similar models that account for more than 2 alternatives. They are not technically random walk models, because a random walk only allows for 2 bounds. But they are computationally equivalent.

> Beck, J. M., Ma, W. J., Kiani, R., Hanks, T., Churchland, A. K., Roitman, J., ... & Pouget, A. (2008). Probabilistic population codes for Bayesian decision making. Neuron, 60(6), 1142-1152. PDF
>
> Bogacz, R., Brown, E., Moehlis, J., Holmes, P., & Cohen, J. D. (2006). The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychological review, 113(4), 700. PDF
>
> Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment. Psychological review, 100(3), 432. PDF
>
> Ratcliff, R. (1978). A theory of memory retrieval. Psychological review, 85(2), 59. PDF
>
> Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: the leaky, competing accumulator model. Psychological review, 108(3), 550. PDF
>
> Vickers, D. (1970). Evidence for an accumulator model of psychophysical discrimination. Ergonomics, 13(1), 37-58.
> 3 votes

---
Tags: cognitive-modeling, methodology, decision-making, mathematical-psychology
---


# Question
Title: Is there an actionable branch of cognitive science?
I'm not sure what is the name of **a branch of cognitive science that deals with the application of cognitive science to the real world**.

I know that in Biology, there are fields like genetic engineering or biomedical engineering, where the principles of science of Biology are applied to development of products.

In physics there's mechanical and electrical engineering.

I did a search and found that there are several *applied cognitive science* labs at various psychology departments in the US.

**Is there some branch of cognitive science that is focused on using the knowledge to create products or services?** Am I thinking of the applied cognitive science, psychology, psychiatry or something else?

Thank you!

# Answer
The answer depends on how you are defining cognitive science and what threshold you use to attribute that actionable component.

If you are talking about cognitive sciences in the broad sense that we use on this site, then you encompass disciplines including psychology, psychiatry, neurobiology, and so on. This opens up a much broader set of applied fields.

There is also the question of what is meant by cognitive science in action. There is a whole spectrum in the degree to which fields would draw on cognitive science findings.

A few applied fields that stand out as having very close links with psychology, cognitive science, psychology, and neurobiology (i.e., what we mean on this site by cognitive sciences) include:

* Applied Psychology including clinical, organisational, educational, counselling, sports psychology, etc. as well as a wide range of counselling, social work, and related roles
* Psychiatry
* Human factors, ergonomics, usability, user experience, etc fields
* Various medical fields related to neurobiology
* Education and related fields

Then there is a whole raft of fields which draw on such knowledge to a greater or lesser degree:

e.g.,

* Management, human resources, marketing, market research, etc.
* Occupational health and safety, work design, product design, etc.
* etc.

No doubt many other fields could be mentioned. I think this is partially explains the popularity of psychology as an undergraduate subject.
> 3 votes

# Answer
Human-Computer Interaction will give you a background for user testing. Compared to UX design, which draws from social psychology and anthropology, I would say HCI is more related to cognitive science. Donald Norman describes HCI as a "solved problem." We know how to make systems usable. Interaction design is a newer field evolving from HCI. There's also Human-Information Interaction, which is even newer. To learn more about HII, look up Information Foraging Theory. For lots of insight on how to apply psychology to web design, check out http://useit.com

Interaction design is amazing because it can be applied outside a business context. Instead of designing for profit, you could design for impact. Problems like poverty and education are going to need to be addressed as design problems. Interaction designers are going to very valuable, working together with politicians, in solving the world's wicked problems. (Look up 'wicked problems') Good luck! –

The best place to start is reading Don Norman's The Design of Everyday Things.
> 2 votes

---
Tags: terminology
---


# Question
Title: Are there professional cognitive science test subjects?
I'm interested in how cognitive science experiments are designed and executed. Up until now, all of the papers I've seen dealing with cognition use **availability of test subjects** as one of the major factors of recruitment for experiments. For example, students are recruited for an experiment, and because the group is so **diverse, they are tested on general skills** that the entire group would possess, like reading or math.

For example, consider this experiment: "Happiness economics" in reverse: Does happiness affect productivity? There were 276 university students and faculty involved. The students are tested with a math task.

Math and reading comprehension are general skills, and I can think of few people who earn their living with those skills, aside from SAT tutors.

I'm interested if there have been other types of cognitive experiments tried, where the **skills used to evaluate experiment are at a "professional" level.** For example professional athletes, soldiers or pilots come to mind.

* Experiments that use scoring percentage as a metric for basketball players
* Experiments that use marksmanship grouping as a metric
* Professional video game players

**Are there any groups of people, whose professional skills are consistently tested as metric of cognitive performance?**

Thank you for your input!

# Answer
> 5 votes
There are a few people who have been very extensively experimented on because they have unusual brain lesions (which is kind of like the opposite of a skill). The most famous is Henry Molaison (HM), who developed a rare case of episodic anterograde amnesia following surgery for severe epilepsy. He spent the entire latter part of his life in a care institute and was a continuous subject of various learning experiments and imaging studies. Another patient with very specific memory-related lesions is KC, who has been the subject of at least 20 case studies.

A couple of other studies that might be of interest are West et al. (2008), who recruited experienced video game players in order to find people with strong skills at rapid visual scanning, and research by Eleanor Maguire and colleagues that examined structural changes in the brains of new London taxi drivers as they acquired their fantastically detailed spatial map of the city's streets.

> West, G. L., Stevens, S. A., Pun, C., & Pratt, J. (2008). Visuospatial experience modulates attentional capture: Evidence from action video game players. Journal of Vision, 8(16).
>
> Maguire, E. A., Gadian, D. G., Johnsrude, I. S., Good, C. D., Ashburner, J., Frackowiak, R. S., & Frith, C. D. (2000). Navigation-related structural change in the hippocampi of taxi drivers. Proceedings of the National Academy of Sciences, 97(8), 4398-4403.

---
Tags: cognitive-psychology, reference-request, methodology
---


# Question
Title: How repeatable are cognitive science findings within the same individual?
Thanks to this website, I've seen a number of papers and scholarly articles that deal with cognition. I'm interested in how repeatable the findings are that are discovered as a result of experiments.

For example, **when a mechanical system is tested**, it is tested to work the same way every single time:

* A lock opens and closes when a key is turned.
* A car door opens up to the maximum degree allowed by design.

**How much is a cognitive system like a mechanical system?** Will it work the same way every time?

For example, I'm thinking about the design of an experiment I've been reading about: The Ikea Effect Experiment. Volunteers are asked to assemble Ikea boxes and bid on the box after assembly. The experiment shows that people value things they built themselves more than things that are just offered to them.

Now if the machine-like definition of repeatability is applied to this experiment: Would test subjects value boxes they built themselves next time they do an experiment? How about next month? What about in 3 years? Would people in the 1940's have the same response?

Have there been any studies that deal with this "repeatability" question? For example, **studies that used the same test subjects doing the same experiment multiple times?**

# Answer
> 5 votes
Humans are, as you would expect, much noisier than physical systems. As such, replication of experiments in psychology is very important. Replication provides evidence that the original results were not spurious.

As pointed out in the comments, replication *with the same subject pool* is not always the way to go. Many tasks have learning effects, and so you would not expect to achieve the same results by re-administering the experiment.

There is a vast statistical literature on reliability testing. A good place to start would be at the Wikipedia article for Reliability (psychometrics)

---
Tags: cognitive-psychology, reference-request
---


# Question
Title: What articles use structural equation modelling in a meta-analytic context to model mediation?
I'm interested in structural equation modelling (SEM) meta-analysis and its application to modelling mediation relationships.

Mike W.-L. Cheung has written a few articles on the topic of SEM Meta-Analysis (e.g., Cheung, 2009). He's also written the metaSEM package in R.

One basic approach to SEM meta-analysis is a simple fixed-effect two-step approach. First, you extract a weighted average correlation matrix from the component studies. Second, you analyse that correlation matrix using standard SEM approaches.

However, in reality, things are more complicated.

* Correlations often vary between studies by more than just sampling error, suggesting that a fixed-effect approach may be invalid. If a random-effects model is adopted, this raises issues of whether to categorise the correlation matrices into groups of fixed-effects matrices or whether to somehow incorporate the random-effects into the overall SEM.
* Some studies may not have all the variables of interest, which introduces a form of missing data, which in turn raises issues of how to define sample size and the standard error.

So in general, I'm interested in reading examples of SEM meta-analyses applied in psychology to get a sense of how others have dealt with the various issues. In particular, I'm interested in meta-analysis SEM that incorporates the modelling of indirect effects (i.e., mediation).

### Questions

* What studies have applied meta-analysis SEM in psychology to model indirect effects?
* How have they dealt with issues such as (a) fixed versus random effects and (b) missing data?

### References

* Cheung, M.W.L. (2009). Meta-analysis: A Structural Equation Modeling Perspective. PDF

# Answer
I'd like to address important issues that Jeromy Anglim raised in the "Personal thoughts" section of his answer, namely that correlation parameters (i.e., true, population, or infinite-sample correlations) often vary and covary among studies, and this between-studies\interstudy heterogeneity implies heterogeneity in studies' parameters for a structural equation model (SEM). I'll describe a method I've proposed to account for this heterogeneity when estimating and making statistical inferences about a SEM, such as a univariate or multivariate regression model, factor model, path model, or model for structural relations among latent variables, or specific quantities based on such a model (e.g., $R^2$, indirect\mediation effect, fit index). **This approach's core idea is simple:**

1. Use random-effects (RE) meta-analysis to estimate attributes of the between-studies distribution of correlation-matrix parameters (e.g., its mean and covariance matrix).
2. Transform those results to estimate attributes of the between-studies distribution of SEM parameters.

Many commonly used MASEM methods are based on fixed-effects (FE) models, and although meta-analyzing heterogeneous correlation matrices using certain FE methods performs reasonably well in some situations, it's probably not advisable for making unconditional inferences about a larger universe of studies (Hafdahl, 2008a).

Here I'll just give overviews of the problem and a three-step version of my proposed meta-analytic SEM (MASEM) method -- actually a collection of methods based on different choices for critical tasks at each step. To keep the exposition relatively accessible I'll gloss over several important technical details. Although this is an active area of my methodological research, the methods I'll sketch here are based largely on unpublished work (I'll cite references). Because some aspects of this approach would be prohibitively difficult for many applied researchers to implement, one aim of my work in this area is to develop user-friendly software. Until that's available, feel free to contact me for help with these techniques; such requests may motivate me to devote more time and other resources to this work.

## Overview of Problem

In this section I'll describe my view of the problems for which MASEM is typically used. Suppose we have from each of $k$ independent studies a sample Pearson-*r* or Fisher-*z* correlation matrix among $p$ variables of interest (e.g., $X\_1, X\_2, \ldots, X\_p$), and suppose we're interested in a particular SEM for those variables or a related quantity that can be expressed as a function of the correlation matrix (e.g., 1 or more \[standardized\] path coefficients, indirect or total effect, squared multiple correlation, fit index). It's useful to distinguish between each study's correlation-matrix ***parameter*** and that study's correlation-matrix ***estimate*** from a particular sample of subjects. Meta-analysts are typically interested in using several studies' estimates to understand their corresponding parameters or distributions thereof.

In this overview I'll mostly ignore several interesting but vexing complications that arise in practice. For instance, studies might use different versions of one or more variables (e.g., different measures similar to $X\_1$), some studies might not contribute all $d = p( p - 1) / 2$ distinct correlation-matrix elements (e.g., due to missing variables or unreported elements), some of a study's correlations might be based on different subsets of its sample (e.g., if some subjects are missing some variables), a study might contribute two or more independent or dependent correlation matrices (e.g., from different groups or the same group in different conditions or at different times), studies' correlations might be influenced by so-called *artifacts* (e.g., \[un\]reliability, range restriction, dichotomization), and study-level covariates\moderators might partially account for between-studies heterogeneity.

Now, if we take the RE view that correlation-matrix parameters vary among studies -- that is, our studies are from a universe of studies whose correlation-matrix parameters follow some multivariate distribution -- then **most functions of that correlation-matrix parameter will also vary among studies**. In particular, an SEM's path-coefficient parameters and other quantities (e.g., indirect effects) will have a distribution over studies. To be clear, these distributions' between-studies (co)variation is *not* due to within-study sampling (co)variation caused by finite samples of subjects; instead, it may be viewed as due to studies' different constellations of features that produce different values for their correlation-matrix parameters and, consequently, most functions of those parameters. (As a simpler example, consider 1 Pearson-*r* correlation parameter from each of several studies: If it varies among studies, then so will various functions of it -- its square, coefficient of alienation, Fisher *z*-transform, common language effect size, etc.)

By analogy with typical tasks in conventional RE meta-analyses, we might be interested in the following for our SEM: estimates of the between-studies mean and variance of each SEM path coefficient or other related quantity (e.g., indirect effect) and inferences about each such quantity (e.g., confidence interval \[CI\], prediction interval \[PrI\], hypothesis test). When we're interested in two or more quantities from the SEM, we might wish to obtain bi- or multivariate generalizations of these estimates or inferences (e.g., confidence or prediction region\set); these might include quantities based on two or more distinct SEMs we'd like to compare, such as fit indices from two or more SEMs. We might also be interested in the entire between-studies distribution of one or more SEM quantities (i.e., not only its mean and \[co\]variance \[matrix\]), which we could in some situations depict graphically (e.g., density plot).

Most commonly used MASEM approaches neglect between-studies heterogeneity in the SEM parameters, which seems especially difficult to justify in the presence of heterogeneous correlation(-matrix) parameters. Certain aspects of the SEM parameters' between-studies (co)variance might be substantively important, such as if we're interested in how much particular path coefficients, indirect or total effects, squared multiple correlations, or fit indices (co)vary among studies. For instance, if we use RMSEA as a fit index, we might wish to know how much RMSEA varies among studies, a plausible range of RMSEA values (e.g., prediction interval), or what proportion of studies have RMSEA values within some "acceptable" interval (e.g., below .05).

Moreover, there's reason to treat heterogeneity cautiously even if we're interested in only the mean of SEM parameters: Because SEM parameters are typically nonlinear functions of the correlation matrix, it's unclear what's estimated by applying the SEM directly to a mean correlation matrix, as in most MASEM approaches; that might estimate the mean SEM parameter poorly. As a simpler example, suppose that for a heterogeneous effect-size parameter $Y$ we know its mean and variance over studies, $\mathrm{E}(Y)$ and $\mathrm{Var}(Y)$, but want to know the mean for $Y$'s square: Because for most distributions of $Y$ it's a fact that $\mathrm{E}(Y^2) = \[\mathrm{E}(Y)\]^2 + \mathrm{Var}(Y)$, simply squaring $\mathrm{E}(Y)$ gives a value lower than the desired $\mathrm{E}(Y^2)$, especially if $\mathrm{Var}(Y)$ is large. This basic problem with applying a nonlinear transformation to a mean of heterogeneous effect sizes was addressed for the case of a univariate correlation -- focusing on the *z*-to-*r* transformation -- by Hafdahl (2009) and Hafdahl and Williams (2009); the analogous situation was addressed for correlation matrices by Hafdahl (2008b, 2009b), for generic univariate effect sizes by Hafdahl (2011), and for generic multivariate effect sizes by Hafdahl (2009c).

## Overview of Proposed Method

In the first paragraph above I mentioned the gist of my proposed two-stage MASEM method. To facilitate explanation, it's useful to divide the second stage -- transformation of results -- into separate steps for estimation and inference. For instance, if we start with $k$ studies' estimates of a Fisher-*z* correlation matrix, the first step might entail estimating the correlation-matrix parameters' between-studies mean and covariance matrix, and the second and third steps might entail transforming those results to obtain estimates of and inferences about the between-studies mean and covariance matrix of the SEM's path coefficients. Below I elaborate a bit on these three steps.

For convenience, let's use the following notation for correlation matrices from Study $i, i = 1, 2, \ldots, k$:

* $\theta\_i$: vector of the $d$ distinct parameters in a correlation matrix for $p$ variables, in either the Pearson-*r* or Fisher-*z* metric
* $y\_i$: vector of the $d$ distinct estimates in sample correlation matrix (i.e., $y\_i$ is an estimate of $\theta\_i$)

For example, if we're interested in the correlation matrix for $p = 5$ variables, then both $\theta\_i$ and $y\_i$ contain $d = 5(5 - 1) / 2 = 10$ correlations. In RE meta-analysis we typically assume that $y\_i$ has a within-study (sampling\conditional) distribution whose mean is approximately $\theta\_i$, and that $\theta\_i$ (or just $\theta$) has the same between-studies distribution for all studies, with mean $\mu\_\theta = \mathrm{E}(\theta)$ and covariance matrix $\Sigma\_\theta = \mathrm{Cov}(\theta)$. (For simplicity I'll slightly abuse notation for random correlation parameters and estimates -- instead of using $\Theta\_i$ and $Y\_i$ -- and I'll ignore other quantities used in certain meta-analytic procedures, such as each study's sample size and conditional covariance matrix, whose inverse \[i.e., precision matrix\] is essentially used as a weight matrix.)

In terms of SEM notation, let's similarly denote the SEM parameters of interest in Study $i$ by $\gamma\_i = g(\theta\_i)$, where the function $g$ transforms a correlation-matrix parameter into SEM parameters. This $g$ might be a fairly complicated function, such as for the parameters of a SEM with respect to a particular objective\loss criterion (e.g., ML, WLS, ADF) or a fit index for that SEM. Also, $\gamma\_i$ might be just one number (e.g., 1 path coefficient, indirect effect, fit index) or a vector (e.g., 2 or more path coefficients). At any rate, our MASEM goal might be to estimate $\gamma$'s between-studies mean or (co)variance (matrix), $\mu\_\gamma = \mathrm{E}(\gamma)$ and $\Sigma\_\gamma = \mathrm{Cov}(\gamma)$, and make inferences about either distributional attribute; we might also wish to estimate $\gamma$'s entire distribution.

Below are the three steps of my proposed method; Steps 2 and 3 presume we have in mind a specific SEM or related quantity that can be expressed as $\gamma\_i = g(\theta\_i)$. This is similar to Hafdahl's (2009a, 2011) univariate methods and Hafdahl's (2008b, 2009b, 2009c) multivariate methods but specific to MASEM.

**1. Meta-Analysis for $\theta$:** Apply multivariate RE meta-analysis to $y\_i$ to obtain estimates of at least $\mu\_\theta$ and $\Sigma\_\theta$, which I'll denote $\hat\mu\_\theta$ and $\hat\Sigma\_\theta$, and perhaps $\theta$'s full between-studies distribution; perhaps also obtain a covariance matrix for only $\hat\mu\_\theta$ or both $\hat\mu\_\theta$ and $\hat\Sigma\_\theta$, depending on how inference is handled in Step 3. Among several proposed methods for estimating $\mu\_\theta$ and $\Sigma\_\theta$, only a few handle incomplete correlation matrices -- nearly unavoidable in MASEM data sets -- in a principled way (e.g., Hafdahl & Wu, 2011; Kalaian, & Raudenbush, 1996; White, 2011). In particular, Hafdahl and Wu's extension of Becker and Schram's (1994) EM algorithm permits any one or more of $y\_i$'s correlations to be missing, doesn't require imputing values for missing correlations, and yields a posterior distribution for each study's entire $\theta\_i$ (given its possibly incomplete $y\_i$); it also yields an estimate of $\theta$'s full between-studies distribution as a mixture of the studies' posterior distributions. Depending on the estimation method, a covariance matrix for $\hat\mu\_\theta$ may be obtained by generalized least-squares (GLS) or other methods (e.g., based on Hessian matrix for maximum-likelihood estimators), some of which also provide a covariance matrix for $\hat\Sigma\_\theta$. Hafdahl (2004) demonstrated substantial differences in performance among different techniques for multivariate RE meta-analysis applied to correlation matrices.

**2. Estimation for $\gamma$:** Use an appropriate transformation method -- based on the function $g$ -- to obtain estimates of at least $\mu\_\gamma$ and $\Sigma\_\gamma$, which I'll denote $\hat\mu\_\gamma$ and $\hat\Sigma\_\gamma$, and perhaps $\gamma$'s full between-studies distribution. One strategy is to use a first- or second-order *Taylor series approximation*, which essentially entails approximating $\gamma = g(\theta)$ by a simpler linear or quadratic function of $\theta$; estimates of this approximating function's mean and covariance can then be computed from Step 1's $\hat\mu\_\theta$ and $\hat\Sigma\_\theta$. Another strategy entails *simulation*: Sample values of $\theta$ from its distribution estimated in Step 1, transform these to values of $\gamma$, and estimate $\mu\_\gamma$ and $\Sigma\_\gamma$ from this simulated distribution; we might treat $\theta$'s distribution as multivariate normal -- such as $\theta \sim \mathcal{N}\_d(\hat\mu\_\theta, \hat\Sigma\_\theta)$ -- or permit it to take some other form estimated from the data (e.g., mixture of posteriors from EM algorithm). Either strategy might also be used to estimate other attributes of $\gamma$'s distribution, such as tail or central areas (e.g., probability that 1 or more path coefficients or other quantities are near 0, positive, large, etc.) or $\gamma$ values bounding regions of interest (e.g., quartiles, middle 95%). (We could in principle transform $\hat\mu\_\theta$ and $\hat\Sigma\_\theta$ to $\hat\mu\_\gamma$ and $\hat\Sigma\_\gamma$ via integration, using the definitions of $\mu\_\gamma$ and $\Sigma\_\gamma$, but that'll often be intractable analytically and infeasible computationally.)

**3. Inference for $\gamma$:** Make inferences about $\mu\_\gamma$ and $\Sigma\_\gamma$, such as CIs, PrIs, or hypothesis tests for single-valued parameters or their multivariate generalizations for vector-valued parameters (e.g., confidence or prediction regions). One strategy is to use the (multivariate) *delta method*, which essentially entails using derivatives to transform the covariance matrix for $\hat\mu\_\theta$ alone or both $\hat\mu\_\theta$ and $\hat\Sigma\_\theta$ to a covariance matrix for $\hat\mu\_\gamma$ or $\hat\Sigma\_\gamma$; the latter covariance matrix can be used to construct CIs or PrIs or test hypotheses. Another strategy, at least for CIs or confidence regions, is to use a bootstrap technique to essentially construct an empirical sampling distribution of $\hat\mu\_\gamma$ or $\hat\Sigma\_\gamma$; numerous bootstrap options are available, depending largely on how the sample of bootstrap replicates -- $\hat\mu\_\gamma$ or $\hat\Sigma\_\gamma$ for each resample from the data -- is constructed (e.g., parametric vs. nonparametric) and how it's used to construct confidence intervals or regions (e.g., standard deviation\error vs. percentile, bias correction or not).

Because this is already a fairly long overview, I'll close with a few remarks. First, despite its advantages over some other MASEM methods, my proposed method also has drawbacks and limitations; I won't elaborate here on these pros and cons, except to caution that my proposed method might perform unacceptably in certain circumstances. Second, my proposed method would benefit from considerably more work, such as refining aspects of each step and studying its performance in realistic situations defined by characteristics of MASEM studies (e.g., number of primary studies, distribution of sample sizes, distribution of correlation-matrix parameters, pattern and mechanism of missing data, choice of function g). To date there's been little evaluation of multivariate RE meta-analysis analytically or by simulation, for either correlation matrices (cf. Hafdahl, 2004, 2008b) or other multivariate effect sizes (cf. Riley, 2009; Riley, Abrams, Sutton, Lambert, & Thompson, 2007), and Hafdahl's (2009c) Monte Carlo studies of meta-analysis for functions of multivariate effect sizes did not include correlation matrices. Third, Bayesian approaches to meta-analysis, such as Prevost, Mason, Griffin, Kinmonth, Sutton, and Spiegelhalter's (2007) proposed method for correlation matrices, might be especially well-suited to MASEM due to their natural -- though computationally challenging -- strategies for constructing posterior distributions for functions of a study's parameters.

## References

Becker, B. J., & Schram, C. M. (1994). Examining explanatory models through research synthesis. In H. Cooper & L. V. Hedges (Eds.), *The handbook of research synthesis* (pp. 357-381). New York: Russell Sage Foundation.

Hafdahl, A. R. (2004, June). *Refinements for random-effects meta-analysis of correlation matrices.* Paper presented at the meeting of the Psychometric Society, Monterey, CA.

Hafdahl, A. R. (2008a). Combining heterogeneous correlation matrices: Simulation analysis of fixed-effects methods. *Journal of Educational and Behavioral Statistics, 33,* 507-533. doi:10.3102/1076998607309472

Hafdahl, A. R. (2008b, July). *Meta-analysis for functions of heterogeneous correlation matrices.* Paper presented at the meeting of the Psychometric Society, Durham, NH.

Hafdahl, A. R. (2009a). Improved Fisher *z* estimators for univariate random-effects meta-analysis of correlations. *British Journal of Mathematical and Statistical Psychology, 62,* 233-261. doi:10.1348/000711008X281633

Hafdahl, A. R. (2009b, May). Meta-analysis for functions of dependent correlations. In A. R. Hafdahl (Chair), *Advances in meta-analysis for multivariable linear models.* Invited symposium presented at the meeting of the Association for Psychological Science, San Francisco, CA.

Hafdahl, A. R. (2009c). *Meta-analysis for functions of heterogeneous multivariate effect sizes.* Unpublished master's thesis, Washington University in St. Louis, St. Louis, Missouri. http://openscholarship.wustl.edu/etd/439/

Hafdahl, A. R. (2011). *Translating meta-analytic results: Techniques to express random-effects estimates in other metrics.* Manuscript in preparation, Washington University in St. Louis.

Hafdahl, A. R., & Williams, M. A. (2009). Meta-analysis of correlations revisited: Attempted replication and extension of Field’s (2001) simulation studies. *Psychological Methods, 14,* 24-42. doi:10.1037/a0014697

Hafdahl, A. R., & Wu, W. (2012, February). *An EM algorithm for multivariate random-effects meta-analysis with incomplete effect estimates.* Manuscript in preparation, ARCH Statistical Consulting, LLC.

Kalaian, H. A., & Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis. *Psychological Methods, 1,* 227-235. doi:10.1037/1082-989X.1.3.227

Prevost, A. T., Mason, D., Griffin, S., Kinmonth, A.-L., Sutton, S., & Spiegelhalter, D. (2007). Allowing for correlations between correlations in random-effects meta-analysis of correlation matrices. *Psychological Methods, 12,* 434-450. doi:10.1037/1082-989X.12.4.434

Riley, R. D. (2009). Multivariate meta-analysis: The effect of ignoring within-study correlation. *Journal of the Royal Statistical Society—Series A, 172,* 789–811. doi:10.1111/j.1467-985X.2008.00593.x

Riley, R. D., Abrams, K. R., Sutton, A. J., Lambert, P. C., & Thompson, J. R. (2007). Bivariate random-effects meta-analysis and the estimation of between-study correlation. *BMC Medical Research Methodology, 7,* 3. doi:10.1186/1471-2288-7-3

White, I. R. (2011). Multivariate random-effects meta-regression: Updates to mvmeta. *Stata Journal, 11,* 255-270.
> 9 votes

# Answer
The following reviews some of the articles that I found discussing and implementing SEM meta-analysis to examine mediation.

## Cheung and Chan (2005)

The authors distinguish three approaches of meta-analytic structural equation modelling (MASEM).

* Univariate two-stage MASEM: This includes a collection of two step approaches (note that Cheung and Chan call it simply *univariate*, but other authors describe it as a two-stage approach). Stage 1 involves first generating a pooled correlation matrix and sample size. Stage 2 involves analysing this correlation matrix and sample size in standard SEM software.
* GLS (Generalised Least Squares for MASEM)
* Cheung and Chan's two-stage MASEM: This approaches use SEM to both synthesise correlation matrices and fit proposed models.

The authors state that if studies have heterogeneous correlation matrices "they cannot be aggregated legitimately" (p.46).

## Shadish (1996)

Shadish (1996) provides an early review of combining meta-analysis and SEM approaches. He discusses a few issues and encourages further statistical work on the topic. He also summarises four early studies that had applied meta-analysis to elucidate causal relationships.

1. Harris and Rosenthal (1985): This early study reported used meta-analysis to obtain eight averaged correlations across a set of studies. Four of the correlations were between an independent variable and theorised mediators, and four were between the mediators and the dependent variable. The presence of correlations between all variables was used as support for mediation.
2. Premack and Hunter (1988): This study averaged correlations for a set of studies and performed path analysis on the resulting correlation matrix.
3. Shadish & Sweeney (1991): This study examined between study correlations.
4. Becker's (1992): This study used generalised least squares to model between and within study variation in correlations.

## Other Studies

**Stajkovic et al (2009)** report a SEM meta-analysis which seemed to use the univariate approach. Pooled correlations were calculated and then entered into SEM software for analysis. The authors also examined invariance of parameters in the SEM mediation model across specified values of specified moderators.

**Colquitt et al (2007)** performed SEM meta-analysis looking at three predictors and four consequences of trust (i.e., the "mediator"). The authors appear to have used the univariate approach. The authors report examining whether correlations varied by the moderators:

> Because the two moderators we examined, type of measure and trust referent, did not significantly impact the trust correlations, we used the overall correlation.

However, they also show that correlations vary between studies suggesting that even if moderators don't explain variation in correlations, variation in correlations presumably existed for other reasons.

**Fried et al (2008)** used the univariate approach to examine mediation effects between work stress, psychological mediators, and job performance. The authors used a random effects model to calculate pooled correlation coefficients and reported using the harmonic mean of cell sample sizes for the sample size in the SEM.

**Dunst and Trivette (2009)** used the univariate approach on studies related to family-centered care. The authors report pooling correlations based on a weighted average "giving more weight to studies with larger sample sizes and by taking into consideration other statistical artifacts." It is unclear what sample size was used in the subsequent SEM.

They addressed the issue of homogeneity of correlation matrices by writing:

> 3.1. Homogeneity of the Correlation Matrices. This is a test of whether the correlation matrices in the 15 different studies can be assumed to be derived from the same population. CFI was 0.91 and RMSEA was 0.09. The results indicate that the different correlation matrices were reasonably similar to produce a pooled correlation matrix.

However, an RMSEA of .09 actually sounds fairly large. I've seen recommendations for an RMSEA of less than .05 for a very good fit. I'm not questioning that the correlation matrices weren't similar. But in general we would expect some true score variance and to my mind such results seem consistent with some variation. The lack of straightforward analysis options for handling heterogeneity discourages exploration of how results would be different if the data was assumed to be heterogeneous.

**Bamberg & Moser (2007)** used the univariate approach to test a model of the theory of planned behaviour applied to environmental behaviour. Because correlations varied significantly between studies according to the Q-test, the authors used the random-effects estimate of the pooled true correlation (but they also report pooled estimates based on the fixed-effects model) and report the mean and 95% confidence intervals for the true correlations. The harmonic mean of cell sample sizes was purportedly used in calculating cell sample sizes.

**Bauer et al (2007)** used the univariate approach to test a model of newcomer adjustment. It was not clear to me whether the pooled correlations were based on fixed or random effects assumptions. Rationale for the sample size is not stated explicitly but looks broadly consistent with something like a harmonic mean of the cell sample sizes.

**Joseph et al (2007)** used the univariate approach to test a model of job turnover. Pooled correlations were formed by weighting sample correlations by sample size and correcting for measurement error. The harmonic mean of of cell sample sizes was used.

**Chang et al (2009)** used the univariate approach looking at organisational politics.

**Haeussler-Keyton (2012)** is a doctoral thesis using the univariate approach that performed meta-analytic path analysis on studies related to breast feeding success. The size used was calculated as such: the average sample size N in studies for each cell was calculated, and the smallest of these was used for analyses. Note that this results in a much smaller sample size than simply taking the average total cell sample size. The author fit models using pooled correlations based on both fixed and random effects calculations.

**Zhang (2011)** wrote a doctoral disertation reviewing the GLS approach and the multivariate two-stage approach of Cheung and Chan (2005).

### Viswesvaran and Ones (1995)

Viswesvaran and Ones (1995) provide a tutorial-style guide for univariate MASEM . They cite several studies that have used the approach including Hunter (1983), Hom, Caranikas-Walker, Prussia, and Griffeth (1992), Peters, Hartke, and Pohlmann (1985), Brown and Peterson (1993), Ones (1993) and Viswesvaran (1993). The general approach is to compute true-score correlations using various standard meta-analytic techniques and use that correlation matrix as input to SEM.

Viswesvaran and Ones (1995) discuss the issue of dealing with **missing cells in the correlation matrix**. They mention several options for dealing with this:

> (a) design a primary study to collect data with sufficiently large sample size, such that the effects of sampling error are reduced, to obtain stable estimates of the correlations not reported in the literature; (b) use the average (across all correlations) correlation in the empty cells; (c) look for patterns of correlations and impute values in the missing cells of the matrix; and (d) modify the test of the theory to include only the constructs for which a full matrix of estimated true score correlations is available in the liter- ature. A final option, as suggested by an anonymous reviewer, is to use Subject Matter Experts to estimate the missing correlation (e.g., Schmidt et al., 1983).

Viswesvaran and Ones (1995) also discuss the issue of **varying sample size per cell** in the correlation matrix. They mention several options: (a) use harmonic mean of the sample sizes across cells; (b) only include studies that include all variables; (c) assume sample is population and ignore standard error and confidence intervals.

Viswesvaran and Ones (1995) also acknowledge the issue that **true correlations may systematically vary between studies**. They suggest several approaches: (a) include moderators until true score variance is reduced to zero and include these moderators in the path analysis; (b) compute the three SEMs, one using lower 90% confidence intervals of meta-analysed correlations, another using upper 90% confidence intervals of meta-analysed correlations, and a third using the mean.

The above issue of variation in true correlations is one of the biggest issues that I'm concerned with. Most meta-analyses that I have read show variation in true correlations (hence various recommendations to prefer random-effects meta analysis). Furthermore, it is unlikely that available moderators would account for all true variation. In many contexts, I doubt that available moderators would even account for most of the true variation. Thus, while I can see that including meaningful moderators would be useful, I predict that in most applications, this would not solve the problem of variation in true correlations.

The second option of computing lower, mean, and upper bound correlations also does not seem to me to provide a solution to the problem. First of all, even if correlations vary between study, it is unlikely that such variation would lead to uniform increases and decreases in all correlations. For examples, in some studies the true correlation may be higher than the mean for one pair of variables and lower for another. That said, the idea of sampling the range of distributions does sound like it could have promise.

### Cheung and Chan (2009)

Cheung and Chan (2009) provide a technical overview with appropriate formulas, a simulation, and an example on performing the two-stage approach to SEM meta-analysis. They acknowledge that heterogeneous correlation matrices are problematic. They suggest the following options: (a) clustering correlation matrices; (b) release some cross-group constraints (e.g., based on moderators) in the SEM.

## Becker (2009)

Becker (2009) sets out how to perform model testing based on the GLS approach.

# Personal thoughts

## Pooling and random effects

In general, the random-effects model seems more reasonable. Unless the meta-analysis is composed of exact replications, studies typically vary in a wide range of ways. And this manifests in varying correlations. Using a random-effects model involves weighting sample correlations differently by incorporating between study variance. The consequence is that studies with smaller sample sizes are weighted more than they would otherwise be using a fixed-effects model.

Variation in correlations may not necessarily be normal. There may be outlier correlations in studies. Observed moderators may explain some of the between study variance in correlations. Similarly artefacts such as reliability, range restrictions, etc. may explain additional variance in effect sizes.

If there is no true-score variance then the two-step approach seems reasonable. Also, if after including moderators, true-score variance is accounted for, the two-step approach seems reasonable. Likewise, the idea of clustering correlation matrices to remove true-score variance might have promise.

Using a random-effects model to weight correlations is a reasonable way to get an estimate of the mean true correlation. However, using such pooled estimates in SEM raises several issues. First, such a procedure does not capture the true variation in effect sizes. Parameter estimates and SEM fit will vary across studies in systematic ways; only analysing the mean true correlations ignores this systematic between study variation. Second, using sample size estimates based on the average, minimum, or harmonic mean, of cell sample sizes seem to all assume a fixed-effects model of precision in estimating the true correlation coefficients.

## Other Approaches

Another strategy would be to perform SEM on each sample and treat the parameters and fit statistics as values that vary between samples. The distribution (e.g.,mean and SD) of these SEM parameters and fit statistics could be summarised. This would be similar to how correlations and other effect sizes are typically modelled as random effects. So, for example the variation in the indirect effect across samples could be examined. The challenge would be to separate what is true score variation and what is due to random sampling.

This seems like a good idea, although often meaningful variance in effects will remain even after controlling for moderator and often the number of studies for a given moderator value can be minimal.

### References

* Bamberg, S. & Moser, G. (2007). Twenty years after Hines, Hungerford, and Tomera: A new meta-analysis of psycho-social determinants of pro-environmental behaviour. Journal of environmental psychology, 27, 14-25.
* Bauer, T.N., Bodner, T., Erdogan, B., Truxillo, D.M. & Tucker, J.S. (2007). Newcomer adjustment during organizational socialization: A meta-analytic review of antecedents, outcomes, and methods.. Journal of applied psychology, 92, 707. PDF
* Becker, B. J. (1992). Models of science achievement: Forces affecting male and female performance in school science. In T. D. Cook, H. M. Cooper, D. S. Cordray, H. Hartmann, L. V. Hedges, R. J. Light, T. A. Louis, & F. Mosteller (Eds.), Meta-analysis for explanation: A casebook (pp. 209-281). New York: Russell Sage Foundation.
* Becker, B. J. (2009) Model-based meta-analysis, chapter 20, in The handbook of research synthesis and meta-analysis, editors: Cooper, H.M. and Hedges, L.V. and Valentine, J.C., Russell Sage Foundation Publications.
* Brown SP, Peterson RA. (1993). Antecedents and consequences of salesperson job satisfaction: Meta-analysis and assessment of causal effects. Journal of Marketing Research, 30,63-77.
* Cheung, M.W.L. & Chan, W. (2005). Meta-analytic structural equation modeling: a two-stage approach. Psychological Methods, 10, 40. PDF
* Cheung, M.W.L. & Chan, W. (2009). A two-stage approach to synthesizing covariance matrices in meta-analytic structural equation modeling. Structural Equation Modeling, 16, 28-53.
* Colquitt, J.A., Scott, B.A. & LePine, J.A. (2007). Trust, trustworthiness, and trust propensity: A meta-analytic test of their unique relationships with risk taking and job performance.. Journal of applied psychology, 92, 909.
* Dunst, C.J. & Trivette, C.M. (2009). Meta-analytic structural equation modeling of the influences of family-centered care on parent and child psychological health. International journal of pediatrics, 2009. PDF
* Fried, Y., Shirom, A., Gilboa, S. & Cooper, C.L. (2008). The mediating effects of job satisfaction and propensity to leave on role stress-job performance relationships: Combining meta-analysis and structural equation modeling.. International Journal of Stress Management, 15, 305.
* Joseph, D., Ng, K.Y., Koh, C. & Ang, S. (2007). Turnover of information technology professionals: a narrative review, meta-analytic structural equation modeling, and model development. MIS Quarterly, 31, 547-577.
* Harris, M. J., & Rosenthal, R. (1985). Mediation of interpersonal expectancy effects: 31 Meta-analysis. Psychological Bulletin, 97, 363-386. Haeussler-Keyton, K. (2012). Psychological and Social Factors Associated with Breastfeeding Success: A Meta-Analytic Path Analysis. Doctoral Thesis at Texas Tech University. PDF
* Hunter JE. (1983). A causal analysis of cognitive ability, job knowledge, job performance, and supervisor ratings. In Landy FJ, Zedeck S, Cleveland J (Eds.), Performance measurement and theory (pp. 257-266). New Jersey: Eribaum.
* Ones DS. (1993). The construct validity of integrity tests. Unpublished doctoral dissertation. University of Iowa, Iowa City.
* Peters LH, Hartke DD, Pohlmann JT. (1985). Fiedler's contingency theory of leadership: An application of the mieta-analysis procedures of Schmidt and Hunter. Psychological Bultetin, 97,274-285.
* Premack, S. L., & Hunter, J. E. (1988). Individual unionization decisions. Psychological Bulletin, 103, 223-234.
* Shadish, W.R. (1996). Meta-analysis and the exploration of causal mediating processes: A primer of examples, methods, and issues.. Psychological Methods, 1, 47.
* Shadish, W. R., & Sweeney, R. (1991). Mediators and moderators in meta-analysis: There's a reason we don't let dodo birds tell us which psychotherapies should have prizes. Journal of Consulting and Clini- cal Psychology, 59, 883-893.
* Stajkovic, A.D., Lee, D. & Nyberg, A.J. (2009). Collective efficacy, group potency, and group performance: Meta-analyses of their relationships, and test of a mediation model.. Journal of Applied Psychology, 94, 814.
* Viswesvaran, C. & Ones, D.S. (1995). THEORY TESTING: COMBINING PSYCHOMETRIC META-ANALYSIS AND STRUCTURAL EQUATIONS MODELING. Personnel Psychology, 48, 865-885.
* Viswesvaran C. (1993). Modeling job performance: Is there a general factor? Unpublished doctoral dissertation, LIniversity of Iowa, Iowa City.
* Zhang, Y. (2011). Meta-analytic Structural Equation Modeling (MASEM): Comparison of the multivariate methods. Doctoral Thesis at Florida State University. PDF
> 7 votes

---
Tags: reference-request, methodology, statistics, meta-analysis
---