Skip navigation

Category Archives: Mediation/Mediator

Article by Stone-Romero, Eugene F., and Rosopa, Patrick J. (2008) in Organizational Research Methods, 11:2.

 

Let say we want to conduct a study with the simplest mediation model, X à M à Y, in order to gain the maximum internal validity inferences, we have to carry out two randomized experiments; one to prove the causal model of X à M and the other one for M à Y. If we choose to conduct quasi-experiments instead of randomized experiments, the internal validity inferences will be reduced; whereas if we choose to conduct non-experimental approach, the causal inferences may not be valid at all (even though we are using ‘causal modeling’ techniques in analyzing the data such as Hierarchical Multiple Regression, Path Analysis and Structural Equation Modeling). This paper outlines the effects of different research design on internal validity inferences:

 

Research Design

Control on Confounding Variables

Internal Validity Inferences

2 randomized experiments

Design

Very Strong

2 quasi-experiments

Statistical

Moderately Strong

1 randomized experiment

Design

Weak

1 quasi experiment

Statistical

Weak

2 non-experiment

Statistical

Very Weak

1 non-experiment

Statistical

Very Weak

 

For more complex mediation model, we need to conduct multiple randomized experiments accordingly.

 

The authors argued that, non-experimental study is only appropriate to prove ‘the consistency between an assumed causal model and the results of the study’, but NOT ‘the consistency between the reality and the results of the study’. That is basically why if non-experimental design approach is selected, we are not legitimately allowed to make any causal inferences. Instead, what we legitimately can claim is the inferences at covariance (e.g. correlation) level only, for example, we can say “…this study has recognized patterns of covariances among measured variables that are consistent with the assumed causal model (specified in the research model)”.

 

In addition to that (inferences at covariance level), the author argued that we also need to acknowledge that: (a) the same pattern of covariances may also consistent with other causal models, and; (b) our findings do not provide a valid basis for making causal inferences specified in the assumed causal model. For example, we can say (this example is taken directly from this paper) “Hypothesis 1 argued that there would be a positive correlation between X and Y. The test of this hypothesis showed that there was. This finding is consistent with the assumed causal model shown in Figure 1. However, it may also be consistent with a number of other causal models.”

 

One of the recommendations given by the authors made me smile, I nodded my head few times and I said “yea, you’re right… indeed you’re right!!” Here is the recommendation:

“…we recommend that individuals who teach undergraduate and/or graduate courses in such areas as statistics, research design, research methods, and causal modeling, instruct their students on the inferences that can and can not be made on the basis of data from nonexperimental research. Moreover, they need to disabuse students of the baseless arguments that appear in various publications about the inferences that are appropriate on the basis of ‘causal modeling’ procedures…”

Advertisements

Article by Wood, Robert E.; Goodman, Jodi S.; Beckmann, Nadin; Cook, Alison (2008) in Organizational Research Methods, 11:2.

 

Do you know that there are 14 different ways to analyze for mediation, intervening variables and indirect effects? To be honest to myself, my answer is “I don’t”!! Before I read this paper, I knew only two ways to conduct such testing, one by using hierarchical regression and another one by using SEM (structural equation modeling). This paper outlines 14 different ways and divides them into three main frameworks, i.e. (i) the causal step approach; (ii) differences in coefficients, and; (iii) products of coefficients. The two methods that I previously knew, they are under the causal step framework. So what I knew was only two methods under one framework, and I didn’t know the other 12 methods. I didn’t even know the existence of the other two frameworks!!!

 

For me, it certainly shows how infinitely-massive is the planet named “knowledge” and how tiny I am in one of the planet’s island, named “research methodology”!!! How about you?

 

I summarize the three frameworks in a diagram at the end of this entry.

 

In this paper, the authors studied all articles that reported mediation testing in the 5 established journals over the past 25 years. Some of the findings (that for me are really interesting) are listed below:

  1. The trend indicated that the number of research with mediation analysis was increasing (within the 25 years study);
  2. Two approaches that mostly cited as the guidance for mediation analysis were Baron and Kenny’s approach (Baron and Kenny, 1986) and James and Brett’s approach (James and Brett, 1984);
  3. Two frameworks that mostly used to conduct the mediation analysis were Causal Steps approach and Product of Coefficients approach. For Product of Coefficients, the mostly cited approach was Sobel’s approach (Sobel, 1982);
  4. Regression has been the most common statistical test used for testing mediation, but the trend depicted the used of SEM (structural equation modeling) has grown significantly over time;
  5. Majority of the study demonstrated nonsignificant results for the inferences of mediation;
  6. Many of the studies were not adhere to the recommended testing procedure; therefore the findings are exposed to a certain degree of potential threats to the validity. They reported that they used certain framework, but failed to fully adhere to any of the recommended approach under the said framework. For instance, they claimed that they used Causal Step framework, but their works didn’t appear to adhere to any approach in the framework they used – either Baron & Kenny (1986) or Kenny, Kashy & Bolger (1998) or James & Brett (1984) or etc. etc.;
  7. Many studies put the basis of their claims of full or partial mediation on the change in the magnitude of coefficient without testing the significance of that change – this would definitely cause potential threats to validity;
  8. Another potential threat to validity was that, the researchers used the simple mediation model (one independent, one mediator and one dependent variables; X ® M ® Y) to test complex models (such as model with more than one mediator);
  9. Many studies made causal claims although conditions for causality were not met (researchers were supposed to use noncausal language and discuss effects in terms of covariation).
  10. The authors suggested a table format for reporting the Causal Steps mediation result which uses regression and the Sobel (1982) test. The table format is shown in a diagram below.

 

 

Mediation Analysis Frameworks

Mediation Analysis Frameworks

Reporting Mediation Result

Reporting Mediation Result