Article by King, William R., and He, Jun (2005) in Communications of the AIS, 16.
There are four methods in conducting literature review, i.e. Narrative Review, Descriptive Review, Vote Counting and MetaAnalysis. Look at the diagram at the end of this entry to view these methods in the qualitativequantitative continuum.
Metaanalysis is considered as the most rigor method; it is the one which closest to the positivist tradition. Here are the advantages of metaanalysis:

The major difference between narrative reviews and quantitative metaanalyses may well be that narrative reviews primarily focus on the conclusions reached in various studies, whereas metaanalyses focus on data, as reflected by the operationalization of variables, the magnitude of effect sizes, and the sample sizes.

Compared to narrative reviewing, metaanalysis procedure is more standardized, less judgmental and less subjective (narrative reviewing is more towards interpretive in nature, thus conclusions drawn are very subjective and dependent to the reviewer’s knowledge and expertise).

Metaanalysis draw conclusion from the data characteristics (i.e. the operationalization of variables, the magnitudes of effect sizes and the sample size) of several studies; whereas traditional methods are more focusing on the end result (conclusions) only. Viewing from this point, metaanalysis result is generated from the richer data; therefore it offers more meaningful and robust findings.

Metaanalysis enables researchers to sample studies that show insignificant effects (note that in traditional methods, researchers are bound to select empirical studies that report significant effects only). With the powerful technique of metaanalysis method, studies with insignificant effects will be analyzed along with others that may at the end show significant effects. The end result is, of course, not necessarily significant, but one thing for sure, it is undoubtedly more accurate and more credible due to the sampling (of studies) that is not bias (to studies with significant effects only).

On the same basis as point #4 above, metaanalysis enables researchers to estimate the cumulative impact of insignificant results that may in turn at the end be significant.

Metaanalysis enables and promotes researchers to search for moderator variables in the less subjective way.
Despite the advantages listed above, it has its own limitations:

Sampling Bias toward Empirical Studies – Metaanalysis is applicable to quantitative studies (such as survey, laboratory experiment, field study, and field experiment) that report the magnitude of the effect size only. Other type of studies such as mathematical modeling, frameworks and conceptual models, case studies, interview, speculation/commentary, secondary data, and qualitative research, are unlikely to be sampled in a metaanalysis study. Even if mixedmethod studies are being sampled, metaanalysis will be based their findings on the quantitative part only.

Garbage In, Garbage Out – Metaanalysis does not generally differentiate studies by their quality (quality of research designs and approaches, sampling units, methods of measuring variables, data analysis methods, and presentations of research findings are not in consideration when sampling is carried out). This may lead metaanalysis to unfounded conclusions. Although some techniques have been introduced to correct this error, it is evident that application of such techniques had introduced other biases (related to the selection and weighting of quality criteria).

Publication Bias – It is normal in the current practice that studies with significant effects are more likely to be published while insignificant studies are tend to be ‘reserved’. This phenomenon may leads to the socalled ‘unrepresentative sampling’, thus conclusion drawn may bias toward certain direction of significance.

Apples and Oranges – All reviewing methods, not restricted to metaanalysis only, are based on the collection of past studies that have common characteristics. However, to find studies with ‘exact replications’ and ‘precisely the same’ are almost impossible. So what researchers can do is to select studies that ‘almost’ the same, and from there, they draw conclusions – this is analogous to taking apples and oranges and averaging such measures as their sizes, weights, tastes, etc.

Small Sample Size – a metaanalysis should include at least 15 studies, otherwise the type I error (accepting a false null hypothesis) could be severely inflated. As point #4 above, it is very difficult to find 15 empirical studies that are the same!!
The statistics that normally used and reported in IS studies are: (i) effect size; (ii) sample size; (iii) possible moderators; (iv) construct reliabilities, and; (v) other measurement error indexes.
In the literature, effects sizes are of various forms. Look at the diagram at the end of this entry to view them.
There are two models of metaanalysis method available. Look at the diagram at the end of this entry to view them.
There are three primary metaanalysis methods available, i.e. (i) HedgesOlkin’s method; (ii) RosenthalRubin’s method, and; (iii) HunterSchmidt’s method. There is another method introduced in the IS domain, named Glass method. This paper does not provide complete procedure on each method, but the authors suggested readers to refer to Lipsey and Wilson (2001) for the thorough description of the whole process.[i]
[i] Lipsey, M.W., and Wilson, D.B. (2001) Practical Metaanalysis, Thousand Oaks, CA: Sage Publications.