Continuous comparison unmatched group data with statistics

By Yashika Kapoor & Priya Chetty on November 4, 2017

The present article takes up the datasheets for the unmatched post and pre or post design and illustrates the results with statistics. The present discussion will focus on the interpretation of the results.

Basic summary statistics

Now, beginning with the interpretation of basic summary statistics, the images below (1 & 2) show the same, for the post and pre or post designs. The studies in the present review exhibit similarity in terms of design and report outcomes over the same scale. This fact is also shown by the nearly same summary effect size of random and fixed models. Thus, the present meta-analysis can use either of the two models.

Unmatched post

Taking into account the summary effect size from the fixed model, the overall standard difference in means is -0.49 (95% CI -0.63 to -0.34) with p-value = 0.00 (<0.05).

Basic summary statistics for Unmatched Post meta-analysis
Image 1: Basic summary statistics for Unmatched Post meta-analysis

Unmatched pre or post

The summary effect size from the fixed model, the overall standard difference in means is -0.44 (95% CI -0.59 to -0.30) with p-value = 0.00 (<0.05).

Basic summary statistics for Unmatched Pre/Post meta-analysis
Image 2: Basic summary statistics for Unmatched pre/Post meta-analysis

In both the results, the negative sign depicts the summary effect size directed towards drug treatment. Thus, drug treatment is more efficacious than placebo. Also, the studies report medium effect, as the magnitude of both summary effect sizes is around 0.5 (Cohen, 1988).

Assesment of heterogeneity

The images below (3 & 4) show the model statistics to assess the heterogeneity, for the post and pre or post designs.

Unmatched post

The I2 statistic for heterogeneity is 31.99 (31.99%), p = 0.173 (>0.05), resulting in rejection of the alternative hypothesis. Thereby, indicating the lack of ‘significant’ heterogeneity within the studies taken for present review.

Model statistics for unmatched post data
Image 3: Model statistics for unmatched post data

Unmatched pre or post

The I2 statistic for heterogeneity is 38.59 (38.59%), p = 0.122 (>0.05), resulting in rejection of the alternative hypothesis. Thereby, indicating the lack of ‘significant’ heterogeneity within the studies taken for present review.

Model statistics for unmatched pre/post data
Image 4: Model statistics for unmatched pre/post data

Forest plots

The images below show the forest plots for both unmatched post and pre/post designs. These forest plots have the line of no effect marked as 0, as the outcome variables are continuous. The means summary effect size is seen on the side of drug treatment, towards the negative side of the graph, as per the set direction of effect size.

Forest plot for unmatched post data
Image 3: Forest plot for unmatched post data
Forest plot for unmatched pre/post data
Image 4: Forest plot for unmatched pre/post data

Publication bias

The publication bias assessment revolves around the visual assessment of funnel plots and the interpretation of formal test statistics. The formal test statistics for rank correlation and intercept test are also shown.

Funnel plots

The images below (5 & 6) show the funnel plots for the unmatched post and pre or post designs. The assessment of funnel plots, indicates the relatively symmetrical distribution of effect estimates, around the central line. This suggests the probable inclusion of relevant trials.

 Funnel plot for unmatched post data
Image 5: Funnel plot for unmatched post data
Funnel plot for unmatched pre/post design
Image 6: Funnel plot for unmatched pre/post design

Rank correlation test

The formal test statistics for rank correlation test, for the unmatched post design are as shown below. The result for rank correlation test show values of p>0.05 thereby indicating the acceptance of null hypothesis, and the presence of bias.

Rank correlation test statistics for unmatched post data
Image 7: Rank correlation test statistics for unmatched post data

The formal test statistics for rank correlation test for the unmatched pre or post design are as shown below. The result for rank correlation test show values of p<0.05 for 1 tailed test and p>0.05 for 2 tailed test. Here, we will take into account the 2 tailed p-value as it allows for testing of either presence or absence of bias (2 directions). It serves as a better parameter for assessment of bias, which is a crucial aspect in the assessment of study efficacy results. Thus, here also the p values lead to the acceptance of null hypothesis and indicate the presence of bias.

Rank correlation test statistics for unmatched pre/post data
Image 8: Rank correlation test statistics for unmatched pre/post data

Intercept test

The formal test statistics for intercept test for the unmatched post design are as shown below. The regression test results indicate significant deviation from zero, although less pronounced. Also, the p-value > 0.05 indicates the presence of significant asymmetry and thus bias.

Intercept test statistics for unmatched post data
Image 6: Intercept test statistics for unmatched post data

The formal test statistics for intercept test for the unmatched pre or post design are as shown below. The regression test results indicate significant deviation from zero, although less pronounced. Here also, p<0.05 for 1 tailed test and p>0.05 for 2 tailed test. We will take into account the 2 tailed p-value which leads to the acceptance of null hypothesis and indicates the presence of significant asymmetry and thus bias.

 Intercept test statistics for unmatched pre/post design
Image7: Intercept test statistics for unmatched pre/post data

Findings

The drug treatment is efficacious than the placebo treatment. The studies show no heterogeneity, although publication bias is present. The next article will now help to learn the procedures for performing analysis with one group pre or post data.

References

  • Cohen J. Statistical Power Analysis for the Behavioral Sciences. Second ed. Hillsdale, NJ: Erlbaum; 1988.

Discuss