We are trying to do a randomized controlled trial on patients. For a certain diseases patients, we want to know whether or not intervention improves their quality of life or not. We're dividing the patients into two groups: one receive the traditional method, other receives the intervention where their vitals and labs are checked with responses to some questionnaires about their quality of life is collected in 2 weeks, 4 weeks, 3 months, and 6 months. Whereas, for the traditional method, we're only going to follow up with patient in 1 month and 6 months. The lab and vitals for the traditional methods are only taken at 6 months.

For this, we'd like to know the power and sample size of the study. I would like to either know how to calculate it, or would like to know what I must know before I ask statistician's help. I read Cohen's article A Power Primer which help me make some sense, but I'm still somewhat confused about effect size


First, if you have the funds, I strongly suggest collecting pre-intervention data for the traditional method group. This will allow a stronger (more valid) study. Your current design suffers from a threat to ecological validity such that you lose ability to make causal claims. Also, again if you can afford it, add a control group.

In making the calculation you want, you need one of the numbers. Normally, you have a minimum detectable effect size that you set before hand, along with your acceptable $\alpha$ type-1 error rate and $1-\beta$ desired power. Having these values gives you the required $n$ sample size Such an analysis a a priori. On the other hand, if you have access to a restricted sample size $n$, acceptable $\alpha$ type-1 error rate, and $1-\beta$ desired power, you would learn your minimum detectable effect size, called sensitivity analysis.

I use G*Power, a free software, to do such calculations.

  • $\begingroup$ thank you. Effect size is what's causing me confusion. We don't have fund for the pre-intervention study, so was hoping to find any statistical method to find an estimate of the sample size. I'll download G*power $\endgroup$ – user1828605 Dec 11 '17 at 21:05
  • $\begingroup$ That was difficult for me, too. Think of it like a microscope. To see something smaller, you need stronger magnification. Likewise, to find a small effect size, you need either: a bigger sample, a bigger power, or a higher acceptable α. $\endgroup$ – Jay Schyler Raadt Dec 11 '17 at 21:38

Elaborating a bit on Jay's excellent answer and assuming that your experiment's outcome of interest is a proportion:

In order to determine a sample Size for a 2-Proportion $Z\text{-test}$ , you need to specify:

  1. the desired $α$ level
    • i.e., your willingness to commit a Type I error
  2. the desired $β$ level
    • i.e., your willingness to commit a Type II error
    • (this equivalent to desired power)
  3. a meaningful difference from the value of the parameter that is specified in the Null
    • i.e., you need to specify an "Alternative" hypothesis value

This is because rearranging the 2-Proportion $Z\text{-test}$ formula (for equal Treatment and Control group sizes) gives us the minimum sample size needed $\, \tilde{n}$:

$$\tilde{n} = (p_1(1-p_1)+p_2(1-p_2))\left(\dfrac{Z_{power}+Z_{1-\alpha}}{p_1-p_2}\right)^2$$


  • $p_1$ is your current proportion that has a high "quality of life"
  • $p_2$ is your current proportion that has a high "quality of life" + minimum effect size
    • this is the prespecified minimum proportional that you choose to be able to detect
  • $ \tilde{n}$ is the minimum total sample size needed
    • i.e., it is the sum of both the Treatment and the Control groups

w.r.t. not doing this calculation manually, one superb open-source solution is to use R-language's pwr library.

The specific function in this case is the pwr.2p.test

  • For any of pwr's functions, you enter three of the four quantities (effect size, sample size, significance level, power) and the fourth is calculated.
  • 1
    $\begingroup$ Excellent answer. Thank you. This is very helpful. $\endgroup$ – user1828605 Dec 13 '17 at 19:54

Your Answer

