A Practical Guide to Statistical Power and Sample Size Calculations in R
Metin Bulus
2023-03-12
If you find the package and related material useful please cite as:
Bulus, M. (2023). pwrss: Statistical Power and Sample Size Calculation Tools. R package version 0.3.0. https://CRAN.R-project.org/package=pwrss
Bulus, M., & Polat, C. (2023). pwrss R paketi ile istatistiksel güç analizi [Statistical power analysis with pwrss R package]. https://osf.io/ua5fc
Install and load pwrss R package:
install.packages("pwrss")
library(pwrss)Please send any bug reports, feedback, or questions to bulusmetin [at] gmail.com
1 Generic Functions
These generic functions calculate and return statistical power along with optional Type I and Type II error plots (as long as the test statistic and degrees of freedom is known). These functions are handy because z, t, \(\chi^{2}\), and F statistics and associated degress of freedom can be readily found in reports, published articles, or standard software output (SAS, SPSS, Jamovi, etc.). Statistical power can be calculated via putting the test statistics for ncp in the functions below; however, do not to overinterpret power rates if calculated post hoc.
1.1 t Test
power.t.test(ncp = 1.96, df = 99, alpha = 0.05,
alternative = "equivalent", plot = TRUE)
## power ncp.alt ncp.null.1 ncp.null.2 alpha df t.crit.1 t.crit.2
## 0.2371389 0 -1.96 1.96 0.05 99 -0.3155295 0.31552951.2 z Test
power.z.test(ncp = 1.96, alpha = 0.05,
alternative = "not equal", plot = TRUE)
## power ncp.alt ncp.null alpha z.crit.1 z.crit.2
## 0.5000586 1.96 0 0.05 -1.959964 1.9599641.3 \(\chi^2\) Test
power.chisq.test(ncp = 2, df = 98,
alpha = 0.05, plot = TRUE)
## power ncp.alt ncp.null alpha df chisq.crit
## 0.06766831 2 0 0.05 98 122.10771.4 F Test
power.f.test(ncp = 2, df1 = 2, df2 = 98,
alpha = 0.05, plot = TRUE)
## power ncp.alt ncp.null alpha df1 df2 f.crit
## 0.2198138 2 0 0.05 2 98 3.0892031.5 Multiple Parameters
Multiple parameters are allowed but plots should be turned off (plot = FALSE).
power.t.test(ncp = c(0.50, 1.00, 1.50, 2.00, 2.50), plot = FALSE,
df = 99, alpha = 0.05, alternative = "not equal")## power ncp.alt ncp.null alpha df t.crit.1 t.crit.2
## 0.07852973 0.5 0 0.05 99 -1.984217 1.984217
## 0.16769955 1.0 0 0.05 99 -1.984217 1.984217
## 0.31785490 1.5 0 0.05 99 -1.984217 1.984217
## 0.50826481 2.0 0 0.05 99 -1.984217 1.984217
## 0.69698027 2.5 0 0.05 99 -1.984217 1.984217power.z.test(alpha = c(0.001, 0.010, 0.025, 0.050), plot = FALSE,
ncp = 1.96, alternative = "greater")## power ncp.alt ncp.null alpha z.crit
## 0.1291892 1.96 0 0.001 3.090232
## 0.3570528 1.96 0 0.010 2.326348
## 0.5000144 1.96 0 0.025 1.959964
## 0.6236747 1.96 0 0.050 1.644854power.chisq.test(df = c(80, 90, 100, 120, 150, 200), plot = FALSE,
ncp = 2, alpha = 0.05)## power ncp.alt ncp.null alpha df chisq.crit
## 0.06989507 2 0 0.05 80 101.8795
## 0.06856779 2 0 0.05 90 113.1453
## 0.06746196 2 0 0.05 100 124.3421
## 0.06571411 2 0 0.05 120 146.5674
## 0.06382959 2 0 0.05 150 179.5806
## 0.06175379 2 0 0.05 200 233.99431.6 Type I and Type II Error Plots
plot() function (S3 method) is a wrapper around the generic functions above. Assign results of any pwrss function to an R object and pass it to plot() function.
# comparing two means
design1 <- pwrss.t.2means(mu1 = 0.20, margin = -0.05, paired = TRUE,
power = 0.80, alpha = 0.05,
alternative = "non-inferior")
plot(design1)
# ANCOVA design
design2 <- pwrss.f.ancova(eta2 = 0.10, n.levels = c(2,3),
power = .80, alpha = 0.05)
plot(design2)
# indirect effect in mediation analysis
design3 <- pwrss.z.med(a = 0.10, b = 0.20, cp = 0.10,
power = .80, alpha = 0.05)
plot(design3)
2 Difference between Two Means
2.1 Parametric Tests
2.1.1 Independent Samples t Test
More often than not, unstandardized means and standard deviations are reported in publications for descriptive purposes. Another reason is that they are more intuative and interpretable (e.g. depression scale). Assume that for the first and second groups expected means are 30 and 12, and expected standard deviations are 12 and 8, respectively.
Calculate Statistical Power
What is the statistical power given that the sample size for group 2 is 50 and groups have equal sample sizes (kappa = n1 / n2 = 1)?
pwrss.t.2means(mu1 = 30, mu2 = 28, sd1 = 12, sd2 = 8, kappa = 1,
n2 = 50, alpha = 0.05,
alternative = "not equal")## Difference between two means (independent samples t test)
## H0: mu1 = mu2
## HA: mu1 != mu2
## ------------------------------
## Statistical power = 0.163
## n1 = 50
## n2 = 50
## ------------------------------
## Alternative = "not equal"
## Degrees of freedom = 98
## Non-centrality parameter = 0.981
## Type I error rate = 0.05
## Type II error rate = 0.837Calculate Minimum Required Sample Size
What is the minimum required sample size given that groups have equal sample sizes (kappa = n1 / n2 = 1)?
pwrss.t.2means(mu1 = 30, mu2 = 28, sd1 = 12, sd2 = 8, kappa = 1,
power = .80, alpha = 0.05,
alternative = "not equal")## Difference between two means (independent samples t test)
## H0: mu1 = mu2
## HA: mu1 != mu2
## ------------------------------
## Statistical power = 0.8
## n1 = 410
## n2 = 410
## ------------------------------
## Alternative = "not equal"
## Degrees of freedom = 818
## Non-centrality parameter = 2.808
## Type I error rate = 0.05
## Type II error rate = 0.2It is sufficient to put pooled standard deviation for sd1 because sd2 = sd1 by default. In this case, for a pooled standard deviation of 10.198 the minimum required sample size can be calculated as
pwrss.t.2means(mu1 = 30, mu2 = 28, sd1 = 10.198, kappa = 1,
power = .80, alpha = 0.05,
alternative = "not equal")## Difference between two means (independent samples t test)
## H0: mu1 = mu2
## HA: mu1 != mu2
## ------------------------------
## Statistical power = 0.8
## n1 = 410
## n2 = 410
## ------------------------------
## Alternative = "not equal"
## Degrees of freedom = 818
## Non-centrality parameter = 2.808
## Type I error rate = 0.05
## Type II error rate = 0.2It is sufficient to put Cohen's d or Hedge's g (standardized difference between two groups) for mu1 because mu2 = 0, sd1 = 1, and sd2 = sd1 by default. For example, for an effect size as small as 0.196 (based on previous example) the minimum required sample size can be calculated as
pwrss.t.2means(mu1 = 0.196, kappa = 1,
power = .80, alpha = 0.05,
alternative = "not equal")## Difference between two means (independent samples t test)
## H0: mu1 = mu2
## HA: mu1 != mu2
## ------------------------------
## Statistical power = 0.8
## n1 = 410
## n2 = 410
## ------------------------------
## Alternative = "not equal"
## Degrees of freedom = 818
## Non-centrality parameter = 2.806
## Type I error rate = 0.05
## Type II error rate = 0.22.1.2 Paired Samples t Test
Assume for the first (e.g. pretest) and second (e.g. posttest) time points expected means are 30 and 12 (a reduction of 8 points), and expected standard deviations are 12 and 8, respectively. Also assume a correlation of 0.50 (the default in the function) between first and second measurements. What is the minimum required sample size?
pwrss.t.2means(mu1 = 30, mu2 = 28, sd1 = 12, sd2 = 8,
paired = TRUE, paired.r = 0.50,
power = .80, alpha = 0.05,
alternative = "not equal")## Difference between two means (paired samples t test)
## H0: mu1 = mu2
## HA: mu1 != mu2
## ------------------------------
## Statistical power = 0.8
## n = 222
## ------------------------------
## Alternative = "not equal"
## Degrees of freedom = 221
## Non-centrality parameter = 2.816
## Type I error rate = 0.05
## Type II error rate = 0.2It is sufficient to put standard deviation of the difference for sd1 because sd2 = sd1 by default. In this case, for a standard deviation of difference of 10.583 the minimum required sample size can be calculated as
pwrss.t.2means(mu1 = 30, mu2 = 28, sd1 = 10.583,
paired = TRUE, paired.r = 0.50,
power = .80, alpha = 0.05,
alternative = "not equal")## Difference between two means (paired samples t test)
## H0: mu1 = mu2
## HA: mu1 != mu2
## ------------------------------
## Statistical power = 0.8
## n = 222
## ------------------------------
## Alternative = "not equal"
## Degrees of freedom = 221
## Non-centrality parameter = 2.816
## Type I error rate = 0.05
## Type II error rate = 0.2It is sufficient to put Cohen's d or Hedge's g (standardized difference between two groups) for mu1 because mu2 = 0, sd1 = sqrt(1/(2*(1-paired.r))), and sd2 = sd1 by default. For example, for an effect size as small as 0.1883 (based on previous example) the minimum required sample size can be calculated as
pwrss.t.2means(mu1 = 0.1883, paired = TRUE, paired.r = 0.50,
power = .80, alpha = 0.05,
alternative = "not equal")## Difference between two means (paired samples t test)
## H0: mu1 = mu2
## HA: mu1 != mu2
## ------------------------------
## Statistical power = 0.8
## n = 224
## ------------------------------
## Alternative = "not equal"
## Degrees of freedom = 223
## Non-centrality parameter = 2.818
## Type I error rate = 0.05
## Type II error rate = 0.22.2 Non-parametric Tests
The variable of interest for which means are compared between two groups (or between two time points) may not follow normal distribution. In this case t test could produce biased results. One reason is that the sample size could be small. Another reason is that the parent distribution (distribution in the population) may not follow normal distribution (e.g., uniform, exponential, double exponential, or logistic).
For non-parametric tests use pwrss.np.2means() function instead of pwrss.t.2means(). All arguments are same. Additionally, however, the parent distribution can be specified. In the following examples the parent distrbution is "normal" (not shown because it is the default) and can be changed with the distribution argument.
2.2.1 Independent Samples (Wilcoxon-Mann-Whitney Test)
pwrss.np.2means(mu1 = 30, mu2 = 28, sd1 = 12, sd2 = 8, kappa = 1,
power = .80, alpha = 0.05,
alternative = "not equal")## Difference between two means (independent samples)
## Mann-Whitney U or Wilcoxon rank-sum test
## (a.k.a Wilcoxon-Mann-Whitney test)
## Method: GUENTHER
## H0: mu1 = mu2
## HA: mu1 != mu2
## ------------------------------
## Statistical power = 0.8
## n1 = 429
## n2 = 429
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = 2.805
## Degrees of freedom = 816.21
## Type I error rate = 0.05
## Type II error rate = 0.22.2.2 Paired Samples (Wilcoxon Signed-rank Test)
pwrss.np.2means(mu1 = 30, mu2 = 28, sd1 = 12, sd2 = 8,
paired = TRUE, paired.r = 0.50,
power = .80, alpha = 0.05,
alternative = "not equal")## Difference between two means (dependent samples)
## Wilcoxon signed-rank test for matched pairs
## Method: GUENTHER
## H0: mu1 = mu2
## HA: mu1 != mu2
## ------------------------------
## Statistical power = 0.8
## n = 233
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = 2.814
## Degrees of freedom = 220.7
## Type I error rate = 0.05
## Type II error rate = 0.2It is sufficient to put Cohen's d or Hedge's g (standardized difference between two groups) for mu1 without specifying mu2, sd1, and sd2.
2.3 Non-inferiority, Superiority, and Equivalence
These tests are useful for testing practically significant difference (non-inferiority, superiority) or practically null difference (equivalence).
2.3.1 Parametric Tests
Non-inferiority: Mean of group 1 is practically not smaller than the mean of group 2. The mu1 - mu2 difference can be as small as -1 (margin = -1). When higher values of the outcome is better margin takes NEGATIVE values for non-inferiority test; whereas when lower values of the outcome is better margin takes POSITIVE values.
pwrss.t.2means(mu1 = 30, mu2 = 28, sd1 = 12, sd2 = 8,
margin = -1, power = 0.80,
alternative = "non-inferior")## Difference between two means (independent samples t test)
## H0: mu1 - mu2 <= margin
## HA: mu1 - mu2 > margin
## ------------------------------
## Statistical power = 0.8
## n1 = 144
## n2 = 144
## ------------------------------
## Alternative = "non-inferior"
## Degrees of freedom = 286
## Non-centrality parameter = 2.496
## Type I error rate = 0.05
## Type II error rate = 0.2Superiority: Mean of group 1 is practically greater than the mean of group 2. The mu1 - mu2 difference is at least greater than 1 (margin = 1). When higher values of the outcome is better margin takes POSITIVE values for non-inferiority test; whereas when lower values of the outcome is better margin takes NEGATIVE values.
pwrss.t.2means(mu1 = 30, mu2 = 28, sd1 = 12, sd2 = 8,
margin = 1, power = 0.80,
alternative = "superior")## Difference between two means (independent samples t test)
## H0: mu1 - mu2 <= margin
## HA: mu1 - mu2 > margin
## ------------------------------
## Statistical power = 0.8
## n1 = 1287
## n2 = 1287
## ------------------------------
## Alternative = "superior"
## Degrees of freedom = 2572
## Non-centrality parameter = 2.487
## Type I error rate = 0.05
## Type II error rate = 0.2Equivalence: Mean of group 1 is practically same as mean of group 2. The mu1 - mu2 difference can be as small as -1 and as high as 1 (margin = 1). Specify the absolute value for the margin.
pwrss.t.2means(mu1 = 30, mu2 = 30, sd1 = 12, sd2 = 8,
margin = 1, power = 0.80,
alternative = "equivalent")## Difference between two means (independent samples t test)
## H0: |mu1 - mu2| >= margin
## HA: |mu1 - mu2| < margin
## ------------------------------
## Statistical power = 0.8
## n1 = 1783
## n2 = 1783
## ------------------------------
## Alternative = "equivalent"
## Degrees of freedom = 3564
## Non-centrality parameter = -2.928
## Type I error rate = 0.05
## Type II error rate = 0.22.3.2 Non-parametric Tests
Non-inferiority: Mean of group 1 is practically not smaller than the mean of group 2. The mu1 - mu2 difference can be as small as -1 (margin = -1).
pwrss.np.2means(mu1 = 30, mu2 = 28, sd1 = 12, sd2 = 8,
margin = -1, power = 0.80,
alternative = "non-inferior")## Difference between two means (independent samples)
## Mann-Whitney U or Wilcoxon rank-sum test
## (a.k.a Wilcoxon-Mann-Whitney test)
## Method: GUENTHER
## H0: mu1 - mu2 <= margin
## HA: mu1 - mu2 > margin
## ------------------------------
## Statistical power = 0.8
## n1 = 151
## n2 = 151
## ------------------------------
## Alternative = "non-inferior"
## Non-centrality parameter = 2.492
## Degrees of freedom = 285.13
## Type I error rate = 0.05
## Type II error rate = 0.2Superiority: Mean of group 1 is practically greater than the mean of group 2. The mu1 - mu2 difference is at least greater than 1 (margin = 1).
pwrss.np.2means(mu1 = 30, mu2 = 28, sd1 = 12, sd2 = 8,
margin = 1, power = 0.80,
alternative = "superior")## Difference between two means (independent samples)
## Mann-Whitney U or Wilcoxon rank-sum test
## (a.k.a Wilcoxon-Mann-Whitney test)
## Method: GUENTHER
## H0: mu1 - mu2 <= margin
## HA: mu1 - mu2 > margin
## ------------------------------
## Statistical power = 0.8
## n1 = 1348
## n2 = 1348
## ------------------------------
## Alternative = "superior"
## Non-centrality parameter = 2.487
## Degrees of freedom = 2571.3
## Type I error rate = 0.05
## Type II error rate = 0.2Equivalence: Mean of group 1 is practically same as mean of group 2. The mu1 - mu2 difference can be as small as -1 and as high as 1 (margin = 1).
pwrss.np.2means(mu1 = 30, mu2 = 30, sd1 = 12, sd2 = 8,
margin = 1, power = 0.80,
alternative = "equivalent")## Difference between two means (independent samples)
## Mann-Whitney U or Wilcoxon rank-sum test
## (a.k.a Wilcoxon-Mann-Whitney test)
## Method: GUENTHER
## H0: |mu1 - mu2| >= margin
## HA: |mu1 - mu2| < margin
## ------------------------------
## Statistical power = 0.8
## n1 = 1867
## n2 = 1867
## ------------------------------
## Alternative = "equivalent"
## Non-centrality parameter = -2.927
## Degrees of freedom = 3561.91
## Type I error rate = 0.05
## Type II error rate = 0.2
3 Multiple Linear Regression
3.1 Omnibus \(F\) Test
3.1.1 \(R^2 > 0\) in Linear Regression
Omnibus F test in multiple liner regression is used to test whether \(R^2\) is greater than 0 (zero). Assume that we want to predict a continuous variable \(Y\) using \(X_{1}\), \(X_{2}\), and \(X_{2}\) variables (a combination of binary or continuous).
\[\begin{eqnarray} Y_{i} &=& \beta_{0} + \beta_{1}X_{1} + \beta_{2}X_{2} + \beta_{3}X_{3} + r_{i}, \quad r_{i} \thicksim N(0,\sigma^2) \newline \end{eqnarray}\]
We are expecting that these three variables explain 30% of the variance in the outcome (\(R^2 = 0.30\)). What is the minimum required sample size?
pwrss.f.reg(r2 = 0.30, k = 3, power = 0.80, alpha = 0.05)## R-squared compared to 0 in linear regression (F test)
## H0: r2 = 0
## HA: r2 > 0
## ------------------------------
## Statistical power = 0.8
## n = 30
## ------------------------------
## Numerator degrees of freedom = 3
## Denominator degrees of freedom = 25.653
## Non-centrality parameter = 12.709
## Type I error rate = 0.05
## Type II error rate = 0.23.1.2 \(\Delta R^2 > 0\) in Hierarchical Regression
Assume that we want to add two more predictors to the earlier regression model (\(X_{4}\), and \(X_{5}\)) and test whether increase in \(R^2\) is greater than 0 (zero). We are expecting an increase of \(\Delta R^2 = 0.15\). What is the minimum required sample size?
\[\begin{eqnarray} Y_{i} &=& \beta_{0} + \beta_{1}X_{1} + \beta_{2}X_{2} + \beta_{3}X_{3} + \beta_{4}X_{4} + \beta_{5}X_{5} + r_{i}, \quad r_{i} \thicksim N(0,\sigma^2) \newline \end{eqnarray}\]
pwrss.f.reg(r2 = 0.15, k = 5, m = 2, power = 0.80, alpha = 0.05)## R-squared change in hierarchical linear regression (F test)
## H0: r2 = 0
## HA: r2 > 0
## ------------------------------
## Statistical power = 0.8
## n = 58
## ------------------------------
## Numerator degrees of freedom = 2
## Denominator degrees of freedom = 51.876
## Non-centrality parameter = 10.213
## Type I error rate = 0.05
## Type II error rate = 0.23.2 Single Regression Coefficient
3.2.1 Standardized versus Unstandardized Input
In the earlier example, assume that we want to predict a continuous variable \(Y\) using a continuous predictor \(X_{1}\) but control for \(X_{2}\), and \(X_{2}\) variables (a combination of binary or continuous). We are mainly interested in the effect of \(X_{1}\) and expect a standardized regression coefficient of \(\beta_{1} = 0.20\).
\[\begin{eqnarray} Y_{i} &=& \beta_{0} + \color{red} {\beta_{1} X_{1}} + \beta_{2}X_{2} + \beta_{3}X_{3} + r_{i}, \quad r_{i} \thicksim N(0,\sigma^2) \newline \end{eqnarray}\]
Again, we are expecting that these three variables explain 30% of the variance in the outcome (\(R^2 = 0.30\)). What is the minimum required sample size? It is sufficient to provide standardized regression coefficient for beta1 because sdx = 1 and sdy = 1 by default.
pwrss.t.reg(beta1 = 0.20, k = 3, r2 = 0.30,
power = .80, alpha = 0.05, alternative = "not equal")## Linear regression coefficient (t test)
## H0: beta1 = beta0
## HA: beta1 != beta0
## ------------------------------
## Statistical power = 0.8
## n = 140
## ------------------------------
## Alternative = "not equal"
## Degrees of freedom = 135.331
## Non-centrality parameter = 2.822
## Type I error rate = 0.05
## Type II error rate = 0.2For unstandardized coefficients specify sdy and sdx. Assume we are expecting an unstandardized regression coefficient of beta1 = 0.60, a standard deviation of sdy = 12 for the outcome and a standard deviation of sdx = 4 for the main predictor. What is the minimum required sample size?
pwrss.t.reg(beta1 = 0.60, sdy = 12, sdx = 4, k = 3, r2 = 0.30,
power = .80, alpha = 0.05, alternative = "not equal")## Linear regression coefficient (t test)
## H0: beta1 = beta0
## HA: beta1 != beta0
## ------------------------------
## Statistical power = 0.8
## n = 140
## ------------------------------
## Alternative = "not equal"
## Degrees of freedom = 135.331
## Non-centrality parameter = 2.822
## Type I error rate = 0.05
## Type II error rate = 0.2If \(X_{1}\) variable is binary (e.g. treatment/control), the standardized regression coefficient is Cohen's d. Standard deviation of the main predictor is \(\sqrt{p(1-p)}\) where p is the proportion of sample in one of the groups. Assume half of the sample is in the first group \(p = 0.50\). What is the minimum required sample size? It is sufficient to provide Cohen's d for beta1 (standardized difference between two groups) but specify sdx = sqrt(p*(1-p)) where p is the proportion of subjects in one of the groups.
pwrss.t.reg(beta1 = 0.20, k = 3, r2 = 0.30, sdx = sqrt(0.50*(1-0.50)),
power = .80, alpha = 0.05, alternative = "not equal")## Linear regression coefficient (t test)
## H0: beta1 = beta0
## HA: beta1 != beta0
## ------------------------------
## Statistical power = 0.8
## n = 552
## ------------------------------
## Alternative = "not equal"
## Degrees of freedom = 547.355
## Non-centrality parameter = 2.807
## Type I error rate = 0.05
## Type II error rate = 0.23.2.2 Non-inferiority, Superiority, and Equivalence
These tests are useful for testing practically significant effects (non-inferiority, superiority) or practically null effects (equivalence).
Non-inferiority: The intervention is expected to be non-inferior to some earlier or other interventions. Assume that the effect of an earlier or some other intervention is beta0 = 0.10. The beta1 - beta0 is expected to be positive and should be at least -0.05 (margin = -0.05). This is the case when higher values of an outcome is better. When lower values of an outcome is better the beta1 - beta0 difference is expected to be NEGATIVE and the margin takes POSITIVE values.
pwrss.t.reg(beta1 = 0.20, beta0 = 0.10, margin = -0.05,
k = 3, r2 = 0.30, sdx = sqrt(0.50*(1-0.50)),
power = .80, alpha = 0.05, alternative = "non-inferior")## Linear regression coefficient (t test)
## H0: beta1 - beta0 <= margin
## HA: beta1 - beta0 > margin
## ------------------------------
## Statistical power = 0.8
## n = 771
## ------------------------------
## Alternative = "non-inferior"
## Degrees of freedom = 766.745
## Non-centrality parameter = 2.489
## Type I error rate = 0.05
## Type II error rate = 0.2Superiority: The intervention is expected to be superior to some earlier or other interventions. Assume that the effect of an earlier or some other intervention is beta0 = 0.10. The beta1 - beta0 is expected to be positive and should be at least 0.05 (margin = 0.05). This is the case when higher values of an outcome is better. When lower values of an outcome is better and the beta1 - beta0 difference is expected to be NEGATIVE and the margin takes NEGATIVE values.
pwrss.t.reg(beta1 = 0.20, beta0 = 0.10, margin = 0.05,
k = 3, r2 = 0.30, sdx = sqrt(0.50*(1-0.50)),
power = .80, alpha = 0.05, alternative = "superior")## Linear regression coefficient (t test)
## H0: beta1 - beta0 <= margin
## HA: beta1 - beta0 > margin
## ------------------------------
## Statistical power = 0.8
## n = 6926
## ------------------------------
## Alternative = "superior"
## Degrees of freedom = 6921.818
## Non-centrality parameter = 2.487
## Type I error rate = 0.05
## Type II error rate = 0.2Equivalence: The intervention is expected to be equivalent to some earlier or other interventions. Assume the effect of an earlier or some other intervention is beta0 = 0.20. The beta1 - beta0 is expected to be within -0.05 and 0.05 (margin = 0.05). margin always takes positive values (absolute) for equivalence.
pwrss.t.reg(beta1 = 0.20, beta0 = 0.20, margin = 0.05,
k = 3, r2 = 0.30, sdx = sqrt(0.50*(1-0.50)),
power = .80, alpha = 0.05, alternative = "equivalent")## Linear regression coefficient (t test)
## H0: |beta1 - beta0| >= margin
## HA: |beta1 - beta0| < margin
## ------------------------------
## Statistical power = 0.8
## n = 9593
## ------------------------------
## Alternative = "equivalent"
## Degrees of freedom = 9588.862
## Non-centrality parameter = -2.927 2.927
## Type I error rate = 0.05
## Type II error rate = 0.2
4 Indirect Effect in Mediation Analysis (z Test)
A simple mediation model can be constructed as in the figure. X is the main predictor, M is the mediator, and Y is the outcome.
Simple Mediation Model
Regression models take the form of
\[\begin{eqnarray} M_i &=& \beta_{0M} + \color{red} {a} X_{i} + e_i\newline Y_i &=& \beta_{0Y} + \color{red} {b} M_{i} + \color{red} {cp} X_{i} + \epsilon_i \newline \end{eqnarray}\]
a, b, and cp path coefficients are needed to calculate the minimum required sample size. They can be standardized or unstandardized. By default, path coefficients are assumed to be standardized because standard deviation of the main predictor, mediator, and outcome is sdx = 1, sdm = 1, and sdy = 1, respectively. For unstandardized regression coefficients sdx, sdm, and sdy should be specified (they can be found in descriptive tables in reports/publications).
4.1 No Covariates
The main predictor is continuous:
# X (cont.), M (cont.) , Y (cont.)
pwrss.z.med(a = 0.25, b = 0.25, cp = 0.10,
power = 0.80, alpha = 0.05)## Indirect effect in mediation model
## H0: a * b = 0
## HA: a * b != 0
## ---------------------------
## test power n ncp
## Sobel 0.8 242 2.802
## Aroian 0.8 250 2.802
## Goodman 0.8 235 2.802
## Joint NA NA NA
## Monte Carlo NA NA NA
## ----------------------------
## Type I error rate = 0.05The main predictor is binary (e.g. treatment/control):
# X (binary), M (cont.) , Y (cont.)
p <- 0.50 # proportion of subjects in one group
pwrss.z.med(a = 0.25, b = 0.25, cp = 0.10,
sdx = sqrt(p*(1-p)),
power = 0.80, alpha = 0.05)## Indirect effect in mediation model
## H0: a * b = 0
## HA: a * b != 0
## ---------------------------
## test power n ncp
## Sobel 0.8 614 2.802
## Aroian 0.8 626 2.802
## Goodman 0.8 602 2.802
## Joint NA NA NA
## Monte Carlo NA NA NA
## ----------------------------
## Type I error rate = 0.054.2 With Covariates
Covariates can be added to the mediator model, outcome model, or both. Explanatory power of the covariates (R-squared values) for the mediator and outcome models can be specified via r2m.x and r2y.mx arguments.
The main predictor is continuous:
# X (cont.), M (cont.) , Y (cont.)
pwrss.z.med(a = 0.25, b = 0.25, cp = 0.10,
r2m.x = 0.50, r2y.mx = 0.50,
power = 0.80, alpha = 0.05)## Indirect effect in mediation model
## H0: a * b = 0
## HA: a * b != 0
## ---------------------------
## test power n ncp
## Sobel 0.8 189 2.802
## Aroian 0.8 194 2.802
## Goodman 0.8 183 2.802
## Joint NA NA NA
## Monte Carlo NA NA NA
## ----------------------------
## Type I error rate = 0.05The main predictor is binary (e.g. treatment/control):
# X (binary), M (cont.) , Y (cont.)
p <- 0.50 # proportion of subjects in one group
pwrss.z.med(a = 0.25, b = 0.25, cp = 0.10,
sdx = sqrt(p*(1-p)),
r2m.x = 0.50, r2y.mx = 0.50,
power = 0.80, alpha = 0.05)## Indirect effect in mediation model
## H0: a * b = 0
## HA: a * b != 0
## ---------------------------
## test power n ncp
## Sobel 0.8 377 2.802
## Aroian 0.8 388 2.802
## Goodman 0.8 366 2.802
## Joint NA NA NA
## Monte Carlo NA NA NA
## ----------------------------
## Type I error rate = 0.05
5 Analysis of (Co)Variance
5.1 One-way
5.1.1 ANOVA
A researcher is expecting a difference of Cohen's d = 0.50 between treatment and control groups (two levels) translating into \(\eta^2 = 0.059\) (eta2 = 0.059). What is the minimum required sample size?
pwrss.f.ancova(eta2 = 0.059, n.levels = 2,
power = .80, alpha = 0.05)## One-way Analysis of Variance (ANOVA)
## H0: 'eta2' or 'f2' = 0
## HA: 'eta2' or 'f2' > 0
## --------------------------------------
## Factor A: 2 levels
## --------------------------------------
## effect power n.total ncp df1 df2
## A 0.8 128 7.971 1 125.132
## --------------------------------------
## Type I error rate: 0.055.1.2 ANCOVA
A researcher is expecting an adjusted difference of Cohen's d = 0.45 between treatment and control groups (n.levels = 2) after controlling for the pretest (n.cov = 1) translating into partial \(\eta^2 = 0.048\) (eta2 = 0.048).
pwrss.f.ancova(eta2 = 0.048, n.levels = 2, n.cov = 1,
alpha = 0.05, power = .80)## One-way Analysis of Covariance (ANCOVA)
## H0: 'eta2' or 'f2' = 0
## HA: 'eta2' or 'f2' > 0
## --------------------------------------
## Factor A: 2 levels
## --------------------------------------
## effect power n.total ncp df1 df2
## A 0.8 158 7.948 1 154.626
## --------------------------------------
## Type I error rate: 0.055.2 Two-way
5.2.1 ANOVA
A researcher is expecting a partial \(\eta^2 = 0.03\) (eta2 = 0.03) for interaction of treatment/control (Factor A: two levels) with gender (Factor B: two levels). Thus, n.levels = c(2,2).
pwrss.f.ancova(eta2 = 0.03, n.levels = c(2,2),
alpha = 0.05, power = 0.80)## Two-way Analysis of Variance (ANOVA)
## H0: 'eta2' or 'f2' = 0
## HA: 'eta2' or 'f2' > 0
## --------------------------------------
## Factor A: 2 levels
## Factor B: 2 levels
## --------------------------------------
## effect power n.total ncp df1 df2
## A 0.8 256 7.909 1 251.73
## B 0.8 256 7.909 1 251.73
## A x B 0.8 256 7.909 1 251.73
## --------------------------------------
## Type I error rate: 0.055.2.2 ANCOVA
A researcher is expecting a partial \(\eta^2 = 0.02\) (eta2 = 0.02) for interaction of treatment/control (Factor A) with gender (Factor B) adjusted for the pretest (n.covariates = 1).
pwrss.f.ancova(eta2 = 0.02, n.levels = c(2,2), n.cov = 1,
alpha = 0.05, power = .80)## Two-way Analysis of Covariance (ANCOVA)
## H0: 'eta2' or 'f2' = 0
## HA: 'eta2' or 'f2' > 0
## --------------------------------------
## Factor A: 2 levels
## Factor B: 2 levels
## --------------------------------------
## effect power n.total ncp df1 df2
## A 0.8 387 7.889 1 381.539
## B 0.8 387 7.889 1 381.539
## A x B 0.8 387 7.889 1 381.539
## --------------------------------------
## Type I error rate: 0.055.3 Three-way
5.3.1 ANOVA
A researcher is expecting a partial \(\eta^2 = 0.02\) (eta2 = 0.02) for interaction of treatment/control (Factor A: two levels), gender (Factor B: two levels), and socio-economic status (Factor C: three levels). Thus, n.levels = c(2,2,3).
pwrss.f.ancova(eta2 = 0.02, n.levels = c(2,2,3),
alpha = 0.05, power = 0.80)## Three-way Analysis of Variance (ANOVA)
## H0: 'eta2' or 'f2' = 0
## HA: 'eta2' or 'f2' > 0
## --------------------------------------
## Factor A: 2 levels
## Factor B: 2 levels
## Factor C: 3 levels
## --------------------------------------
## effect power n.total ncp df1 df2
## A 0.8 387 7.889 1 374.576
## B 0.8 387 7.889 1 374.576
## C 0.8 476 9.697 2 463.167
## A x B 0.8 387 7.889 1 374.576
## A x C 0.8 476 9.697 2 463.167
## B x C 0.8 476 9.697 2 463.167
## A x B x C 0.8 476 9.697 2 463.167
## --------------------------------------
## Type I error rate: 0.055.3.2 ANCOVA
A researcher is expecting a partial \(\eta^2 = 0.01\) (eta2 = 0.01) for interaction of treatment/control (Factor A), gender (Factor B), and socio-economic status (Factor C: three levels) adjusted for the pretest (n.covariates = 1).
pwrss.f.ancova(eta2 = 0.01, n.levels = c(2,2,3), n.cov = 1,
alpha = 0.05, power = .80)## Three-way Analysis of Covariance (ANCOVA)
## H0: 'eta2' or 'f2' = 0
## HA: 'eta2' or 'f2' > 0
## --------------------------------------
## Factor A: 2 levels
## Factor B: 2 levels
## Factor C: 3 levels
## --------------------------------------
## effect power n.total ncp df1 df2
## A 0.8 779 7.869 1 765.990
## B 0.8 779 7.869 1 765.990
## C 0.8 957 9.665 2 943.868
## A x B 0.8 779 7.869 1 765.990
## A x C 0.8 957 9.665 2 943.868
## B x C 0.8 957 9.665 2 943.868
## A x B x C 0.8 957 9.665 2 943.868
## --------------------------------------
## Type I error rate: 0.05n.cov argument has trivial effect on the results. The difference between ANOVA and ANCOVA procedure depends on whether the effect (eta2) is unadjusted or covariate-adjusted.
6 Repeated Measures
6.1 Between Effect (Group)
Example 1: Posttest only design with treatment and control groups.
A researcher is expecting a difference of Cohen's d = 0.50 on the posttest score between treatment and control groups (n.levels = 2), translating into \(\eta^2 = 0.059\) (eta2 = 0.059). The test is administered at a single time point; thus, the 'number of repeated measures' is n.rm = 1.
pwrss.f.rmanova(eta2 = 0.059, n.levels = 2, n.rm = 1,
power = 0.80, alpha = 0.05,
type = "between")## One-way repeated measures analysis of variance (F test)
## H0: eta2 = 0 (or f2 = 0)
## HA: eta2 > 0 (or f2 > 0)
## ------------------------------
## Number of levels (groups) = 2
## Number of measurement time points = 1
## ------------------------------
## Statistical power = 0.8
## Total n = 128
## ------------------------------
## Type of the effect = "between"
## Numerator degrees of freedom = 1
## Denominator degrees of freedom = 125.132
## Non-centrality parameter = 7.971
## Type I error rate = 0.05
## Type II error rate = 0.26.2 Within Effect (Time)
Example 2: Pretest-posttest design with treatment group only.
A researcher is expecting a difference of Cohen's d = 0.30 between posttest and pretest scores, translating into \(\eta^2 = 0.022\) (eta2 = 0.022). The test is administered before and after the treatment; thus, the 'number of repeated measures' is n.rm = 2. There is treatment group but no control group (n.levels = 1). The researcher also expects a correlation of 0.50 (corr.rm = 0.50) between pretest and posttest scores.
pwrss.f.rmanova(eta2 = 0.022, n.levels = 1, n.rm = 2,
power = 0.80, alpha = 0.05,
corr.rm = 0.50, type = "within")## One-way repeated measures analysis of variance (F test)
## H0: eta2 = 0 (or f2 = 0)
## HA: eta2 > 0 (or f2 > 0)
## ------------------------------
## Number of levels (groups) = 1
## Number of measurement time points = 2
## ------------------------------
## Statistical power = 0.8
## Total n = 90
## ------------------------------
## Type of the effect = "within"
## Numerator degrees of freedom = 1
## Denominator degrees of freedom = 88.169
## Non-centrality parameter = 8.023
## Type I error rate = 0.05
## Type II error rate = 0.26.3 Between-within Interaction (Group x Time)
Example 3: Pretest-posttest control-group design.
A researcher is expecting a difference of Cohen's d = 0.40 on the posttest scores between treatment and control groups (n.levels = 2) after controlling for pretest, translating into partial \(\eta^2 = 0.038\) (eta2 = 0.038). The test is administered before and after the treatment; thus, the 'number of repeated measures' is n.rm = 2. The researcher also expects a correlation of 0.50 (corr.rm = 0.50) between pretest and posttest scores.
pwrss.f.rmanova(eta2 = 0.038, n.levels = 2, n.rm = 2,
power = 0.80, alpha = 0.05,
corr.rm = 0.50, type = "between")## One-way repeated measures analysis of variance (F test)
## H0: eta2 = 0 (or f2 = 0)
## HA: eta2 > 0 (or f2 > 0)
## ------------------------------
## Number of levels (groups) = 2
## Number of measurement time points = 2
## ------------------------------
## Statistical power = 0.8
## Total n = 151
## ------------------------------
## Type of the effect = "between"
## Numerator degrees of freedom = 1
## Denominator degrees of freedom = 148.97
## Non-centrality parameter = 7.951
## Type I error rate = 0.05
## Type II error rate = 0.2A researcher is expecting a difference of Cohen's d = 0.30 between posttest and pretest scores (n.rm = 2) after controlling for group membership, translating into partial \(\eta^2 = 0.022\) (eta2 = 0.022). There is both treatment and control groups (n.levels = 2). The researcher also expects a correlation of 0.50 (corr.rm = 0.50) between pretest and posttest scores.
pwrss.f.rmanova(eta2 = 0.022, n.levels = 2, n.rm = 2,
power = 0.80, alpha = 0.05,
corr.rm = 0.50, type = "within")## One-way repeated measures analysis of variance (F test)
## H0: eta2 = 0 (or f2 = 0)
## HA: eta2 > 0 (or f2 > 0)
## ------------------------------
## Number of levels (groups) = 2
## Number of measurement time points = 2
## ------------------------------
## Statistical power = 0.8
## Total n = 90
## ------------------------------
## Type of the effect = "within"
## Numerator degrees of freedom = 1
## Denominator degrees of freedom = 87.191
## Non-centrality parameter = 8.025
## Type I error rate = 0.05
## Type II error rate = 0.2The rationale for inspecting the interaction is that the benefit of the treatment may depend on the pretest score (e.g. those with higher scores on the pretest improve or deteriorate more). A researcher is expecting an interaction effect of partial \(\eta^2 = 0.01\) (eta2 = 0.01). The test is administered before and after the treatment; thus, the 'number of repeated measures' is n.rm = 2. There is both treatment and control groups (n.levels = 2). The researcher also expects a correlation of 0.50 (corr.rm = 0.50) between pretest and posttest scores.
pwrss.f.rmanova(eta2 = 0.01, n.levels = 2, n.rm = 2,
power = 0.80, alpha = 0.05,
corr.rm = 0.50, type = "interaction")## One-way repeated measures analysis of variance (F test)
## H0: eta2 = 0 (or f2 = 0)
## HA: eta2 > 0 (or f2 > 0)
## ------------------------------
## Number of levels (groups) = 2
## Number of measurement time points = 2
## ------------------------------
## Statistical power = 0.8
## Total n = 197
## ------------------------------
## Type of the effect = "interaction"
## Numerator degrees of freedom = 1
## Denominator degrees of freedom = 194.199
## Non-centrality parameter = 7.927
## Type I error rate = 0.05
## Type II error rate = 0.2
7 Goodness-of-fit or Independence (\(\chi^2\) Test)
7.1 Vector of k Cells
Effect size: Cohen's W for goodness-of-fit test for 1 x k or k x 1 table.
How many subjects are needed to claim that girls choose STEM related majors less than males? Check the original article at https://www.aauw.org/resources/research/the-stem-gap/
Option 1: Use cell probabilities
Alternative hypothesis state that 28% of the workforce in STEM field is women whereas 72% is men (from the article).
The null hypothesis assume 50% is women and 50% is men.
prob.mat <- c(0.28, 0.72)
pwrss.chisq.gofit(p1 = c(0.28, 0.72), p0 = c(0.50, 0.50),
alpha = 0.05, power = 0.80)## Pearson's Chi-square goodness of fit test
## for contingency tables
## ------------------------------
## Statistical power = 0.8
## Total n = 41
## ------------------------------
## Degrees of freedom = 1
## Non-centrality parameter = 7.849
## Type I error rate = 0.05
## Type II error rate = 0.2Option 2: Use Cohen's W = 0.44.
Degrees of freedom is k - 1 for Cohen's W.
pwrss.chisq.gofit(w = 0.44, df = 1,
alpha = 0.05, power = 0.80)## Pearson's Chi-square goodness of fit test
## for contingency tables
## ------------------------------
## Statistical power = 0.8
## Total n = 41
## ------------------------------
## Degrees of freedom = 1
## Non-centrality parameter = 7.849
## Type I error rate = 0.05
## Type II error rate = 0.27.2 Contingency Table: 2 (row) x 2 (column)
Effect size: Phi Coefficient (or Cramer's V or Cohen's W) for independence test for 2 x 2 contingency table.
How many subjects are needed to claim that girls are underdiagnosed with ADHD?
Check the original article at https://time.com/growing-up-with-adhd/
Option 1: Use cell probabilities.
5.6% of girls and 13.2% of boys are diagnosed with ADHD (from the article).
prob.mat <- rbind(c(0.056, 0.132),
c(0.944, 0.868))
colnames(prob.mat) <- c("Girl", "Boy")
rownames(prob.mat) <- c("ADHD", "No ADHD")
prob.mat## Girl Boy
## ADHD 0.056 0.132
## No ADHD 0.944 0.868pwrss.chisq.gofit(p1 = prob.mat,
alpha = 0.05, power = 0.80)## Pearson's Chi-square goodness of fit test
## for contingency tables
## ------------------------------
## Statistical power = 0.8
## Total n = 463
## ------------------------------
## Degrees of freedom = 1
## Non-centrality parameter = 7.849
## Type I error rate = 0.05
## Type II error rate = 0.2Option 2: Use Phi coefficient = 0.1302134.
Degrees of freedom is 1 for Phi coefficient.
pwrss.chisq.gofit(w = 0.1302134, df = 1,
alpha = 0.05, power = 0.80)## Pearson's Chi-square goodness of fit test
## for contingency tables
## ------------------------------
## Statistical power = 0.8
## Total n = 463
## ------------------------------
## Degrees of freedom = 1
## Non-centrality parameter = 7.849
## Type I error rate = 0.05
## Type II error rate = 0.27.3 Contingency Table: j (row) x k (column)
Effect size: Cramer's V (or Cohen's W) for independence test for j x k contingency tables.
How many subjects are needed to detect the relationship between depression severity and gender?
Check the original article at https://doi.org/10.1016/j.jad.2019.11.121
Option 1: Use cell probabilities (from the article).
prob.mat <- cbind(c(0.6759, 0.1559, 0.1281, 0.0323, 0.0078),
c(0.6771, 0.1519, 0.1368, 0.0241, 0.0101))
rownames(prob.mat) <- c("Normal", "Mild", "Moderate", "Severe", "Extremely Severe")
colnames(prob.mat) <- c("Female", "Male")
prob.mat## Female Male
## Normal 0.6759 0.6771
## Mild 0.1559 0.1519
## Moderate 0.1281 0.1368
## Severe 0.0323 0.0241
## Extremely Severe 0.0078 0.0101pwrss.chisq.gofit(p1 = prob.mat,
alpha = 0.05, power = 0.80)## Pearson's Chi-square goodness of fit test
## for contingency tables
## ------------------------------
## Statistical power = 0.8
## Total n = 13069
## ------------------------------
## Degrees of freedom = 4
## Non-centrality parameter = 11.935
## Type I error rate = 0.05
## Type II error rate = 0.2Option 2: Use Cramer's V = 0.03022008 based on 5 x 2 contingency table
Degrees of freedom is (nrow - 1) * (ncol - 1) for Cramer's V.
pwrss.chisq.gofit(w = 0.03022008, df = 4,
alpha = 0.05, power = 0.80)## Pearson's Chi-square goodness of fit test
## for contingency tables
## ------------------------------
## Statistical power = 0.8
## Total n = 13069
## ------------------------------
## Degrees of freedom = 4
## Non-centrality parameter = 11.935
## Type I error rate = 0.05
## Type II error rate = 0.2
8 Logistic Regression (Wald's z Test)
In logistic regression a binary outcome variable (0/1: failed/passed, dead/alive, absent/present) is modeled by predicting probability of being in group 1 (\(P_1\)) via logit (natural logarithm of odds) transformation. The base rate \(P_0\) is the overall probability of being in group 1 without influence of predictors in the model. When predictors are added, under alternative hypothesis, the probability of being in group 1 (\(P_1\)) deviate from \(P_0\); whereas under null it is same as \(P_0\). A model with one main predictor (\(X_1\)) and two other covariates (\(X_2\) and \(X_3\)) can be constructed as
\[\begin{eqnarray} ln(\frac{P_1}{1- P_1}) &=& \beta_{0} + \color{red} {\beta_{1} X_{1}} + \beta_{2}X_{2} + \beta_{3}X_{3} \newline \end{eqnarray}\]
where
\[\beta_0 = ln(\frac{P_0}{1- P_0})\]
\[\beta_1 = ln(\frac{P_1}{1- P_1} / \frac{P_0}{1- P_0})\]
Odds ratio is defined as \[OR = exp(\beta_1) = \frac{P_1}{1- P_1} / \frac{P_0}{1- P_0}\]
Assume that the multiple squared correlation between \(X_1\) and (\(X_2\),\(X_3\)) is \(R^2_{X_1, (X_2,X_3)} = 0.20\). This is 0 (zero) by default implying that there are no covariates in the model besides the main predictor. It can be found in the form of adjusted \(R^2\) via regressing \(X_1\) on \(X_2\) and \(X_3\).
There are three types of specification to power calculations; (i) probability specification, (ii) odds ratio specification, and (iii) regression coefficient specification (as in standard software output).
Assume that the expected base rate is \(P_0 = 0.15\). When a continuous predictor \(X_1\) (follows normal) is added along with other covariates, the probability of being in group 1 becomes \(P_1 = 0.10\). This means that increasing predictor \(X_1\) by one unit reduces probability of being in group 1 from 0.15 to 0.10. What is the minimum required sample size?
Probability specification:
pwrss.z.logreg(p0 = 0.15, p1 = 0.10, r2.other.x = 0.20,
power = 0.80, alpha = 0.05,
dist = "normal")## Logistic regression coefficient (large sample approx. Wald's z test)
## H0: beta1 = 0
## HA: beta1 != 0
## Distribution of X = 'normal'
## Method = DEMIDENKO(VC)
## ------------------------------
## Statistical power = 0.8
## n = 365
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.766
## Type I error rate = 0.05
## Type II error rate = 0.2Odds ratio specification: \[OR = \frac{P_1}{1-P1} / \frac{P_0}{1-P_0} = \frac{0.10}{1-0.10} / \frac{0.15}{1-0.15} = 0.6296\]
pwrss.z.logreg(p0 = 0.15, odds.ratio = 0.6296, r2.other.x = 0.20,
alpha = 0.05, power = 0.80,
dist = "normal")## Logistic regression coefficient (large sample approx. Wald's z test)
## H0: beta1 = 0
## HA: beta1 != 0
## Distribution of X = 'normal'
## Method = DEMIDENKO(VC)
## ------------------------------
## Statistical power = 0.8
## n = 365
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.766
## Type I error rate = 0.05
## Type II error rate = 0.2Regression coefficient specification: \[\beta_1 = ln(\frac{P_1}{1-P1} / \frac{P_0}{1-P_0}) = ln(0.6296) = -0.4626\]
pwrss.z.logreg(p0 = 0.15, beta1 = -0.4626, r2.other.x = 0.20,
alpha = 0.05, power = 0.80,
dist = "normal")## Logistic regression coefficient (large sample approx. Wald's z test)
## H0: beta1 = 0
## HA: beta1 != 0
## Distribution of X = 'normal'
## Method = DEMIDENKO(VC)
## ------------------------------
## Statistical power = 0.8
## n = 365
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.766
## Type I error rate = 0.05
## Type II error rate = 0.2By default, the mean and standard deviation of a normally distributed main predictor is 0 and 1, respectively. They can be changed. In the following example the mean and standard deviation is 10 and 2, respectively.
Change parameters of distribution of predictor X:
dist.x <- list(dist = "normal", mean = 10, sd = 2)
pwrss.z.logreg(p0 = 0.15, beta1 = -0.4626, r2.other.x = 0.20,
alpha = 0.05, power = 0.80,
dist = dist.x)## Logistic regression coefficient (large sample approx. Wald's z test)
## H0: beta1 = 0
## HA: beta1 != 0
## Distribution of X = 'normal'
## Method = DEMIDENKO(VC)
## ------------------------------
## Statistical power = 0.8
## n = 4403
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.796
## Type I error rate = 0.05
## Type II error rate = 0.2The function accomodates other types of distribution. For example, the main predictor can be binary (e.g. treatment/control). Often a balanced sample is desired where half of the sample is in treatment group and the other half is in control group (prob = 0.50 by default).
Change distribution family of predictor X:
pwrss.z.logreg(p0 = 0.15, beta1 = -0.4626, r2.other.x = 0.20,
alpha = 0.05, power = 0.80,
dist = "bernoulli")## Logistic regression coefficient (large sample approx. Wald's z test)
## H0: beta1 = 0
## HA: beta1 != 0
## Distribution of X = 'bernoulli'
## Method = DEMIDENKO(VC)
## ------------------------------
## Statistical power = 0.8
## n = 1723
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.789
## Type I error rate = 0.05
## Type II error rate = 0.2Change treatment group allocation rate of binary predictor X (prob = 0.40):
dist.x <- list(dist = "bernoulli", prob = 0.40)
pwrss.z.logreg(p0 = 0.15, beta1 = -0.4626, r2.other.x = 0.20,
alpha = 0.05, power = 0.80,
dist = dist.x)## Logistic regression coefficient (large sample approx. Wald's z test)
## H0: beta1 = 0
## HA: beta1 != 0
## Distribution of X = 'bernoulli'
## Method = DEMIDENKO(VC)
## ------------------------------
## Statistical power = 0.8
## n = 1826
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.766
## Type I error rate = 0.05
## Type II error rate = 0.2
9 Poisson Regression (Wald's z Test)
In poisson regression a count outcome variable (e.g. number of hospital/store/website visits, number of absence/dead/purchase in a day/week/month) is modeled by predicting incidence rate (\(\lambda\)) via logarithmic (natural logarithm of rates) transformation. A model with one main predictor (\(X_1\)) and two other covariates (\(X_2\) and \(X_3\)) can be constructed as
\[\begin{eqnarray} ln(\lambda) &=& \beta_{0} + \color{red} {\beta_{1} X_{1}} + \beta_{2}X_{2} + \beta_{3}X_{3} \newline \end{eqnarray}\]
where \(exp(\beta_0)\) is the base incidence rate.
\[\beta_1 = ln(\frac{\lambda(X_1=1)}{\lambda(X_1=0)})\]
Rate ratio is defined as \[exp(\beta_1) = \frac{\lambda(X_1=1)}{\lambda(X_1=0)}\]
There are two types of specification; (i) rate ratio specification (exponentiated regression coefficient), and (ii) the raw regression coefficient specification (as in standard software output).
Assume that the expected base rate is \(exp(0.50) = 1.65\) and when a continuous predictor \(X_1\) (follows normal) is added to the model along with other covariates the expected rate becomes \(exp(-0.10) = 0.905\). This means that increasing predictor \(X_1\) by one unit reduces reduces the mean incidence rate from 1.65 to 0.905. What is the minimum required sample size?
Regression coefficient specification:
pwrss.z.poisreg(beta0 = 0.50, beta1 = -0.10,
power = 0.80, alpha = 0.05,
dist = "normal")## Poisson regression coefficient (large sample approx. Wald's z test)
## H0: beta1 = 0
## HA: beta1 != 0
## Distribution of X = 'normal'
## Method = DEMIDENKO(VC)
## ------------------------------
## Statistical power = 0.8
## n = 474
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.802
## Type I error rate = 0.05
## Type II error rate = 0.2Rate ratio specification:
pwrss.z.poisreg(exp.beta0 = exp(0.50),
exp.beta1 = exp(-0.10),
power = 0.80, alpha = 0.05,
dist = "normal")## Poisson regression coefficient (large sample approx. Wald's z test)
## H0: beta1 = 0
## HA: beta1 != 0
## Distribution of X = 'normal'
## Method = DEMIDENKO(VC)
## ------------------------------
## Statistical power = 0.8
## n = 474
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.802
## Type I error rate = 0.05
## Type II error rate = 0.2Change parameters of distribution of predictor X:
dist.x <- list(dist = "normal", mean = 10, sd = 2)
pwrss.z.poisreg(exp.beta0 = exp(0.50),
exp.beta1 = exp(-0.10),
alpha = 0.05, power = 0.80,
dist = dist.x)## Poisson regression coefficient (large sample approx. Wald's z test)
## H0: beta1 = 0
## HA: beta1 != 0
## Distribution of X = 'normal'
## Method = DEMIDENKO(VC)
## ------------------------------
## Statistical power = 0.8
## n = 318
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.802
## Type I error rate = 0.05
## Type II error rate = 0.2The function accomodates other types of distribution. For example, the main predictor can be binary (e.g. treatment/control).
Regression coefficient specification:
pwrss.z.poisreg(beta0 = 0.50, beta1 = -0.10,
alpha = 0.05, power = 0.80,
dist = "bernoulli")## Poisson regression coefficient (large sample approx. Wald's z test)
## H0: beta1 = 0
## HA: beta1 != 0
## Distribution of X = 'bernoulli'
## Method = DEMIDENKO(VC)
## ------------------------------
## Statistical power = 0.8
## n = 2003
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.801
## Type I error rate = 0.05
## Type II error rate = 0.2Rate ratio specification:
pwrss.z.poisreg(exp.beta0 = exp(0.50),
exp.beta1 = exp(-0.10),
alpha = 0.05, power = 0.80,
dist = "bernoulli")## Poisson regression coefficient (large sample approx. Wald's z test)
## H0: beta1 = 0
## HA: beta1 != 0
## Distribution of X = 'bernoulli'
## Method = DEMIDENKO(VC)
## ------------------------------
## Statistical power = 0.8
## n = 2003
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.801
## Type I error rate = 0.05
## Type II error rate = 0.2Change treatment group allocation rate of the binary predictor X (prob = 0.40):
dist.x <- list(dist = "bernoulli", prob = 0.40)
pwrss.z.poisreg(exp.beta0 = exp(0.50),
exp.beta1 = exp(-0.10),
alpha = 0.05, power = 0.80,
dist = dist.x)## Poisson regression coefficient (large sample approx. Wald's z test)
## H0: beta1 = 0
## HA: beta1 != 0
## Distribution of X = 'bernoulli'
## Method = DEMIDENKO(VC)
## ------------------------------
## Statistical power = 0.8
## n = 2095
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.792
## Type I error rate = 0.05
## Type II error rate = 0.2
10 Single Correlation
One-sided Test: Assume that the expected correlation is 0.20 and it is greater than 0.10. What is the minimum required sample size?
pwrss.z.corr(r = 0.20, r0 = 0.10,
power = 0.80, alpha = 0.05,
alternative = "greater")## One correlation compared to a constant (one sample z test)
## H0: r = r0
## HA: r > r0
## ------------------------------
## Statistical power = 0.8
## n = 593
## ------------------------------
## Alternative = "greater"
## Non-centrality parameter = 2.486
## Type I error rate = 0.05
## Type II error rate = 0.2Two-sided Test: Assume that the expected correlation is 0.20 and it is different from 0 (zero). The correlation could be 0.20 as well as -0.20. What is the minimum required sample size?
pwrss.z.corr(r = 0.20, r0 = 0,
power = 0.80, alpha = 0.05,
alternative = "not equal")## One correlation compared to a constant (one sample z test)
## H0: r = r0
## HA: r != r0
## ------------------------------
## Statistical power = 0.8
## n = 194
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = 2.802
## Type I error rate = 0.05
## Type II error rate = 0.2
11 Two Correlations (Independent Samples)
Assume that the expected correlations in the first and second groups are 0.30 and 0.20, respectively (r1 = 0.30 and r2 = 0.20).
One-sided Test: Expecting r1 - r2 greater than 0 (zero). The difference could be 0.10 but could not be -0.10. What is the minimum required sample size?
pwrss.z.2corrs(r1 = 0.30, r2 = 0.20,
power = .80, alpha = 0.05,
alternative = "greater")## Difference between two correlations (independent samples z test)
## H0: r1 = r2
## HA: r1 > r2
## ------------------------------
## Statistical power = 0.8
## n1 = 1088
## n2 = 1088
## ------------------------------
## Alternative = "greater"
## Non-centrality parameter = 2.486
## Type I error rate = 0.05
## Type II error rate = 0.2Two-sided Test: Expecting r1 - r2 different from 0 (zero). The difference could be -0.10 as well as 0.10. What is the minimum required sample size?
pwrss.z.2corrs(r1 = 0.30, r2 = 0.20,
power = .80, alpha = 0.05,
alternative = "not equal")## Difference between two correlations (independent samples z test)
## H0: r1 = r2
## HA: r1 != r2
## ------------------------------
## Statistical power = 0.8
## n1 = 1380
## n2 = 1380
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = 2.802
## Type I error rate = 0.05
## Type II error rate = 0.2
12 Single Proportion
In the following examples p is the proportion under alternative hypothesis and p0 is the proportion under null hypothesis.
One-sided Test: Expecting p - p0 smaller than 0 (zero).
# normal approximation
pwrss.z.prop(p = 0.45, p0 = 0.50,
alpha = 0.05, power = 0.80,
alternative = "less",
arcsin.trans = FALSE)## Approach: Normal approximation
## One proportion compared to a constant (z test)
## H0: p = p0
## HA: p < p0
## ------------------------------
## Statistical power = 0.8
## n = 613
## ------------------------------
## Alternative = "less"
## Non-centrality parameter = -2.486
## Type I error rate = 0.05
## Type II error rate = 0.2# arcsine transformation
pwrss.z.prop(p = 0.45, p0 = 0.50,
alpha = 0.05, power = 0.80,
alternative = "less",
arcsin.trans = TRUE)## Approach: Arcsine transformation
## One proportion compared to a constant (z test)
## H0: p = p0
## HA: p < p0
## ------------------------------
## Statistical power = 0.8
## n = 617
## ------------------------------
## Alternative = "less"
## Non-centrality parameter = -2.486
## Type I error rate = 0.05
## Type II error rate = 0.2Two-sided Test: Expecting p - p0 smaller than 0 (zero) or greater than 0 (zero).
pwrss.z.prop(p = 0.45, p0 = 0.50,
alpha = 0.05, power = 0.80,
alternative = "not equal")## Approach: Normal approximation
## One proportion compared to a constant (z test)
## H0: p = p0
## HA: p != p0
## ------------------------------
## Statistical power = 0.8
## n = 778
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.802
## Type I error rate = 0.05
## Type II error rate = 0.2Non-inferiority Test: The case when smaller proportion is better. Expecting p - p0 smaller than 0.01.
pwrss.z.prop(p = 0.45, p0 = 0.50, margin = 0.01,
alpha = 0.05, power = 0.80,
alternative = "non-inferior")## Approach: Normal approximation
## One proportion compared to a constant (z test)
## H0: p - p0 <= margin
## HA: p - p0 > margin
## ------------------------------
## Statistical power = 0.8
## n = 426
## ------------------------------
## Alternative = "non-inferior"
## Non-centrality parameter = -2.486
## Type I error rate = 0.05
## Type II error rate = 0.2Non-inferiority Test: The case when bigger proportion is better. Expecting p - p0 greater than -0.01.
pwrss.z.prop(p = 0.55, p0 = 0.50, margin = -0.01,
alpha = 0.05, power = 0.80,
alternative = "non-inferior")## Approach: Normal approximation
## One proportion compared to a constant (z test)
## H0: p - p0 <= margin
## HA: p - p0 > margin
## ------------------------------
## Statistical power = 0.8
## n = 426
## ------------------------------
## Alternative = "non-inferior"
## Non-centrality parameter = 2.486
## Type I error rate = 0.05
## Type II error rate = 0.2Superiority Test: The case when smaller proportion is better. Expecting p - p0 smaller than -0.01.
pwrss.z.prop(p = 0.45, p0 = 0.50, margin = -0.01,
alpha = 0.05, power = 0.80,
alternative = "superior")## Approach: Normal approximation
## One proportion compared to a constant (z test)
## H0: p - p0 <= margin
## HA: p - p0 > margin
## ------------------------------
## Statistical power = 0.8
## n = 957
## ------------------------------
## Alternative = "superior"
## Non-centrality parameter = -2.486
## Type I error rate = 0.05
## Type II error rate = 0.2Superiority Test: The case when bigger proportion is better. Expecting p - p0 greater than 0.01.
pwrss.z.prop(p = 0.55, p0 = 0.50, margin = 0.01,
alpha = 0.05, power = 0.80,
alternative = "superior")## Approach: Normal approximation
## One proportion compared to a constant (z test)
## H0: p - p0 <= margin
## HA: p - p0 > margin
## ------------------------------
## Statistical power = 0.8
## n = 957
## ------------------------------
## Alternative = "superior"
## Non-centrality parameter = 2.486
## Type I error rate = 0.05
## Type II error rate = 0.2**Equivalence Test*: Expecting p - p0 between -0.01 and 0.01.
pwrss.z.prop(p = 0.50, p0 = 0.50, margin = 0.01,
alpha = 0.05, power = 0.80,
alternative = "equivalent")## Approach: Normal approximation
## One proportion compared to a constant (z test)
## H0: |p - p0| >= margin
## HA: |p - p0| < margin
## ------------------------------
## Statistical power = 0.8
## n = 21410
## ------------------------------
## Alternative = "equivalent"
## Non-centrality parameter = -2.926 2.926
## Type I error rate = 0.05
## Type II error rate = 0.2
13 Two Proportions (Independent Samples)
In the following examples p1 and p2 are proportions for the first and second groups under alternative hypothesis. The null hypothesis state p1 = p2 or p1 - p2 = 0.
One-sided Test: Expecting p1 - p2 smaller than 0 (zero).
# normal approximation
pwrss.z.2props(p1 = 0.45, p2 = 0.50,
alpha = 0.05, power = 0.80,
alternative = "less",
arcsin.trans = FALSE)## Approach: Normal approximation
## Difference between two proportions (z test)
## H0: p1 = p2
## HA: p1 < p2
## ------------------------------
## Statistical power = 0.8
## n1 = 1231
## n2 = 1231
## ------------------------------
## Alternative = "less"
## Non-centrality parameter = -2.486
## Type I error rate = 0.05
## Type II error rate = 0.2# arcsine transformation
pwrss.z.2props(p1 = 0.45, p2 = 0.50,
alpha = 0.05, power = 0.80,
alternative = "less",
arcsin.trans = TRUE)## Approach: Arcsine transformation
## Cohen's h = -0.1
## Difference between two proportions (z test)
## H0: p1 = p2
## HA: p1 < p2
## ------------------------------
## Statistical power = 0.8
## n1 = 1233
## n2 = 1233
## ------------------------------
## Alternative = "less"
## Non-centrality parameter = -2.486
## Type I error rate = 0.05
## Type II error rate = 0.2Two-sided Test: Expecting p1 - p2 smaller than 0 (zero) or greater than 0 (zero).
pwrss.z.2props(p1 = 0.45, p2 = 0.50,
alpha = 0.05, power = 0.80,
alternative = "not equal")## Approach: Normal approximation
## Difference between two proportions (z test)
## H0: p1 = p2
## HA: p1 != p2
## ------------------------------
## Statistical power = 0.8
## n1 = 1562
## n2 = 1562
## ------------------------------
## Alternative = "not equal"
## Non-centrality parameter = -2.802
## Type I error rate = 0.05
## Type II error rate = 0.2Non-inferiority Test: The case when smaller proportion is better. Expecting p1 - p2 smaller than 0.01.
pwrss.z.2props(p1 = 0.45, p2 = 0.50, margin = 0.01,
alpha = 0.05, power = 0.80,
alternative = "non-inferior")## Approach: Normal approximation
## Difference between two proportions (z test)
## H0: p1 - p2 <= margin
## HA: p1 - p2 > margin
## ------------------------------
## Statistical power = 0.8
## n1 = 855
## n2 = 855
## ------------------------------
## Alternative = "non-inferior"
## Non-centrality parameter = -2.486
## Type I error rate = 0.05
## Type II error rate = 0.2Non-inferiority Test: The case when bigger proportion is better. Expecting p1 - p2 greater than -0.01.
pwrss.z.2props(p1 = 0.55, p2 = 0.50, margin = -0.01,
alpha = 0.05, power = 0.80,
alternative = "non-inferior")## Approach: Normal approximation
## Difference between two proportions (z test)
## H0: p1 - p2 <= margin
## HA: p1 - p2 > margin
## ------------------------------
## Statistical power = 0.8
## n1 = 855
## n2 = 855
## ------------------------------
## Alternative = "non-inferior"
## Non-centrality parameter = 2.486
## Type I error rate = 0.05
## Type II error rate = 0.2Superiority Test: The case when smaller proportion is better. Expecting p1 - p2 smaller than -0.01.
pwrss.z.2props(p1 = 0.45, p2 = 0.50, margin = -0.01,
alpha = 0.05, power = 0.80,
alternative = "superior")## Approach: Normal approximation
## Difference between two proportions (z test)
## H0: p1 - p2 <= margin
## HA: p1 - p2 > margin
## ------------------------------
## Statistical power = 0.8
## n1 = 1923
## n2 = 1923
## ------------------------------
## Alternative = "superior"
## Non-centrality parameter = -2.486
## Type I error rate = 0.05
## Type II error rate = 0.2Superiority Test: The case when bigger proportion is better. Expecting p1 - p2 greater than 0.01.
pwrss.z.2props(p1 = 0.55, p2 = 0.50, margin = 0.01,
alpha = 0.05, power = 0.80,
alternative = "superior")## Approach: Normal approximation
## Difference between two proportions (z test)
## H0: p1 - p2 <= margin
## HA: p1 - p2 > margin
## ------------------------------
## Statistical power = 0.8
## n1 = 1923
## n2 = 1923
## ------------------------------
## Alternative = "superior"
## Non-centrality parameter = 2.486
## Type I error rate = 0.05
## Type II error rate = 0.2Equivalence Test: Expecting p1 - p2 between -0.01 and 0.01.
pwrss.z.2props(p1 = 0.50, p2 = 0.50, margin = 0.01,
alpha = 0.05, power = 0.80,
alternative = "equivalent")## Approach: Normal approximation
## Difference between two proportions (z test)
## H0: |p1 - p2| >= margin
## HA: |p1 - p2| < margin
## ------------------------------
## Statistical power = 0.8
## n1 = 42820
## n2 = 42820
## ------------------------------
## Alternative = "equivalent"
## Non-centrality parameter = -2.926 2.926
## Type I error rate = 0.05
## Type II error rate = 0.2The end of the document.
Please send any bug reports, feedback or questions to bulusmetin [at] gmail.com.