Daniel Weiand – Consultant Medical Microbiologist – Newcastle upon Tyne NHS Foundation Trust
Hello!
This is my first blog post for the NHS R Community, which I stumbled across in the course of my work as a consultant medical microbiologist at Newcastle upon Tyne Hospitals NHS Foundation Trust.
At work, I’ve been trying to use R to create column charts featuring 95% confidence intervals. I approached the friendly people on the NHS R Community’s Slack channel for further information and guidance.
I must add here that the Community’s Slack channel has been extremely helpful to me, as a novice R user, in solving some of the issues I’ve experienced, and highlighting R packages of potential interest. This is the first time I’ve tried to create a ReprEx and now I understand why people love (?) the mtcars database as much as they do!
Step 1: Calculate some summary statistics
I wanted to calculate some summary statistics, including the mean, and standard error or 95% confidence intervals.
Initially I came across the summary() function of Base R, which is helpful as it calculates the Min., 1st Qu., Median, Mean, 3rd Qu., and Max.
However, the summary() function of {base} R does not calculate either the standard error or the 95% confidence intervals
#calculate summary statistics for all numeric data using summary() and where(is.numeric())
mtcars %>%
select(where(is.numeric)) %>%
summary()
#calculate summary statistics for mpg using summary() and where(is.numberic())
mtcars %>%
select(mpg) %>%
summary()
Then zx8754 very kindly pointed me towards a method for calculating the standard error on StackOverflow: https://stackoverflow.com/q/2676554/680068
#create stderr function
stderr <- function(x, na.rm=TRUE) {
if (na.rm) x <- na.omit(x)
sqrt(var(x)/length(x))
}
Then I used this function to calculate summary statistics, incl. mean and standard error, using the summarise() and across() functions of {dplyer}
#calculate summary statistics using summarise() and across() and n/mean/min/median/max/sd/stderr
# stderr <- function(x, na.rm=TRUE) {
# if (na.rm) x <- na.omit(x)
# sqrt(var(x)/length(x))
# }
mtcars %>%
group_by(cyl) %>%
mutate(
across(mpg,
list(
n = ~ n(),
mean = ~ mean(.x, na.rm = TRUE),
min = ~ min(.x, na.rm = TRUE),
median = ~ median(.x, na.rm = TRUE),
max = ~ max(.x, na.rm = TRUE),
sd = ~ sd(.x, na.rm = TRUE),
stderr = ~ stderr(.x)),
.names = NULL)) %>%
select(starts_with("mpg")) %>%
summarise(mean = mean(mpg_mean),
min = mean(mpg_min),
median = mean(mpg_median),
max = mean(mpg_max),
sd = mean(mpg_sd),
stderr = mean(mpg_stderr)) %>%
#create column chart with error bars (using stderr)
ggplot(aes(cyl, mean))+
geom_col(na.rm = TRUE)+
geom_errorbar(aes(ymin = mean-stderr, ymax = mean+stderr), position = "dodge", width = 0.25)
#calculate summary statistics using summarise() and across() and n/mean/min/median/max/sd/stderr
# stderr <- function(x, na.rm=TRUE) {
# if (na.rm) x <- na.omit(x)
# sqrt(var(x)/length(x))
# }
mtcars %>%
group_by(cyl) %>%
mutate(
across(mpg,
list(
n = ~ n(),
mean = ~ mean(.x, na.rm = TRUE),
min = ~ min(.x, na.rm = TRUE),
median = ~ median(.x, na.rm = TRUE),
max = ~ max(.x, na.rm = TRUE),
sd = ~ sd(.x, na.rm = TRUE),
stderr = ~ stderr(.x)),
.names = NULL)) %>%
select(starts_with("mpg")) %>%
summarise(mean = mean(mpg_mean),
min = mean(mpg_min),
median = mean(mpg_median),
max = mean(mpg_max),
sd = mean(mpg_sd),
stderr = mean(mpg_stderr)) %>%
#create column chart with error bars (using stderr)
ggplot(aes(cyl, mean))+
geom_col(na.rm = TRUE)+
geom_errorbar(aes(ymin = mean-stderr, ymax = mean+stderr), position = "dodge", width = 0.25)
Step 2: Create column charts with error bars (using 95% confidence intervals)
Then Seb Fox pointed me towards a method for calculating 95% confidence intervals using the {PHEindicatormethods} package, available on CRAN: https://cran.r-project.org/web/packages/PHEindicatormethods/index.html
#create MEAN column chart with error bars (using 95% confidence intervals)
require(PHEindicatormethods)
mtcars %>%
filter(!is.na(cyl)) %>%
group_by(cyl) %>%
#use phe_mean()
phe_mean(x = mpg, #field name from data containing the values to calculate the means for
type = "full", #defines the data and metadata columns to be included in output; can be "value", "lower", "upper", "standard" (for all data) or "full" (for all data and metadata); quoted string; default = "full"
confidence = 0.95) %>% #required level of confidence expressed as a number between 0.9 and 1
#create column chart with error bars (using 95% CI calculated using phe_mean())
ggplot(aes(cyl, value))+
geom_col(na.rm = TRUE)+
geom_errorbar(aes(ymin = lowercl, ymax = uppercl), position = "dodge", width = 0.25)
#create PROPORTION column chart with error bars (using 95% confidence intervals)
require(PHEindicatormethods)
mtcars %>%
group_by(cyl) %>%
summarise(n = n(),
sum = sum(n)) %>%
mutate(sum = sum(n)) %>%
#phe_proportion()
phe_proportion(x = n, #numerator
n = sum, #denominator
type = "full", #defines the data and metadata columns to be included in output; can be "value", "lower", "upper", "standard" (for all data) or "full" (for all data and metadata); quoted string; default = "full"
confidence = 0.95, #required level of confidence expressed as a number between 0.9 and 1
multiplier = 100) %>% #the multiplier used to express the final values (eg 100 = percentage); numeric; default 1
#create column chart with error bars (using 95% CI calculated using phe_proportion())
ggplot(aes(cyl, value))+
geom_col(na.rm = TRUE)+
geom_errorbar(aes(ymin = lowercl, ymax = uppercl), position = "dodge", width = 0.25)
I hope that the code, above, helps a few colleagues of mine across the NHS, in some small way.
Thank you, again, to all members of the NHS R Community, for all your help. Particular thanks go to everyone who has helped me, to date, on the NHS R Community’s Slack channel.