I’ve a data frame that summarises the number of missing and non-missing observations in a data frame that is passed to it[1]. I then have been asked to test for differences between two treatment arms in the data I have (personally I disagree with the need or utility of doing so, but its what I’ve been asked to do). So I’ve written a small function to do this…
quick.test <- function(x, y){
chisq <- chisq.test(x = x, y = y)
fisher <- fisher.test(x = x, y = y)
results <- cbind(chisq = chisq$statistic,
df = chisq$parameter,
p = chisq$p.value,
fisher = fisher$p.value)
results
}
I then use apply() to pass the relevant columns to this function as follows…
apply(miss.t1, 1, function(x) quick.test(x[2:3], x[4:5]))
This is fine for the above specified miss.t1 data frame, but I’m working with time-series data and have three time-points I wish to summarise so have miss.t2 and miss.t3 (each of which is summarising the number of present/missing data for each time point, and have been created in the same manner using the function described in [1]).
miss.t2 fails with the following error…
apply(miss.t2, 1, function(x) quick.test(x[2:3], x[4:5]))
Error in chisq.test(x = x, y = y) :
'x' and 'y' must have at least 2 levels
My initial thought was that one of the columns had a missing value for some reason, but this doesn’t appear to be the case…
> describe(miss.t2)
miss.t2
5 Variables 171 Observations
--------------------------------------------------------------------------------
variable
n missing unique
171 0 171
lowest : Abtotal Abyn agg_ment agg_phys All.score
highest: z_pf z_re z_rp z_sf z_vt
--------------------------------------------------------------------------------
nmiss.1
n missing unique Mean
171 0 4 8.649
0 (6, 4%), 8 (9, 5%), 9 (153, 89%), 10 (3, 2%)
--------------------------------------------------------------------------------
npresent.1
n missing unique Mean
171 0 4 9.351
8 (3, 2%), 9 (153, 89%), 10 (9, 5%), 18 (6, 4%)
--------------------------------------------------------------------------------
nmiss.2
n missing unique Mean
171 0 4 10.65
0 (6, 4%), 11 (160, 94%), 12 (4, 2%), 13 (1, 1%)
--------------------------------------------------------------------------------
npresent.2
n missing unique Mean
171 0 4 14.35
12 (1, 1%), 13 (4, 2%), 14 (160, 94%), 25 (6, 4%)
--------------------------------------------------------------------------------
Next thing I tried was trying subsets of miss.t2 by taking head(miss.t2, n=XX) and it works fine upto row 54…
> apply(head(miss.t2, n=53), 1, function(x) quick.test(x[2:3], x[4:5]))
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
[1,] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[2,] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
[3,] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
[4,] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53
[1,] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[2,] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
[3,] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
[4,] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
There were 50 or more warnings (use warnings() to see the first 50)
> apply(head(miss.t2, n=54), 1, function(x) quick.test(x[2:3], x[4:5]))
Error in chisq.test(x = x, y = y) :
'x' and 'y' must have at least 2 levels
> miss.t2[54,]
variable nmiss.1 npresent.1 nmiss.2 npresent.2
54 psq 10 8 11 14
> traceback()
5: stop("'x' and 'y' must have at least 2 levels") at #2
4: chisq.test(x = x, y = y) at #2
3: quick.test(x[2:3], x[4:5])
2: FUN(newX[, i], ...)
1: apply(head(miss.t2, n = 54), 1, function(x) quick.test(x[2:3],
x[4:5]))
Similarly with the ‘bottom’ of the data frame the last 26 rows are parsed fine, but not the 27th from last…
> apply(tail(miss.t2, n=26), 1, function(x) quick.test(x[2:3], x[4:5]))
146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163
[1,] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[2,] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
[3,] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
[4,] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
164 165 166 167 168 169 170 171
[1,] 0 0 0 0 0 0 0 0
[2,] 1 1 1 1 1 1 1 1
[3,] 1 1 1 1 1 1 1 1
[4,] 1 1 1 1 1 1 1 1
There were 26 warnings (use warnings() to see them)
> apply(tail(miss.t2, n=27), 1, function(x) quick.test(x[2:3], x[4:5]))
Error in chisq.test(x = x, y = y) :
'x' and 'y' must have at least 2 levels
In addition: Warning message:
In chisq.test(x = x, y = y) : Chi-squared approximation may be incorrect
> miss.t2[118,]
variable nmiss.1 npresent.1 nmiss.2 npresent.2
118 sf16 9 9 11 14
I can’t see anything wrong with these two lines that means they should fail and the traceback() shown above doesn’t reveal anything useful (to my mind).
Can anyone offer any suggestions as to why or where things are going wrong?
Many thanks in advance,
Neil
EDIT : Formatted reply to Vincent Zoonekynd …
I opted for the chisq.test(x = x, y = y) version described in ?chisq.test(), using cbind() as you suggest to produce a matrix results in
Error in sum(x) : invalid ‘type’ (character) of argument.
Putting print statements and showing the length of x and y results in the same error, but shows the values and lenghts as being…
> miss.t2.res <- data.frame(t(apply(miss.t2, 1, function(x) quick.test(x[2:3], x[4:5]))))
[1] "Your x is : 9" "Your x is : 9"
[1] 2 ### < Length of x
[1] "Your y is : 11" "Your y is : 14"
[1] 2 ### < Length of y
Error in chisq.test(x = x, y = y) : 'x' and 'y' must have at least 2 levels
EDIT 2 : Thanks to Vincent Zoonekynd pointers the problem was that because the counts were identical for two cells the call to chisq.test() treats these as factors and collapses them. The solution was to modify the quick.test() function and coerce the arguments that are being passed into a matrix, so the function that worked is now….
quick.test <- function(x, y){
chisq <- chisq.test(rbind(as.numeric(x), as.numeric(y)))
fisher <- fisher.test(rbind(as.numeric(x), as.numeric(y)))
results <- cbind(chisq = chisq$statistic,
df = chisq$parameter,
p = chisq$p.value,
fisher = fisher$p.value)
results
}
Many thanks for the help & pointers Vincent, very much appreciated.
[1] http://gettinggeneticsdone.blogspot.co.uk/2011/02/summarize-missing-data-for-all.html
The solution suggested by Vincent Zoonkeynd in the comments above, was to modify the quick.test() function and coerce the arguments that are being passed into a matrix, so the function that worked is now….