Warnings When Running an Lmer in R

Glmer warning message to use lmer

The warning is very exact in it's messaging.

When fitting a mixed-effect model with gaussian(link = "identity") it is equivalent to fitting a linear mixed effect model with normal random effects.

glmer simply changes the call to lmer(yield ~ Location + treatment + (1|block),data=df) and gives a warning.

The warning has been there for a very long time, and I would bet that it wont be deprecated in any near future, but for all intends and purposes you should use lmer(...) instead of glmer(..., family = gaussian(link = 'identity'))

warning messages when trying to run glmer in r

tl;dr at least based on the subset of data you provided, this is a fairly unstable fit. The warnings about near unidentifiability go away if you scale the continuous predictors. Trying with a wide variety of optimizers, we get about the same log-likelihoods, and parameter estimates that vary by a few percent; two optimizers (nlminb from base R and BOBYQA from the nloptr package) converge without warnings, and are probably giving the "correct" answer. I haven't computed confidence intervals, but I suspect that they're very wide. (Your mileage may differ somewhat with your full data set ...)

source("SO_23478792_dat.R")  ## I put the data you provided in here

Basic fit (replicated from above):

library(lme4)
df$SUR.ID <- factor(df$SUR.ID)
df$replicate <- factor(df$replicate)
Rdet <- cbind(df$ValidDetections,df$FalseDetections)
Unit <- factor(1:length(df$ValidDetections))
m1 <- glmer(Rdet ~ tm:Area + tm:c.distance +
            c.distance:Area + c.tm.depth:Area +
            c.receiver.depth:Area + c.temp:Area +
            c.wind:Area +
            c.tm.depth + c.receiver.depth +
            c.temp +c.wind + tm + c.distance + Area +
            replicate +
            (1|SUR.ID) + (1|Day) + (1|Unit) ,
            data = df, family = binomial(link=logit))

I get more or less the same warnings you did, slightly fewer since the development version has been a little improved/tweaked:

## 1: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
##   Model failed to converge with max|grad| = 1.52673 (tol = 0.001, component 1)
## 2: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
##   Model is nearly unidentifiable: very large eigenvalue
##  - Rescale variables?;Model is nearly unidentifiable: large eigenvalue ratio
## - Rescale variables?

I tried various little things (restarting from the previous fitted values, switching optimizers) without much change in the results (i.e. the same warnings).

ss <- getME(m1,c("theta","fixef"))
m2 <- update(m1,start=ss,control=glmerControl(optCtrl=list(maxfun=2e4)))
m3 <- update(m1,start=ss,control=glmerControl(optimizer="bobyqa",
                         optCtrl=list(maxfun=2e4)))

Following the advice in the warning message (rescaling the continuous predictors):

numcols <- grep("^c\\.",names(df))
dfs <- df
dfs[,numcols] <- scale(dfs[,numcols])
m4 <- update(m1,data=dfs)

This gets rid of scaling warnings, but the warning about large gradients persists.

Use some utility code to fit the same model with many different optimizers:

afurl <- "https://raw.githubusercontent.com/lme4/lme4/master/misc/issues/allFit.R"
## http://tonybreyal.wordpress.com/2011/11/24/source_https-sourcing-an-r-script-from-github/
library(RCurl)
eval(parse(text=getURL(afurl)))
aa <- allFit(m4)
is.OK <- sapply(aa,is,"merMod")  ## nlopt NELDERMEAD failed, others succeeded
## extract just the successful ones
aa.OK <- aa[is.OK]

Pull out warnings:

lapply(aa.OK,function(x) x@optinfo$conv$lme4$messages)

(All but nlminb and nloptr BOBYQA give convergence warnings.)

Log-likelihoods are all approximately the same:

summary(sapply(aa.OK,logLik),digits=6)
##     Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
## -107.127 -107.114 -107.111 -107.114 -107.110 -107.110

(again, nlminb and nloptr BOBYQA have the best fits/highest log-likelihoods)

Compare fixed effect parameters across optimizers:

aa.fixef <- t(sapply(aa.OK,fixef))
library(ggplot2)
library(reshape2)
library(plyr)
aa.fixef.m <- melt(aa.fixef)
models <- levels(aa.fixef.m$Var1)
(gplot1 <- ggplot(aa.fixef.m,aes(x=value,y=Var1,colour=Var1))+geom_point()+
    facet_wrap(~Var2,scale="free")+
    scale_y_discrete(breaks=models,
                     labels=abbreviate(models,6)))
## coefficients of variation of fixed-effect parameter estimates:
summary(unlist(daply(aa.fixef.m,"Var2",summarise,sd(value)/abs(mean(value)))))
##     Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
## 0.003573 0.013300 0.022730 0.019710 0.026200 0.035810

Compare variance estimates (not as interesting: all optimizers except N-M give exactly
zero variance for Day and SUR.ID)

aa.varcorr <- t(sapply(aa.OK,function(x) unlist(VarCorr(x))))
aa.varcorr.m <- melt(aa.varcorr)
gplot1 %+% aa.varcorr.m

I tried to run this with lme4.0 ("old lme4"), but got various "Downdated VtV" errors, even with the scaled data set. Perhaps that problem would go away with the full data set?

I haven't yet explored why drop1 doesn't work properly if the initial fit returns warnings ...

warning messages in lme4 for survival analysis that did not arise 3 years ago

tl;dr the "non-integer successes" warning is accurate; it's up to you to decide whether fitting a binomial model to these data really makes sense. The other warnings suggest that the fit is a bit unstable, but scaling and centering some of the input variables can make the warnings go away. It's up to you, again, to decide whether the results from these different formulations are different enough for you to worry about ...

data2 <- read.csv("1005cs.csv")
library(lme4)

Fit model (slightly more compact model formulation)

dat.asr3<-glmer(
   csns~Day*Region*treatment+
        (1|tub)+(1|aquarium),
            weights=start, family=binomial, data=data2)

I do get the warnings you report.

Let's take a look at the data:

library(ggplot2); theme_set(theme_bw())
ggplot(data2,aes(Day,csns,colour=factor(treatment)))+
        geom_point(aes(size=start),alpha=0.5)+facet_wrap(~Region)

Sample Image

Nothing obviously problematic here, although it does clearly show that the data are very close to 1 for some treatment combinations, and that the treatment values are far from zero. Let's try scaling & centering some of the input variables:

data2sc <- transform(data2,
                     Day=scale(Day),
                     treatment=scale(treatment))
dat.asr3sc <- update(dat.asr3,data=data2sc)

Now the "very large eigenvalue" warning is gone, but we still have the "non-integer # successes" warning, and a max|grad|=0.082. Let's try another optimizer:

dat.asr3scbobyqa <- update(dat.asr3sc,
                     control=glmerControl(optimizer="bobyqa"))

Now only the "non-integer #successes" warning remains.

d1 <- deviance(dat.asr3)
d2 <- deviance(dat.asr3sc)
d3 <- deviance(dat.asr3scbobyqa)
c(d1,d2,d3)
## [1] 12597.12 12597.31 12597.56

These deviances don't differ by very much (0.44 on the deviance scale is more than could be accounted for by round-off error, but not much difference in goodness of fit); actually, the first model gives the best (lowest) deviance, suggesting that the warnings are false positives ...

resp <- with(data2,csns*start)
plot(table(resp-floor(resp)))

Sample Image

This makes it clear that there really are non-integer responses, so the warning is correct.

Hide warning message in lme4

It's a message (a milder form of informational message than a warning: warnings use the prefix Warning message:), so you can use suppressMessages():

library(lme4)
ss <- transform(sleepstudy,Days2=Days)  ## create duplicate variable
m1 <- lmer(Reaction~Days+Days2+(1|Subject),ss)
## fixed-effect model matrix is rank deficient so dropping 1 column / coefficient
m2 <- suppressMessages(lmer(Reaction~Days+Days2+(1|Subject),ss))

However, you can tell lmer to suppress just this message by including control=lmerControl(check.rankX="silent.drop.cols"):

m3 <- update(m1, control=lmerControl(check.rankX="silent.drop.cols"))

In general I would say it's better to avoid these collinear terms in your model in the first place if possible (although they're harmless). (I see that the OP says they know what's happening, so this information is more for future readers.) You can see what variables are being dropped by looking at the attributes of the model matrix:

attr(getME(m2,"X"),"col.dropped")
## Days2 
#3    3

glmer model from early 2013: warning message about convergence when re-running it

(Upgraded from a comment)

the class of objects produced by glmer has indeed changed (from mer to merMod). You can install a back-compatibility lme4.0 package from http://lme4.r-forge.r-project.org/repos :

install.packages("lme4.0",repos="http://lme4.r-forge.r-project.org/repos")

and see ?convert_old_lme4 for a utility function to convert the saved model fit so you can use it in lme4.0.

Your convergence warning is almost certainly a false positive: see this mailing list thread or the tail end of this Github issue. You can either install the development version from Github (via devtools::install_github("lme4","lme4")) or the repository listed above, or just try this:

relgrad <- with(mixmod1@optinfo$derivs, solve(Hessian, gradient))
max(abs(relgrad))

and see if the scaled gradient is small.

How to get rid of lmer warning message?

You should try check.nobs.vs.rankZ="ignore".

lmerControl doesn't need to specify anything other than the non-default options: at a quick glance, these are your non-default values:

lmerControl(check.nobs.vs.nlev = "ignore",check.nobs.vs.rankZ =   
   "ignore",check.nlev.gtreq.5 = "ignore",check.nobs.vs.nRE="ignore",
   check.rankX =    c("ignore"),
   check.scaleX = "ignore",
   check.formula.LHS="ignore",
   check.conv.grad   = .makeCC("warning", tol = 1e-3, relTol = NULL))

In general I would suggest it's wise to turn off only the specific warnings and errors you know you want to override -- the settings above look like they could get you in trouble.

I haven't checked this since you didn't give a reproducible example ...