Issue with Interactions in Formula
Issue with Interactions in Formula
runMLwiN is inconsistent accepting interactions in my formulas and I don't see a pattern. For example, interactions that work when in separate models create an error when included in the same formula. Here's an example. (Unfortunately, I haven't been able to recreate the problem with MLwiN's example data sets so far.)
#set up formula for three models
form.raceInteract1 < formula(z_pcm3partr ~ 1 + d0locIndeg + d0extIndeg + zN0mapDDpopBXisol + zN0mapDDextpopBXisol + d1BlackNonHisp_scs + d1BlackNonHisp_scs:d0locIndeg + (1scid) + (1commID) + (1aid) )
form.raceInteract2 < formula(z_pcm3partr ~ 1 + d0locIndeg + d0extIndeg + zN0mapDDpopBXisol + zN0mapDDextpopBXisol + d1BlackNonHisp_scs + d1BlackNonHisp_scs:d0extIndeg + (1scid) + (1commID) + (1aid) )
form.raceInteract3 < formula(z_pcm3partr ~ 1 + d0locIndeg + d0extIndeg + zN0mapDDpopBXisol + zN0mapDDextpopBXisol + d1BlackNonHisp_scs + d1BlackNonHisp_scs:d0locIndeg + d1BlackNonHisp_scs:d0extIndeg + (1scid) + (1commID) + (1aid) )
#fit models
fit1 < runMLwiN(form.raceInteract1, data=DF, D="Normal")
fit2 < runMLwiN(form.raceInteract2, data=DF, D="Normal")
fit3 < runMLwiN(form.raceInteract1, data=DF, D="Normal")
The first two models run as they should but the third returns.
Error in `[.data.frame`(indata, , outvars) : undefined columns selected
All three models include the same variables. The only difference is that both interaction terms are included in the third. The variables with the d prefix are dummy variables. All the fixed parameters are at level 1.
I've also tried turning the dummies to factors but with the same result.
Can anyone offer advice as to what may be going on?
Thank you in advance!
#set up formula for three models
form.raceInteract1 < formula(z_pcm3partr ~ 1 + d0locIndeg + d0extIndeg + zN0mapDDpopBXisol + zN0mapDDextpopBXisol + d1BlackNonHisp_scs + d1BlackNonHisp_scs:d0locIndeg + (1scid) + (1commID) + (1aid) )
form.raceInteract2 < formula(z_pcm3partr ~ 1 + d0locIndeg + d0extIndeg + zN0mapDDpopBXisol + zN0mapDDextpopBXisol + d1BlackNonHisp_scs + d1BlackNonHisp_scs:d0extIndeg + (1scid) + (1commID) + (1aid) )
form.raceInteract3 < formula(z_pcm3partr ~ 1 + d0locIndeg + d0extIndeg + zN0mapDDpopBXisol + zN0mapDDextpopBXisol + d1BlackNonHisp_scs + d1BlackNonHisp_scs:d0locIndeg + d1BlackNonHisp_scs:d0extIndeg + (1scid) + (1commID) + (1aid) )
#fit models
fit1 < runMLwiN(form.raceInteract1, data=DF, D="Normal")
fit2 < runMLwiN(form.raceInteract2, data=DF, D="Normal")
fit3 < runMLwiN(form.raceInteract1, data=DF, D="Normal")
The first two models run as they should but the third returns.
Error in `[.data.frame`(indata, , outvars) : undefined columns selected
All three models include the same variables. The only difference is that both interaction terms are included in the third. The variables with the d prefix are dummy variables. All the fixed parameters are at level 1.
I've also tried turning the dummies to factors but with the same result.
Can anyone offer advice as to what may be going on?
Thank you in advance!

 Posts: 1094
 Joined: Mon Oct 19, 2009 10:34 am
Re: Issue with Interactions in Formula
I'm not sure whether it is a typo copying the code into your message but your third fit line:
appears to refer to the first formula (form.raceInteract1) instead of the third (form.raceInteract3). If you are using different data between the fits then this might explain the error. If not then can you try the fixed part of the model with other R fit functions (e.g. lm() or lmer()) to see whether you see similar behaviour there?
Code: Select all
fit3 < runMLwiN(form.raceInteract1, data=DF, D="Normal")
Re: Issue with Interactions in Formula
Thanks for the prompt reply.
The form.raceInteract1 for the third model was just a typo copying to the post. The fitted model was correct.
It's the same data set for all models and the formula works fine with lmer.
The form.raceInteract1 for the third model was just a typo copying to the post. The fitted model was correct.
It's the same data set for all models and the formula works fine with lmer.

 Posts: 1094
 Joined: Mon Oct 19, 2009 10:34 am
Re: Issue with Interactions in Formula
Could you please provide further information about the variables in the data, e.g. number of factor levels with their labels and number of unique values for each variable?

 Posts: 1094
 Joined: Mon Oct 19, 2009 10:34 am
Re: Issue with Interactions in Formula
I think that I can see a possible cause. If I run the following:
You will see that the term order of the interaction d1BlackNonHisp_scs:d0locIndeg has been flipped around to d0locIndeg:d1BlackNonHisp_scs. The extra data variables are generated from this, however R2MLwN will be looking for them as they are in the original formula. Could you try adjusting your third formula to match the variables as above to see if that fixes it?
Code: Select all
> form.raceInteract3 < formula(z_pcm3partr ~ 1 + d0locIndeg + d0extIndeg + zN0mapDDpopBXisol + zN0mapDDextpopBXisol + d1BlackNonHisp_scs + d1BlackNonHisp_scs:d0locIndeg + d1BlackNonHisp_scs:d0extIndeg + (1scid) + (1commID) + (1aid) )
> terms.formula(form.raceInteract3)
z_pcm3partr ~ 1 + d0locIndeg + d0extIndeg + zN0mapDDpopBXisol +
zN0mapDDextpopBXisol + d1BlackNonHisp_scs + d1BlackNonHisp_scs:d0locIndeg +
d1BlackNonHisp_scs:d0extIndeg + (1  scid) + (1  commID) +
(1  aid)
attr(,"variables")
list(z_pcm3partr, d0locIndeg, d0extIndeg, zN0mapDDpopBXisol,
zN0mapDDextpopBXisol, d1BlackNonHisp_scs, 1  scid, 1  commID,
1  aid)
attr(,"factors")
d0locIndeg d0extIndeg zN0mapDDpopBXisol zN0mapDDextpopBXisol d1BlackNonHisp_scs 1  scid 1  commID 1  aid d0locIndeg:d1BlackNonHisp_scs d0extIndeg:d1BlackNonHisp_scs
z_pcm3partr 0 0 0 0 0 0 0 0 0 0
d0locIndeg 1 0 0 0 0 0 0 0 1 0
d0extIndeg 0 1 0 0 0 0 0 0 0 1
zN0mapDDpopBXisol 0 0 1 0 0 0 0 0 0 0
zN0mapDDextpopBXisol 0 0 0 1 0 0 0 0 0 0
d1BlackNonHisp_scs 0 0 0 0 1 0 0 0 1 1
1  scid 0 0 0 0 0 1 0 0 0 0
1  commID 0 0 0 0 0 0 1 0 0 0
1  aid 0 0 0 0 0 0 0 1 0 0
attr(,"term.labels")
[1] "d0locIndeg" "d0extIndeg" "zN0mapDDpopBXisol" "zN0mapDDextpopBXisol" "d1BlackNonHisp_scs" "1  scid"
[7] "1  commID" "1  aid" "d0locIndeg:d1BlackNonHisp_scs" "d0extIndeg:d1BlackNonHisp_scs"
attr(,"order")
[1] 1 1 1 1 1 1 1 1 2 2
attr(,"intercept")
[1] 1
attr(,"response")
[1] 1
attr(,".Environment")
<environment: R_GlobalEnv>
Re: Issue with Interactions in Formula
Apologies for the delayed response. I tried that solution. Unless I misunderstood the suggestion, the same error comes up,
Code: Select all
>
> form.raceInteract3B < formula(z_pcm3partr ~ 1 + d0locIndeg + d0extIndeg + zN0mapDDpopBXisol + zN0mapDDextpopBXisol + d1BlackNonHisp_scs + d0locIndeg:d1BlackNonHisp_scs + d0extIndeg:d1BlackNonHisp_scs + (1scid) + (1commID) + (1aid) )
> attributes(terms.formula(form.raceInteract3B))
$variables
list(z_pcm3partr, d0locIndeg, d0extIndeg, zN0mapDDpopBXisol,
zN0mapDDextpopBXisol, d1BlackNonHisp_scs, 1  scid, 1  commID,
1  aid)
$factors
d0locIndeg d0extIndeg zN0mapDDpopBXisol
z_pcm3partr 0 0 0
d0locIndeg 1 0 0
d0extIndeg 0 1 0
zN0mapDDpopBXisol 0 0 1
zN0mapDDextpopBXisol 0 0 0
d1BlackNonHisp_scs 0 0 0
1  scid 0 0 0
1  commID 0 0 0
1  aid 0 0 0
zN0mapDDextpopBXisol d1BlackNonHisp_scs 1  scid
z_pcm3partr 0 0 0
d0locIndeg 0 0 0
d0extIndeg 0 0 0
zN0mapDDpopBXisol 0 0 0
zN0mapDDextpopBXisol 1 0 0
d1BlackNonHisp_scs 0 1 0
1  scid 0 0 1
1  commID 0 0 0
1  aid 0 0 0
1  commID 1  aid d0locIndeg:d1BlackNonHisp_scs
z_pcm3partr 0 0 0
d0locIndeg 0 0 1
d0extIndeg 0 0 0
zN0mapDDpopBXisol 0 0 0
zN0mapDDextpopBXisol 0 0 0
d1BlackNonHisp_scs 0 0 1
1  scid 0 0 0
1  commID 1 0 0
1  aid 0 1 0
d0extIndeg:d1BlackNonHisp_scs
z_pcm3partr 0
d0locIndeg 0
d0extIndeg 1
zN0mapDDpopBXisol 0
zN0mapDDextpopBXisol 0
d1BlackNonHisp_scs 1
1  scid 0
1  commID 0
1  aid 0
$term.labels
[1] "d0locIndeg" "d0extIndeg"
[3] "zN0mapDDpopBXisol" "zN0mapDDextpopBXisol"
[5] "d1BlackNonHisp_scs" "1  scid"
[7] "1  commID" "1  aid"
[9] "d0locIndeg:d1BlackNonHisp_scs" "d0extIndeg:d1BlackNonHisp_scs"
$order
[1] 1 1 1 1 1 1 1 1 2 2
$intercept
[1] 1
$response
[1] 1
$class
[1] "terms" "formula"
$.Environment
<environment: R_GlobalEnv>
> fit.raceInteract3B < runMLwiN(form.raceInteract3B, data=DF, D="Normal")
Error in `[.data.frame`(indata, , outvars) : undefined columns selected
Calls: runMLwiN > [ > [.data.frame
Execution halted
Last edited by jpcmu5002 on Wed Sep 13, 2017 8:48 pm, edited 1 time in total.
Re: Issue with Interactions in Formula
I tried to bypass the problem entirely by using a character string instead of a formula object, which the manual says can be done. I can't find examples of this approach, so I'm confused why I'm getting the following error. I think I'm following the syntax rules on page 22 of the manual correctly.
Code: Select all
> charform.raceInteract3B < 'z_pcm3partr ~ 1 + d0locIndeg + d0extIndeg + zN0mapDDpopBXisol + zN0mapDDextpopBXisol + d1BlackNonHisp_scs + d0locIndeg:d1BlackNonHisp_scs + d0extIndeg:d1BlackNonHisp_scs + (1scid) + (1commID) + (1aid)'
> fitchar.raceInteract3B < runMLwiN(charform.raceInteract3B, data=DF)
Error in parse(text = x, keep.source = FALSE) :
<text>:2:0: unexpected end of input
1: ~

 Posts: 1094
 Joined: Mon Oct 19, 2009 10:34 am
Re: Issue with Interactions in Formula
I don't think that specifying the formula as a string is likely to make a difference as it will be converted back to a formula object internally anyway. Could you try adding the option drop.data=FALSE to your estoptions list in case one of the variables is being inadvertently removed and see if that makes a difference?

 Posts: 1094
 Joined: Mon Oct 19, 2009 10:34 am
Re: Issue with Interactions in Formula
I believe that we have now managed to track down the cause of this bug. Could you please let us know whether the attached test version of the package fixes the problem for you?
 Attachments

 R2MLwiN_0.86.tar.gz
 (339.76 KiB) Downloaded 62 times
Re: Issue with Interactions in Formula
Hmmm... Well with this version, the model that was producing the error before (form.raceInteract3 in the original post) now runs.
Unfortunately, the two models that worked before (form.raceInteract1 and form.raceInteract2) no longer work. They now produce
I also tried including drop.data=FALSE in estoptions, but to no avail.
Unfortunately, the two models that worked before (form.raceInteract1 and form.raceInteract2) no longer work. They now produce
Code: Select all
Error in '[.data.frame'(indata, ,outvars):undefined columns selected