2LevelImpute

Welcome to the forum for Stat-JR users. Feel free to post your question about Stat-JR here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

We will add further support to the Stat-JR website, such as FAQs and tutorials, as soon as it is available; the Stat-JR website can be found here: http://www.bristol.ac.uk/cmm/software/statjr/
shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

2LevelImpute

Post by shakespeare » Thu Aug 07, 2014 12:11 pm

If one reads the imputation literature, inclusion of the MOI outcome measure in the imputation model is suggested, either as a response (when incomplete), or explanatory (when complete) variable. I was able to successfully do this in REALCOM. I wanted to make sure that the 2LevelImpute template is written to accommodate this point of view. Thx.

shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

Re: 2LevelImpute

Post by shakespeare » Thu Aug 07, 2014 6:49 pm

Follow-up: I'm not sure including my outcome as an explanatory variable is feasible, given the documentation. For now, I've included it only as a response in the MOI. I'm attempting to run a random intercept/random slope model. I get the error below. I'm not sure if it's because the program is not working correctly (I had some difficulty getting it to run the example problems) or if this indicates my set up is faulty. Would appreciate feedback.

Traceback (most recent call last):
File "webtest.py", line 416, in go
File "C:\StatJR\packages\Python_script.py", line 71, in run
self.eng.run('script.py')
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Engine.py", line 43, in decorated
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Python.py", line 16, in run
File "", line 189, in
File "O:\StatJR\estat\trunk\src\apps\cmdtest\cmdtest.py", line 127, in RunStatJR
File "C:\StatJR\packages\CustomC.py", line 587, in init
self.applydata()
File "C:\StatJR\packages\CustomC.py", line 318, in applydata
missmask[d] = missmask[d] | self.dsets[d].variables[v]['data'].mask
KeyError: 'r'

richardparker
Posts: 57
Joined: Fri Oct 23, 2009 1:49 pm

Re: 2LevelImpute

Post by richardparker » Fri Aug 08, 2014 8:59 am

Hi - the 2LevelImpute template does support the inclusion of the MOI's outcome variable in the imputation model (e.g. in the accompanying eBook, the MOI's outcome variable 'normexam' is included as an explanatory variable in the imputation model).

Sorry you got an error message. Do your variable names have any non-alphanumeric characters in them (or are you able to send a list of their names)? In addition, can I just check if you're adding any variables with missing values as explanatory variables in your imputation model?

Thanks,

Richard

shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

Re: 2LevelImpute

Post by shakespeare » Fri Aug 08, 2014 10:46 am

No missing values in my explanatory variables, but some of my variable names have underscores. Should I remove the underscores and use only alphanumeric characters in the variable names?

richardparker
Posts: 57
Joined: Fri Oct 23, 2009 1:49 pm

Re: 2LevelImpute

Post by richardparker » Fri Aug 08, 2014 12:51 pm

Hi - yes, that would be worth trying.

Richard

shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

Re: 2LevelImpute

Post by shakespeare » Tue Aug 12, 2014 8:58 pm

Removing the underscores seems to have had the desired effect. My program ran for about 24 hours until it threw the error below. I had the same problem when I tried to run the example problem. It eventually ran, but I was never really clear what causes this error and how to fix it. One suggestion was a virus scanner. I'll check with my IT department to see if there is virus software. Any other ideas would be appreciated. Thx.


Traceback (most recent call last): File "webtest.py", line 416, in go File "C:\StatJR\packages\Python_script.py", line 71,
in run self.eng.run('script.py') File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Engine.py", line 43,
in decorated File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Python.py", line 16,
in run File "", line 259, in File "c:\Python27\lib\multiprocessing\pool.py", line 554,
in get CompileError: error: c:\users\sc014459\appdata\local\temp\4\tmpv1qly6\zkfzgu:
The process cannot access the file because it is being used by another process

shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

Re: 2LevelImpute

Post by shakespeare » Fri Aug 22, 2014 1:04 am

We changed the version of our anti-virus program (McAfee) to one that does no on-box scanning. I'm still getting this message after running about 24 hours:

Traceback (most recent call last):
File "webtest.py", line 416, in go
File "C:\StatJR\packages\Python_script.py", line 71, in run
self.eng.run('script.py')
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Engine.py", line 43, in decorated
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Python.py", line 16, in run
File "", line 259, in
File "c:\Python27\lib\multiprocessing\pool.py", line 554, in get
CompileError: error: c:\users\sc014459\appdata\local\temp\1\tmp6dsdrs\b3z8uf: The process cannot access the file because it is being used by another process

My IT department is requesting a list of exceptions. I have some SAS data on this machine, but was not accessing at it the time of the software crash. My SAS software runs on another box anyway. Is this a compiler error? I have R and SRCware, the stand alone version of IVEware, installed on this box, but neither were running at the time of the crash, so I don't think that's the problem. Seems like the ant-virus is still the best suspect, but I'd appreciate feedback. Thoughts? Thx.

shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

Re: 2LevelImpute

Post by shakespeare » Fri Aug 22, 2014 2:18 pm

Follow-up: My IT department clarified the exceptions in which they are interested:

"a list of files/folders, executables, and/or services that they know need to be excluded from being scanned by the virus scanner"

One additional question about the nature of this file: c:\users\sc014459\appdata\local\temp\1\tmp6dsdrs\b3z8uf

I can see the folder c:\users\sc014459, but nothing else. Is this a temporary folder created as Stat-JR processes the data?

shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

Re: 2LevelImpute

Post by shakespeare » Wed Aug 27, 2014 1:23 pm

Second follow-up: Had my IT department exclude from McAfee TREE.exe, DEEP.exe and c:\users\sc014459\appdata along with subfolders. I'm still getting the same error at the same time (around 30 hours):

Traceback (most recent call last):
File "webtest.py", line 416, in go
File "C:\StatJR\packages\Python_script.py", line 71, in run
self.eng.run('script.py')
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Engine.py", line 43, in decorated
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Python.py", line 16, in run
File "", line 259, in
File "c:\Python27\lib\multiprocessing\pool.py", line 554, in get
CompileError: error: c:\users\sc014459\appdata\local\temp\1\tmp6dsdrs\lt3vxu:
The process cannot access the file because it is being used by another process

Guidance appreciated.

shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

Re: 2LevelImpute

Post by shakespeare » Tue Sep 02, 2014 1:47 pm

Third follow-up: Completely disabled McAfee. Am still getting this error:

Traceback (most recent call last):
File "webtest.py", line 416, in go
File "C:\StatJR\packages\Python_script.py", line 71, in run
self.eng.run('script.py')
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Engine.py", line 43, in decorated
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Python.py", line 16, in run
File "", line 259, in
File "c:\Python27\lib\multiprocessing\pool.py", line 554, in get
CompileError: error: c:\users\sc014459\appdata\local\temp\1\tmpsykg2l\llmx7w:
The process cannot access the file because it is being used by another process

Not sure what to do.

Post Reply