Page 1 of 3

2LevelImpute

Posted: Thu Aug 07, 2014 12:11 pm
by shakespeare
If one reads the imputation literature, inclusion of the MOI outcome measure in the imputation model is suggested, either as a response (when incomplete), or explanatory (when complete) variable. I was able to successfully do this in REALCOM. I wanted to make sure that the 2LevelImpute template is written to accommodate this point of view. Thx.

Re: 2LevelImpute

Posted: Thu Aug 07, 2014 6:49 pm
by shakespeare
Follow-up: I'm not sure including my outcome as an explanatory variable is feasible, given the documentation. For now, I've included it only as a response in the MOI. I'm attempting to run a random intercept/random slope model. I get the error below. I'm not sure if it's because the program is not working correctly (I had some difficulty getting it to run the example problems) or if this indicates my set up is faulty. Would appreciate feedback.

Traceback (most recent call last):
File "webtest.py", line 416, in go
File "C:\StatJR\packages\Python_script.py", line 71, in run
self.eng.run('script.py')
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Engine.py", line 43, in decorated
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Python.py", line 16, in run
File "", line 189, in
File "O:\StatJR\estat\trunk\src\apps\cmdtest\cmdtest.py", line 127, in RunStatJR
File "C:\StatJR\packages\CustomC.py", line 587, in init
self.applydata()
File "C:\StatJR\packages\CustomC.py", line 318, in applydata
missmask[d] = missmask[d] | self.dsets[d].variables[v]['data'].mask
KeyError: 'r'

Re: 2LevelImpute

Posted: Fri Aug 08, 2014 8:59 am
by richardparker
Hi - the 2LevelImpute template does support the inclusion of the MOI's outcome variable in the imputation model (e.g. in the accompanying eBook, the MOI's outcome variable 'normexam' is included as an explanatory variable in the imputation model).

Sorry you got an error message. Do your variable names have any non-alphanumeric characters in them (or are you able to send a list of their names)? In addition, can I just check if you're adding any variables with missing values as explanatory variables in your imputation model?

Thanks,

Richard

Re: 2LevelImpute

Posted: Fri Aug 08, 2014 10:46 am
by shakespeare
No missing values in my explanatory variables, but some of my variable names have underscores. Should I remove the underscores and use only alphanumeric characters in the variable names?

Re: 2LevelImpute

Posted: Fri Aug 08, 2014 12:51 pm
by richardparker
Hi - yes, that would be worth trying.

Richard

Re: 2LevelImpute

Posted: Tue Aug 12, 2014 8:58 pm
by shakespeare
Removing the underscores seems to have had the desired effect. My program ran for about 24 hours until it threw the error below. I had the same problem when I tried to run the example problem. It eventually ran, but I was never really clear what causes this error and how to fix it. One suggestion was a virus scanner. I'll check with my IT department to see if there is virus software. Any other ideas would be appreciated. Thx.


Traceback (most recent call last): File "webtest.py", line 416, in go File "C:\StatJR\packages\Python_script.py", line 71,
in run self.eng.run('script.py') File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Engine.py", line 43,
in decorated File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Python.py", line 16,
in run File "", line 259, in File "c:\Python27\lib\multiprocessing\pool.py", line 554,
in get CompileError: error: c:\users\sc014459\appdata\local\temp\4\tmpv1qly6\zkfzgu:
The process cannot access the file because it is being used by another process

Re: 2LevelImpute

Posted: Fri Aug 22, 2014 1:04 am
by shakespeare
We changed the version of our anti-virus program (McAfee) to one that does no on-box scanning. I'm still getting this message after running about 24 hours:

Traceback (most recent call last):
File "webtest.py", line 416, in go
File "C:\StatJR\packages\Python_script.py", line 71, in run
self.eng.run('script.py')
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Engine.py", line 43, in decorated
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Python.py", line 16, in run
File "", line 259, in
File "c:\Python27\lib\multiprocessing\pool.py", line 554, in get
CompileError: error: c:\users\sc014459\appdata\local\temp\1\tmp6dsdrs\b3z8uf: The process cannot access the file because it is being used by another process

My IT department is requesting a list of exceptions. I have some SAS data on this machine, but was not accessing at it the time of the software crash. My SAS software runs on another box anyway. Is this a compiler error? I have R and SRCware, the stand alone version of IVEware, installed on this box, but neither were running at the time of the crash, so I don't think that's the problem. Seems like the ant-virus is still the best suspect, but I'd appreciate feedback. Thoughts? Thx.

Re: 2LevelImpute

Posted: Fri Aug 22, 2014 2:18 pm
by shakespeare
Follow-up: My IT department clarified the exceptions in which they are interested:

"a list of files/folders, executables, and/or services that they know need to be excluded from being scanned by the virus scanner"

One additional question about the nature of this file: c:\users\sc014459\appdata\local\temp\1\tmp6dsdrs\b3z8uf

I can see the folder c:\users\sc014459, but nothing else. Is this a temporary folder created as Stat-JR processes the data?

Re: 2LevelImpute

Posted: Wed Aug 27, 2014 1:23 pm
by shakespeare
Second follow-up: Had my IT department exclude from McAfee TREE.exe, DEEP.exe and c:\users\sc014459\appdata along with subfolders. I'm still getting the same error at the same time (around 30 hours):

Traceback (most recent call last):
File "webtest.py", line 416, in go
File "C:\StatJR\packages\Python_script.py", line 71, in run
self.eng.run('script.py')
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Engine.py", line 43, in decorated
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Python.py", line 16, in run
File "", line 259, in
File "c:\Python27\lib\multiprocessing\pool.py", line 554, in get
CompileError: error: c:\users\sc014459\appdata\local\temp\1\tmp6dsdrs\lt3vxu:
The process cannot access the file because it is being used by another process

Guidance appreciated.

Re: 2LevelImpute

Posted: Tue Sep 02, 2014 1:47 pm
by shakespeare
Third follow-up: Completely disabled McAfee. Am still getting this error:

Traceback (most recent call last):
File "webtest.py", line 416, in go
File "C:\StatJR\packages\Python_script.py", line 71, in run
self.eng.run('script.py')
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Engine.py", line 43, in decorated
File "O:\StatJR\estat\trunk\src\lib\EStat\engines\Python.py", line 16, in run
File "", line 259, in
File "c:\Python27\lib\multiprocessing\pool.py", line 554, in get
CompileError: error: c:\users\sc014459\appdata\local\temp\1\tmpsykg2l\llmx7w:
The process cannot access the file because it is being used by another process

Not sure what to do.