Page 1 of 1

memory problem

Posted: Tue Dec 16, 2014 10:54 am
by Gattatkb
Hi,

I have memory problems with my imputations. When I have few variables to be imputed and few auxiliary variables, the imputation works. However, when I increase the number of variables imputed and/or the number of auxiliary variables, the imputation does not run. I receive the following error message: “Error using + Matrix dimensions must agree.” How can I solve the memory issue? Is there for example a Linux version, so that I could run it on a computer cluster? Or does Stat-JR have more memory?

Best wishes.

Re: memory problem

Posted: Tue Dec 16, 2014 2:22 pm
by Gattatkb
Hi,

In 2012 Chris Charlton replied the following on the message "Index exceeds matrix dimensions query":
As far as the maximum memory it can use goes, Realcom-Impute is currently compiled as a 32 bit windows executable. This should mean that on a 32 bit system it can use up to 2Gb and on a 64 bit system up to 4Gb. This of course assumes that Windows is able to provide it with this much using a combination of RAM/Swapfile.
Could someone confirm that the limit of 4GB still exists?

Best wishes.

Re: memory problem

Posted: Tue Dec 16, 2014 2:34 pm
by ChrisCharlton
Yes, this limitation still exists. How many records and variables do you have in your data? How many variables of each type do you have in the model that works versus the one that gives you the error message?

Re: memory problem

Posted: Tue Dec 16, 2014 3:29 pm
by Gattatkb
Hi,

I am working on my smallest cohort at the moment. That cohort has 16744 records. It has one second-level identifier that has 280 categories. I want to impute five variables:

• Y1: continuous, 26% missing
• Y2: continuous, 26% missing
• Y3: continuous, 36% missing
• Y4: binary, 37% missing
• Y5: ordered categorical (3 categories), 29% missing

There are 24 auxiliary variables, including the constant.

If I keep all the auxiliary variables, I can impute 3 out of 5 variables. It does not matter which three variables I impute. If I keep all the imputed variables, I can use up to sixteen auxiliary variables.

Best wishes.

Re: memory problem

Posted: Tue Dec 16, 2014 8:39 pm
by ChrisCharlton
I have checked with Professor Harvey Goldstein, the author of Realcom, and he said that it shouldn't have any difficulties dealing with data of this size. The most likely cause of the problem that you are seeing is due to characteristics of the data such as linear dependencies, or possible model miss-specification.

Re: memory problem

Posted: Wed Dec 17, 2014 6:28 pm
by Gattatkb
Hi Chris,

You are right, I have linear dependencies and it is not a memory problem. Thank you for your help!

Best wishes.