memory problem

Welcome to the forum for REALCOM users. Feel free to post your question about REALCOM here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Go REALCOM (Developing multilevel models for REAListically COMplex social science data) >> http://www.bristol.ac.uk/cmm/software/realcom/
Post Reply
Gattatkb
Posts: 4
Joined: Mon Dec 15, 2014 3:49 pm

memory problem

Post by Gattatkb » Tue Dec 16, 2014 10:54 am

Hi,

I have memory problems with my imputations. When I have few variables to be imputed and few auxiliary variables, the imputation works. However, when I increase the number of variables imputed and/or the number of auxiliary variables, the imputation does not run. I receive the following error message: “Error using + Matrix dimensions must agree.” How can I solve the memory issue? Is there for example a Linux version, so that I could run it on a computer cluster? Or does Stat-JR have more memory?

Best wishes.

Gattatkb
Posts: 4
Joined: Mon Dec 15, 2014 3:49 pm

Re: memory problem

Post by Gattatkb » Tue Dec 16, 2014 2:22 pm

Hi,

In 2012 Chris Charlton replied the following on the message "Index exceeds matrix dimensions query":
As far as the maximum memory it can use goes, Realcom-Impute is currently compiled as a 32 bit windows executable. This should mean that on a 32 bit system it can use up to 2Gb and on a 64 bit system up to 4Gb. This of course assumes that Windows is able to provide it with this much using a combination of RAM/Swapfile.
Could someone confirm that the limit of 4GB still exists?

Best wishes.

ChrisCharlton
Posts: 1103
Joined: Mon Oct 19, 2009 10:34 am

Re: memory problem

Post by ChrisCharlton » Tue Dec 16, 2014 2:34 pm

Yes, this limitation still exists. How many records and variables do you have in your data? How many variables of each type do you have in the model that works versus the one that gives you the error message?

Gattatkb
Posts: 4
Joined: Mon Dec 15, 2014 3:49 pm

Re: memory problem

Post by Gattatkb » Tue Dec 16, 2014 3:29 pm

Hi,

I am working on my smallest cohort at the moment. That cohort has 16744 records. It has one second-level identifier that has 280 categories. I want to impute five variables:

• Y1: continuous, 26% missing
• Y2: continuous, 26% missing
• Y3: continuous, 36% missing
• Y4: binary, 37% missing
• Y5: ordered categorical (3 categories), 29% missing

There are 24 auxiliary variables, including the constant.

If I keep all the auxiliary variables, I can impute 3 out of 5 variables. It does not matter which three variables I impute. If I keep all the imputed variables, I can use up to sixteen auxiliary variables.

Best wishes.

ChrisCharlton
Posts: 1103
Joined: Mon Oct 19, 2009 10:34 am

Re: memory problem

Post by ChrisCharlton » Tue Dec 16, 2014 8:39 pm

I have checked with Professor Harvey Goldstein, the author of Realcom, and he said that it shouldn't have any difficulties dealing with data of this size. The most likely cause of the problem that you are seeing is due to characteristics of the data such as linear dependencies, or possible model miss-specification.

Gattatkb
Posts: 4
Joined: Mon Dec 15, 2014 3:49 pm

Re: memory problem

Post by Gattatkb » Wed Dec 17, 2014 6:28 pm

Hi Chris,

You are right, I have linear dependencies and it is not a memory problem. Thank you for your help!

Best wishes.

Post Reply