drop data that is stored in the r2mlwin object?

[adeldaoud]

Hi

I wonder if it is possible to drop the data that is stored in the r2mlwin object? As I am estimating many models, these objects get very large (~4gb each), and start eating my ram very quickly.

Thanks in advance

[ChrisCharlton]

If you do something like:

Code: Select all

mymodel@data <- data.frame()

then it will replace the stored data with an empty data frame, freeing up the memory used. Note however that if you do this then functions which require the data to be present (such as predictions) will no longer work correctly on the object. You can of course save the data to a file and then manually reinstate it if required.

[adeldaoud]

Thanks, Chris. Two follow-up questions:

1) removing the data reduced the object size by ~15 %. Could you advice about what else I could remove to save space? Checked through the s4 object but it is difficult to know what one can through away if one only wants to keep the model output.

2) If I want to remove the data for all objects in a list, should not the following code work?

Code: Select all

# "result" is a list object produced with doParrallel and R2mlwin. 

a <- lapply(result, function(x){
  x@data <- data.frame()
})

But it doesn´t.

[ChrisCharlton]

1) The other large parts of the object would be the various parameter chains, and residuals if you are storing them. The chains will be needed in order to display the results, so it might be better to just decide which information you want to keep from each model, copy that somewhere else and then remove the model output object from memory.

2) It looks like lapply is copying the object when calling the function. Using the following loop instead works for me:

Code: Select all

for (i in 1:length(result)) {
    result[[i]]@data <- data.frame()
}

[adeldaoud]

Thanks, Chris, for the input. Your for-loop works fine. Just for the sake of details, how would you write a function that removes the data, chains, etc and only keeps the absolute minimum to show the results with the print function?

Experimented a bit, but I am still left with a rather large object.

Many thanks in advance

[ChrisCharlton]

This currently isn't possible, as the print function uses the chains to display the estimates and related statistics. For this to work these quantities would need to be added to the output object and the print function would need to be modified to use this if the chains are not available. I will add this to the TODO list, but I can't make any promises as to when this would be implemented.