www.cmm.bristol.ac.uk/forum

Welcome to the MLwiN User Forum
https://www.cmm.bristol.ac.uk/forum/

Issues when storing residual chains

https://www.cmm.bristol.ac.uk/forum/viewtopic.php?t=3600

Page 1 of 1

Issues when storing residual chains

Posted: Tue May 18, 2021 9:36 am
by PhilJaehn
Dear all,

I am having trouble storing the residual chains from MCMC estimation in a two-level logistic regression model. I want to do further calculations with the level 2 residual chains. However, when specifying resi.store.levs=2, I get the following error:
Error in matrix(stats::na.omit(residata[, name]), nrow = iterations/thinning, :
length of 'dimnames' [2] not equal to array extension
This is my R code:

Code: Select all

F0=logit(resp) ~ 1 + pred + (1 | level2units)

m=runMLwiN(F0, D="Binomial", data=df, estoptions = list(EstM = 1, resi.store.levs=2))
Residual chains are stored when I run a linear regression with the tutorial data.

You have an idea what the problem could be?

Best wishes,
Philipp

Re: Issues when storing residual chains

Posted: Wed May 19, 2021 11:47 am
by ChrisCharlton
Would it be possible for you to confirm that the following, which should be an equivalent model using the MLwiN example data works for you?

Code: Select all

> library(R2MLwiN)
> data(bang, package = "R2MLwiN")
> f <- logit(use) ~ 1 + age + (1 | district)
> m <- runMLwiN(f, D = "Binomial", data = bang, estoptions = list(EstM = 1, resi.store.levs = 2))
MLwiN is running, please wait......
/nogui option ignored
ECHO    0


Echoing is ON
BATC 1

Batch mode is ON
MAXI 2
STAR
iteration 0
iteration 1

Convergence not achieved
TOLE 2
MAXI 20
BATC 1

Batch mode is ON
NEXT
iteration 2
iteration 3

Convergence achieved
ECHO 0

Echoing is ON
MCMC   0 500 1 5.8 50 10 G30[1] G30[2]  2 2 2 1 1 2
Adapting for 100 iterations (Maximum 5000)
-2 * Loglike = 3664.513281
Adapting for 200 iterations (Maximum 5000)
-2 * Loglike = 3675.038874
Adapting for 300 iterations (Maximum 5000)
-2 * Loglike = 3649.538587
Adapting for 400 iterations (Maximum 5000)
-2 * Loglike = 3667.799866
Adapting for 500 iterations (Maximum 5000)
-2 * Loglike = 3664.345650
Adapting for 600 iterations (Maximum 5000)
-2 * Loglike = 3674.114156
Adapting finished and took 700 iterations
Adapting took 700 iterations
Burning in for 50 iterations out of 500
-2 * Loglike = 3686.568077
Burning in for 100 iterations out of 500
-2 * Loglike = 3661.594681
Burning in for 150 iterations out of 500
-2 * Loglike = 3667.214322
Burning in for 200 iterations out of 500
-2 * Loglike = 3699.944267
Burning in for 250 iterations out of 500
-2 * Loglike = 3679.787350
Burning in for 300 iterations out of 500
-2 * Loglike = 3668.379318
Burning in for 350 iterations out of 500
-2 * Loglike = 3667.081363
Burning in for 400 iterations out of 500
-2 * Loglike = 3686.565542
Burning in for 450 iterations out of 500
-2 * Loglike = 3654.455906
Burning in for 500 iterations out of 500
-2 * Loglike = 3677.422240

ERAS G30
LINK 0 G30
ERAS  c1090 c1091

MCMC 1 5000 1 c1090 c1091 c1003 c1004 1 2
Actual update 50 of 5000, Stored update 50 of 5000
-2 * Loglike = 3669.157477
Actual update 100 of 5000, Stored update 100 of 5000
-2 * Loglike = 3690.762939
Actual update 150 of 5000, Stored update 150 of 5000
-2 * Loglike = 3659.555989
Actual update 200 of 5000, Stored update 200 of 5000
-2 * Loglike = 3667.042260
Actual update 250 of 5000, Stored update 250 of 5000
-2 * Loglike = 3663.279540
Actual update 300 of 5000, Stored update 300 of 5000
-2 * Loglike = 3656.446703
Actual update 350 of 5000, Stored update 350 of 5000
-2 * Loglike = 3666.999108
Actual update 400 of 5000, Stored update 400 of 5000
-2 * Loglike = 3668.527913
Actual update 450 of 5000, Stored update 450 of 5000
-2 * Loglike = 3664.416065
Actual update 500 of 5000, Stored update 500 of 5000
-2 * Loglike = 3673.814288
Actual update 550 of 5000, Stored update 550 of 5000
-2 * Loglike = 3669.170097
Actual update 600 of 5000, Stored update 600 of 5000
-2 * Loglike = 3663.119560
Actual update 650 of 5000, Stored update 650 of 5000
-2 * Loglike = 3669.742923
Actual update 700 of 5000, Stored update 700 of 5000
-2 * Loglike = 3655.928702
Actual update 750 of 5000, Stored update 750 of 5000
-2 * Loglike = 3651.024209
Actual update 800 of 5000, Stored update 800 of 5000
-2 * Loglike = 3676.265238
Actual update 850 of 5000, Stored update 850 of 5000
-2 * Loglike = 3668.511777
Actual update 900 of 5000, Stored update 900 of 5000
-2 * Loglike = 3680.540518
Actual update 950 of 5000, Stored update 950 of 5000
-2 * Loglike = 3676.362890
Actual update 1000 of 5000, Stored update 1000 of 5000
-2 * Loglike = 3683.259330
Actual update 1050 of 5000, Stored update 1050 of 5000
-2 * Loglike = 3671.928428
Actual update 1100 of 5000, Stored update 1100 of 5000
-2 * Loglike = 3670.340797
Actual update 1150 of 5000, Stored update 1150 of 5000
-2 * Loglike = 3673.658199
Actual update 1200 of 5000, Stored update 1200 of 5000
-2 * Loglike = 3649.047230
Actual update 1250 of 5000, Stored update 1250 of 5000
-2 * Loglike = 3655.178298
Actual update 1300 of 5000, Stored update 1300 of 5000
-2 * Loglike = 3676.099379
Actual update 1350 of 5000, Stored update 1350 of 5000
-2 * Loglike = 3675.893582
Actual update 1400 of 5000, Stored update 1400 of 5000
-2 * Loglike = 3656.202755
Actual update 1450 of 5000, Stored update 1450 of 5000
-2 * Loglike = 3648.981335
Actual update 1500 of 5000, Stored update 1500 of 5000
-2 * Loglike = 3671.907135
Actual update 1550 of 5000, Stored update 1550 of 5000
-2 * Loglike = 3677.990007
Actual update 1600 of 5000, Stored update 1600 of 5000
-2 * Loglike = 3665.339884
Actual update 1650 of 5000, Stored update 1650 of 5000
-2 * Loglike = 3682.618214
Actual update 1700 of 5000, Stored update 1700 of 5000
-2 * Loglike = 3674.281851
Actual update 1750 of 5000, Stored update 1750 of 5000
-2 * Loglike = 3660.587857
Actual update 1800 of 5000, Stored update 1800 of 5000
-2 * Loglike = 3670.883736
Actual update 1850 of 5000, Stored update 1850 of 5000
-2 * Loglike = 3668.332403
Actual update 1900 of 5000, Stored update 1900 of 5000
-2 * Loglike = 3671.818755
Actual update 1950 of 5000, Stored update 1950 of 5000
-2 * Loglike = 3670.815446
Actual update 2000 of 5000, Stored update 2000 of 5000
-2 * Loglike = 3664.924050
Actual update 2050 of 5000, Stored update 2050 of 5000
-2 * Loglike = 3661.779303
Actual update 2100 of 5000, Stored update 2100 of 5000
-2 * Loglike = 3681.575585
Actual update 2150 of 5000, Stored update 2150 of 5000
-2 * Loglike = 3664.805556
Actual update 2200 of 5000, Stored update 2200 of 5000
-2 * Loglike = 3663.249264
Actual update 2250 of 5000, Stored update 2250 of 5000
-2 * Loglike = 3669.328327
Actual update 2300 of 5000, Stored update 2300 of 5000
-2 * Loglike = 3671.101815
Actual update 2350 of 5000, Stored update 2350 of 5000
-2 * Loglike = 3654.358531
Actual update 2400 of 5000, Stored update 2400 of 5000
-2 * Loglike = 3657.859758
Actual update 2450 of 5000, Stored update 2450 of 5000
-2 * Loglike = 3643.240362
Actual update 2500 of 5000, Stored update 2500 of 5000
-2 * Loglike = 3674.082202
Actual update 2550 of 5000, Stored update 2550 of 5000
-2 * Loglike = 3684.617607
Actual update 2600 of 5000, Stored update 2600 of 5000
-2 * Loglike = 3668.119802
Actual update 2650 of 5000, Stored update 2650 of 5000
-2 * Loglike = 3658.854313
Actual update 2700 of 5000, Stored update 2700 of 5000
-2 * Loglike = 3663.035269
Actual update 2750 of 5000, Stored update 2750 of 5000
-2 * Loglike = 3665.758922
Actual update 2800 of 5000, Stored update 2800 of 5000
-2 * Loglike = 3686.073061
Actual update 2850 of 5000, Stored update 2850 of 5000
-2 * Loglike = 3660.994303
Actual update 2900 of 5000, Stored update 2900 of 5000
-2 * Loglike = 3686.100308
Actual update 2950 of 5000, Stored update 2950 of 5000
-2 * Loglike = 3675.657456
Actual update 3000 of 5000, Stored update 3000 of 5000
-2 * Loglike = 3664.530635
Actual update 3050 of 5000, Stored update 3050 of 5000
-2 * Loglike = 3677.985721
Actual update 3100 of 5000, Stored update 3100 of 5000
-2 * Loglike = 3672.863453
Actual update 3150 of 5000, Stored update 3150 of 5000
-2 * Loglike = 3658.006029
Actual update 3200 of 5000, Stored update 3200 of 5000
-2 * Loglike = 3662.529219
Actual update 3250 of 5000, Stored update 3250 of 5000
-2 * Loglike = 3685.448858
Actual update 3300 of 5000, Stored update 3300 of 5000
-2 * Loglike = 3658.961703
Actual update 3350 of 5000, Stored update 3350 of 5000
-2 * Loglike = 3692.839117
Actual update 3400 of 5000, Stored update 3400 of 5000
-2 * Loglike = 3671.752911
Actual update 3450 of 5000, Stored update 3450 of 5000
-2 * Loglike = 3661.961702
Actual update 3500 of 5000, Stored update 3500 of 5000
-2 * Loglike = 3662.482558
Actual update 3550 of 5000, Stored update 3550 of 5000
-2 * Loglike = 3667.867827
Actual update 3600 of 5000, Stored update 3600 of 5000
-2 * Loglike = 3675.799613
Actual update 3650 of 5000, Stored update 3650 of 5000
-2 * Loglike = 3654.314314
Actual update 3700 of 5000, Stored update 3700 of 5000
-2 * Loglike = 3694.211916
Actual update 3750 of 5000, Stored update 3750 of 5000
-2 * Loglike = 3687.939762
Actual update 3800 of 5000, Stored update 3800 of 5000
-2 * Loglike = 3656.649947
Actual update 3850 of 5000, Stored update 3850 of 5000
-2 * Loglike = 3645.126272
Actual update 3900 of 5000, Stored update 3900 of 5000
-2 * Loglike = 3671.422200
Actual update 3950 of 5000, Stored update 3950 of 5000
-2 * Loglike = 3673.229740
Actual update 4000 of 5000, Stored update 4000 of 5000
-2 * Loglike = 3657.888886
Actual update 4050 of 5000, Stored update 4050 of 5000
-2 * Loglike = 3669.300364
Actual update 4100 of 5000, Stored update 4100 of 5000
-2 * Loglike = 3680.648578
Actual update 4150 of 5000, Stored update 4150 of 5000
-2 * Loglike = 3658.576698
Actual update 4200 of 5000, Stored update 4200 of 5000
-2 * Loglike = 3660.124417
Actual update 4250 of 5000, Stored update 4250 of 5000
-2 * Loglike = 3653.214706
Actual update 4300 of 5000, Stored update 4300 of 5000
-2 * Loglike = 3668.825515
Actual update 4350 of 5000, Stored update 4350 of 5000
-2 * Loglike = 3668.790373
Actual update 4400 of 5000, Stored update 4400 of 5000
-2 * Loglike = 3666.745836
Actual update 4450 of 5000, Stored update 4450 of 5000
-2 * Loglike = 3666.585785
Actual update 4500 of 5000, Stored update 4500 of 5000
-2 * Loglike = 3660.311943
Actual update 4550 of 5000, Stored update 4550 of 5000
-2 * Loglike = 3673.205105
Actual update 4600 of 5000, Stored update 4600 of 5000
-2 * Loglike = 3662.646247
Actual update 4650 of 5000, Stored update 4650 of 5000
-2 * Loglike = 3662.548578
Actual update 4700 of 5000, Stored update 4700 of 5000
-2 * Loglike = 3657.482480
Actual update 4750 of 5000, Stored update 4750 of 5000
-2 * Loglike = 3661.774784
Actual update 4800 of 5000, Stored update 4800 of 5000
-2 * Loglike = 3674.294638
Actual update 4850 of 5000, Stored update 4850 of 5000
-2 * Loglike = 3659.444877
Actual update 4900 of 5000, Stored update 4900 of 5000
-2 * Loglike = 3660.129958
Actual update 4950 of 5000, Stored update 4950 of 5000
-2 * Loglike = 3659.726315
Actual update 5000 of 5000, Stored update 5000 of 5000
-2 * Loglike = 3682.569027
PUPN c1003 c1004
AVER c1091 b99 b100

Count   =           5000
Average =          3668.0
S.D.    =          10.962
S.E.M.  =         0.15503

ECHO 0
Execution completed

> summary(m@resi.chains$resi_lev2)

Iterations = 1:5000
Thinning interval = 1 
Number of chains = 1 
Sample size per chain = 5000 

1. Empirical mean and standard deviation for each variable,
   plus standard error of the mean:

            Mean     SD Naive SE Time-series SE
u_0_1  -0.458134 0.1825 0.002581       0.007736
u_0_2  -0.448389 0.3250 0.004597       0.010665
u_0_3   0.205684 0.4881 0.006903       0.015413
u_0_4   0.177572 0.2684 0.003796       0.008854
u_0_5   0.105015 0.2635 0.003727       0.008975
u_0_6  -0.333035 0.2182 0.003086       0.007187
u_0_7  -0.230364 0.3355 0.004744       0.010042
u_0_8   0.206828 0.2435 0.003443       0.007803
u_0_9   0.007713 0.2993 0.004233       0.009437
u_0_10 -0.701055 0.4020 0.005685       0.013758
u_0_11 -1.025663 0.3973 0.005619       0.012805
u_0_12 -0.181517 0.2754 0.003895       0.008726
u_0_13 -0.119728 0.2862 0.004048       0.008678
u_0_14  0.990507 0.1736 0.002455       0.009352
u_0_15  0.154600 0.3070 0.004342       0.009532
u_0_16  0.478458 0.3172 0.004486       0.009942
u_0_17 -0.191479 0.2908 0.004112       0.008986
u_0_18 -0.107678 0.2460 0.003479       0.008312
u_0_19 -0.091118 0.2735 0.003868       0.008784
u_0_20 -0.064001 0.3332 0.004712       0.010453
u_0_21 -0.065431 0.3203 0.004530       0.010529
u_0_22 -0.415348 0.3473 0.004912       0.011125
u_0_23 -0.372913 0.3469 0.004906       0.010675
u_0_24 -0.610874 0.4048 0.005724       0.012710
u_0_25  0.278112 0.2085 0.002948       0.009072
u_0_26  0.002928 0.3647 0.005157       0.011061
u_0_27 -0.688298 0.2669 0.003774       0.009743
u_0_28 -0.437259 0.2302 0.003256       0.008235
u_0_29 -0.234661 0.2809 0.003973       0.009530
u_0_30  0.352584 0.1966 0.002780       0.008944
u_0_31  0.205744 0.2400 0.003394       0.007355
u_0_32 -0.449208 0.3197 0.004521       0.010492
u_0_33  0.176187 0.3564 0.005041       0.011094
u_0_34  0.890955 0.2957 0.004181       0.010886
u_0_35  0.426823 0.2427 0.003432       0.007670
u_0_36 -0.562676 0.3288 0.004649       0.010300
u_0_37  0.544954 0.3797 0.005370       0.013180
u_0_38 -0.275673 0.3824 0.005407       0.011869
u_0_39  0.617692 0.2974 0.004205       0.009935
u_0_40  0.264772 0.2400 0.003394       0.007408
u_0_41  0.231205 0.2939 0.004156       0.009555
u_0_42  0.470649 0.3909 0.005529       0.011900
u_0_43  0.709908 0.2366 0.003346       0.009872
u_0_44 -0.549111 0.3296 0.004662       0.010142
u_0_45  0.104335 0.2749 0.003888       0.010568
u_0_46  0.465912 0.1906 0.002695       0.008120
u_0_47  0.384260 0.3587 0.005073       0.012064
u_0_48  0.526294 0.2433 0.003441       0.009490
u_0_49 -0.115629 0.4521 0.006394       0.013768
u_0_50  0.393097 0.3252 0.004599       0.010766
u_0_51  0.308005 0.2645 0.003740       0.009198
u_0_52  0.322801 0.2145 0.003033       0.008390
u_0_53  0.012495 0.3448 0.004876       0.011205
u_0_54 -0.244853 0.4413 0.006241       0.013109
u_0_55  1.044065 0.2608 0.003688       0.010718
u_0_56 -0.487930 0.2868 0.004056       0.008953
u_0_57  0.158462 0.2738 0.003872       0.009115
u_0_58 -0.250409 0.3799 0.005373       0.011763
u_0_59 -0.160288 0.2688 0.003801       0.008044
u_0_60 -0.358112 0.2637 0.003729       0.009379

2. Quantiles for each variable:

           2.5%      25%       50%       75%     97.5%
u_0_1  -0.83197 -0.58377 -0.453769 -0.334627 -0.100284
u_0_2  -1.12877 -0.65640 -0.437310 -0.225007  0.158028
u_0_3  -0.74740 -0.11301  0.202043  0.534916  1.158240
u_0_4  -0.34109 -0.01065  0.177372  0.354776  0.702519
u_0_5  -0.38614 -0.07932  0.100110  0.287144  0.622100
u_0_6  -0.77384 -0.47556 -0.326650 -0.181899  0.077483
u_0_7  -0.88078 -0.46032 -0.238205  0.003834  0.428506
u_0_8  -0.27723  0.03915  0.207953  0.369147  0.682832
u_0_9  -0.57917 -0.20049  0.014003  0.211992  0.578642
u_0_10 -1.55828 -0.94964 -0.682941 -0.424791  0.046348
u_0_11 -1.85638 -1.27404 -1.011984 -0.736860 -0.295525
u_0_12 -0.71764 -0.36460 -0.184496  0.006459  0.358724
u_0_13 -0.68288 -0.31336 -0.115741  0.079314  0.417048
u_0_14  0.66453  0.87125  0.990110  1.101741  1.352699
u_0_15 -0.45629 -0.04616  0.149643  0.367483  0.754268
u_0_16 -0.16430  0.26660  0.484531  0.689499  1.107564
u_0_17 -0.75157 -0.39521 -0.182218  0.009870  0.358007
u_0_18 -0.59486 -0.26747 -0.106194  0.052425  0.388038
u_0_19 -0.61506 -0.28471 -0.092972  0.109771  0.437304
u_0_20 -0.75075 -0.27536 -0.056439  0.166027  0.571247
u_0_21 -0.68449 -0.28831 -0.063846  0.149168  0.561796
u_0_22 -1.11530 -0.64838 -0.400777 -0.171878  0.224982
u_0_23 -1.03669 -0.61878 -0.367690 -0.134439  0.309260
u_0_24 -1.41809 -0.88389 -0.603174 -0.316183  0.121217
u_0_25 -0.11725  0.13673  0.275865  0.421773  0.696074
u_0_26 -0.72756 -0.23291  0.001719  0.233864  0.730602
u_0_27 -1.22769 -0.86759 -0.687703 -0.502590 -0.168136
u_0_28 -0.88190 -0.59322 -0.425266 -0.279324  0.003607
u_0_29 -0.79458 -0.41425 -0.231115 -0.056453  0.309746
u_0_30 -0.02640  0.22019  0.349911  0.481584  0.763770
u_0_31 -0.25966  0.04735  0.210114  0.353872  0.695036
u_0_32 -1.07169 -0.67259 -0.444090 -0.220104  0.160235
u_0_33 -0.49495 -0.06633  0.172255  0.413151  0.894935
u_0_34  0.30936  0.69788  0.890459  1.077885  1.498042
u_0_35 -0.05708  0.25704  0.430821  0.591454  0.901804
u_0_36 -1.21573 -0.77462 -0.547616 -0.338077  0.057216
u_0_37 -0.20408  0.28793  0.536989  0.798905  1.303553
u_0_38 -1.04568 -0.53406 -0.265525 -0.027311  0.486679
u_0_39  0.04935  0.42217  0.613493  0.810934  1.228556
u_0_40 -0.19347  0.09499  0.254834  0.423536  0.720623
u_0_41 -0.35384  0.02948  0.237657  0.426341  0.815847
u_0_42 -0.27362  0.21170  0.456172  0.710916  1.275744
u_0_43  0.25557  0.54841  0.709225  0.861571  1.182722
u_0_44 -1.22909 -0.76740 -0.523897 -0.327473  0.079335
u_0_45 -0.47146 -0.07065  0.109485  0.287100  0.633941
u_0_46  0.10165  0.34009  0.466894  0.586928  0.857343
u_0_47 -0.31163  0.13049  0.379903  0.618616  1.120280
u_0_48  0.05188  0.37080  0.521297  0.689609  1.004935
u_0_49 -0.96252 -0.41598 -0.125847  0.188332  0.766450
u_0_50 -0.22287  0.17086  0.381537  0.596876  1.063914
u_0_51 -0.21611  0.13615  0.309293  0.484640  0.831334
u_0_52 -0.09621  0.17584  0.325612  0.465715  0.744192
u_0_53 -0.65524 -0.22721  0.020296  0.235135  0.678905
u_0_54 -1.14092 -0.54273 -0.224948  0.045622  0.619311
u_0_55  0.51828  0.87202  1.034364  1.225256  1.550763
u_0_56 -1.06305 -0.67854 -0.477751 -0.302533  0.078348
u_0_57 -0.36142 -0.02857  0.154032  0.334027  0.693253
u_0_58 -0.98589 -0.50944 -0.238941  0.004218  0.525048
u_0_59 -0.66585 -0.34349 -0.162197  0.017408  0.346201
u_0_60 -0.90571 -0.53458 -0.355667 -0.175062  0.149264

Re: Issues when storing residual chains

Posted: Sun May 23, 2021 12:09 pm
by PhilJaehn
Dear Chris,

thank you for the quick answer. Yes, the code you posted works with the example data.

I tried to work with MLwiN and figured out that I need a data frame without missing values. When I delete missings before running runMLwiN in R, I can store and view the residual chains. Sorry, I overlooked this requirement.

Best wishes,
Philipp
All times are UTC
Page 1 of 1

Powered by phpBB® Forum Software © phpBB Limited