Scene Background Initialization (SBI) dataset

The SBI dataset has been assembled in order to move the first steps towards evaluating and comparing the results of background initialization algorithms and adopted for the SBMI2015 Workshop and the Special Issue of Pattern Recognition Letters Journal on Scene Background Modeling and Initialization (2016). A description and some results can be found in our work [1]. The dataset includes:

A) a dataset of 14 image sequences and the corresponding 14 ground truth backgrounds (updated on January 25, 2016);

B) Matlab scripts (updated on July 26, 2016) for evaluating background initialization results.

Results of methods reported in [2] are also available (updated on November 30, 2016).

A) Dataset of 14 image sequences and corresponding ground truth backgrounds

The 14 image sequences have been extracted by original publicly available sequences that are frequently used in the literature to evaluate background initialization algorithms:

Name Dataset Original
frames		 Used
frames		 Original
resolution		 Final
resolution		 Brief description
and issues
BoardPBI0-227 0-227 200x164 200x164Man moving in front of a dashboard, with mild shadows
Candela_m1.10Candela 0-855 85-435 352x288 352x288Man entering and leaving a room, abandoning a bag for most of the frames
CAVIAR1CAVIAR 0-725 115-724 384x288 384x256People slowly walking along a corridor, with mild shadows
CAVIAR2CAVIAR 0-1500 900-1360 384x288 384x256People entering and leaving a store, standing only for few frames
CaVignalPBI0-257 0-257 200x136 200x136Man standing for most of the frames and then moving
FoliagePBI0-399 6-399 200x148 200x144Parked cars occluded by big waving leaves
Hall&MonitorCOST 2110-299 4-299 352x240 352x240Walking person and abandoned bag in the same image region for most of the frames
HighwayIATON 0-439 0-439 320x240 320x240Fast motion of cars along a highway, with strong shadows and small camera jitter
HighwayIIATON 0-499 0-499 320x240 320x240Fast motion of cars along a highway, with strong shadows and small camera jitter
HumanBody2RPI ISL 0-898 70-810 320x240 320x240People quickly walking indoor, with mild shadows
IBMtest2IBM0-1750 1027-1117 320x240 320x240People quickly walking along indoor corridors
People&FoliagePBI0-349 0-340 320x240 320x240Parked cars occluded by moving people and big waving leaves
SnellenPBI 0-333 0-320 146x150 144x144Stationary Snellen chart occluded and shadowed by big waving leaves
ToscanaMPI Informatik 0-5 0-5 2272x1704 800x600Very few outdoor pictures of pedestrians taken at different times

The subsets of used frames have been selected in order to avoid the inclusion into the testing sequences of empty frames (frames not including foreground objects), while the final resolution has been chosen in order to avoid problems in the computation of boundary patches for block-based methods.

The ground truths have been manually obtained by choosing one of the sequence frames free of foreground objects (not included into the subsets of used frames), by stitching together empty background regions from different sequence frames, or by temporal median of background regions.
1. Board
(PBI data set)
Sequence: PNG video frames (zipped)
Ground truth: Background
2. Candela_m1.10
(Candela data set)
Sequence: PNG video frames (zipped)
Ground truth: Background
3. CAVIAR1
(CAVIAR Test Case Scenarios)
Sequence: PNG video frames (zipped)
Ground truth: Background
4. CAVIAR2
(CAVIAR Test Case Scenarios)
Sequence: PNG video frames (zipped)
Ground truth: Background
5. CaVignal
(PBI data set)
Sequence: PNG video frames (zipped)
Ground truth: Background
6. Foliage
(PBI data set)
Sequence: PNG video frames (zipped)
Ground truth: Background
7. Hall&Monitor
(COST 211 data set)
Sequence: PNG video frames (zipped)    
Ground truth: Background
8. HighwayI
(ATON data set)
Sequence: PNG video frames (zipped)
Ground truth: Background
9. HighwayII
(ATON data set)
Sequence: PNG video frames (zipped)
Ground truth: Background
10. HumanBody2
(RPI ISL data set)
Sequence: PNG video frames (zipped)
Ground truth: Background
11. IBMtest2
(IBM Research - PeopleVision data set)
Sequence: PNG video frames (zipped)
Ground truth: Background
12. People&Foliage
(PBI data set)
Sequence: PNG video frames (zipped)
Ground truth: Background
13. Snellen
(PBI data set)
Sequence: PNG video frames (zipped)
Ground truth: Background
14. Toscana
(MPI Informatik data set)
Sequence: PNG video frames (zipped)
Ground truth: Background
Click here to download all the 14 ground truths (zipped).

B) Matlab scripts for evaluating background initialization results

Matlab scripts (updated on July 26, 2016) are provided for evaluating results in terms of six metrics that include those used in the literature for background estimation. Denoting with GT an image containing the true background and with CB the estimated background image computed with a background initialization method, these six metrics are defined as follows:

  1. Average Gray-level Error (AGE): It is the average of the gray-level absolute difference between GT and CB images. Its values range in [0, L-1], where L is the maximum number of grey levels; the lower the AGE value, the better is the background estimate.

  2. Percentage of Error Pixels (pEPs): An error pixel (EP) is a pixel of CB whose value differs from the value of the corresponding pixel in GT by more than some threshold th (in the experiments the value th=20 has been suggested). pEPs is the ratio between the EPs and the number N of image pixels. Its values range in [0, 1]; the lower the pEPs value, the better is the background estimate.

  3. Percentage of Clustered Error Pixels (pCEPs): A clustered error pixel (CEP) is defined as any error pixel whose 4-connected neighbors are also error pixels. pCEPs is the ratio between the CEPs and the number N of image pixels. Its values range in [0,1]; the lower the pCEPs value, the better is the background estimate.

  4. Peak-Signal-to-Noise-Ratio (PSNR): It is defined as PSNR = 10 log10((L-1)2/MSE), where L is the maximum number of grey levels and MSE is the Mean Squared Error between GT and CB images. It assumes values in decibels; the higher the PSNR value, the better is the background estimate.

  5. Multi-Scale Structural Similarity Index (MS-SSIM): This is the metric defined by Z. Wang, E. P. Simoncelli and A. C. Bovik (link), that uses structural distortion as an estimate of the perceived visual distortion. It assumes values in [-1; 1]; the higher the value of MS-SSIM, the better is the estimated background.

  6. Color image Quality Measure (CQM): This is the metric recently proposed by Y. Yalman and I. Erturk (link), based on a reversible transformation of the YUV color space and on the PSNR computed in the single YUV bands. As for the PSNR, it assumes values in decibels; the higher the CQM value, the better is the background estimate.

If you have problems downloading, please contact lucia.maddalena@cnr.it; if you use the SBI dataset, please cite our works [1], [2].

Results of methods reported in [2]

Here, we report all the results of the background initialization methods compared in [2] for each sequence of the SBI dataset, so that each new method can be easily compared with those considered in [2]. Table 1 reports average accuracy results obtained by the compared methods according to the adopted metrics, while Tables 2 through 15 report accuracy results of all the compared methods on each SBI sequence, as well as their average per sequence. In all the Tables, the best and the second best results for each metric and each sequence appear in red and blue, respectively.

Table 1: Average accuracy results of the compared methods on all sequences of the SBI dataset.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 14.1944 22.5150 18.4428 0.8737 25.6980 43.5839
Color Median 10.3744 13.4008 10.5571 0.8533 28.0044 42.4746
MOG2 14.3579 4.0847 2.8080 0.8935 25.9576 38.1916
KNN 20.6968 7.5118 4.5180 0.7595 18.4701 26.3836
BE-AAPSA 11.4846 12.5518 10.0605 0.9247 27.8024 41.8124
WS2006 5.2885 3.5335 1.2118 0.9349 28.8791 39.6334
IMBS-MT 4.2092 3.8819 2.2602 0.9598 33.4090 44.9362
LaBGen 2.99451.39720.9246 0.9764 35.2028 47.2947
RSL2011 5.8228 5.3511 4.0186 0.9172 29.9272 40.5713
Photomontage 5.8238 4.6952 3.7274 0.9334 31.8573 43.9038
LRGeomCG 8.7644 14.1305 11.0810 0.9302 28.9596 45.5625
TMac 8.8685 14.3577 11.2884 0.9284 28.7507 45.4125
SC-SOBS_1 3.5023 4.1508 2.2295 0.9765 35.2723 50.1138
SC-SOBS_2 4.6049 4.7435 2.5370 0.9645 32.2024 45.7614
BEWIS 3.8665 2.4286 1.4238 0.9675 32.0143 44.3728

Table 2: Accuracy results of all the compared methods on sequence Board and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 24.9527 45.4055 37.7530 0.6100 17.9834 44.0696
Color Median 17.9861 23.4360 20.1433 0.4549 17.7707 43.5605
MOG2 21.5981 23.3689 15.2652 0.8433 17.0541 29.2805
KNN 31.1259 26.8963 17.2561 0.7734 13.4368 21.3434
BE-AAPSA 21.6749 0.2904 0.2459 0.7705 16.9839 32.3073
WS2006 8.1210 6.0945 0.6799 0.8007 23.5534 34.6697
IMBS-MT 2.2537 0.3201 0.0061 0.9836 36.8244 52.4920
LaBGen 5.7214 2.7287 0.7561 0.9054 29.6545 50.7244
RSL2011 7.3911 5.3079 2.6616 0.9310 25.3463 29.0085
Photomontage 6.0480 2.4116 0.6829 0.9529 29.4804 50.4760
LRGeomCG 18.8829 35.9543 27.5213 0.6754 20.1325 43.7282
TMac 19.0408 36.2409 27.8293 0.6726 20.0735 43.7571
SC-SOBS_1 4.7184 4.5671 0.1616 0.9273 29.9489 54.7757
SC-SOBS_2 6.5834 5.0000 0.1829 0.8898 28.6532 51.2832
BEWIS 5.5714 2.2165 0.5274 0.9514 29.9511 50.4528
Average 13.4447 14.6826 10.1115 0.8095 23.7898 42.1286

Table 3: Accuracy results of all the compared methods on sequence Candela\_m1.10 and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 3.1270 3.0086 1.9423 0.9503 29.4636 45.0242
Color Median 3.3225 3.3519 2.1928 0.9382 27.5054 39.9418
MOG2 1.7044 0.7694 0.6185 0.9914 34.0895 46.9634
KNN 11.2176 10.0507 5.8179 0.8158 17.3467 23.1109
BE-AAPSA 2.2656 0.0116 0.0065 0.9733 31.9643 47.2827
WS2006 2.5528 1.9048 0.9657 0.9636 29.6869 40.4489
IMBS-MT 1.3823 0.4705 0.0957 0.9893 35.4288 44.2374
LaBGen 2.5700 1.6809 1.2676 0.9709 30.3140 39.7975
RSL2011 1.5767 0.3748 0.2358 0.9916 36.3572 43.9371
Photomontage 3.6780 3.5837 2.3763 0.9332 26.8665 38.8983
LRGeomCG 1.9037 0.6579 0.4991 0.9912 33.8805 45.0354
TMac 2.0456 1.0150 0.7842 0.9888 32.5507 43.7920
SC-SOBS_1 1.8472 0.8986 0.5080 0.9775 32.6782 49.9181
SC-SOBS_2 3.0125 2.2204 1.2015 0.9532 28.9964 40.4835
BEWIS 1.9049 0.7931 0.4350 0.9852 34.0806 41.6700
Average 2.9407 2.0528 1.2631 0.9609 30.7473 42.0361

Table 4: Accuracy results of all the compared methods on sequence CAVIAR1 and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 5.2178 5.1097 4.0141 0.9382 29.3866 51.3920
Color Median 2.6858 0.3438 0.2421 0.9918 34.8191 51.6298
MOG2 3.1274 2.8412 2.3621 0.9722 29.3055 41.4591
KNN 4.6259 4.0415 2.8463 0.9348 24.3250 29.9878
BE-AAPSA 3.6881 0.0091 0.0037 0.9667 32.4477 51.0556
WS2006 2.6638 0.1261 0.0071 0.9932 35.8184 49.3176
IMBS-MT 1.2267 0.0539 0.0214 0.9967 42.2244 55.0816
LaBGen 3.8243 0.6327 0.4679 0.9819 31.5534 49.2123
RSL2011 2.3295 0.1597 0.0397 0.9947 37.9348 52.3607
Photomontage 2.6498 0.1333 0.0651 0.9933 37.1385 50.0340
LRGeomCG 5.6735 6.6274 5.4830 0.9120 28.1790 51.1680
TMac 5.6945 6.7017 5.5593 0.9116 28.1425 51.1681
SC-SOBS_1 3.0788 0.8199 0.4944 0.9781 32.0824 51.6212
SC-SOBS_2 4.1143 1.0376 0.6571 0.9724 30.2392 48.7241
BEWIS 3.5539 0.4588 0.3103 0.9813 32.2702 50.0786
Average 3.6103 1.9398 1.5049 0.9679 32.3911 48.9527

Table 5: Accuracy results of all the compared methods on sequence CAVIAR2 and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 1.1967 0.1689 0.0356 0.9979 40.7302 59.6979
Color Median 0.6987 0.0000 0.0000 0.9994 47.5113 59.5439
MOG2 1.4154 0.1658 0.0997 0.9974 40.1120 53.4368
KNN 7.1935 4.8910 1.4404 0.8469 18.9970 25.5783
BE-AAPSA 1.1718 0.0000 0.0000 0.9983 43.7194 54.6637
WS2006 0.7138 0.0387 0.0000 0.9991 44.1003 59.6514
IMBS-MT 1.2948 0.0102 0.0000 0.9986 43.0235 53.7161
LaBGen 0.8131 0.0000 0.0000 0.9993 46.8425 58.3582
RSL2011 0.8678 0.1312 0.0763 0.9962 39.7804 57.3866
Photomontage 1.1047 0.0000 0.0000 0.9984 44.4508 54.3645
LRGeomCG 1.1822 0.3265 0.1038 0.9971 39.8982 59.1772
TMac 1.1877 0.3286 0.1058 0.9970 39.8569 59.1736
SC-SOBS_1 0.7550 0.0000 0.0000 0.9994 47.2190 59.1624
SC-SOBS_2 0.9428 0.0000 0.0000 0.9992 45.6705 57.4730
BEWIS 0.7389 0.0000 0.0000 0.9994 47.6100 58.7359
Average 1.4185 0.4041 0.1241 0.9882 41.9681 55.3413

Table 6: Accuracy results of all the compared methods on sequence CaVignal and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 8.7869 10.4890 8.2537 0.8338 21.6405 50.2703
Color Median 10.3082 10.4632 8.1066 0.7984 18.1355 33.1438
MOG2 16.9327 0.1114 0.0837 0.8136 18.5891 34.5104
KNN 15.9267 0.0813 0.0127 0.8241 18.2332 30.9930
BE-AAPSA 10.0755 4.8100 3.1200 0.9711 26.1972 39.4600
WS2006 2.5403 1.5000 0.4743 0.9289 27.1089 37.0609
IMBS-MT 0.76920.0147 0.00000.9982 45.9202 57.1044
LaBGen 0.4542 0.0147 0.00000.9981 45.5789 55.9161
RSL2011 0.9106 0.0147 0.0000 0.9973 43.9322 53.7718
Photomontage 11.2665 11.2206 8.8529 0.7919 17.6257 32.0570
LRGeomCG 5.4839 6.4118 3.9890 0.9111 28.3288 52.9853
TMac 5.4979 6.5074 4.0625 0.9109 28.2877 53.0188
SC-SOBS_1 0.9590 0.6728 0.0000 0.9947 37.4679 55.9939
SC-SOBS_2 1.1434 0.6949 0.0000 0.9935 37.0992 54.1992
BEWIS 0.3990 0.0147 0.0000 0.9984 46.5616 59.8510
Average 6.0969 3.5347 2.4637 0.9176 30.7138 46.6891

Table 7: Accuracy results of all the compared methods on sequence Foliage and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 30.0992 65.8542 46.9549 0.6785 17.3201 26.8996
Color Median 27.0135 47.3125 30.4583 0.6444 16.7842 28.7321
MOG2 32.3624 0.6685 0.5526 0.8038 16.5991 31.5282
KNN 34.5615 0.3962 0.0385 0.6281 14.1761 25.6845
BE-AAPSA 26.2190 59.9800 43.0900 0.8015 18.4317 30.2999
WS2006 6.8649 2.8507 0.0069 0.9754 27.2438 34.9776
IMBS-MT 7.5809 9.8507 3.1319 0.9090 22.7278 34.0028
LaBGen 1.6172 0.0000 0.0000 0.9982 40.6051 46.7662
RSL2011 9.0230 12.3090 8.1250 0.8370 20.9844 30.4461
Photomontage 1.8592 0.0000 0.0000 0.9974 39.1779 45.6052
LRGeomCG 11.6932 20.8924 14.9861 0.9535 23.9826 39.0643
TMac 12.0335 21.9826 15.9861 0.9498 23.7072 38.7549
SC-SOBS_1 3.0825 0.0625 0.0000 0.9939 35.6936 39.6256
SC-SOBS_2 3.3587 0.0660 0.0000 0.9931 35.1103 39.5048
BEWIS 1.7767 0.0174 0.0000 0.9978 39.5441 46.1713
Average 13.9430 16.1495 10.8887 0.8774 26.1392 35.8709

Table 8: Accuracy results of all the compared methods on sequence Hall&Monitor and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 3.7238 3.0516 1.6643 0.9545 29.7571 42.6596
Color Median 2.7105 0.9931 0.5339 0.9640 30.4656 42.6705
MOG2 2.4506 0.0109 0.0045 0.9833 34.3943 45.9714
KNN 3.9413 0.0121 0.0021 0.9519 28.2208 37.4907
BE-AAPSA 2.4425 0.3200 0.0400 0.9892 36.4218 45.2466
WS2006 2.6644 0.5563 0.0308 0.9821 30.9313 40.0949
IMBS-MT 1.5350 0.0923 0.0000 0.9954 38.6214 48.5224
LaBGen 2.4008 0.1302 0.0000 0.9916 37.1746 45.1416
RSL2011 3.2937 1.6489 0.7931 0.9377 26.9214 36.7046
Photomontage 2.7986 0.3610 0.0817 0.9819 33.3715 41.7323
LRGeomCG 2.0476 0.2237 0.0000 0.9938 38.0243 46.3813
TMac 2.0599 0.2415 0.0000 0.9937 37.5664 46.2214
SC-SOBS_1 1.8125 0.6641 0.2166 0.9832 34.2985 44.2863
SC-SOBS_2 2.6930 0.7599 0.2166 0.9798 33.1795 42.5386
BEWIS 3.6217 1.4347 0.0154 0.9626 27.1794 35.6121
Average 2.6797 0.7000 0.2399 0.9763 33.1019 42.7516

Table 9: Accuracy results of all the compared methods on sequence HighwayI and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 6.4127 0.2995 0.0156 0.9700 30.3580 60.3090
Color Median 1.4275 0.1563 0.0143 0.9924 40.1432 62.5723
MOG2 2.6031 0.0023 0.0002 0.9753 35.8635 58.2889
KNN 6.1277 0.0616 0.0003 0.8506 25.1521 34.8174
BE-AAPSA 4.3721 2.7600 0.6900 0.9442 31.1332 52.3623
WS2006 2.5185 0.6849 0.0247 0.9816 35.6885 56.9113
IMBS-MT 1.4913 0.0612 0.0026 0.9939 41.7728 58.8328
LaBGen 1.9054 0.4362 0.0286 0.9877 37.4928 53.1613
RSL2011 1.5918 0.2344 0.0195 0.9899 38.8728 59.4531
Photomontage 2.1745 0.4076 0.0482 0.9830 37.1250 59.0270
LRGeomCG 2.6535 0.2018 0.0130 0.9779 36.2808 58.2359
TMac 2.6788 0.1992 0.0130 0.9777 36.1917 58.2417
SC-SOBS_1 0.9917 0.0026 0.0000 0.9968 44.3343 66.0819
SC-SOBS_2 2.1209 0.3216 0.0221 0.9870 37.3789 53.6069
BEWIS 2.1070 0.4661 0.0169 0.9886 36.8023 54.4956
Average 2.7451 0.4197 0.0606 0.9731 36.3060 56.4265

Table 10: Accuracy results of all the compared methods on sequence HighwayII and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 3.3414 0.3607 0.0000 0.9915 34.3045 47.1299
Color Median 1.7278 0.3190 0.0013 0.9961 34.6639 42.3162
MOG2 2.0893 0.0040 0.0000 0.9946 36.1190 45.2643
KNN 3.2112 0.0085 0.0001 0.9851 32.0981 39.6454
BE-AAPSA 2.5181 0.2800 0.0100 0.9903 36.2738 47.3613
WS2006 2.4906 0.4883 0.0130 0.9927 33.9515 40.5088
IMBS-MT 1.8684 0.0260 0.0000 0.9960 40.1098 48.8094
LaBGen 2.4240 0.3034 0.0026 0.9921 35.5876 42.8025
RSL2011 2.3000 0.5130 0.0846 0.9907 33.8305 42.4937
Photomontage 2.4306 0.5885 0.0052 0.9909 34.3975 41.7656
LRGeomCG 2.7526 0.3555 0.0026 0.9908 35.3406 46.3161
TMac 2.7697 0.3763 0.0039 0.9908 35.2287 46.2181
SC-SOBS_1 0.7100 0.0000 0.0000 0.9991 46.8739 56.6012
SC-SOBS_2 2.3946 0.2982 0.0039 0.9926 35.7688 43.2384
BEWIS 2.1932 0.4141 0.0013 0.9942 34.6264 41.4061
Average 2.3481 0.2890 0.0086 0.9925 35.9450 44.7918

Table 11: Accuracy results of all the compared methods on sequence HumanBody2 and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 6.3783 5.5065 3.0443 0.9736 27.9022 38.4071
Color Median 2.9408 0.2995 0.0391 0.9970 35.4279 46.6252
MOG2 11.2767 13.4609 9.9427 0.8752 19.5258 32.1251
KNN 20.9423 18.5130 15.2188 0.7783 14.5871 21.4805
BE-AAPSA 6.3274 0.0797 0.0550 0.9528 24.9434 36.6271
WS2006 3.9876 0.6393 0.0026 0.9923 30.7994 42.8966
IMBS-MT 1.9190 0.5794 0.0534 0.9958 34.0997 45.2074
LaBGen 3.8273 0.2630 0.0013 0.9975 34.4291 46.8653
RSL2011 3.1154 0.3099 0.0013 0.9959 35.5261 46.2671
Photomontage 11.4203 13.0052 9.4375 0.8751 19.2008 30.9881
LRGeomCG 5.7621 4.6497 2.4414 0.9788 28.7047 40.1984
TMac 5.8044 4.7292 2.4583 0.9786 28.6019 40.2122
SC-SOBS_1 1.8126 0.0990 0.0000 0.9980 39.3952 47.3105
SC-SOBS_2 3.3927 0.3411 0.0000 0.9969 35.0465 45.4292
BEWIS 4.2667 1.5013 0.0260 0.9866 27.9740 41.7024
Average 6.2116 4.2651 2.8481 0.9582 29.0776 40.1561

Table 12: Accuracy results of all the compared methods on sequence IBMtest2 and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 5.1227 3.7643 1.8112 0.9800 30.2187 41.6366
Color Median 2.2862 0.0391 0.00000.9939 36.6967 48.5607
MOG2 3.1981 1.5039 0.7083 0.9717 30.518 38.1524
KNN 21.3572 16.2995 2.3099 0.6671 14.1235 20.5705
BE-AAPSA 5.7290 0.0012 0.0000 0.9914 31.7541 44.5344
WS2006 4.6744 1.9531 0.0495 0.9410 24.2631 32.8595
IMBS-MT 7.3508 3.2734 0.1328 0.9721 24.6275 36.4310
LaBGen 3.7491 0.0872 0.0000 0.9906 33.5923 45.9029
RSL2011 6.1074 2.7005 0.9922 0.9303 24.4272 36.0603
Photomontage 3.1954 0.0690 0.0000 0.9898 35.1813 45.3905
LRGeomCG 3.6413 1.4544 0.6081 0.9868 32.8930 44.9516
TMac 3.6575 1.4831 0.6289 0.9868 32.8424 44.9660
SC-SOBS_1 2.3424 0.0143 0.0000 0.9954 37.9515 50.5923
SC-SOBS_2 3.9729 0.0964 0.0000 0.9919 33.7736 46.3316
BEWIS 3.9848 1.5013 0.0651 0.9602 25.6501 39.8792
Average 5.3579 2.2827 0.4871 0.956629.9009 41.1213

Table 13: Accuracy results of all the compared methods on sequence People&Foliage and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 34.4507 61.6576 54.0924 0.7555 15.3719 31.9727
Color Median 24.4211 32.2396 25.3203 0.6114 15.1870 27.4979
MOG2 33.8442 0.7108 0.6134 0.8584 16.2252 27.4728
KNN 48.4920 0.4718 0.2966 0.4238 10.9196 19.8121
BE-AAPSA 20.1865 31.0000 24.1900 0.9256 19.7152 29.3564
WS2006 5.4243 3.5716 0.0924 0.9269 22.6952 31.3847
IMBS-MT 8.3982 7.3568 3.2305 0.8514 20.0658 32.5231
LaBGen 1.7751 0.0026 0.0000 0.9968 39.7161 46.2148
RSL2011 8.1966 9.4023 7.8867 0.8628 21.2093 27.1459
Photomontage 1.4103 0.0039 0.0000 0.9973 41.0866 47.1517
LRGeomCG 29.2393 57.7812 48.2135 0.8332 16.8399 32.1823
TMac 29.4402 57.1888 47.6719 0.8233 16.6676 32.1267
SC-SOBS_1 7.5889 11.5482 6.2734 0.9333 23.6413 36.3079
SC-SOBS_2 7.9408 11.5586 6.2734 0.9329 23.6085 36.1581
BEWIS 11.9685 13.0182 10.2018 0.8823 17.6743 26.7312
Average 18.1851 19.8341 15.6238 0.8410 21.3749 32.2692

Table 14: Accuracy results of all the compared methods on sequence Snellen and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 54.2865 87.7025 81.0282 0.7154 12.6049 34.4532
Color Median 42.3981 62.2010 56.9734 0.6932 13.6573 36.0691
MOG2 58.8159 0.7615 0.6839 0.5336 11.4143 27.0312
KNN 61.9389 0.6832 0.4328 0.4493 10.6164 22.5804
BE-AAPSA 46.7580 76.0800 69.3300 0.7615 13.6310 37.1410
WS2006 23.0010 23.1674 12.2685 0.7481 15.6158 24.9930
IMBS-MT 14.4480 25.3279 19.7290 0.8668 19.7436 40.1151
LaBGen 4.6412 6.3368 5.9317 0.9792 27.1445 47.9560
RSL2011 16.0515 14.4290 12.6640 0.7190 16.7070 28.4869
Photomontage 29.9797 33.4973 30.4688 0.5926 14.1466 26.9210
LRGeomCG 24.4846 50.4340 42.8337 0.9250 18.6585 42.1307
TMac 24.8743 51.9965 44.3528 0.9206 18.5311 41.8552
SC-SOBS_1 16.1433 35.4504 21.8412 0.9332 21.6050 46.0165
SC-SOBS_2 16.5042 35.4745 21.9088 0.9322 21.3953 44.9320
BEWIS 4.6386 5.2758 3.3131 0.9692 25.7540 42.7116
Average 29.2643 33.9212 28.2507 0.7826 17.4150 36.2262

Table 15: Accuracy results of all the compared methods on sequence Toscana and their Average.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 11.6247 22.8308 17.5896 0.8831 22.7298 36.2525
Color Median 5.3148 6.4562 3.7742 0.8707 23.2941 31.7804
MOG2 9.5929 12.806 8.3773 0.8947 23.5968 23.1972
KNN 19.0935 22.7581 17.5800 0.7034 16.3492 16.2754
BE-AAPSA 7.3553 0.1033 0.0658 0.9095 25.6164 37.6754
WS2006 5.8222 5.8935 2.3500 0.8623 22.8504 29.0927
IMBS-MT 7.4109 6.9096 5.2394 0.8903 22.5367 22.0319
LaBGen 6.1993 6.9440 4.4881 0.8805 23.1537 33.3061
RSL2011 18.7636 27.3794 22.6806 0.6662 17.1506 24.4755
Photomontage 1.5175 0.4517 0.1652 0.9892 36.7526 50.2416
LRGeomCG 7.3009 11.8569 8.4394 0.8959 24.2914 36.3206
TMac 7.3742 12.0163 8.5815 0.8958 24.2609 36.2690
SC-SOBS_1 3.1898 3.3113 1.7183 0.9616 30.6221 43.3002
SC-SOBS_2 6.2949 8.5400 5.0521 0.8880 24.9143 36.7567
BEWIS 7.4054 6.8877 5.0215 0.8878 22.5227 31.7212
Average 8.2840 10.3430 7.4082 0.8719 24.0428 32.5798

Please, observe that the above CMQ values were evaluated using a previous version of the Matlab scripts, that included a bug (you can still download the Old version of Matlab scripts to compare with the above results).

References

[1] L. Maddalena, A. Petrosino, Towards Benchmarking Scene Background Initialization, in V. Murino et al. (eds), New Trends in Image Analysis and Processing -- ICIAP 2015 Workshops, Lecture Notes in Computer Science, Vol. 9281, Springer International Publishing Switzerland, DOI 10.1007/978-3-319-23222-5_57#, pp. 469–476, 2015.

[2] T. Bouwmans, L. Maddalena, A. Petrosino, Scene background initialization: A taxonomy, Pattern Recognition Letters 96, DOI 10.1016/j.patrec.2016.12.024, 3-11, 2017.