Detection ablations, PASLCD, 20 instances (MV3DCD evaluate.py metric per image, mean per instance, then over instances) | variant | F1 | IoU | precision | recall | |---|---|---|---|---| | no_agent | 0.508 | 0.368 | 0.464 | 0.823 | | stage1_boxes | 0.527 | 0.371 | 0.378 | 0.963 | | no_points | 0.846 | 0.754 | 0.901 | 0.819 | | pre_only | 0.580 | 0.461 | 0.777 | 0.505 | | post_only | 0.624 | 0.513 | 0.818 | 0.541 | | full | 0.912 | 0.846 | 0.899 | 0.931 | | stored | 0.912 | 0.846 | 0.899 | 0.931 | Per-instance F1 | instance | no_agent | stage1_boxes | no_points | pre_only | post_only | full | stored | |---|---|---|---|---|---|---|---| | cantinaG_op | 0.640 | 0.597 | 0.853 | 0.707 | 0.747 | 0.881 | 0.881 | | garden_op | 0.398 | 0.447 | 0.879 | 0.766 | 0.463 | 0.930 | 0.930 | | lounge_op | 0.471 | 0.369 | 0.885 | 0.689 | 0.654 | 0.930 | 0.930 | | lunchroom_op | 0.384 | 0.421 | 0.760 | 0.473 | 0.686 | 0.879 | 0.879 | | meetingroom_op | 0.530 | 0.544 | 0.750 | 0.712 | 0.605 | 0.917 | 0.917 | | playground_op | 0.164 | 0.408 | 0.701 | 0.865 | 0.000 | 0.865 | 0.865 | | porch_op | 0.688 | 0.566 | 0.970 | 0.700 | 0.751 | 0.970 | 0.970 | | pots_op | 0.652 | 0.703 | 0.930 | 0.000 | 0.959 | 0.959 | 0.959 | | printingarea_op | 0.650 | 0.709 | 0.863 | 0.710 | 0.477 | 0.898 | 0.898 | | zen_op | 0.590 | 0.547 | 0.910 | 0.245 | 0.880 | 0.972 | 0.972 | | cantina2_op | 0.611 | 0.602 | 0.840 | 0.700 | 0.716 | 0.870 | 0.870 | | garden2_op | 0.401 | 0.423 | 0.869 | 0.757 | 0.464 | 0.925 | 0.925 | | lounge2_op | 0.445 | 0.392 | 0.830 | 0.643 | 0.617 | 0.879 | 0.879 | | lunchroom2_op | 0.377 | 0.415 | 0.802 | 0.490 | 0.686 | 0.868 | 0.868 | | meetingroom2_op | 0.506 | 0.553 | 0.740 | 0.673 | 0.600 | 0.890 | 0.890 | | playground2_op | 0.159 | 0.442 | 0.680 | 0.826 | 0.000 | 0.826 | 0.826 | | porch2_op | 0.686 | 0.552 | 0.971 | 0.700 | 0.752 | 0.971 | 0.971 | | pots2_op | 0.634 | 0.658 | 0.931 | 0.000 | 0.956 | 0.956 | 0.956 | | printingarea2_op | 0.589 | 0.669 | 0.821 | 0.689 | 0.576 | 0.882 | 0.882 | | zen2_op | 0.591 | 0.530 | 0.930 | 0.246 | 0.882 | 0.975 | 0.975 |