Serge populated a database consisting of all the columns in the false detection fits files that were sent to us.

I made some cuts and found some things:

There are 397521 false detections. Of these, there are only 45 different 'flags' that are used:

mysql> select count(*), flags, flags2 from DetectionFeatures group by flags;

+----------+-----------+--------+
| count(*) | flags     | flags2 |
+----------+-----------+--------+
|      128 |   2107393 |    128 | 
|      791 |  35653633 |    128 | 
|        1 |  35653645 |    160 | 
|        1 |  35653889 |    224 | 
|      454 |  35661825 |    128 | 
|       24 | 102762501 |    192 | 
|   108851 | 169869313 |    208 | 
|      191 | 169869325 |    208 | 
|       11 | 169869333 |    192 | 
|        1 | 169869389 |    128 | 
|    26004 | 169869441 |    192 | 
|       51 | 169869453 |    192 | 
|       61 | 169869569 |    192 | 
|       53 | 169869577 |    192 | 
|        1 | 169869641 |    128 | 
|       69 | 169869697 |    192 | 
|       23 | 169869705 |    192 | 
|   224897 | 169885697 |    240 | 
|      315 | 169885701 |    224 | 
|        1 | 169885761 |    192 | 
|      444 | 169885953 |    192 | 
|       56 | 169885961 |    192 | 
|        2 | 169885969 |    192 | 
|        8 | 170917889 |    192 | 
|        5 | 170917901 |    240 | 
|        8 | 170918017 |    208 | 
|        1 | 170918029 |    224 | 
|        6 | 203423749 |    128 | 
|        4 | 203423877 |    128 | 
|     2865 | 236978181 |    128 | 
|      315 | 236978245 |    128 | 
|      119 | 236978309 |    128 | 
|     3780 | 236994567 |    192 | 
|      294 | 236994597 |    128 | 
|        4 | 236994631 |    192 | 
|       26 | 238026757 |    144 | 
|       19 | 238026885 |    224 | 
|        2 | 238043143 |    240 | 
|        1 | 241172485 |    192 | 
|      618 | 268437504 |    128 | 
|        3 | 268437760 |    128 | 
|        1 | 268437768 |    128 | 
|    21738 | 402685952 |    128 | 
|     5273 | 402686080 |    128 | 
|        1 | 403734656 |    128 | 
+----------+-----------+--------+
45 rows in set (1.40 sec)

Most lump into only a handful of flags.

I did investigations of flag2 first, because there are fewer of those. I did cuts to remove 64,32,16,8, since those all had to do with mask, cosmicray, burntool, starspikes (ON_SPIKE, ON_STARCORE, ON_BURNTOOL, ON_CONVPOOR), which are not necessary objects we want to count.

I then cut on DEFECT, SATURATED, CR_LIMIT, EXT_LIMIT, and SIZE_SKIPPED on flag. These seemed like reasonable candidates for flags that maybe should be used to filter out bad detections.

I also did cuts to remove NULL mags (is there any mag which should ALWAYS be calculated? I can cut on that instead of the mag I cut)

Here's what we have left: mysql> select count(*), flags, flags2 from DetectionFeatures where (!(flags2 & 64) and !(flags2 & 32) and !(flags2 & 16) and !(flags2&8) and !(flags & 2048) and !(flags & 4096) and !(flags&8192) and !(flags&16384) and !(flags&268435456)) and !(cal_psf_mag is null) group by flags,flags2;

+----------+-----------+--------+
| count(*) | flags     | flags2 |
+----------+-----------+--------+
|    55035 | 169869313 |    128 | 
|       12 | 169869325 |    128 | 
|        1 | 169869389 |    128 | 
|      191 | 169869441 |    128 | 
|        2 | 169869569 |    128 | 
|        8 | 169869577 |    128 | 
|        1 | 169869641 |    128 | 
|        3 | 169869697 |    128 | 
|        6 | 203423749 |    128 | 
|        4 | 203423877 |    128 | 
|      959 | 236978181 |    128 | 
|      252 | 236978245 |    128 | 
|       75 | 236978309 |    128 | 
|        8 | 238026757 |    128 | 
|        1 | 238026885 |    128 | 
+----------+-----------+--------+
15 rows in set (0.24 sec)

for a total of 56558 false detections.

I investigated flag 169869313, which contains most of the false detections. Unfortunately, the flag is not interesting (it did a fit, it calculated a mag, etc), there are no further things to cut out. So, I created region files for the remaining false detections (see /data/ipp009.0/heather/heather), and looked at a few of the ones that had the largest number of detections. I see nothing that stands out:

The numbers represent the flag setting, and the circles are the remaining false detections.

This is AFTER I did the flag cuts. Prior to that there were 8x as many detections, often landing on burntools and edges.