pxinject gives a Bus Error (NULL value for field 'epoch' on some hardware)

[Michael Wood-Vasey]

When running the simtest module on ipp-2.6.beta (20080716) I can't inject images. The basic pantasks job failed with

no data in db TESS/Photcodes.dat
directory TESS does not exist, creating...
exp_name : simtest.000.000.
1 files in fileset.
Running -newExp -tmp_exp_name simtest.000.000 -tmp_inst SIMTEST -tmp_telescope SIMTEST -workdir /Volumes/data/PS1/data/simtest/work -dvodb /Volumes/data/PS1/data/simtest/DVO -tess_id /Volumes/data/PS1/data/simtest/TESS -label wait -dbname simtest...
Unable to inject simtest.000.000: 10

In more detail from gdb

(gdb) run -newExp -tmp_exp_name simtest.000.000 -tmp_inst SIMTEST -tmp_telescope SIMTEST -workdir /Volumes/data/PS1/data/simtest/work -dvodb /Volumes/data/PS1/data/simtest/DVO -tess_id /Volumes/data/PS1/data/simtest/TESS -label wait -dbname simtest
Starting program: /Volumes/data/PS1/code/ipp-2.6.beta/default.darwin/bin/pxinject -newExp -tmp_exp_name simtest.000.000 -tmp_inst SIMTEST -tmp_telescope SIMTEST -workdir /Volumes/data/PS1/data/simtest/work -dvodb /Volumes/data/PS1/data/simtest/DVO -tess_id /Volumes/data/PS1/data/simtest/TESS -label wait -dbname simtest
Reading symbols for shared libraries ++++++++++++++++++++++..... done

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
0x009499ac in net_store_datetime ()
(gdb) bt
#0 0x009499ac in net_store_datetime ()
#1 0x00949abc in store_param_datetime ()
#2 0x0094c500 in cli_stmt_execute ()
#3 0x0094bfec in mysql_stmt_execute ()
#4 0x005fa2ac in p_psDBRunQueryPrepared ()
#5 0x005f8bc8 in psDBInsertRows ()
#6 0x005f89b4 in psDBInsertOneRow ()
#7 0x00564528 in newExpInsert (dbh=0x18056b4, exp_id=0, tmp_exp_name=0x1815214 "simtest.000.000", tmp_camera=0x18139a4 "SIMTEST", tmp_telescope=0x1804974 "SIMTEST", state=0x3bf4 "reg", workdir=0x1804b64 "/Volumes/data/PS1/data/simtest/work", workdir_state=0x3bf8 "dirty", reduction=0x0, dvodb=0x1815344 "/Volumes/data/PS1/data/simtest/DVO", tess_id=0x18153a4 "/Volumes/data/PS1/data/simtest/TESS", end_stage=0x0, label=0x1815474 "wait", epoch=0x0) at ippdb.c:2535
#8 0x00002900 in newExpMode (config=0xe005d4) at pxinject.c:92
#9 0x00002198 in main (argc=18, argv=0xbfffa628) at pxinject.c:53

Is this related to ippdb being mismatched to the rest of the code as noted earlier today or is this some problem with my DB configuration? What -trace should I enable?

[Michael Wood-Vasey]

I think I've traced this down to my MySQL (5.0.51a) being unhappy that 'epoch' is NULL rather than a valid psTime struct.

[Michael Wood-Vasey]

Yes. When I create dummy 'epoch' psTime variables and pass those instead of NULL to and newExpInsert and newImfileInsert in 'ippTools/pxinject.c' the images are successfully inserted into the database.

So the suggested fix is to either be explicit that these are being set to dummy datetime values or to take it to the next step and figure out what the times should really be.

[Paul Price]

That's a workaround, not a fix.

Do you know if the problem is in mysql or in ippdb? I can't reproduce it on my box.

[Michael Wood-Vasey]

Here's the -trace psLib.db 10 output without the workaround:

[wwoodvas@serenity simtest] gdb /Volumes/data/PS1/code/ipp-2.6.beta/default.darwin/bin/pxinject
[...]
(gdb) run -newExp -tmp_exp_name simtest.000.000 -tmp_inst SIMTEST -tmp_telescope SIMTEST -workdir /Volumes/data/PS1/data/simtest/work -dvodb /Volumes/data/PS1/data/simtest/DVO -tess_id /Volumes/data/PS1/data/simtest/TESS -label wait -dbname simtest -trace psLib.db 10
Starting program: /Volumes/data/PS1/code/ipp-2.6.beta/default.darwin/bin/pxinject -newExp -tmp_exp_name simtest.000.000 -tmp_inst SIMTEST -tmp_telescope SIMTEST -workdir /Volumes/data/PS1/data/simtest/work -dvodb /Volumes/data/PS1/data/simtest/DVO -tess_id /Volumes/data/PS1/data/simtest/TESS -label wait -dbname simtest -trace psLib.db 10
Reading symbols for shared libraries ++++++++++++++++++++++..... done

calling mysql_autocommit(): 1
connected to database simtest
Preparing SQL:

INSERT INTO newExp (exp_id, tmp_exp_name, tmp_camera, tmp_telescope, state, workdir, workdir_state, reduction, dvodb, tess_id, end_stage, label, epoch) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)

statment is not in query cache
Binding:

exp_id S64 0
tmp_exp_name STR simtest.000.000
tmp_camera STR SIMTEST
tmp_telescope STR SIMTEST
state STR reg
workdir STR /Volumes/data/PS1/data/simtest/work
workdir_state STR dirty
reduction STR NULL
dvodb STR /Volumes/data/PS1/data/simtest/DVO
tess_id STR /Volumes/data/PS1/data/simtest/TESS
end_stage STR NULL
label STR wait
epoch TAI NULL

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
0x009479ac in net_store_datetime ()
(gdb) bt
#0 0x009479ac in net_store_datetime ()
#1 0x00947abc in store_param_datetime ()
#2 0x0094a500 in cli_stmt_execute ()
#3 0x00949fec in mysql_stmt_execute ()
#4 0x005f82ac in p_psDBRunQueryPrepared ()
#5 0x005f6bc8 in psDBInsertRows ()
#6 0x005f69b4 in psDBInsertOneRow ()
#7 0x00563aa0 in newExpInsert (dbh=0x1812664, exp_id=0, tmp_exp_name=0x1815404 "simtest.000.000", tmp_camera=0x1813b94 "SIMTEST", tmp_telescope=0x1804b64 "SIMTEST", state=0x3bf4 "reg", workdir=0x1804d54 "/Volumes/data/PS1/data/simtest/work", workdir_state=0x3bf8 "dirty", reduction=0x0, dvodb=0x1815534 "/Volumes/data/PS1/data/simtest/DVO", tess_id=0x1815594 "/Volumes/data/PS1/data/simtest/TESS", end_stage=0x0, label=0x1815664 "wait", epoch=0x0) at ippdb.c:2534
#8 0x00002900 in newExpMode (config=0xe005d4) at pxinject.c:94
#9 0x00002198 in main (argc=21, argv=0xbfffa4ac) at pxinject.c:53

[eugene]

I'd like to bump this and see if MWV can still reproduce this error on his hardware.

I've adjusted the summary to clarify the source of the problem.

[Michael Wood-Vasey]

Yes, the bug remains. Same MySQL lib 5.0.51a as is currently distributed in extlibs dir.

IPP-2.8, svn revision 23340

[wwoodvas@serenity ipp-2.8] which pxinject
/Volumes/data/PS1/code/ipp-2.8/ipp-2.8.darwin/bin/pxinject
[wwoodvas@serenity ipp-2.8] gdb which pxinject
[...]
(gdb) run -newExp -tmp_exp_name simtest.000.000 -tmp_inst SIMTEST -tmp_telescope SIMTEST -workdir /Volumes/data/PS1/data/simtest/work -dvodb /Volumes/data/PS1/data/simtest/DVO -tess_id /Volumes/data/PS1/data/simtest/TESS -label wait -dbname simtest -trace psLib.db 10
Starting program: /Volumes/data/PS1/code/ipp-2.8/ipp-2.8.darwin/bin/pxinject -newExp -tmp_exp_name simtest.000.000 -tmp_inst SIMTEST -tmp_telescope SIMTEST -workdir /Volumes/data/PS1/data/simtest/work -dvodb /Volumes/data/PS1/data/simtest/DVO -tess_id /Volumes/data/PS1/data/simtest/TESS -label wait -dbname simtest -trace psLib.db 10
Reading symbols for shared libraries ++++++++++++++++++++++...... done

calling mysql_autocommit(): 1
connected to database simtest
Preparing SQL:

INSERT INTO newExp (exp_id, tmp_exp_name, tmp_camera, tmp_telescope, state, workdir, workdir_state, reduction, dvodb, tess_id, end_stage, label, epoch) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)

statment is not in query cache
Binding:

exp_id S64 0
tmp_exp_name STR simtest.000.000
tmp_camera STR SIMTEST
tmp_telescope STR SIMTEST
state STR reg
workdir STR /Volumes/data/PS1/data/simtest/work
workdir_state STR dirty
reduction STR NULL
dvodb STR /Volumes/data/PS1/data/simtest/DVO
tess_id STR /Volumes/data/PS1/data/simtest/TESS
end_stage STR NULL
label STR wait
epoch TAI NULL

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
0x009e13b4 in net_store_datetime ()
(gdb) bt
#0 0x009e13b4 in net_store_datetime ()
#1 0x009e14c4 in store_param_datetime ()
#2 0x009e3f08 in cli_stmt_execute ()
#3 0x009e39f4 in mysql_stmt_execute ()
#4 0x006698bc in p_psDBRunQueryPrepared ()
#5 0x00668008 in psDBInsertRows ()
#6 0x00667df4 in psDBInsertOneRow ()
#7 0x005c4b4c in newExpInsert (dbh=0x185e054, exp_id=0, tmp_exp_name=0x1861cf4 "simtest.000.000", tmp_camera=0x1860364 "SIMTEST", tmp_telescope=0x185db24 "SIMTEST", state=0x3bf4 "reg", workdir=0x185db84 "/Volumes/data/PS1/data/simtest/work", workdir_state=0x3bf8 "dirty", reduction=0x0, dvodb=0x1861e24 "/Volumes/data/PS1/data/simtest/DVO", tess_id=0x1861e84 "/Volumes/data/PS1/data/simtest/TESS", end_stage=0x0, label=0x1861f54 "wait", epoch=0x0) at ippdb.c:2562
#8 0x00002900 in newExpMode (config=0xf00604) at pxinject.c:92
#9 0x00002198 in main (argc=21, argv=0xbfffa254) at pxinject.c:53

[Paul Price]

Bus error was due to apparent endian bug when using MYSQL_BIND.is_null in psDBPackMySQLRow (casting bool* to my_bool*).

Fixed in r#23507 on trunk.