May 1

OK, I am trying hard to get working code this week. I am stuck at a problem with libfw.so.

lib/libfw.so: undefined reference to `boost::basic_regex ...
lib/libfw.so: undefined reference to `boost::re_detail::perl_matcher ...
collect2: ld returned 1 exit status

Go back into fw and re-Scons the lib, see if anything looks out of place.

becker158: scons -c lib
becker159: scons lib
  ...
  Checking for C++ header file boost/version.hpp... (cached) yes
  Checking for C++ library boost_filesystem... (cached) yes
  Checking for C++ header file boost/regex.hpp... (cached) yes
  Checking for C++ library boost_regex... (cached) yes
  ...
  g++ -o lib/libfw.so -Wl,-rpath-link -Wl,/local/becker/lsst_root/Linux/external/vw/1.0.1/lib:/usr/X11R6/lib:/usr/lib:/usr/local/lib:/home/becker/lib:/local/becker/lsst_root/Linux/external/boost/1.33.1/lib:/local/becker/lsst_root/Linux/external/cfitsio/3006/lib:/local/becker/lsst_root/Linux/external/xpa/2.1.7b2/lib:/local/becker/lsst_root/Linux/external/tcltk/8.5a4/lib:/local/becker/lsst_root/Linux/external/termcap/1.3.1/lib:/local/becker/lsst_root/Linux/external/readline/5.2/lib:/local/becker/lsst_devel/DC2/fw/lib -shared src/Citizen.os src/DataProperty.os src/Demangle.os src/Trace.os src/Utils.os -L/local/becker/lsst_root/Linux/external/boost/1.33.1/lib -L/local/becker/lsst_root/Linux/external/vw/1.0.1/lib -L/local/becker/lsst_root/Linux/external/python/2.5/lib/python2.5/site-packages -L/local/becker/lsst_root/Linux/external/cfitsio/3006/lib -L/local/becker/lsst_root/Linux/external/wcstools/3.6.2/lib -L/local/becker/lsst_root/Linux/external/xpa/2.1.7b2/lib -Llib -lboost_filesystem -lboost_regex

Aaah, I also see a libreadline problem...

running tests/DataProperty_1.py
python: error while loading shared libraries: libreadline.so.5: cannot open shared object file: No such file or directory

Anyways, Tim suggest checking out his ticket. So I...

becker6: svn co $LSST_SVN/DC2/fw/tickets/49
becker7: setup -v -r $LSST_DEVEL/DC2/49 fw
becker9: cd 49/
becker11: scons
  ...
  lib/libfw.so: undefined reference to `boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::do_assign(char const*, char const*, unsigned int)'

Still broken..! Can't build anything. I guess I will have to look into it. Track down do_assign in boost. Its in regex/v4/basic_regex.hpp.

template <class charT, class traits>
basic_regex<charT, traits>& basic_regex<charT, traits>::do_assign(const charT* p1,
                        const charT* p2,
                        flag_type f)

Is it unsigned int vs. flag_type f? I will operate on this premise for now. Find where its used in fw.

AHHHH... I got things to compile by adding -lboost_regex-gcc to the g++ line. Note that -lboost_regex did not cut it. There are 2 culprits

  • Boost was compiled improperly, not leaving me with a -lboost_regex.
    becker48: ls $LSST_HOME/Linux/external/boost/1.33.1/lib/libboost_regex*
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-1_33_1.a
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-1_33_1.so@
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-1_33_1.so.1.33.1*
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc.a
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-d-1_33_1.a
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-d-1_33_1.so@
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-d-1_33_1.so.1.33.1*
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-d.a
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-d.so*
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-mt-1_33_1.a
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-mt-1_33_1.so@
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-mt-1_33_1.so.1.33.1*
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-mt.a
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-mt-d-1_33_1.a
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-mt-d-1_33_1.so@
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-mt-d-1_33_1.so.1.33.1*
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-mt-d.a
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-mt-d.so*
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc-mt.so*
      /local/becker/lsst_root/Linux/external/boost/1.33.1/lib/libboost_regex-gcc.so*
    

  • Scons needs to explicitly ask for libboost_regex-gcc instead of libboost_regex. In particular, this is necessary when libfw.so is built.
    g++ -o lib/libfw.so ... -lboost_regex-gcc  # add in that last -gcc by hand.
    scons examples                             # WORKS!
    scons tests                                # also works
    
    I can accomplish this by editing the SConstruct file and saying
    env.libs = dict([
        ("boost",   Split("boost_filesystem-gcc boost_regex-gcc")),
        ...
    

Anyways, this problem seems to be localized to my machine. Tim sent me his libfw.so line, and it says g++ -o lib/libfw.so ... -lboost_regex. So he *must* have a $LSST_HOME/Linux/external/boost/1.33.1/lib/libboost_regex.so. I don't.

Wow, Tim does have -gcc.so libraries. So how in the hell did he get his code to build!?!? It should not work, unless he has another libboost somewhere on his machines!? Indeed, both have local versions in

/usr/lib/libboost_regex.a
/usr/lib/libboost_regex.so
/usr/lib/libboost_regex.so.1.33.1
/usr/lib/libboost_regex.so.2

which his code must be linking against.


Anyways, there is a potential solution for 64-bit machines, so I am re-trying this on darkstar. Summary of how to get this thing set up

becker1:  setenv LSST_HOME /lsst/lsst_root_new/
becker4:  curl -o newinstall.sh http://dev.lsstcorp.org/pkgs/newinstall.sh
becker31: chmod a+x newinstall.sh
becker32: ./newinstall.sh
becker33: cd /lsst/lsst_root_new
becker34: source loadLSST.csh
becker36: eups distrib -i -C LSSTPipe

Anyways, I can also try and compile my code now!

becker69: scons tests
  Checking for C++ header file lsst/fw/Trace.h... no
  Checking for C header file lsst/fw/Trace.h... no
becker70: more config.log
  scons: Configure: Checking for C++ header file lsst/fw/Trace.h...
  .sconf_temp/conftest_5.cpp <-
    |#include <lsst/fw/MaskedImage.h>
    |
    |#include <lsst/fw/Trace.h>
    |
    |
  g++ -o .sconf_temp/conftest_5.o -c -g -Wall -O0 -DLSST_LITTLE_ENDIAN=1 -I/local/becker/lsst_root/Linux/external/boost/1.33.1/include -I/local/becker/lsst_root/Linux/external/vw/1.0.1/include -I/local/becker/lsst_devel/DC2/fw/include .sconf_temp/conftest_5.cpp
  In file included from /local/becker/lsst_devel/DC2/fw/include/lsst/fw/Mask.h:22,
                   from /local/becker/lsst_devel/DC2/fw/include/lsst/fw/MaskedImage.h:24,
                   from .sconf_temp/conftest_5.cpp:1:
  /local/becker/lsst_devel/DC2/fw/include/lsst/fw/LSSTFitsResource.h:23:33: LSSTFitsResource.cc: No such file or directory
  In file included from /local/becker/lsst_devel/DC2/fw/include/lsst/fw/MaskedImage.h:24,
                   from .sconf_temp/conftest_5.cpp:1:
  /local/becker/lsst_devel/DC2/fw/include/lsst/fw/Mask.h:132:21: Mask.cc: No such file or directory
  In file included from /local/becker/lsst_devel/DC2/fw/include/lsst/fw/MaskedImage.h:25,
                   from .sconf_temp/conftest_5.cpp:1:
  /local/becker/lsst_devel/DC2/fw/include/lsst/fw/Image.h:81:20: Image.cc: No such file or directory
  In file included from .sconf_temp/conftest_5.cpp:1:
  /local/becker/lsst_devel/DC2/fw/include/lsst/fw/MaskedImage.h:141:26: MaskedImage.cc: No such file or directory
  scons: Configure: no

So it has to do with this deal of #include "MaskedImage.cc" in MaskedImage.h. I don't get that one... The test scons code is

 becker74: more conftest_5.cpp
   #include <lsst/fw/MaskedImage.h>
   #include <lsst/fw/Trace.h>

and Scons wants to compile it like

 g++ -o conftest_5.o -c -g -Wall -O0 -DLSST_LITTLE_ENDIAN=1 -I/local/becker/lsst_root/Linux/external/boost/1.33.1/include -I/local/becker/lsst_root/Linux/external/vw/1.0.1/include -I/local/becker/lsst_devel/DC2/fw/include conftest_5.cpp

We'll need to tell Scons to compile it like so if we include .cc files in .h files

 g++ -o conftest_5.o -c -g -Wall -O0 -DLSST_LITTLE_ENDIAN=1 -I/local/becker/lsst_root/Linux/external/boost/1.33.1/include -I/local/becker/lsst_root/Linux/external/vw/1.0.1/include -I/local/becker/lsst_devel/DC2/fw/include -I/local/becker/lsst_devel/DC2/fw/src conftest_5.cpp

So apparently this is needed for templating. Meaning we need to tell Scons that when using 'fw' as a product we also need to look at ${PRODUCT_DIR}/src. I tried adding in to fw/ups/fw.table the line

envAppend(CPPPATH, ${PRODUCT_DIR}/src)

but no go. It still only looks in fw/include. Hmmm...