Apr 27

Couple of Ess-cons tricks

# Equivalant of 'make clean'
scons -c

# Getting the DC2/scons fw branch into my path
# Edit SConscript
env.Program(["lsstimageproc01.cc"], LIBS=env.libs["vw"] + ["fw"],
            LIBPATH=os.path.join(os.environ["LSST_DEVEL"], "DC2", "scons", "lib"),
            CPPPATH=os.path.join(os.environ["LSST_DEVEL"], "DC2", "scons", "include"))

# OOPS, the above gets rid of everything else from LIBPATH and CPPPATH, what I want is the equivalent of LIBPATH.append
# this is done like...
env.Append(LIBPATH = os.path.join(os.environ["LSST_DEVEL"], "DC2", "scons", "lib"))
env.Append(CPPPATH=os.path.join(os.environ["LSST_DEVEL"], "DC2", "scons", "include"))
env.Program(["lsstimageproc01.cc"], LIBS=env.libs["vw"] + ["fw"])

So yay! I get the file tests/lsstimageproc01. I run it and find

becker233: tests/lsstimageproc01
  tests/lsstimageproc01: error while loading shared libraries: libfw.so: cannot open shared object file: No such file or directory

Hmmm...

becker238: setup fw
becker239: echo $FW_DIR
  /lsst/lsst_root/Linux/fw/0.3
becker97: eups list -d fw
  /lsst/lsst_root/Linux/fw/0.3

But I want to use /lsst/becker/lsst_devel/DC2/scons. So I say

becker240: echo 'pathAppend(LD_LIBRARY_PATH, ${PRODUCT_DIR}/lib)' | eups declare -c fw scons -M - -r /lsst/becker/lsst_devel/DC2/scons
becker241: eups list -d fw
  /lsst/lsst_root/Linux/fw/0.3
  /lsst/becker/lsst_devel/DC2/scons
becker245: setup fw scons -v
becker247: tests/lsstimageproc01
# WORKS!  I am the master of the build system...

And of course my success is immediately shot down by RHL. Sigh... Apparently the place to do this is in SConstruct instead of SConscript. Says RHL

All that you should have to do is add fw to the dependencies in the
top level SConstruct file, and maybe specify some extra libraries.

A nicely perfunctory answer. I would have answered this like so

echo 'pathAppend(LD_LIBRARY_PATH, ${PRODUCT_DIR}/lib)' | eups declare -c fw trunk -M - -r /lsst/becker/lsst_devel/DC2/fw/trunk
setup fw trunk
add ["fw", "lsst/fw/LsstBase.h", "fw"] to SConstruct's `env = scons.makeEnv()`
add ("fw", Split("fw")) to SConstruct's `env.libs = dict()`

In the end the fact that I was unable to run tests/lsstimageproc01 even though I was able to compile it should have been a sign that what I did was a bit too much of a hack.

ANYWAYS, I also got yelled at because I was using an old branch (scons) of fw. So get the real deal.

becker319: svn co $LSST_SVN/DC2/fw/trunk fw
becker323: cd fw/
becker325: scons src       # OK
becker329: scons include   # OK
becker337: scons lib       # NOT OK, BOOST CAUSING PROBLEMS ON MY 64 BIT MACHINE.  I'M NOT SUPPOSED TO LOOK INTO THIS AT ALL...
  /usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libboost_regex.a(cpp_regex_traits.o): relocation R_X86_64_32S against `boost::re_detail::re_default_error_messages' can not be used when making a shared object; recompile with -fPIC
  /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libboost_regex.a: could not read symbols: Bad value
  collect2: ld returned 1 exit status
  scons: *** [lib/libfw.so] Error 1

So fine, darkstar is hosed because of the build system. We need another machine for R&D. Which one?

While we try and figure this machine deal out, an explicit answer from RHL on the correct way to compile against libraries in $LSST_DEVEL. So since I can't use the trunk for now lets use scons as the branch (and note I kind of screwed up where I put the scons branch). There are 2 codes we need to deal with, Eups and Scons.

**** EUPS
* we need to declare a FW product in $LSST_DEVEL that we can compile our IMAGEPROC code against.
* the shortest solution is

  eups declare -c -r /lsst/becker/lsst_devel/DC2/scons fw scons

* if I had things set up right, in DC2/fw/scons, than this would work

  eups declare -c -r /lsst/becker/lsst_devel/DC2/fw/scons

* since i said "-c" i made the scons branch the current branch *for everyone*.  Probably not what i wanted to do.

  setup fw

* will then get the current (scons) branch.  a better way to do this is to declare my own

  eups declare -r /lsst/becker/lsst_devel/DC2/scons fw becker
  setup fw becker

* next up, and i have always wanted to figure out how to do this, is to set up my own 'imageproc' product ***along with its dependencies***!!!
* so in my imageproc branch directory /lsst/becker/lsst_devel/imageproc/tickets/7

  cd ups
  vi imageproc.table
     setupRequired(fw)

* and i think that this will allow me to setup 'fw' when i setup 'imageproc'.
* declare my imageproc product

  eups declare -r /lsst/becker/lsst_devel/imageproc/tickets/7 imageproc becker

* and set it up along with all its dependencies

  setup imageproc becker

* however, what i really want to do is setup "fw becker", right?  so you can do this by editing imageproc.table again, but saying
  
  setupRequired(fw becker)

* however, you have to be careful when you merge this back into the main branch.
*
**** SCONS
*
* Next up, we need to tell Scons that we want to compile our test code using FW.  Here is my test code in its entirety
* /lsst/becker/lsst_devel/imageproc/tickets/7/tests/lsstimageproc01.cc

#include "lsst/fw/LsstBase.h"
#include "lsst/fw/Trace.h"

#include <iostream>
using namespace std;
using namespace lsst::fw;

int main( int argc, char** argv )
{
    return 1;
}

* Scons has 2 files that I can edit, SConstruct and tests/SConscript
* The right place to make the changes is in SConstruct
* You need to tell it what include files to use, and what libraries use
* First up, we update MakeEnv
* Its used like : def MakeEnv(eups_product, versionString=None, dependencies=[], traceback=False):
* and dependencies are tuples of [product, include files, libs, symbols]
* so we have

  env = scons.makeEnv("imageproc",
                      r"$HeadURL: svn+ssh://svn.lsstcorp.org/DC2/imageproc/tickets/7/SConstruct $",
                      [ ...
                        ["fw", "lsst/fw/LsstBase.h lsst/fw/Trace.h", "fw"],
                        ...
                      ])

* Next up we have to edit env.libs.  Why this is the case, I am not sure.  This is an outstanding issue.  

  env.libs = dict([
                    ...
                    ("fw",  Split("fw")),
                    ...
                 ])

* These two "fw" seem redundant with the "fw" above.
*
* However, note that you can "hack" in a change to tests/SConscript to add the location of your lib/include files with

  env.Append(LIBPATH = os.path.join(os.environ["LSST_DEVEL"], "DC2", "scons", "lib"))
  env.Append(CPPPATH = os.path.join(os.environ["LSST_DEVEL"], "DC2", "scons", "include"))

* but you will probably not be able to run any of your code, because your LD_LIBRARY_PATH is not fully set.