Uploaded image for project: 'OpenVDB'
  1. OpenVDB
  2. OVDB-134

Issues with Jemalloc >= 5.0.0 and python

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • None
    • Build, Core
    • None

      Currently the python module will fail to import if dynamically linked against jemalloc >= 5.0.0. This is to do with how python dynamically loads extension modules and a change that was made to jemalloc in 5.0.0. It manifests in the following error

      libjemalloc.so: cannot allocate memory in static TLS block
      

      There are a few runtime solutions to this but I can't see a nice way of solving this within our builder.

      1) Set LD_PRELOAD to point to the jemalloc.so i.e.

      > export LD_PRELOAD=/path/to/jemalloc.so
      > python
      >> import pyopenvdb
      

      2) Share symbols across extension libraries (only works form python 3.6)

      > python
      >> import sys, dl
      >> sys.setdlopenflags(dl.RTLD_NOW | dl.RTLD_GLOBAL)
      >> import pyopenvdb
      

      Note that jemalloc can be built in such a way that the initial execution model for jemalloc's TLS is disabled, which I imagine will also fix the issue - but I don't count this as a solution. I suspect linking against jemalloc statically will also fix the issue.

      Reference:
      https://github.com/jemalloc/jemalloc/issues/937
      https://github.com/jemalloc/jemalloc/blob/dev/INSTALL.md (--disable-initial-exec-tls)
      https://stackoverflow.com/questions/19663366/dlopen-fails-in-boost-python-module

            Unassigned Unassigned
            nna nna
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: