Compiling Python Extensions on MinGW

I’ve spent most of the weekend trying to compile the python-ldap extension on MinGW. It’s not a trivial task no.

It appears that I’ve got surprisingly close to have it working. Well, actually it looks like it might just work with some minor fiddling but I really wanted to avoid this fiddling.

The problem I’m facing is that the final .pyd extension is depending liblber.dll and friends, and I really wanted to avoid that. So I will describe the process I’ve followed in the hope that someone with more knowledge can point out what I’ve done wrong. First, some introduction about compiling python extensions with MinGW.

That was the easy part. Surprisingly (or not?) it just worked.

Then I had to follow a trail of non-obvious clues about how to get the needed dependencies in place to be able to compile the python extension itself.

After banging my head against the wall for a little and googling around for openldap windows I’ve stumbled on Lucas Bergman’s homepage, which linked to a devel tarball containing some patches and build scripts for compiling OpenLDAP on Windows.

‘Cool’, I’ve thought. This might just work after all.

I went on, downloading the tarballs for the exact versions of OpenLDAP, OpenSSL and Sleepycat’s DB 4.3.

Next step, extract Lucas’ devel tarball, put the tarballs for OpenLDAP, OpenSSL and Sleepycat’s DB 4.3 inside the openldap-mingw-build-5 directory, put cygwin’s bin directory on the path so MinGW finds perl.exe, run ./build-it and cross fingers.

After a couple tries and tweaking ./configure arguments I got the damn beast to finish compiling. That took me all of sunday.

Copied setup.cfg.mingw32 from python-ldap Build directory next to setup.py and tweaked the library_dirs and include_dirs to point to the right paths on my machine.

Ran python setup.py bdist_wininst and it surely worked on the first try! What a good surprise after all. The problem then was that the final .pyd depended on liblber.dll and a couple more. Copying those beasts next to the _ldap.pyd sure made it work but was far from optimal.

My best idea at this point was to ask Mark Hammond about any clues. All he told me was to make sure that OpenLDAP was compiled statically, which to my best knowledge it was. I’ve tried changing the ./configure options a couple times and a few rebuilds but to no effect.

Update: Now, after writing this, I realized that maybe distutils was not doing the Right Thing as one would expect. Looked at the output of running python setup.py build and surely it was doing something like this:

c:\cygwin\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.4\...

How odd is that? After all everything but distutils seemed to be doing the right thing, so digging in the source was the next step.

A quick look at distutils/cygwinccompiler.py and this is what I’ve found there:

# ld_version >= "2.13" support -shared so use it instead of
# -mdll -static
if self.ld_version >= "2.13":
    shared_option = "-shared"
else:
    shared_option = "-mdll -static"

Cool. So after all it was intentional. My version of ld is greater than 2.13:

C:\src\python-ldap-2.2.0>ld -v
GNU ld version 2.17.50 20060817

Taking a guess at what those parameters meant to do, I’ve changed the -shared to -mdll -static just like it’s supposed to do for versions of ld smaller or equal to 2.13:

if self.ld_version >= "2.13":
    shared_option = "-mdll -static"
else:
    shared_option = "-mdll -static"

And what a surprise! After re-running python setup.py bdist_wininst I end up with a python-ldap that does not depend on the nasty .dll! That’s amazing.

Uhm. Maybe distutils should grow an option for toggling static vs. dynamic building of extensions.

Not being a master into the intricate majik of compilers and linkers I cannot do but hope that the nasty thing works after all this trouble. So I’m making this installer available for people that want to give it a poke, and hopefully someone will complain loudly if it doesn’t work.