21 March 2009

No module named _md5 - Missing OpenSSL?

I'm working a Python project on an embedded Linux device. As usual, I've had my struggles but yesterday I ran into one a tough one. I was afraid that I was going to have to dump Python and go to another language, ruining a week+ of effort.

I'm using OpenWrt kamikaze 8.09 and have been making excellent progress until I slammed into the wall when attempting to import md5. This spins off an odd trace that ends with "No module named _md5". It seems that in Python 2.5, the md5 module was deprecated for hashlib but something seems fishy.

I'm not the only dweeb with this problem. Mr Google has pages of people whining and tons of people talking about patches. Maybe they're not having the exact problem I did, but lots of problems and no one is reporting success.

Here's my exact issue:
# python
Python 2.5.4 (r254:67916, Feb 2 2009, 22:32:58)
[GCC 3.4.6 (OpenWrt-2.0)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> import md5
Traceback (most recent call last):
File "", line 1, in
File "/opt/usr/lib/python2.5/md5.py", line 6, in
from hashlib import md5
File "/opt/usr/lib/python2.5/hashlib.py", line 133, in
md5 = __get_builtin_constructor('md5')
File "/opt/usr/lib/python2.5/hashlib.py", line 60, in __get_builtin_constructor
import _md5
ImportError: No module named _md5
Using the various tips in the Google searches, I mucked with my LD_LIBRARY_PATH and considered rebuilding Python. Finally, I actually started looking at the source referenced in the trace. Let's see what's there.

/opt/usr/lib/python2.5/md5.py, line 6
0 # $Id: md5.py 39316 2005-08-21 18:45:59Z greg $
1 #
2 # Copyright (C) 2005 Gregory P. Smith (greg@electricrain.com)
3 # Licensed to PSF under a Contributor Agreement.
4
5 from hashlib import md5
6 new = md5
A simple import and new. Nothing exciting there. Next.

/opt/usr/lib/python2.5/hashlib.py, line 133
131 # lookup the C function to use directly for the named constructors
132 md5 = __get_builtin_constructor('md5')
133 sha1 = __get_builtin_constructor('sha1')
134 sha224 = __get_builtin_constructor('sha224')
Ok, we're calling a built-in contructor. I'm a complete Python newb. WTFO? Next.

/opt/usr/lib/python2.5/hashlib.py, line 60
54 def __get_builtin_constructor(name):
55 if name in ('SHA1', 'sha1'):
56 import _sha
57 return _sha.new
58 elif name in ('MD5', 'md5'):
59 import _md5
60 return _md5.new
Alright, here's where the built-in constructor is defined and sure enough, it imports _md5. What sleuthing, I'm amazing. We know that the whole frickin problem is that module _md5 can't be found, this tangent really helped.

Scratch head, ear, and other parts. Let's look at that code again. Why are we calling that built-in constructor? Actually, I have no clue. But I did notice line 126 that's conveniently located above our error at line 133.

126 except ImportError:
127 # We don't have the _hashlib OpenSSL module?
128 # use the built in legacy interfaces via a wrapper function
129 new = __py_new
130
131 # lookup the C function to use directly for the named constructors
132 md5 = __get_builtin_constructor('md5')

An ImportError exception. Yup, that's what we have. But a comment that mentions a missing OpenSSL module. Woo Hoo, eureka!

Frantically do this:
# opkg update
# opkg install openssl-util
And like magic, no more _md5 errors. For me, it was a simple missing dependency on OpenSSL. I hope your error is this easy to correct.

FYI, the top line of my opkg.conf file is:
BTW, opkg is a newer version of ipkg. Either is simply "the package manager on the system".

What do you think? Did this help you? Leave a comment.

7 comments:

gracious said...

Yes, that fixed my problem. Thanks for your help!!

Riccardo said...

I LOVE YOU

Fredrik said...

Hi,
I'm having the same problem but I'm running ipkg on a QNAP NAS. I can't get openssl-util with ipkg. It's an ARM cpu... Any alternatives?

Thanks in advance :)

Darko M. said...

Great, except that on my system I also had to update/install package python-openssl (besides openssl).

Anonymous said...

I've got the same problem on a Solaris 10 x86 box and I can't figure out how to apply your solution. (My Unix-fu is weak.)

Can anybody help? (Yes - I'm desperate)

mcnutt at utk.edu

Anonymous said...

Thanks a million! Your info
"No module named _md5"
== "Missing OpenSSL"
saved my day (or rather night ;-).

In my case, the solution would be to build Pyhton without OpenSSL (don't ask why...)

Regards, B'l

TeradataAsterQA said...

Thank you so much for your blog. I faced same issue on solaris-sparc. Your hint to install openssl helped me resolve the issue. There were couple of missing libraries which I had to download and add their path to LD_LIBRARY_PATH to get it fully working. Thanks again!