Gahhh!! Django, virtualenv, and cx_Oracle

Published on 2009-11-24 13:47:00+00:00
apache   cx_oracle   django   virtualenv  

UPDATE: Thanks in advance to the comments from Graham Dumpleton whose comments below pointed me in the right direction!

This was~~is~~ a plea for help.

I've got django installed and configured with apache and virtualenv. I also have one particular app (named myapp) that queries an Oracle database directly (django is configured to use MySQL). All of the apps work, except for anything thatbrequires the myapp app... which includes the admin!

Requesting any view that uses cx_Oracle results in a cryptic error simiar to:

ViewDoesNotExist at /some/url/
Could not import myproject.myapp.views. Error was: cannot open shared object file: No such file or directory

The full traceback follows:

Request Method: GET  
Request URL:  
Django Version: 1.1.1  
Python Version: 2.6.2  
Installed Applications:  
 ... *snip*...  
Installed Middleware:  
File "/home/django/.virtualenvs/myproject/lib/python2.6/site-packages/django/core/handlers/" in get_response  
  83.                     request.path_info)  
File "/home/django/.virtualenvs/myproject/lib/python2.6/site-packages/django/core/" in resolve  
  218.                     sub_match = pattern.resolve(new_path)  
File "/home/django/.virtualenvs/myproject/lib/python2.6/site-packages/django/core/" in resolve  
  218.                     sub_match = pattern.resolve(new_path)  
File "/home/django/.virtualenvs/myproject/lib/python2.6/site-packages/django/core/" in resolve  
  125.             return self.callback, args, kwargs  
File "/home/django/.virtualenvs/myproject/lib/python2.6/site-packages/django/core/" in _get_callback  
  134.             raise ViewDoesNotExist, "Could not import %s. Error was: %s" % (mod_name, str(e))  
Exception Type: ViewDoesNotExist at /myapp/foo/  
Exception Value: Could not import myproject.myapp.views. Error was: cannot open shared object file: No such file or directory

The strange thing is that tests using cx_Oracle run correctly (as does code typed into the interpreter via ./ shell), so cx_Oracle is installed correctly in the virtualenv. Orignally, I thought that apache was somehow misconfigured. The problem, however, is that Apache just doesn't know where to find the shared libraries for cx_Oracle's C extension modules. On my system (Ubuntu), shared libraries are located in /usr/lib and /usr/local/lib .

When building cx_Oracle, you must include an environment variable for ORACLE_HOME and update the LD_LIBRARY_PATH so that it also includes ORACLE_HOME. While logged in as the django user, I updated .bashrc so that it includes the following:

export ORACLE\_HOME=/home/django/oracle/instantclient_10_2  

For me, all of the cx_Oracle shared libraries are located in /home/django/oracle/instantclient_10_2. So, (with my virtualenv activated) I built, installed, and tested cx_Oracle, and everything worked fine, because my user environment was set up so that these shared libraries are available.

To make these available to Apache, I created hard links to all of the shared libraries from /usr/local/lib, and then ran ldconfig. Note that ldconfig ignores symbolic links!

cd /usr/local/lib  
sudo ln /home/django/oracle/instantclient_10_2/  
# also creating links to all other libs  
sudo ldconfig  

After restarting Apache, my django apps worked as expected. For the curious, my wsgi script and apache config follows.

My .wsgi script

import sys  
import site  
import os  
vepath = '/home/django/.virtualenvs/myproject/lib/python2.6/site-packages'  
prev_sys_path = list(sys.path)  
# add the site-packages of our virtualenv as a site dir  
# add the app's directory to the PYTHONPATH  
os.environ['PYTHON\_EGG\_CACHE'] = '/home/django/.python-eggs'  
new_sys_path = [p for p in sys.path if p not in prev_sys_path]  
for item in new_sys_path:   
sys.path[:0] = new_sys_path  
from django.core.handlers.wsgi import WSGIHandler  
os.environ['DJANGO\_SETTINGS\_MODULE'] = 'myproject.settings'  
application = WSGIHandler()  

The Pertinent part of the Apache Config, located within a VirtualHost Directive:

Alias /media/ /var/www/media/  
Alias /site\_media/ /var/www/static/  
# Run WSGI in Daemon Mode:  
WSGIDaemonProcess myproject user=www-data group=www-data threads=25  
WSGIProcessGroup myproject  
WSGIScriptAlias / /home/django/django\_projects/myproject\_root/apache\_mod\_wsgi\_conf/myproject.wsgi  

NOTE: Graham Dumpleton suggests four options to get around the original problem (and I paraphrase):

  1. Rebuild cx_Oracle setting the LD_RUN_PATH - I tried this, but was unnsuccessful.

  2. Put the shared libraries in /usr/lib - I hard linked to the libraries from /usr/local/lib. This is the solution that worked.

  3. Modify /etc/ld.conf so that it includes the path to the shared libraries - I considered adding a file to /etc/ which would have included the path to my ORACLE_HOME directory. However, the above options worked, so I went with that instead.

  4. Edit /etc/apache/envvars so that it includes the appropriate environment variables - I also tried this approach, including the same environment variables that I added to my .bashrc, but this also did not work

Hope this helps someone else!