Introducing YippieMove '09. Easy email transfers. Now open for all destinations.

Update: Please note that this does not apply to FreeBSD 8.x as POSIX Semaphores are now enabled by default. Hence you do not need to recompile the kernel.

Celery is a great product. Unfortunately getting it to work on FreeBSD is a bit of a hassle. After some troubleshooting we managed to get it running, here’s how we did it.

We are running FreeBSD 7.2, but this is likely to work on 7.x.

Step 1: Recompile kernel

I assume that you know how to compile the kernel on FreeBSD. If not, you can find out how to do it here. I also assume that you have the source code for FreeBSD installed.

Let’s head over to the code.
#cd /usr/src/sys/i386/conf

Make a copy of the standard kernel-config.
#cp GENERIC YOUR_KERNEL

Now we need to edit the kernel-config. You can obviously use any editor, but I prefer joe.
#joe YOUR_KERNEL

Change the ‘ident’ from GENERIC to YOUR_KERNEL
ident YOUR_KERNEL

This is the most important part. We need to add support for POSIX Semaphores. Add the following line:
options P1003_1B_SEMAPHORES # POSIX-style semaphores

That’s it for the kernel-config. Now just save the changes and exit (^K+X in joe).

Next up is the compiling and installing the actual kernel.
#cd /usr/src
#make buildkernel KERNCONF=YOUR_KERNEL
#make installkernel KERNCONF=YOUR_KERNEL
#reboot

That’s it. If everything went well, you should now have a kernel with POSIX Semaphores enabled.

Step 1: Install Python

I assume you got the ports installed, so here we go.

#cd /usr/ports/lang/python26
#make config

Make sure to select SEM and deselect PTH. If PTH is enabled, Celery won’t work.

If you’re running FreeBSD <7.2, you will need to edit Makefile and comment out the following:

Line 63-71:
#.if defined(WITH_SEM)
#.if ${OSVERSION} >= 701106
#SEM_MSG= ""
#.else
#IGNORE= POSIX semaphore support only works in FreeBSD 7-STABLE and later
#.endif # ${OSVERSION} >= 701106
#.else # !defined(WITH_SEM)
#SEM_MSG= "@comment "
.#endif # defined(WITH_SEM)

and line 186:
#.if ${OSVERSION} >= 701106

Next up, install Python:
#make clean install

Step 3: Test with simple app

With Python installed, let’s see if it actually works. To accomplish that, we will create a simple app.

Create a new file: test_mp.py

import multiprocessing as mp
p = mp.Pool(16)
import time
p.apply_async(time.sleep, 1)

Let’s run it:
#python test_mp.py

If the application silently exits within a second or so, you’re good to go. If not, please re-visit Step 1 and Step 2 to make sure you configured everything properly.

Step 4: Install Celery

This is probably the easiest step:
#easy_install celery

That’s it! Good luck!

Credits: Thanks asksol and raysl over at #celery.

Be Sociable, Share!
Author:

Trackbacks/Pingbacks

  1. Tweets that mention Playing With Wire » How to get Celeryd to work on FreeBSD -- Topsy.com
  2. uberVU - social comments

© 2006-2009 WireLoad, LLC.
Logo photo by William Picard. Theme based on BlueMod © 2005 - 2009 FrederikM.de, based on blueblog_DE by Oliver Wunder.