Update: Please note that this does not apply to FreeBSD 8.x as POSIX Semaphores are now enabled by default. Hence you do not need to recompile the kernel.
Celery is a great product. Unfortunately getting it to work on FreeBSD is a bit of a hassle. After some troubleshooting we managed to get it running, here’s how we did it.
We are running FreeBSD 7.2, but this is likely to work on 7.x.
Step 1: Recompile kernel
I assume that you know how to compile the kernel on FreeBSD. If not, you can find out how to do it here. I also assume that you have the source code for FreeBSD installed.
Let’s head over to the code.
#cd /usr/src/sys/i386/conf
Make a copy of the standard kernel-config.
#cp GENERIC YOUR_KERNEL
Now we need to edit the kernel-config. You can obviously use any editor, but I prefer joe.
#joe YOUR_KERNEL
Change the ‘ident’ from GENERIC to YOUR_KERNEL
ident YOUR_KERNEL
This is the most important part. We need to add support for POSIX Semaphores. Add the following line:
options P1003_1B_SEMAPHORES # POSIX-style semaphores
That’s it for the kernel-config. Now just save the changes and exit (^K+X in joe).
Next up is the compiling and installing the actual kernel.
#cd /usr/src
#make buildkernel KERNCONF=YOUR_KERNEL
#make installkernel KERNCONF=YOUR_KERNEL
#reboot
That’s it. If everything went well, you should now have a kernel with POSIX Semaphores enabled.
Step 1: Install Python
I assume you got the ports installed, so here we go.
#cd /usr/ports/lang/python26
#make config
Make sure to select SEM and deselect PTH. If PTH is enabled, Celery won’t work.
If you’re running FreeBSD <7.2, you will need to edit Makefile and comment out the following:
Line 63-71:
#.if defined(WITH_SEM)
#.if ${OSVERSION} >= 701106
#SEM_MSG= ""
#.else
#IGNORE= POSIX semaphore support only works in FreeBSD 7-STABLE and later
#.endif # ${OSVERSION} >= 701106
#.else # !defined(WITH_SEM)
#SEM_MSG= "@comment "
.#endif # defined(WITH_SEM)
and line 186:
#.if ${OSVERSION} >= 701106
Next up, install Python:
#make clean install
Step 3: Test with simple app
With Python installed, let’s see if it actually works. To accomplish that, we will create a simple app.
Create a new file: test_mp.py
import multiprocessing as mp
p = mp.Pool(16)
import time
p.apply_async(time.sleep, 1)
Let’s run it:
#python test_mp.py
If the application silently exits within a second or so, you’re good to go. If not, please re-visit Step 1 and Step 2 to make sure you configured everything properly.
Step 4: Install Celery
This is probably the easiest step:
#easy_install celery
That’s it! Good luck!
Credits: Thanks asksol and raysl over at #celery.
Author: Viktor Petersson