Question

So far, I’ve been working only with python manage.py celeryd, starting it like this:

python manage.py celeryd -l info --settings=settings

The code from my view, does this:

BinaryExecTask.delay(request.POST["binary_path"])

And the code from my settings.py, is this:

import djcelery
djcelery.setup_loader()

BROKER_BACKEND = "djkombu.transport.DatabaseTransport"
#celery
BROKER_HOST = "localhost"
BROKER_PORT = 5672
BROKER_USER = "guest"
BROKER_PASSWORD = "guest"
BROKER_VHOST = "/"

and it will execute some binaries in the background. The thing is, some of the binaries take pretty short time to run, while others may take up to half an hour. Working with celeryd, all my tasks are blocked until the current one finishes it’s execution. I saw here some examples of starting celeryd_multi, but running:

python manage.py celeryd_multi start 3 --settings=settings -l info

gives this error:

celeryd-multi v2.3.1
> Starting nodes...
        > celery1.x: Traceback (most recent call last):
  File "manage.py", line 14, in <module>
    execute_manager(settings)
  File "c:\code\python27\lib\site-packages\django-1.3-py2.7.egg\django\core\management\_
 line 438, in execute_manager
    utility.execute()
  File "c:\code\python27\lib\site-packages\django-1.3-py2.7.egg\django\core\management\_
 line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "c:\code\python27\lib\site-packages\django_celery-2.3.3-py2.7.egg\djcelery\manage
s\celeryd_multi.py", line 22, in run_from_argv
    ["%s %s" % (argv[0], argv[1])] + argv[2:])
  File "c:\code\python27\lib\site-packages\celery-2.3.1-py2.7.egg\celery\bin\celeryd_mul
 172, in execute_from_commandline
    self.commands[argv[0]](argv[1:], cmd)
  File "c:\code\python27\lib\site-packages\celery-2.3.1-py2.7.egg\celery\bin\celeryd_mul
 205, in start
    retcode = self.waitexec(argv)
  File "c:\code\python27\lib\site-packages\celery-2.3.1-py2.7.egg\celery\bin\celeryd_mul
 354, in waitexec
    pipe = Popen(argstr, env=self.env)
  File "c:\code\python27\lib\subprocess.py", line 672, in __init__
    errread, errwrite)
  File "c:\code\python27\lib\subprocess.py", line 882, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified

The celeryd-multi start 3 -c 3 throws the same error. What should I do so that I could succesfully start a celery instance that will allow me to run more tasks in parallel? Also, would I need to do something different in my view?

EDIT: some debugging led me here ( site-packages\celery-2.3.1-py2.7.egg\celery\bin\celeryd_multi.py(354)waitexec )

351         def waitexec(self, argv, path=sys.executable):
352             args = " ".join([path] + list(argv))
353  ->         argstr = shlex.split(args.encode("utf-8"))
354             pipe = Popen(argstr, env=self.env)
(Pdb) p argstr
['c:codepython27python.exe', 'manage.py', 'celeryd_detach', '-l', 'info', '--pidfile=[e-mail]',
 '-n', 'celery1.x', '--logfile=[e-mail]']
(Pdb) p Popen(argstr, env=self.env)
*** WindowsError: WindowsError(2, 'The system cannot find the file specified')
(Pdb)

So, as we can see, the path to Python gets destroyed :). What should I do next?

EDIT2: I opened an issue here

Answer 1

Looks like smth Windows-specific… Did you try to provide full path to python executable in command line like

C:\code\python27\bin\python.exe manage.py celeryd_multi start 3 --settings=settings -l info

Also, use full paths for python file can be usefull

C:\code\python27\bin\python.exe  C:\path\to\your\project\manage.py celeryd_multi start 3 --settings=settings -l info