Home/ Questions/Q 8772599
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-13T18:04:24+00:00

I’ve tried to simplify this as much as possible but I’m still getting an

  • 0

I’ve tried to simplify this as much as possible but I’m still getting an error. I have a simple http server (bottle) that upon receiving a post request executes a function which is supposed to quickly fork itself. The parent process simply returns a job ID and closes while the child process continues to process same data (which is a list of URLs). I’ve removed all the input and output functions and hard coded the data but my program is still crashing. The funny part is when I alter the program to run directly at the command line rather then start an http server and wait for bottle to execute it everything works fine!

#!/usr/bin/python
#This is a comment
import sys, time, bottle, os
from threading import Thread
from Queue import Queue
from bottle import route, run, request, abort

num_fetch_threads = 2
url_queue = Queue()

def fetchURLContent(i, q):
  while True:
    #print '%s: Looking for URLs in queue' % i
    url = q.get()
    #print 'URL found: %s' % url[0]
    q.task_done()
    time.sleep(1)

@route('/', method='POST') # or @route('/login', method='POST')
def main():

  urls = ['http://www.yahoo.com', 'http://www.google.com']

  newpid = os.fork()
  if newpid == 0:

    for i in range(num_fetch_threads):
      worker = Thread(target=fetchURLContent, args=(i, url_queue))
      worker.setDaemon(True)
      worker.start()
    print 'Queuing: ', url
    for url in urls:
      url_queue.put(url)
    time.sleep(2)
    print 'main thread waiting...'
    url_queue.join()
    print 'Done'

  else:
    print "Your job id is 5"
    return
def webServer():
  run(host='33.33.33.10', port=8080)

if __name__ == "__main__":
  print 'Listening on 8080...'
  webServer()

The error message I get is as follows:

Listening on 8080...
Bottle v0.11.3 server starting up (using WSGIRefServer())...
Listening on http://33.33.33.10:8080/
Hit Ctrl-C to quit.

33.33.33.1 - - [19/Oct/2012 21:21:24] "POST / HTTP/1.1" 200 0
Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 86, in run
    self.finish_response()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 128, in finish_response
    self.finish_content()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 246, in finish_content
    self.send_headers()
  9 url_queue = Queue()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 268, in send_headers
    self.send_preamble()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 189, in send_preamble
    self._write('HTTP/%s %s\r\n' % (self.http_version,self.status))
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 389, in _write
    self.stdout.write(data)
  File "/usr/lib/python2.7/socket.py", line 324, in write
    self.flush()
  File "/usr/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
----------------------------------------
Exception happened during processing of request from ('33.33.33.1', 57615)
Traceback (most recent call last):
  File "/usr/lib/python2.7/SocketServer.py", line 284, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 310, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
    self.finish()
  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish
    self.wfile.flush()
  File "/usr/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
----------------------------------------

Any ideas?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Your main() function terminates immediately without returning anything. Bottle writes an empty HTTP response to the socket and the web server closes the connection.

    Your forked off process stays a bit longer in main(), but then terminates too and causes Bottle to write another empty response to the already closed socket. Thats the error you get (broken pipe).

    Forking at that point cannot work. HTTP does not allow more than one response per request. You can either block until all work is done and then send a response, or send the response immediately and do the work in a differed thread.

    • 0