Question

I have a socket which receives information in packets of 1024 characters or less. In those packets I some messages or parts thereof. Each message is ended with 2 characters.

The problem appears when there is fragmentation of messages across two packages. What algorithm do you recommend to handle this fragmentation? (I’m not looking to reinvent the wheel here, because I think this is a ubiquitous problem).

Example:

|011 012 013 014 \r\r 021 022 023 | 024 \r\r 031 032 033 | 034 \r\r 041 042 043 044 \r\r |

| … | -> packet

xxx xxx … \r\r -> a message

Answer 1

buffer = ''

# reading loop
while True:

  data = socket.recv(1024)
  if not data:
    break

  # add the current data read by the socket to a temporary buffer
  buffer += data

  # search complete messages
  messages = buffer.split('\r\r')

  # we need at least 2 messages to continue
  if len(messages) == 1:
    continue

  # seperator found, iterate across complete messages
  for message in messages [:-1]:
    # handle here the message
    print message

  # set the buffer with the last cutted message
  buffer = messages [-1]