Question

I have a string that has several date values in it, and I want to parse them all out. The string is natural language, so the best thing I’ve found so far is dateutil.

Unfortunately, if a string has multiple date values in it, dateutil throws an error:

>>> s = "I like peas on 2011-04-23, and I also like them on easter and my birthday, the 29th of July, 1928"
>>> parse(s, fuzzy=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/pymodules/python2.7/dateutil/parser.py", line 697, in parse
    return DEFAULTPARSER.parse(timestr, **kwargs)
  File "/usr/lib/pymodules/python2.7/dateutil/parser.py", line 303, in parse
    raise ValueError, "unknown string format"
ValueError: unknown string format

Any thoughts on how to parse all dates from a long string? Ideally, a list would be created, but I can handle that myself if I need to.

I’m using Python, but at this point, other languages are probably OK, if they get the job done.

PS – I guess I could recursively split the input file in the middle and try, try again until it works, but it’s a hell of a hack.

Answer 1

Looking at it, the least hacky way would be to modify dateutil parser to have a fuzzy-multiple option.

parser._parse takes your string, tokenizes it with _timelex and then compares the tokens with data defined in parserinfo.

Here, if a token doesn’t match anything in parserinfo, the parse will fail unless fuzzy is True.

What I suggest you allow non-matches while you don’t have any processed time tokens, then when you hit a non-match, process the parsed data at that point and start looking for time tokens again.

Shouldn’t take too much effort.

---

Update

While you’re waiting for your patch to get rolled in…

This is a little hacky, uses non-public functions in the library, but doesn’t require modifying the library and is not trial-and-error. You might have false positives if you have any lone tokens that can be turned into floats. You might need to filter the results some more.

from dateutil.parser import _timelex, parser

a = "I like peas on 2011-04-23, and I also like them on easter and my birthday, the 29th of July, 1928"

p = parser()
info = p.info

def timetoken(token):
  try:
    float(token)
    return True
  except ValueError:
    pass
  return any(f(token) for f in (info.jump,info.weekday,info.month,info.hms,info.ampm,info.pertain,info.utczone,info.tzoffset))

def timesplit(input_string):
  batch = []
  for token in _timelex(input_string):
    if timetoken(token):
      if info.jump(token):
        continue
      batch.append(token)
    else:
      if batch:
        yield " ".join(batch)
        batch = []
  if batch:
    yield " ".join(batch)

for item in timesplit(a):
  print "Found:", item
  print "Parsed:", p.parse(item)

Yields:

Found: 2011 04 23
Parsed: 2011-04-23 00:00:00
Found: 29 July 1928
Parsed: 1928-07-29 00:00:00

---

Update for Dieter

Dateutil 2.1 appears to be written for compatibility with python3 and uses a “compatability” library called six. Something isn’t right with it and it’s not treating str objects as text.

This solution works with dateutil 2.1 if you pass strings as unicode or as file-like objects:

from cStringIO import StringIO
for item in timesplit(StringIO(a)):
  print "Found:", item
  print "Parsed:", p.parse(StringIO(item))

If you want to set option on the parserinfo, instantiate a parserinfo and pass it to the parser object. E.g:

from dateutil.parser import _timelex, parser, parserinfo
info = parserinfo(dayfirst=True)
p = parser(info)