Question

What have people used to catch, log, and report multiple data validation errors at once in Python?

I’m building an application in Python 3 that first validates input data and then processes it. Reporting errors in the first step is part of the intended functionality of the program, so I don’t want my validator to give up on the first exception. In particular, the data are tabular and I want be able to return — rather than raise — an exception for each line of the table that does not validate.

A forum discussion from a couple of years ago contemplates multiple solutions, including the following, which seems the cleanest to me:

errors = []
for item in data:
    try:
        process(item)
    except ValidationError as e:
        errors.append(e)
if errors:
    raise MultipleValidationErrors(errors)

where the MultipleValidationErrors class would have an appropriate __str__ method to list useful information about all the ValidationErrors in it.

Others recommend using the traceback module, but since the exceptions I want to catch are data validation errors rather than program errors, that seems inappropriate. Getting the logging module involved might be appropriate, though.

Answer 1

I’ve used this idiom in both C++ and Python. It’s the cleanest solution that I know of when what you want is an exception, rather than a log message. The downside to it is that the combined exception takes up linear space in general, which can be problematic when processing large datasets with many errors.