Home/ Questions/Q 9110017
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-17T03:10:39+00:00

I have an input file which is in a Fortran namelist format which I

  • 0

I have an input file which is in a Fortran “namelist” format which I would like to parse with python regular expressions. Easiest way to demonstrate is with a ficticious example:

$VEHICLES
 CARS= 1,
 TRUCKS = 0,
 PLAINS= 0, TRAINS = 0,
 LIB='AUTO.DAT',
C This is a comment
C Data variable spans multiple lines
 DATA=1.2,2.34,3.12,
      4.56E-2,6.78,
$END
$PLOTTING
 PLOT=T,
 PLOT(2)=12,
$END

So the keys can contain regular variable-name characters as well as parenthesis and numbers. The values can be strings, boolean (T, F, .T., .F., TRUE, FALSE, .TRUE., .FALSE. are all possible), integers, floating-point numbers, or comma-separated lists of numbers. Keys are connected to their values with equal signs. Key-Value pairs are separated by commas, but can share a line. Values can span multiple lines for long lists of numbers. Comments are any line beginning with a C. There is generally inconsistent spacing before and after ‘=’ and ‘,’.

I have come up with a working regular expression for parsing the keys and values and getting them into an Ordered Dictionary (need to preserve order of inputs).

Here’s my code so far. I’ve included everything from reading the file to saving to a dictionary for thoroughness.

import re
from collections import OrderedDict

f=open('file.dat','r')
file_str=f.read()

#Compile regex pattern for requested namelist
name='Vehicles'

p_namelist = re.compile(r"\$"+name.upper()+"(.*?)\$END",flags=re.DOTALL|re.MULTILINE)

#Execute regex on file string and get a list of captured tokens
m_namelist = p_namelist.findall(file_str)

#Check for a valid result
if m_namelist:
    #The text of the desired namelist is the first captured token
    namelist=m_namelist[0]

#Split into lines
lines=namelist.splitlines()

#List comprehension which returns the list of lines that do not start with "C"
#Effectively remove comment lines
lines = [item for item in lines if not item.startswith("C")]

#Re-combine now that comment lines are removed
namelist='\n'.join(lines)

#Create key-value parsing regex
p_item = re.compile(r"([^\s,\=]+?)\s*=\s*([^=]+)(?=[\s,][^\s,\=]+\s*\=|$)",flags=re.DOTALL|re.MULTILINE)

#Execute regex
items = p_item.findall(namelist)

#Initialize namelist ordered dictionary
n = OrderedDict()

#Remove undesired characters from value    
for item in items:
    n[item[0]] = item[1].strip(',\r\n ')

My question is whether I’m going about this correctly. I realize there is a ConfigParser library, which I have not yet attempted. My focus here is the regular expression:

([^\s,\=]+?)\s*=\s*([^=]+)(?=[\s,][^\s,\=]+\s*\=|$)

but I went ahead and included the other code for thoroughness and to demonstrate what I’m doing with it. For my Regular Expression, because the values can contain commas, and the key-value pairs are also separated by commas, there is no simple way to isolate the pairs. I chose to use a forward look-ahead to find the next key and “=”. This allows everything between the “=” and the next key to be the value. Finally, because this doesn’t work for the last pair, I threw in “|$” into the forward look-ahead meaning that if another “VALUE=” isn’t found, look for the end of the string. I figured matching the value with [^=]+ followed by a look-ahead was better than trying to match all possible value types.

While writing this question I came up with an alternative Regular Expresson that takes advantage of the fact that numbers are the only value that can be in lists:

 ([^\s,\=]+?)\s*=\s*((?:\s*\d[\d\.\E\+\-]*\s*,){2,}|[^=,]+)

This one matches either a list of 2 or more numbers with (?:\s*\d[\d\.\E\+\-]*\s*,){2,} or anything before the next comma with [^=,].

Are these somewhat messy Regular Expressions the best way to parse a file like this?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I would suggest to develop little more sophisticated parser.

    I stumble upon the project on google code hosting that implements very similar parser functionality: Fortran Namelist parser for Python prog/scripts but it was build for little different format.
    I played with it a little and updated it to support structure of the format in your example:

    Please see my version on gist:
    Updated Fortran Namelist parser for python https://gist.github.com/4506282

    I hope this parser will help you with your project.

    Here is example output produced by the script after parsing FORTRAN code example:

    {'PLOTTING': 
    {'par': 
        [OrderedDict([('PLOT', ['T']), ('PLOT(2) =', ['12'])])],
    'raw': ['PLOT=T', 'PLOT(2)=12']},
 'VEHICLES': 
    {'par': 
        [OrderedDict([('TRUCKS', ['0']), ('PLAINS', ['0']), ('TRAINS', ['0']), ('LIB', ['AUTO.DAT']), ('DATA', ['1.2', '2.34', '3.12', '4.56E-2', '6.78'])])],
  'raw': 
                ['TRUCKS = 0',
                  'PLAINS= 0, TRAINS = 0',
                  "LIB='AUTO.DAT'",
                  'DATA=1.2,2.34,3.12',
                  '4.56E-2,6.78']}}
    • 0