Bug of the day: How ConfigParser ruined my calculations

December 7, 2011

For my thesis, I have implemented a series of numerical applications. The core application (written in C++) uses simple INI-like configuration files for reading system parameters. For example:


[system]
name=example2
r1=1.0
r2=1.5
V1=-0.05
V2=0.1
[...]

Nothing exciting there, reading this is a breeze with QSettings. Now I needed to do some symbolic calculations, for which I chose Python and the excellent SymPy module. To read the system configuration, I chose the standard ConfigParser module.


parser = ConfigParser.RawConfigParser()
parser.read("scatttd.conf")
sysType = parser.get("system", "name")
# read system parameters
sysParams = dict()
for key, value in parser.items("system"):
    if value != "name":
        sysParams[key] = float(value)

That reads the “system/name” key, and puts all other keys (which are assumed to be floats) into a dictionary, so the system-specific code does not need to deal with the ConfigParser object.


r1, r2 = params.get("r1", 1.0), params.get("r2", 1.0)
V1, V2 = params.get("V1", -0.1), params.get("V2", +0.1)

This code gets the parameters in the system-specific code. Nothing exciting here. Except, stuff doesn’t work. The results of the following calculations suddenly don’t match my expectations (or the predictions made by my other programs).

Since the code after the snippet above is about 30 lines of SymPy magic (accumulating expressions that print dozens of lines onto my terminal), I suspected that some error occurred there. After about two days of searching for the problem there, and calculating everything by hand, something occurred to me when I looked at the debug output:


DEBUG V1 = -0.1

Didn’t the configuration file say “V1=-0.05”? Let’s have a look at the parameter dict:


{'v1': -0.05, 'v2': 0.1, 'r1': 1.0, 'r2': 1.5}

See the problem? “v1” has a lower-case “v”, so params.get("V1", -0.1) fails and returns the default value (-0.1). One glimpse at the documentation says that


parser.optionxform = str

solves the problem. Gah!

6 Responses to “Bug of the day: How ConfigParser ruined my calculations”

  1. Hyperion Says:

    Your “solution” could be quite dangerous, since you convert Unicode-Strings into Byte-Strings (in Python 2.x). So it would be better to use the proposed solution in the documentation:
    “parser.optionxform = lambda option: option“
    That way you do not transform anything.

  2. gunni Says:

    RTFM
    SCNR
    😉

  3. Henry Miller Says:

    This is why in industry we test modules separately. That way you know your algorithm is working (or not), even if the load/store fails. Unit tests work wonders.

    For all the nay-sayers, a unit test may not have caught this particular bug. However it would have narrowed the place where the bug could exist closer to the right spot, and therefore make it easier to fix.


Leave a comment