File test operators, the garden path of race conditions

[ Perl tips index ]
[ Subscribe to Perl tips ]

A race condition exists when the result of several interrelated events depends on the ordering of those events, but that order cannot be guaranteed due to nondeterministic timing effects.

Programming Perl, 3rd Ed -- Larry Wall, Tom Christiansen and Jon Orwant

When we write code, we often assume that our program will run continuously until the program terminates. In fact this is rarely the case. On most modern machines, the operating system multi-tasks, which is a fancy way of saying appears to run many things at the same time. Since single-CPU machines can only really run a single process at a time, individual programs are paused and restarted many times during their invocation. Fortunately this typically happens so fast that it is rarely noticeable to the user.

The effects of multi-tasking can mean that your system state can change unexpectedly between one statement and the next. For example, imagine that one code statement tests that a certain file exists, and the next statement executes code to open this file. If the program is paused by the operating system before executing the second statement, then the file could be deleted or moved by another program, and the assumption that the file exists is now false. This is called a race condition.

At the best race conditions can result in unexpected and difficult to debug behaviour from your programs. Usually they work fine, but sometimes they will have problems. Two processes may start writing to the same file, resulting in data corruption, or two processes may start processing the same information, resulting in duplicated work. At their worst, race-conditions can result in security holes on your system. There's a long tradition of attackers using name-guessing and symbolic links to trick privileged programs into opening and potentially rewriting critical files.

Unfortunately, it's easy to fall prey to race conditions -- they're not always obvious, and they are rarely if ever detected during testing. In fact, Perl comes with a set of operators that are so often associated with race conditions that they are sometimes referred to as the 'garden path' of race conditions. These are Perl's file test operators.

The file test operators are extremely easy to use, and by themselves they present no problems. However, file test operators are commonly used to decide whether or not a file should be opened:

        # Create a file if it does not already exist.
        if (not -e $file ) {
                open(my $fh, ">", $file") or die "Cannot open $file - $!";

The above example contains a race condition. At the time of the exists test (-e), the file may well not exist. However, by the time we reach the open another process may have created it. Our program would then destroy the contents of that file, which may be in use by another instance of the same program. If our program was running with privileges, this code could potentially chase a symbolic link (a type of shortcut) and destroy a critical file on our system.

This code provides a false sense of security, as it appears that we're taking measures against accidental overwriting of files, but in reality we're still vulnerable. Due to the difficulty of reproducing the race condition, it can be extremely difficult to diagnose and debug.

This doesn't mean that you can't use file-test operators when coding securely, but you must be fully aware that files can change in between the testing of a file and any other operation which you may perform upon it.

Avoiding the need for file-test operators

We used a file-test operator to test the existence of a file before opening it, but as we've just seen this is a mistake. Not only can this result in a race condition, but sometimes it's also unnecessary. If all we want is to check file access permissions, then we should just go ahead and try to open the file. If we don't have permission, this operation will result in failure.

To avoid the more difficult problems, such as following symbolic-links, or truncating files which already exist, Perl gives us the sysopen command. This allows us to specify conditions on the file at the same time as we access it. The atomicity of this test and open is guaranteed, thus race conditions are avoided.

Here are some examples of using sysopen and their test/open equivalent:

        use Fcntl;

        # Open file for writing, only if it doesn't already exist.
        # This also avoids chasing symbolic links on Unix systems.

        sysopen(my $fh, $outfile, O_WRONLY|O_CREAT|O_EXCL)
                or die "Failed to open $outfile: $!";
        # equivalent to (race condition version)
        if (-e $outfile) {
                die "$outfile already exists on system";
        open(my $fh, ">", $outfile)
                or die "Failed to open $outfile: $!";

        # Open file for writing,  create it if does not exist, truncate if
        # it does, and do not follow symbolic links.  Not all
        # operating systems support the O_NOFOLLOW option.

        sysopen(my $fh, $outfile, O_WRONLY|O_CREAT|O_TRUNC|O_NOFOLLOW)
                or die "Failed to open $outfile: $!";

        # equivalent to (race condition version)
        if (-l $outfile) {
                die "$outfile is a sym-link";
        open(my $fh, ">", $outfile)
                or die "Failed to open $outfile: $!";

The sysopen function allows many other combinations of options for file access, and should be considered whenever you need fine-grained control over how a file is opened. To find out more, read perldoc -f sysopen.

The topic of race conditions and filesystem security is examined in-depth as part of Perl Training Australia's Perl Security course ( ).

[ Perl tips index ]
[ Subscribe to Perl tips ]

This Perl tip and associated text is copyright Perl Training Australia. You may freely distribute this text so long as it is distributed in full with this Copyright noticed attached.

If you have any questions please don't hesitate to contact us:

Phone: 03 9354 6001 (Australia)
International: +61 3 9354 6001

Valid XHTML 1.0 Valid CSS