If you just want to match an exact string, using the index() builtin is faster, but sometimes you want to match more or less of a particular string. That’s when you want to use quantifiers in your regular expression. For example, to match the letter a followed by an optional letter b, and then the letter c, use the ? quantifier to show that the b is optional. The following will match both abc and ac:

if ( $word =~ /ab?c/ ) { ... }

The * is used to show that you can match zero or more of a given letter:

if ( $word =~ /ab*c/ ) { ... }

The + is used to show that you can match one or more of a given letter:

if ( $word =~ /ab+c/ ) { ... }

This sample code should make this clear. We will use the qr() quote-like operator. This allows us to properly quote a regular expression without trying to match it to anything before we’re ready.

my @strings = qw(
    abba
    abacus
    abbba
    babble
    Barbarella
    Yello
);
my @regexes = (
    qr/ab?/,
    qr/ab*/,
    qr/ab+/,
);
foreach my $string (@strings) {
    foreach my $regex (@regexes) {
        if ( $string =~ $regex ) {
            print "'$regex' matches '$string'\n";
        }
    }
}

And that prints out:

'(?-xism:ab?)' matches 'abba'
'(?-xism:ab*)' matches 'abba'
'(?-xism:ab+)' matches 'abba'
'(?-xism:ab?)' matches 'abacus'
'(?-xism:ab*)' matches 'abacus'
'(?-xism:ab+)' matches 'abacus'
'(?-xism:ab?)' matches 'abbba'
'(?-xism:ab*)' matches 'abbba'
'(?-xism:ab+)' matches 'abbba'
'(?-xism:ab?)' matches 'babble'
'(?-xism:ab*)' matches 'babble'
'(?-xism:ab+)' matches 'babble'
'(?-xism:ab?)' matches 'Barbarella'
'(?-xism:ab*)' matches 'Barbarella'

Sadly, nothing matches Yello, an excellent music group, but studying the rest of the matches should make it clear what is going on.

However, you may be wondering what that bizarre (?-xism:ab*) is doing on the regex we’ve printed out? Those are regular expression modifiers, which we’ll cover later in this chapter.

If you need to be more precise, you can use the {n,m} syntax. This tells the Perl that you want to match at least n times and no more than m times. There are three variants of this:

/ab{3}c/    # 1 a, 3 'b's, 1 c (only "abbbc")
/ab{3,}c/   # 1 a, 3 or more 'b's, 1 c
/ab{3,6}c/  # 1 a, 3 to 6 'b's, 1 c

Table 8.1, “Regex quantifiers” summarizes the different types of regex quantifiers and their meaning:

By default, all quantifiers in Perl are greedy. That means they’ll try to match as much as possible. For example, the dot metacharacter (.) means “match anything” except newlines and .* will match the rest of the string up to a newline. Later in the chapter when you learn to print out just the bits you’ve matched, you’ll discover that for the word cataract, the regular expression a.+a matches atara and not just ata. If you want a quantifier to be lazy (match as little as possible) instead of greedy, just follow it with a question mark:

if ( "cataract" =~ /a.+?a/ ) {
    # the first match is now "ata" instead of "atara"
}

{}[]()^$.|*+?\

Sometimes you want to match a wide variety of different things that are difficult to type, or may match a wide range of characters. Many of the common cases are handled with escape sequences. Table 8.2, “Common escape sequences” explains some of these sequences and we’ll give a few practical examples. This is not an exhaustive list, just a list of the more common sequences.

Of those, the ones you’ll most commonly see are \w (“word” characters), \d (digits), \s (whitespace) and \b (“word” boundary).

So let’s say you have some strings and you want to find all strings containing phone numbers matching the pattern XXX-XXX-XXXX where X can be any digit. You might use the following regular expression:

for my (@strings) {
    if ( /\d{3}-\d{3}-\d{4}/ ) {
        print "Phone number found: $string\n";
    }
}

And that will indeed match 555-867-5309. Unfortunately, it will also match a string containing 555555555-867-444444444 and that, presumably, is not a phone number. There are a several ways of dealing with this. If you know the phone number has whitespace on either side, you could do try to match whitespace with the \s escape:

for my (@strings) {
    if ( /\s\d{3}-\d{3}-\d{4}\s/ ) {
        print "Phone number found: $string\n";
    }
}

But maybe you don’t know what is on either side of the phone number. You might make a mistake and try to match “non-digits” with \D:

for my (@strings) {
    if ( /\D\d{3}-\d{3}-\d{4}\D/ ) {
        print "Phone number found: $string\n";
    }
}

That looks reasonable, but try this:

print "Phone: 123-456-7890" =~ /\D\d{3}-\d{3}-\d{4}\D/
    ? "Yes"
    : "No";

That prints No. Why? Because \D has to match something. The first \D matches a space, but the second one has nothing to match. What you actually want is the \b. That matches a word boundary. A word is matched by \w and that’s any alphanumeric character, plus the underscore. A word boundary matches no characters, but matches when there is a transition between a word and non-word character (this means that \w\b\w can never match anything).

print "Phone: 123-456-7890" =~ /\b\d{3}-\d{3}-\d{4}\b/
   ? "Yes"
   : "No";

That prints Yes because the final \b matches between the final digit and the end of the string.

At this point, you’re probably thinking “that’s nice, but what good is that data if you can’t get it?” It’s simple: just put parentheses around any data in a regular expression that you want to extract. For every set of capturing parentheses, use a $1, $2, $3, and so on, to access that data.

if ( "Phone: 123-456-7890" =~ /(\b\d{3}-\d{3}-\d{4}\b)/ ) {
    my $phone = $1;
    print "The phone number is $phone\n";
}

And that will print:

The phone number is 123-456-7890

You can use this to populate data structures. Consider the following block of text. You want to create a hash of names and their ages. Example 8.1, “Building data structures with regexes” shows an example of this.

use strict;
use warnings;
use diagnostics;
use Data::Dumper;
my $text = <<'END';
Name: Alice Allison Age: 23
Occupation: Spy
Name: Bob Barkely   Age: 45
Occupation: Fry Cook
Name: Carol Carson  Age: 44
Occupation: Manager
Name: Prince        Age: 53
Occupation: World Class Musician
END
my %age_for;
foreach my $line (split /\n/, $text) {
    if ( $line =~ /Name:\s+(.*?)\s+Age:\s+(\d+)/ ) {
        $age_for{$1} = $2;
    }
}
print Dumper(\%age_for);

And that will print something like:

$VAR1 = {
          'Bob Barkely' => '45',
          'Alice Allison' => '23',
          'Carol Carson' => '44',
          'Prince' => '53'
        };

If that regular expression is confusing, here’s a way to make it read easier: put an /x modifier at the end and all whitespace (unless escaped with a backslash) will be ignored. You can then put comments at the end of each part to explain it.

my $name_and_age = qr{
   Name:
   \s+      # 1 or more whitespace
   (.*?)    # The name in $1
   \s+      # 1 or more whitespace
   Age:
   \s+      # 1 or more whitespace
   (\d+)    # The age in $2
}x;
foreach my $line (split /\n/, $text) {
    if ( $line =~ $name_and_age ) {
        $age_for{$1} = $2;
    }
}

That makes regexes much easier to read.

As was explained earlier, the . metacharacter will match anything except newlines (but see the /s modifier later in this chapter). So .* means match zero or more of anything. Note that we made the .* lazy by adding a question mark after it. If we didn’t do this, it would have matched greedily and pulled in all of whitespace it could before the \s+. The resulting data structure would have looked like this:

$VAR1 = {
          'Carol Carson ' => '44',
          'Alice Allison' => '23',
          'Bob Barkely  ' => '45',
          'Prince       ' => '53'
        };

You can also use those “digit” variables in a regular expression. However, you precede them with a backslash. The $1 captured by the first set of parentheses is matched by \1. Here’s how you can find double words:

print "Four score score and seven years ago" =~ /\b(\w+)\s+\1\b/
    ? "The word ($1) was doubled"
    : "No doubles found";

And that will print:

The word (score) was doubled

We use the \b (word boundary) after the \1 to ensure that strings like the theramin will not be reported as doubled words.

A regular expression modifier is one or more characters appended to the end of the regular expression that modifies it.

Earlier, when printing a regular expression, you saw (?-xism:ab?). The (?-) syntax shows the modifiers in effect for the regular expression. If the modifying letter is after the minus sign (-), then it does not apply to the regex. For the $name_and_age regular expression we used earlier, let’s also add an /i modifier at the end of it. When that’s added, it makes the regular expression case-insensitive. /name/i will match Name, name, nAMe, and so on.

For the (?-) syntax, if a modifying letter is before the minus sign, it means that it applies to this regex:

my $name_and_age = qr{
   Name:
   \s+      # 1 or more whitespace
   (.*?)    # The name in $1
   \s+      # 1 or more whitespace
   Age:
   \s+      # 1 or more whitespace
   (\d+)    # The age in $2
}xi;
print $name_and_age;

And that will print out:

(?ix-sm:
    Name:
    \s+      # 1 or more whitespace
    (.*?)    # The name in $1
    \s+      # 1 or more whitespace
    Age:
    \s+      # 1 or more whitespace
    (\d+)    # The age in $2%
 )

The most common modifiers you will see are explained in Table 8.3, “Common Regex Modifiers”.

You already know about the /x and /i modifiers, so let’s look at the /g modifier. That allows you to globally match something. For example, to print every non-number in a string:

my $string = '';
while ("a1b2c3dddd444eee66" =~ /(\D+)/g ) {
    $string .= $1;
}
print $string;

And that will print out abcddddeee, as we expect.

You can also use this to count things, if you’re so inclined. Here’s how to count every occurrence of a word ending in the letters at.

my $silly = 'The fat cat sat on the mat';
my $at_words = 0;
$at_words++ while $silly =~ /\b\w+at/g;

The $at_words variable will contain the number 4 after that code runs. If you don’t like statement modifiers (putting the while at the end of the statement), you can write it this way:

while ( $silly =~ /\b\w+at/g ) {
    $at_words++;
}

You might recall that while loops are often used with iterators. the /g modifier effectively turns the regular expression in to an iterator.

The /m and /s modifiers look a bit strange, but to discuss those, we should explain anchor metacharacters first.

Anchor metacharacters are used to “anchor” a regular expression to a particular place in a string. They do not match an actual character. You’ve already seen one anchor: \b. The ^ is used to match the start of the string and the $ is used to match the end of the string. They are synonymous with \A and \Z. Note that both $ and \Z match the end of a string or before a newline. Thus, if you have a newline in your string, the $ will match immediately before the newline.

my $prisoner = <<"END";
I will not be pushed, filed, stamped, indexed, briefed, debriefed or numbered.
My life is my own.
END
print $prisoner =~ /^I/          ? "Yes\n" : "No\n";
print $prisoner =~ /^My/         ? "Yes\n" : "No\n";
print $prisoner =~ /numbered\.$/ ? "Yes\n" : "No\n";
print $prisoner =~ /own\.$/      ? "Yes\n" : "No\n";
That prints:
Yes
No
No
Yes

In other words, only /^I/ and /own\.$/ matched. If you want /^My/ and /numbered\.$/ to match, use the /m switch to force “multiline” mode. That will force the ^ and $ to match at the beginning and end of every string (separated by newlines) instead of the beginning and end of the entire string.

It’s also important to be aware that if the $ is not the last character in the regular expression, then Perl will assume that this is the sigil introducing a scalar variable:

my $match = "aa";
if ( $some_string =~ /$match/ ) {
    # match words containing aa
}

Later, we’ll see how we can take advantage of this to build complicated regular expressions that would ordinarily be too difficult to write.

Sometimes you want to match a few characters, as even numbers. You can do with a character class. You put the characters you want in square brackets, []. Here’s a silly way of extracting all positive even integers from a string.

my $string = '42 85 abcd 8 4ever foobar 666 43';
my @even;
push @even => $1 while $string =~ /\b(\d*[02468])\b/g;

That will leave @even containing the numbers 42, 8 and 666. Here’s how it works.

By now you already know that the \b matches a word boundary, so the 4 in 4ever cannot be matched because not only is that an abomination to the English language, there is no “boundary” between the 4 and ever.

The \d*[02468] means “zero or more digits, followed by a zero, two, four, six or eight”. In other words, a positive even integer.

In a character class, only the -]\^$ characters are considered “special”. So a . will match a literal dot, not “any character except newline”. If the first character is a caret, ^, then it’s a negated character class. This means it will match anything except what’s listed in the character class and we can use this to match odd numbers:

my $string = '42 85 abcd 8 4ever foobar 666 43';
my @odd;
push @odd => $1 while $string =~ /\b(\d*[^02468])\b/g;

That will push 85 and 43 onto the @odd array (of course, you could have simply used [13579] for the character class).

The dash, -, if used any place after the first character in a character class, tries to create a range. For example, we’ve mentioned earlier that \d matches any Unicode character (Unicode will be discussed in Chapter 9, Files and Directories). If you want to to only match the 0 through 9 ASCII digits, you can use [0-9]. This is generally easier to read than [0123456789], though they mean the same thing.

You can have multiple ranges in a character class. [0-9a-fA-F] will match all hexadecimal digits.

Perl also supports POSIX character classes. These have the form [:name:]. Despite the square brackets around them, you must use an additional set of square brackets around them. For example, to match all alphabetical and numeric characters (the same as \w, but without the underscore), you could use [[:alnum:]]. You can combine these, too. To match all digits and punctuation characters: [[:digit:][:punct:]]. Table 8.4, “POSIX Character Classes” explains Perl’s POSIX-style character classes and their meaning.

if ( $string =~ /[:alnum:]/ ) {
    ...
}

Note only does that not work, but it doesn’t generate an error. This is because Perl sees [:alnum:] as being a character class matching :, a, l, n, u, m (it’s OK to list a character more than once in a character class). You must write that [[:alnum:]] for Perl to recognize the regex correctly.

As a Perl extension to POSIX character classes, you can include a ^ after the [: to indicate negation. So to match anything that is not a control character, use [[:^cntrl:]].

For a character class, you list what types of characters you’re looking for. For a group, you can list what types of words you’re looking for. To group words (or patterns), just put parentheses around them. With that, you can do all sorts of interesting thing, including using quantifiers:

# cat, optionally followed by atastrophe
/cat(atastrophe)?/

You can use a | character in the group to alternate between different patterns:

# matches catastrophe, cataract and catapult, but not cat
/cat(atastrophe|aract|apult)/

We’ve already seen parentheses before. These are used when we want to extract data into the $1, $2, $3 variables and so on. If you want to group but don’t want to extract the data (perhaps you’re inserting a group in an existing regex and don’t want to change all of your match variables), use the (?:...) syntax:

# matches catastrophe or cataract, but without setting $1
/cat(?:atastrophe|aract)/

As you’ve already seen (?-xism:...) earlier in this chapter, you may wonder if the (?:...) syntax is related. In fact, it’s the same thing. You can set those modifiers yourself to tell Perl how to behave. For example, let’s make part of a regex case-insensitive. Maybe you’re writing code to list everyone who is a volunteer. Unfortunately, the people who typed in the data typed volunteer, Volunteer, VOLUNTEER.

use Data::Dumper;
my $text = <<'END';
Name: Alice Allison Position: VOLUNTEER
Name: Bob Barkely   Position: Manager
Name: Carol Carson  Position: Volunteer
Name: David Dark    Position: Geek
Name: e.e. cummings Position: Volunteer
name: Fran Francis  Position: volunteer
END
my @volunteers;
foreach my $line (split /\n/, $text) {
    if ( $line =~ m<Name:\s+(.*?)\s+Position:\s+(?i-xsm:volunteer)\b> ) {
        push @volunteers => $1;
    }
}
print Dumper(\@volunteers);

And that prints:

$VAR1 = [
          'Alice Allison',
          'Carol Carson',
          'e.e. cummings'
        ];

(And you’ll note how we sneakily put the . in the Name pattern so we could still match e.e. cummings).

Why didn’t it add Fran Francis to that list? Because she has name: in front of her name but we didn’t make that part of the regular expression case-insensitive.

You might find that typing (?i-xsm:volunteer) to be a bit cumbersome. If the entire regular expression is not using the /x, /s or /m modifiers, you don’t need the -xsm in the group. You only need them if you need to explicitly disable them (and you don’t need to list all of them, either). So we could have written (?i:volunteer), which is cleaner.

Try it out Using /g and [[:alpha:]]

Example 8.1

Earlier you extracted names and ages from a section of text by splitting the text on newlines and matching.

foreach my $line (split /\n/, $text) {
    if ( $line =~ /Name:\s+(.*?)\s+Age:\s+(\d+)/ ) {
        $age_for{$1} = $2;
    }
}

If $text is huge, that’s rather inefficient and can better be handled with a while loop and the /g modifier. Also, we mentioned earlier that the (.*?) should be avoided. We’re going to be a little more precise.

1. Type in the following program and save it as name_and_age.pl.

   use strict;
   use warnings;
   use Data::Dumper;
   my $text = <<'END';
   Name: Alice Allison Age: 23
   Occupation: Spy
   Name: Bob Barkely   Age: 45
   Occupation: Fry Cook
   Name: Carol Carson  Age: 44
   Occupation: Manager
   Name: Prince        Age: 53
   Occupation: World Class Musician
   END
   my %age_for;
   while ( $text =~ m<Name:\s+([[:alpha:] ]+?)\s+Age:\s+(\d+)>g ) {
       $age_for{$1} = $2;
   }
   print Dumper(\%age_for);

   Note

   name_and_age.pl available for download at Wrox.com.

2. Run the program with perl name_and_age.pl. You should see the following output:

   $VAR1 = {
             'Bob Barkely' => '45',
             'Alice Allison' => '23',
             'Carol Carson' => '44',
             'Prince' => '53'
           };

How it works

This works almost exactly the same as the previous version, but with some important differences. First, the /g turns the regular expression match into an iterator you can use with the while loop. It forces the regular expression to keep matching until no more matches are found.

We’ve use m<...> for the regular expression just to remind you that you can use different delimiters.

The [[:alpha:] ] matches alphabetic characters plus a space character (note the space before the trailing square bracket.

In short, there’s nothing terribly magical here, but now we’ll move on to some more advanced regular expression techniques.

While perhaps not an “advanced” feature of regexes, substitutions are the next logical step in your programming journey. They have the following form:

s/regular expression/replacement text/

You prefer a rare steak to a well-done steak (as you should), so you need to fix this menu item:

my $main_course = "A well-done filet mignon";
$main_course =~ s/well-done/rare/;
print $main_course.

And that prints A rare filet mignon.

As with the normal m//, you can use the /g modifier to make substitutions global. Here’s a (very stupid) technique to remove all doubled words from a text:

my $text = "a a b b c cat dd dd";
$text =~ s/\b(\w+)\s+\1\b/$1/g;
print $text;

And that leaves us with a b c cat dd.

Let’s use the /x modifier to make this a bit clearer.

$text =~ s/
    \b          # word boundary
    (\w+)       # capture to $1
    \s+         # whitespace
    \1          # doubled word (matches $1)
    \b          # word boundary
/$1/gx;         # replace doubled with $1

The left side of the substitution is a regular expression and the right side is not. Thus, we use \1 inside the regex and $1 outside the regex.

As you know by now, an anchor matches a particular place in a string without actually matching a character. Lookahead/behind anchors (and their negative counterparts) are primarily used with substitutions (and sometimes split()) to allow fine-grained control over matching. A positive lookahead allows you to match text following a regular expression, but not including it in the regular expression. The positive lookahead syntax is

(?=$regex)

For example, if you want to replace all xxx followed by yyy with ---, but not replacing the yyy itself, you can do this:

my $string = 'xxxyyy xxxbbb xxxyyy';
$string =~ s/
    xxx      # match xxx
    (?=yyy)  # followed by yyy, but not included in the match
  /---/xg;
print $string;

And that prints out:

---yyy xxxbbb ---yyy

The negative lookahead syntax is (?!$regex). That will allow you to match a regular expression not followed by another regular expression, but the negative lookahead will not be included in the match. So let’s say your young child is writing a “compare and contrast” essay regarding Queen Elizabeth of the United Kingdom and queen bees and ants. She writes this:

The queen rules over the United Kingdom and is loved by
her subjects but a queen ant just lays a lot of eggs.
The queen lives in a palace and the queen bee lives
in a hive.

Obviously you are horrified because the queen of the United Kingdom should be referred to as Queen Elizabeth in this context. So you write this:

my $childs_essay = <<'END_ESSAY';
The queen rules over the United Kingdom and is loved by
her subjects but a queen ant just lays a lot of eggs.
The queen lives in a palace and the queen bee lives
in a hive.
END_ESSAY
$childs_essay =~ s/the queen/Queen Elizabeth/gi;
print $childs_essay;

And that prints out:

Queen Elizabeth rules over the United Kingdom and is loved by
her subjects but a queen ant just lays a lot of eggs.
Queen Elizabeth lives in a palace and Queen Elizabeth bee lives
in a hive.

Obviously that’s not going to earn your daughter a good grade, so let’s use a negative lookahead to only replace those instances of queen not followed by the words ant or bee.

my $childs_essay = <<'END_ESSAY';
The queen rules over the United Kingdom and is loved by
her subjects but a queen ant just lays a lot of eggs.
The queen lives in a palace and the queen bee lives
in a hive.
END_ESSAY
$childs_essay =~
  s/
    the
    \s+
    queen
    \s+
    (?!ant|bee)
   /Queen Elizabeth /gxi;
print $childs_essay;

And that prints out the desired paragraph:

Queen Elizabeth rules over the United Kingdom and is loved by
her subjects but a queen ant just lays a lot of eggs.
Queen Elizabeth lives in a palace and the queen bee lives
in a hive.

Your daughter may not get a wonderful grade for the essay, but at least she’ll be following proper editorial style.

Positive lookbehinds are designated with (?<=$regex) and negative lookbehinds are written as (?<!$regex). They are identical to their lookahead counterparts with two exceptions:

If you are using Perl 5.10 or better, you can also use named subexpressions. Ordinarily you refer to a captured group in the regex with \1, \2 and so on. After a successful match, those are $1, $2 and so on. With named subexpressions you can name them and make things easier to read.

To name a subexpression, use the syntax (?<name>...). To refer to it again inside of the regex, use \g{name}. To refer to the match outside of the regex, be aware that it’s a key in the special %+ hash. For example, our double-word stripper would look something like this:

use v5.10;
my $text = "a a b b c cat dd dd";
$text =~
   s/
    \b
    (?<word>\w+)
    \s+
    \g{word}
    \b
    /$+{word}/gx;
print $text;

For a clearer example consider matching dates. You may remember our code for converting a date to the ISO 8601 format. Here we rewrite it with named subexpressions.

Before:

my $provided_date = '28-9-2011';
    $provided_date =~
    s{
        (\d\d?)      # day
        [-/]         # - or /
        (\d\d?)      # month
        [-/]         # - or /
        (\d\d\d\d)   # year
    }
    {
        sprintf "$3-%02d-%02d", $2, $1
    }ex;
    print;

After:

my $provided_date = '28-9-2011';
    $provided_date =~
    s{
        (?<day>\d\d?)
        [-/]
        (?<month>\d\d?)
        [-/]
        (?<year>\d\d\d\d)
    }
    {
        sprintf "$+{year}-%02d-%02d", $+{month}, $+{day}
    }ex;
    print;

This has an added advantage of no longer requiring you to keep track of the number of the capture. Thus, if you needed to switch the day and month around:

s{
        (?<month>\d\d?)      # month
        [-/]
        (?<day>\d\d?)
        [-/]
        (?<year>\d\d\d\d)
    }
    {
        sprintf "$+{year}-%02d-%02d", $+{month}, $+{day}
    }ex;

You’ll note that the regular expression changed, but the substitution did not.

You can also use the named parameters outside of the substitution as long as you’re in the same scope. For example:

print LOGFILE "converted provided date to ",
     sprintf "$+{year}-%02d-%02d", $+{month}, $+{day};

Try it out Converting date formats

Example 8.2

Substitutions are very common in and it’s important that you get used to them. We’ll use a very simple example converting the US style MM/DD/YYYY dates to the more common (outside the US) DD/MM/YYYY dates.

1. Type in the following program and save it as dates.pl.

   use strict;
   use warnings;
   use Data::Dumper;
   my @dates = qw(
      01/23/1987
      11/30/2000
      02/29/1980
   );
   foreach (@dates) {
       s{\A(\d\d)/(\d\d)/}
        {$2/$1/};
   }
   print Dumper(\@dates);

   Note

   dates.pl available for download at Wrox.com.

2. Run the program with perl dates.pl. You should see the following output:

   $VAR1 = [
             '23/01/1987',
             '30/11/2000',
             '29/02/1980'
           ];

How it works

This simple example has several subtleties you should be aware of as they crop up often in Perl code.

The substitution operator, s///, when the binding operator, =~, is not used, defaults to $_. Further, like the regular expressions you’ve seen earlier, you’re permitted to use alternate delimiters. We didn’t use / for the delimiter because that would have forced us to escape the other forward slashes and made the substitution harder to read:

s/\A(\d\d)\/(\d\d)\//$2\/$1\//;

As a side-effect of using balanced delimiters ([] and <> would also have been nice options), we can put the regular expression and the substitution value on separate lines.

s{\A(\d\d)/(\d\d)/}
     {$2/$1/};

This is not required, but it can also help improve readability.

Another very interesting thing you can do with substitutions is executing code via the /e modifier. When using this modifier, the substitution code is not considered to just be a string; it’s Perl code to be evaluated. For example, consider our substitution to change American style dates:

s{\A(\d\d)/(\d\d)/}
     {$2/$1/};

What if the US style date had been written 28/2/2011? Well, you know that’s February 28th, but your regex doesn’t. And maybe it was sometimes entered as 12-12-1999. That’s a pain, too. Let’s fix that. While we’re at it, we’ll also convert the data to the ISO 8601 unambiguous date format of YYYY-MM-DD.

local $_ = '28-9-2011';
    s{
        (\d\d?)      # day
        [-/]         # - or /
        (\d\d?)      # month
        [-/]         # - or /
        (\d\d\d\d)   # year
    }
    {
        sprintf "$3-%02d-%02d", $2, $1
    }ex;
    print;

And that correctly prints 2011-09-28. We use the /x modifier to make it easy to read. Note that the /x modifier never applies to the right-hand side of the substitution. Instead, we can use extra whitespace here because the /e modifier turns the right side into Perl code instead of a simple string.

All of the other regex modifiers, including the /g, can be used with substitutions.

You know a number might be represented as 2, 2.3, .4, −3e17 and so on. There are a variety of ways in which you can legally write a number and writing a regular expression for it is hard. So don’t write it. When you need a regular expression that you think someone else has already written, look at the Regexp::Common module and see if it’s in there. Here’s how to match a real number:

use Regexp::Common;
print "yes" if '-3e17' =~ $RE{num}{real};

Here’s how to blank out profanity (knowing it would never get through the editorial process, I regretfully omitted the full example).

use Regexp::Common;
my $text = 'something awful or amusing';
$text =~ s/($RE{profanity})/'*' x length($1)/eg;
print $text;

There’s plenty more in this module, so go install it and have fun reading the docs.

If you’ve never read RFC 822 (http://tools.ietf.org/html/rfc822), your author recommends that you do. It’s a great way to get to sleep. It’s also a great way to realize that if you’ve been trying to validate email addresses with a regular expression, you’ve been doing it wrong.

Email addresses can contain comments. The localpart of the domain name cannot contain spaces (unless they’re in comments), but they can contain dashes. They can even start with dashes. Lots of people with last names like O’Malley have trouble sending and receiving email because <o’malley@example.com> is a perfectly valid email address, but many email validation tools think that apostrophe is naughty.

So whatever you do, don’t do this:

if ( $email =~ /^\w+\@(?:\w+\.)+\w$/ ) {
    # Congrats. Many good emails rejected!
}

We see that a lot in code. It doesn’t work. In fact, you can’t use regular expressions to match email addresses. The one your author knows of that is closest to being correct is an almost one-hundred line (beautiful) monstrosity written by Jeffrey Friedl. You can see it in the source code of Email::Valid, the module you should use instead.

use Email::Valid;
print (Email::Valid->address($maybe_email) ? 'yes' : 'no');

Actually, Email::Valid will only tell you if the email address is well-formed. If you ask nicely, it will try to tell you if the host exists. It cannot tell you if the email is valid.

Sooner or later every programmer hears about people trying to parse HTML with regular expressions. Here’s an attempt your author tried to make once:

$html =~ s{
             (<a\s(?:[^>](?!href))*href\s*)
             (&(&[^;]+;)?(?:.(?!\3))+(?:\3)?)
             ([^>]+>)
          }
          {$1 . decode_entities($2) .  $4}gsexi;

Do you know what that does? Neither does your author. He can’t remember, but he doesn’t care because it didn’t work. He learned to use a proper HTML parser. HTML does not have a regular grammar and thus cannot be properly parsed with regular expressions.

That being said, if you are using a well-defined subset of HTML and you are writing small, one-off scripts for extracting data, feel free to use regexes. Just don’t blame anyone but yourself when it breaks. Instead, consider HTML::TreeBuilder, HTML::TokeParser::Simple, or any of a variety of other great HTML parsing modules. Just don’t do it with regular expressions.

Sometimes you find that a regular expression gets very complicated. For example, you might want to match an employee number in the format “department-grade-number”, where is one of 4 different valid department codes for a company, AC, IT, MG, JA, the grade is a two digit number from 00 to 20 and the number is any five or six digit number. The regular expression might look like this:

if ( /\b(AC|IT|MG|JA)-([01]\d|20)-(\d{5,6})\b/ ) {
    my $dept       = $1;
    my $grade      = $2;
    my $emp_number = $3;
    ...
}

As far as regular expressions go, that one really isn’t too bad, but maybe you still want it to be a bit easier to read. You can compose regular expressions easily by using variables and the qr// operator.

my $depts         = join '|' => qw(AC IT MG JA);
my $dept_re       = qr/$depts/;
my $grade_re      = qr/[01]\d|20/;
my $emp_number_re = qr/\d{5,6}/;
if ( /\b($dept_re)-($grade_re)-($emp_number_re)\b/ ) {
    my $dept       = $1;
    my $grade      = $2;
    my $emp_number = $3;
    ...
}

The qr() operator will “quote” your regular expression and, in some cases, pre-compile it, leading to significant performance gains when you later use it in a match.

As a more complicated example, your author was writing a pre-processor for Prolog code (Prolog is a programming language) and wanted to match math expressions. The following are all valid math expressions in Prolog (the actual code is far more complicated):

2 + 3
Var
-3.2e5 % SomeVar / Var

The code to match those is presented in Example 8.2, “Building complex regular expressions from smaller ones.”.

use strict;
use warnings;
use diagnostics;
use Regexp::Common;
my $num_re = $RE{num}{real};
my $var_re = qr/[[:upper:]][[:alnum:]_]*/;
my $op_re  = qr{[-+*/%]};
my $math_term_re = qr/$num_re|$var_re/;
my $expression_re       = qr/
    $math_term_re
    (?:
        \s*
        $op_re
        \s*
        $math_term_re
    )*
/x;
my @expressions = (
    '2 + 3',
    ' + 2 - 3',
    'Var',
    '-3.2e5 % SomeVar / Var',
    'not_a_var + 2',
);
foreach my $expression (@expressions) {
    if ( $expression =~ /^$expression_re$/ ) {
        print "($expression) is a valid expression\n";
    }
    else {
        print "($expression) is not a valid expression\n";
    }
}

And that will print our desired output:

(2 + 3) is a valid expression
( + 2 - 3) is not a valid expression
(Var) is a valid expression
(-3.2e5 % SomeVar / Var) is a valid expression
(not_a_var + 2) is not a valid expression

You may be thinking that the regular expression isn’t that complicated, but if you print out the entire thing, it looks like this (formatted for to fit this page and still be a valid regular expression):

/(?x-ism:(?-xism:(?:(?i)(?:[+-]?)(?:(?=[.]?[0123456789])(?:[
0123456789]*)(?:(?:[.])(?:[0123456789]{0,}))?)(?:(?:[
E])(?:(?:[+-]?)(?:[0123456789]+))|))|(?-xism:[[:upper:]][
[:alnum:]_]*))(?:\s*(?-xism:[-+*/%])\s*(?-xism:(?:(?i)(?:[
+-]?)(?:(?=[.]?[0123456789])(?:[0123456789]*)(?:(?:[.])
(?:[0123456789]{0,}))?)(?:(?:[E])(?:(?:[+-]?)(?:[0123456789
]+))|))|(?-xism:[[:upper:]][[:alnum:]_]*)))*)/x

If you want to write that by hand, be my guest, but don’t ask anyone (including yourself) to debug it.