Question

Let’s say you have a HTML file with a couple duplicate scripts, meaning multiple external script tags for the same resource, like loading jquery 3 times on the page. Is there an efficient regular expression that can remove the duplicates but keep the first one in place. The duplicates will be all with the same exact src name.

Language is PHP and here is a good example:

Before:

<script src="js/jquery.js" type="text/javascript"></script>
    some content
<script src="js/jquery.js" type="text/javascript"></script>
    more content
<script src="js/jquery.js" type="text/javascript"></script>

After:

<script src="js/jquery.js" type="text/javascript"></script>
    some content
    more content

Answer 1

Disclaimer:

Many will rightfully state that using regular expressions to parse non-regular languages such as HTML is fraught with peril. And they are correct. The only way to reliably parse these languages is with a parser specifically designed for the task. A solution using regular expressions will typically have many special cases of subject text that will cause it to fail, resulting in false positives, and missing matches.

That said…

If one insists upon using regular expressions to process HTML/XML markup, and they are aware of the inherent limitations, there are ways to craft a regex solution that can minimize these potential pitfalls, and do a “pretty good” job (depending on the specific requirements of the question). However, to correctly handle many of the rare (but valid and possible) edge cases (e.g. correctly handling HTML tag attributes containing <> angle brackets for instance), the correct regex can frequently be rather complex and not for the faint-of-heart.

Understanding the following regex solution requires a fairly deep understanding of the regex language and the underlying mechanics of the regex engine. There are certainly examples of markup text that will cause it to fail, but the following solution should do pretty good job for many cases of typical markup.

Here is a tested PHP function that removes SCRIPT elements having duplicate SRC attribute values:

// Strip all SCRIPT elements having duplicate SRC URLs.
function stripDuplicateScripts($text) {
    $re = '%
        # Match duplicate SCRIPT element having same SRC attribute URL.
        (                   # $1: Everything up to duplicate SCRIPT element.
          <script           # literal start of script open tag
          (?:               # Zero or more attributes before SRC.
            \s+             # Whitespace required before attribute.
            (?!src\b)       # Assert this attribute is not "SRC".
            [\w\-.:]+       # Non-SRC attribute name.
            (?:             # Attribute value is optional.
              \s*=\s*       # Value separated by =, optional ws.
              (?:           # Group attribute value alternatives.
                "[^"]*"     # Either a double quoted value,
              | \'[^\']*\'  # or a single quoted value,
              | [\w\-.:]+   # or an unquoted value.
              )             # End group of value alternatives.
            )?              # Attribute value is optional.
          )*                # Zero or more attributes before SRC.
          \s+               # Whitespace required before SRC attrib.
          src               # Required SRC attribute name.
          \s*=\s*           # Value separated by =, optional ws.
          ([\'"])           # $2: Attrib value opening quote.
          ((?:(?!\2).)+)    # $3: SRC attribute value (a URL).
          \2                # Attrib value closing quote.
          (?:               # Zero or more attributes after SRC.
            \s+             # Whitespace required before attribute.
            [\w\-.:]+       # Attribute name.
            (?:             # Attribute value is optional.
              \s*=\s*       # Value separated by =, optional ws.
              (?:           # Group attribute value alternatives.
                "[^"]*"     # Either a double quoted value,
              | \'[^\']*\'  # or a single quoted value,
              | [\w\-.:]+   # or an unquoted value.
              )             # End group of value alternatives.
            )?              # Attribute value is optional.
          )*                # Zero or more attributes after SRC.
          \s*               # Optional whitespace before tag close.
          >                 # End of SCRIPT open tag.
          </script\s*>      # SCRIPT close tag.
          .*?               # Stuff up to duplicate script element.
        )                   # End $1: Everything up to duplicate SCRIPT.
        <script             # literal start of script open tag
        (?:                 # Zero or more attributes before SRC.
          \s+               # Whitespace required before attribute.
          (?!src\b)         # Assert this attribute is not "SRC".
          [\w\-.:]+         # Non-SRC attribute name.
          (?:               # Attribute value is optional.
            \s*=\s*         # Value separated by =, optional ws.
            (?:             # Group attribute value alternatives.
              "[^"]*"       # Either a double quoted value,
            | \'[^\']*\'    # or a single quoted value,
            | [\w\-.:]+     # or an unquoted value.
            )               # End group of value alternatives.
          )?                # Attribute value is optional.
        )*                  # Zero or more attributes before SRC.
        \s+                 # Whitespace required before SRC attrib.
        src                 # Required SRC attribute name.
        \s*=\s*             # Value separated by =, optional ws.
        ([\'"])             # $4: Attrib value opening quote.
        \3                  # This script must have duplicate SRC URL.
        \4                  # Attrib value closing quote.
        (?:                 # Zero or more attributes after SRC.
          \s+               # Whitespace required before attribute.
          [\w\-.:]+         # Attribute name.
          (?:               # Attribute value is optional.
            \s*=\s*         # Value separated by =, optional ws.
            (?:             # Group attribute value alternatives.
              "[^"]*"       # Either a double quoted value,
            | \'[^\']*\'    # or a single quoted value,
            | [\w\-.:]+     # or an unquoted value.
            )               # End group of value alternatives.
          )?                # Attribute value is optional.
        )*                  # Zero or more attributes after SRC.
        \s*                 # Optional whitespace before tag close.
        >                   # End of SCRIPT open tag.
        </script\s*>        # SCRIPT close tag.
        \s*                 # Strip whitespace following duplicate.
        %six';
    while (preg_match($re, $text)) {
        $text = preg_replace($re, '$1', $text);
    }
    return $text;
}

The function above uses one regex which is applied recursively until no matches are found. Although at first glance the regex looks like a monster, its actually quite straight-forward (if you are well versed in regex syntax) and most of the text consists of descriptive comments. The complexity of this regex is required to handle the variety of attribute/value formats allowed by HTML. For example, the SCRIPT tags may have any number of attributes before and after the SRC attribute. The SRC attribute value may be single or double quoted. All other attributes may have values that are either quoted or unquoted and may have no value at all. Quoted attributes may contain <> angle brackets.