Question

I’m using a 3rd party app that uses regex to searches html documents, in this case it doesn’t have proper structure (no head or body), and returns the matches as a form of properties in an excel file. It does not parse them. I already know the horrors induced by attempting to parse html with regex.

So I wrote a regex that is supposed to capture every sentence within a paragraph or list item but after checking the matches I noticed that sometimes it wouldn’t match all the sentences and would stop matching once that sentence or list item gave an error. Almost always with the list item but occasionally with sentences. After realizing this was due to human error, I added the optional non-capture group that totally screwed everything up.

This was the initial regex I wrote which worked in most cases:

([^<>]*?)[.!?<]|[ <"'/]

Because of some sentences having an error where the writer put a space before the punctuation, I added the optional non-capture group:

([^<>]*?)(?:[ ])?[.!?<]|[ <"/l]

Here is an example of the text it was searching:

Buy this because it is soooooooooooooooooooo freaking awesome! If you buy this 
everyone will think you're "cool." You'll get all the babes !<br><br><ul><li>It 
will make you smell better<li>It will make you preform better.</li><li>Will make
you last longer in bed!<li>Will fix any acne problem.</li> <li>It will reduce the
amount you perspire to .01% your normal amount!<br><li>It will make you 
"invincible."</li></ul>

Because their is nothing to use as an anchor (text starts at beginning of the html file) I just had it start capturing immediately. As you see, it is poorly coded and has grammar errors, which is why I ended it the way I did.

The first one captured most all the sentences but left out some… The second one returns a bunch of blank of null match which screw up the arrays made with the captures. It’s like its disregarding everything after the non-capture group.

I thought about doing it this way but this returned every single word as a match:

([^<>]*?)[ .!?<]|[ .!?<"/l]

Only problem was this cut some sentences off in the middle and would require a third range which I think would have a bunch of different options (notice the random <br> tag) and would take a while to find them all.

From appearances, it is not using the optional non-capture group! Why is this? Or am I overlooking something extremely simple? I feel like the latter is probably the case.

Answer 1

I came up with this beast:

(?:^|\s+|>)((?:[^<>.!?\s])(?:[^<>.!?]|\.\d)+(?:\.(?!\d)"?|!|\?)?)

Let me try to explain what I am doing here.

(?:^|\s+|>)       # only start after at the string's beginning, after a row of
                  # spaces, or after closing a tag
                  # this eliminates all in-tag matches (like "li" and "br")
(                 # opening a capturing group that will contain the actual match
(?:[^<>.!?\s])    # require at least one character that is not in the given group
                  # this eliminates matching a single space between two <li>s
                  # NOTE: there are probably better ways to do this
(?:[^<>.!?]|\.\d) # defines possible sentence characters; allow everything but
                  # <, >, ., !, ? EXCEPT FOR . followed by a digit
(?:\.(?!\d)"?|!|\?)?
                  # include possible sentence endings; that is . not followed by
                  # a digit (hence, the negative lookahead), but possibly
                  # followed by ", or !, or ?, or nothing at all
)                 # close the main matching group

Now you should be able to access your sentences at captured index 1.

I believe you might have cases where my assumptions about what a sentence can look like break. But I could only work from the example you have given, and all its oddities are included.