How to Become a Regular Expression Power User

If you don’t know what regular expression (also known as “regex” or “regexp”) is, it’s an unbelievably powerful language for executing a search and/or replace through any kind of text. Regex has had a long and glorious history (it’s origins date back to the ‘50s) and even now, you use it every day … you just might not know it. Regex is baked into most major programming languages and computer systems these days and is used in a wide variety of apps.

Regex is no longer just for programmers: it’s showing up in all sorts of places today. Some of the places you might have seen it so far is for matching URLs in Google Analytics, search & replace in your favorite text editor (some popular editors supporting regex include Sublime, Atom, Cloud9, Notepad++, Google Docs and Microsoft Word … although Word has a regex syntax that is very non-standard) and even matching file names in our awesome WordPress Membership Plugin, MemberPress.

Even though regex is now showing up in surprisingly new places, its seemingly cryptic nature will probably always make it more predominantly used by programmers and “power users.”

So here's the point in this article where you need to decide if you’re going to continue life without knowing the untold power of regex, or if you’ll swallow the red pill and wield the sword of a power user! In this post, I’ll give you the basic knowledge you’ll need to start putting this extremely useful tool to work for you. So let’s get started!

Some Examples

What do regex statements look like? If you saw one without having any idea what it was, it would look like complete gobbledygook.

Here’s a sample regexp statement to match a phone number:

\(?\d{3}\)?[- ]\d{3}-\d{4}

Here’s one that matches a URL:

https?:\/\/[\w-]+(\.[\w-]{2,})*(:\d{1,5})?

And one that can match an email address:

[\w\d._%+-]+@[\w\d.-]+\.[\w]{2,4}

Finally, here’s a simple search and replace that will find any domain names with the ‘test’ subdomain and change them to a ‘www’ subdomain:

Search: test\.([^\.]+)\.com
Replace: www.$1.com

Intimidated yet? Don’t be. I’ll go through how regex statements are interpreted, breaking down what we’re doing in each of these statements and explaining how to create your own.

How to Read Regex

Regular expressions are like normal find statements, but on steroids. For instance, if you wanted to find the name “Harry” on a web page, you’d click control-f and type “Harry” to find it. With regex you can do more than simply search for “Harry.” You could search for “Harry” or “Bob” with (Harry|Bob) or you could search for any word starting with “Ha” with Ha\S*.

To your computer, text is represented as a “string” of characters. In fact, if you’ve ever overheard programmers talking about code you’ve probably heard them use the term “string” when referring to some variable.

Regular expressions read left to right (just like English and many other languages) and they explain to the computer what to match as it scans a string left to right. As the computer looks at each character it follows this flow:

If you look at this flowchart, as complicated as it might seem, you’ll see something very familiar here. It’s pretty much identical to how you’d expect any search to happen on a computer and how searches that you’re currently comfortable with happen already. Regex just adds some extremely powerful features on top of normal search.

The Basics

Now before we start dissecting our example regex statements from above, let’s get through some of the very basic command characters of regex:

Repeaters: *, + and {…}

The asterisk (*) and plus (+) characters are repeaters. A repeater is only used after another character or enclosed statement and tells the computer to match the preceding character X number of times. The asterisk will match the preceding character 0 to infinite times; the plus, 1 to infinite times; {2} will match it 2 times exactly; {4,6} will match it between 4 and 6 times exactly; and {7,} will match it between 7 and infinite times.

Wildcard: .

You’re probably used to the star symbol being used as a wildcard, but it’s a period (.) in regex. The period will simply match any character.

Optional: ?

Like the repeater, the optional character, a question mark (?), is only used after another character or enclosed statement. When this is in place, it tells the computer that the preceding character may or may not be present in search results.

Beginning and End of String: ^ and $

When the caret (^) is used at the beginning of the regex, it indicates that the first of the string should be matched. And the dollar sign ($), when used at the end of the regex, will match the end of the string.

Possible Characters: […]

Enclosing characters within brackets indicates the characters that could be matched in this position. For example, if you wanted to match an ‘n’,’m’,’l’ or ‘_’ then you could add this to your regex [nml_]. If you just wanted to match any letter or number you could add this [A-Za-z0-9].

If you want to match any character except the possible characters you can just put a caret (^) after the opening bracket. For instance, if you wanted to match any character except ‘x’,’y’,’z’ or ‘_’ you could do so with the statement: [^xyz_]. As you can see, this can give you fine grain control over what you’re matching on a character-by-character basis.

Enclosed Statements: (…)

Sometimes you want to make sections of your regex behave as a block or save them for later. For this, all you need to do is wrap the statement in parentheses.

Escape: \

Now, what if you want to match for the actual ‘+’, ‘.’, etc characters? Well, if you throw a backslash in front of it then that will tell your computer that you’re trying to match an actual search character, and to not interpret it as a command character.

Shorthand Characters: /s, /S, /d, /D, /w, /W and /b

These characters are very useful and will help you match certain sets of characters. Here’s how these break down:

• /s matches any whitespace characters such as space and tab
• /S matches any non-whitespace characters
• /d matches any digit character
• /D matches any non-digit characters
• /w matches any word character (basically alpha-numeric)
• /W matches any non-word character
• /b matches any word boundary (this would include spaces, dashes, commas, semi-colons, etc)

Of course there are many other, more powerful things that you can do in regex but these basics will get you the tools you need for 90% of your advanced search needs.

Breakdown of Our Phone Number Example

Okay, now that you have the tools, let’s look through our examples to see how they work, starting with our match for the phone number:

\(?\d{3}\)?[- ]\d{3}-\d{4}

So we’ll break it down by statement:

\(?
This is the first statement that the computer will look for. It starts with an escape character (\) followed by an open parentheses (() followed by an optional character (?). This tells the regex to look for an actual open parentheses to signal the start of a phone number, but that it may not be present in every instance of a phone number.

\d{3}
The second statement starts with the shorthand character for digit (\d) then it has a repeater ({3}) that will require there to be 3 digits. It does not end with a question mark because these three digits are not optional – they must be present for the search result to qualify as a phone number.

\)?
Now the computer will check for an optional closing parenthesis, but if it’s not present it can still match.

[- ]
After that, we have a possible characters statement that will look for either a space or a dash to match.

\d{3}-\d{4}
Finally, we actually have some statements that will expect 3 digits followed by a dash and then followed by 4 digits.

You can see this in action and play around with it here: http://regex101.com/r/nW9iC5/3

A few things you might notice about this regex is that it will only match two acceptable formats for phone numbers:

• (888) 888-8888
• 888-888-8888

It won’t match a multitude of unacceptable phone number formats such as:

• 8888888888
• 88-8888-8888

But it will also match a few formats that might not be appropriate:

• (888-888-8888
• 888) 888-8888
• (888)-888-8888

What are some things you think you could do to ensure that this regex only matched the appropriate formats?

Breakdown of our URL Regex

First, I’d like to state that the following regex is not a comprehensive URL matching regex by any means. It will only match a web address with no arguments and has several other problems, but would serve to match many URLs effectively.

https?:\/\/[\w-]+(\.[\w-]+)*(:\d{1,5})?

Now for the breakdown:

https?:\/\/
This first statement matches the protocol of the URL. Because the “s” has a question mark after it, it’s optional. There are also escape characters in front of the forward slashes, which is usually what has to happen in regex to match forward slashes. So really, the only two ways a URL could match this statement would be to begin with http:// or https://.

[\w-]+
Here we have a possible characters statement followed by a plus. This means that the next characters must be 1 or more word characters or dashes.

(\.[\w-]+)*
Here we have the statement that will match a period followed by word characters or dashes. This whole statement is wrapped in parentheses and followed by a plus which tells the computer that we can repeat this whole sequence zero or more times. So, this section could match .memberpress or .memberpress.com or .memberpress.co.uk, and so on.

(:\d{1,5})?
Finally, this statement will provide for an optional port number to be appended to the end of the URL which includes a colon followed by 1 to 5 digits then wrapped in parentheses and followed by a question mark to make the entire statement optional.

Here’s where you can play around with this one: http://regex101.com/r/vL5uZ2/2

This regex will match:

• https://memberpress.com
• https://memberpress.com
• http://localhost:3000

BUT it won’t match any parameters trailing a URL like this:

• http://localhost:3000?test=1&page=5

What could you add to this regex to match some parameters at the end of a URL?

If you're interested in the most comprehensive URL matching regex (one that will match parameters, unicode characters, large TLDs, etc) there are plenty of them that you can look at but apparently the most comprehensive and accurate is the one created by Diego Perini which you can look at on his Github gist.

Breakdown of our Email Regex

Another common use of regex is to match email addresses. Our email matching regex will actually match a substantial number of email addresses:

[\w\d\._%+-]+@[\w\d.-]+\.[\w]{2,4}

Here’s the breakdown:

[\w\d\._%+-]+
This statement will match one or more word characters, digits, periods, underscores, percentage symbols, plus symbols, or dashes.

@
Every email address needs an ‘@’ symbol and that’s what this matches of course.

[\w\d.-]+
After the ‘@’ symbol we need to start on the domain name. This statement will match one or more word characters, digits, periods, or dashes.

\.[\w]{2,4}
Now we need to match the top level domain, which is a period followed by 2 to 4 word characters long.

You can also play around with this regex here: http://regex101.com/r/yQ2wP9/1

Even though this regex isn’t 100% comprehensive it should be able to match most addresses.

Regex Search and Replace

So in many cases you’ll care not only about matching patterns in your text with regex but also utilizing the power of search and replace with regex.

Let's use the search and replace example from above:

Search: test\.([^\.]+)\.com
Replace: www.$1.com

Here's the breakdown, starting with the search pattern:

test\.
This first part will match any part of the string that starts with “test.”

([^\.]+)
This statement will match one or more characters that are anything except a period. The entire statement is wrapped in parentheses for a different reason than we’ve seen so far: this time it’s to save whatever is matched by this statement to be used in the replacement.

\.com
Finally, it must match a “.com” at the end.

Now, in the replacement we can use “$1” to include whatever we saved between the parentheses in the search pattern. If you had a second statement like this in the search pattern you could use it in the replacement by using “$2.”

You can play around with this search and replace regex here: http://regex101.com/r/bW5vX8/1

This search pattern has some real limitations though. It can only match URLS starting with “test.” and ending with “.com”. How do you think you could modify it to match URLs ending in “.com”, “.net” or “.org”?

Learning more about Regex

This primer has given you a crash course on regular expressions but there’s plenty more to learn. You can get more info about regex on Wikipedia or regular-expressions.info.

Blair Williams

Developer and entrepreneur Blair Williams is the visionary and creative force behind MemberPress, the world's #1 monetization, LMS, and membership plugin for WordPress. His knack for identifying market gaps led him to develop several other top-tier WordPress plugins, elevating him as an industry expert. A seasoned founder with a unique blend of technical and business acumen, Blair is a sought-after authority in the WordPress and ecommerce spaces. His insights offer more than just useful information – they deliver groundbreaking wisdom that's a total game changer.

Ver biografía completa