Regular Expressions (Regex) - Your Thoughts?

[Renegade]

Perhaps freakishly, I love regular expressions. I use them all the time. They literally give me godlike power over text. I find it hard to believe that there are developers out there that don't know what they are, much less that there are developers that can't use them.

They are so useful for more than development though. Anyone that needs to deal with large amounts of text could benefit so much from them. I've helped myself and other people to do hours, days, weeks, months, years, decades, centuries, and even millenia worth of work through regular expressions. (Imagine a human can process/organize 1 line of text in 10 seconds, and then imagine 100 GB of data. 1 line is about 80 characters and you get a bit over 2,000 years worth of work.)

Now, I'm sure a lot of people here have never heard of them, and some use them daily. I'm sure that some people would never have a use, while others that have never heard of them could easily find numerous uses for them to help save time.

My wife doesn't know regular expressions, but I recently helped her out with some work where I used them. It all took me a couple hours, but would have taken her days or more to do.

The uses for Excel jockeys are endless.

Anyways, just looking to see what people think about them.

Now, depending on a few factors...

^(good)?bye$

[40hz]

Where would a NIXer be without grep

And if you love regex, you might also like the Awk Programming Language. If you regularly need to manipulate large amounts of text data, Awk is a great way to do it.

 

[Renegade]

Where would a NIXer be without grep

And if you love regex, you might also like the Awk Programming Language. If you regularly need to manipulate large amounts of text data, Awk is a great way to do it.

 

-40hz (November 28, 2010, 01:14 AM)

Interesting. I've never really considered AWK anything more than some "obscure" system administrator language for UNIX. I should have a look at it as it sounds like it's right up my alley.

[f0dder]

I use regex occasionally, and they've saved me a fair amount of time (even though I need a reference when doing some of the more funky stuff). Pretty useful for massaging text beyond what normal search/replace can do, and invaluable when doing some quick webscraping hackjob (if you've found a security flaw in a social networking site you don't want to spend hours coding a proper solution, but you do want people's private image galleries - or so I've heard).

But I also believe that regexes are over-used. It's way too easy cooking up something that's write-once-never-understand-again, it's utterly unusable for production-quality parsing of external data sources you don't have control over, and for really big input data with complicated regexes you're probably a lot better off writing some high-speed code.

If you're ever using regular expressions for user-supplied data, you should probably take a look at SDL Regex Fuzzer

[rjbull]

And if you love regex, you might also like the Awk Programming Language. If you regularly need to manipulate large amounts of text data, Awk is a great way to do it.

-40hz (November 28, 2010, 01:14 AM)

Back in 1988/1989 the first program I started to learn was the shareware word processor/editor PC-Write.  It was a great program, but only had simple repeat-key macros, i.e., nothing like flow control.  When I began to feel a need for automated editing, the first thing I tried was the stream editor SED.  I couldn't work out how to do something in it, popped up a question on a bulletin board, and someone kindly said, "I don't think SED can do that - try AWK."  So I did.  It became the only programming language I have a passing familiarity with and did huge amounts of work for me (mostly back when I had a job).  A lot of data I handled was fairly ordered, like patent abstracts and files of formulations, and AWK was invaluable for manipulating it.  The book The Awk Programming Language by its original three authors, Aho, Kernighan and Weinberger, is not only good on AWK itself, but a very good tutorial on regular expressions.

You can find out more, and download AWK executables, at either of Prof. Brian Kernighan's home pages:

• Brian Kernighan at Princeton
• Brian Kernighan at Bell Labs

The most popular version of AWK is probably Gnu AWK, Gawk.  Another remarkable extended version, TAWK by Thompson Automation Software, includes a compiler.  Other versions abound, many listed on the AWK FAQ page.

[Renegade]

I *think* I read a number of years ago about AWK and it being common to have TCL/Tk front ends for it. Is that right, or am I confused there?

[kyrathaba]

I find it hard to believe that there are developers out there that don't know what they are, much less that there are developers that can't use them.

Well, I've only used them a couple of times, but I admit there've been plenty of other times they would have been useful, if I had greater facility with them.  I cut myself some slack, since I'm only a hobbyist programmer, and only that for the past 4-5 years.  Still, I am pleased with an email address validity checker I developed for my entry in this year's N.A.N.Y. competition.

[timns]

Cound not be without RegEx. Like kyrathaba, I used RegEx in this year's NANY, in my case to allow some pretty clever pattern matching when tracking user's input to see if it can be expanded / replaced on the fly.

If anyone needs probably THE most powerful RegEx search and replace tool, I suggest PowerGrep.

http://www.powergrep.com/

I have no affiliations - just a satisfied user.

[cranioscopical]

Cound not be without RegEx...I suggest PowerGrep.

-timns (November 28, 2010, 11:01 PM)

Either you're typing too fast or there's a faunt in your RegEx.
You should try this neat new thing, Auspex…
 

[timns]

Cound not be without RegEx...I suggest PowerGrep.

-timns (November 28, 2010, 11:01 PM)

Either you're typing too fast or there's a faunt in your RegEx.
You should try this neat new thing, Auspex…
 

-cranioscopical (November 29, 2010, 09:55 AM)

Dang, you're right. I'll simply add another autocorrection to that marvellous piece of software we call 'Auspex' using a regex such as cou[k|n]d -> could

[phitsc]

To all you regex lovers: if you happen to also be a FARR user, please install my FarrWebMetaSearch plugin and write a search for your favourite web site. Then post it on the forum or send it to me directly for inclusion with FarrWebMetaSearch

By the way, the following web site has served me well when working with (not too complicated) regexes: http://www.regextester.com/index2.html

[40hz]

I *think* I read a number of years ago about AWK and it being common to have TCL/Tk front ends for it. Is that right, or am I confused there?

-Renegade (November 28, 2010, 08:13 PM)

Never saw that although there's no reason why you couldn't wrap a GUI around an AWK program's functionality. Don't know why you'd bother since there's already things like GUI search and replace utilities where it's already been done. Also sorta defeats the purpose of AWK since it's really intended to be used as a programming language rather than an app.

Note: you can do much (or all) of what AWK does with PERL. If my memory serves, I seem to remember Larry Wall cited AWK (along with SED and shell) as contributing influences on the design of his baby.

One interesting thing about AWK is that it's like riding a bike. It takes practice to use it effectively. But once you learn it, you never seem to forget. I haven't done any serious AWK-ing for a number of years. But I had an oddball project a few months ago where an AWK script was the perfect way to get the job done.

After a few false starts, and a look in the manual (RTFM right? ), I was back in business. I doubt I'd be able to do that with many other languages I've learned and allowed to lapse over the years. That alone is testimonial to just how logical and well designed a tool AWK is IMHO.



Also +1 w/rjbull on The AWK Programming Language book. One of the clearest and best written computer books of all time. Too bad they couldn't all be like that.