Question for Bloggers: Do You Even Try to Check The Spam Filter?

51 Comments

Adam Steinbaugh • Dec 27, 2012 @9:21 am

Get the Wordfence plugin and pony up for the advanced version. Use that to block IPs from certain countries from accessing your site. Sure, you might lose the occasional legitimate visitor, but you can modify the default "you've been blocked" page to include a link to an anonymizer.

It won't stop all the spam, but do you really have many legit viewers from central Asia?
JDDrew • Dec 27, 2012 @9:22 am

My blog does get spam comments, but not nearly as many as you do, so it hasn't been a problem for me. I'd seriusly consider shortening up the "live" period for posts, though – 60 days seems very long. 7 days? 14?
John • Dec 27, 2012 @9:23 am

We've got a few websites that use Disqus (which we don't recommend) that offload the spam issue mostly to them, but we have the same issue of false positives being caught up in their filters.
Lizard • Dec 27, 2012 @9:24 am

I use Conditional Capcha on top of Akismet. Conditional Capcha pops up a second capcha to manually fill in, evading the automated capha-solvers. It's reduced my Akismet spam queue from a few hundred a day to one a week or so.
TJIC • Dec 27, 2012 @9:28 am

I recall that Askimet or some side-plugin for it can be set to either "X days" or "Y days since last comment". I agree with JDDrew: set it to 7 days, or within 7 days of a comment.

Also: don't manually go through the comments, but DO have a contact form that folks can use to ask about rejected comments. Spammers won't use it, and the 1 or two legitimate folks per day will.
TJIC • Dec 27, 2012 @9:29 am

Oh, also: please never ever EVER use a capcha as the default. I utterly hate them and avoid most blogs that use them.
colson • Dec 27, 2012 @9:32 am

You might also try the Bad Behavior plugin (with Akismet alongside) – worth it for even a modest trial run if nothing else:

http://wordpress.org/extend/plugins/bad-behavior/
AlphaCentauri • Dec 27, 2012 @9:32 am

The problem is that the spammers now hire real people to bypass CAPTCHAs and other filtering methods. All you can do is require registration with confirmation, require moderation on the first x number of posts by a user, and use a method to block troublesome IP ranges. You don't have to block whole countries to block the ISPs which don't do anything about spam. But unfortunately, US ISPs can be the worst offenders, because spammers covet their users' high speed access and dynamic IP addresses.

If your forum software allows you to create different levels of access, you can relegate spammers to their own little world where once registered they can post to their hearts content but no other users or search engines can view their posts. Then you can even use that information to mock them and their sponsors.

Since so many of them are promoting counterfeits, you can report their sponsors to the Intellectual Property Rights agency and let them take down those website for you:
http://www.iprcenter.gov/referral
Maggie McNeill • Dec 27, 2012 @9:33 am

I check it every time I visit my blog; I make a pass through the spam filter and scan it quickly for real comments, then dump the rest. Only rarely do I find a real comment trapped there, but it's enough to trigger my paranoia so I keep doing it. It's definitely a pain in the rear, but by doing it several times a day as part of routine blog control I keep it from getting too full (generally about 20 comments per pass, half a dozen times a day). For me it's kind of like looking at my gauges while driving: something I do every so often without thinking much about it.
AlphaCentauri • Dec 27, 2012 @9:33 am

Also, Cloudflare is very helpful
Bearman • Dec 27, 2012 @9:34 am

@Lizard, I agree with @tjic. If your primary purpose is engagement, then I would never use manual capcha's as they actually discourage comments.

I tend to click on 1-3 pages and do a quick scan. typically someone legit will email me and tell me they aren't coming through.
AlphaCentauri • Dec 27, 2012 @9:36 am

Part of the reason your blog is so popular is you spend so much time moderating it. Nobody's ever asked me to run a guest post.
Andrew • Dec 27, 2012 @10:11 am

The least obtrusive plugin I've found so far is G.A.S.P. (Growmap Anti Spambot Plugin) but it won't do much against spammers who have hired real-live people. Still, it stops spambots cold.
David Aubke • Dec 27, 2012 @10:19 am

@TJIC and others:
Why do you dislike captchas so much? I use the WP plugin simply named Captcha and it all but eliminated my spam. It makes you solve a simple math problem before posting a comment. I sometimes find word-as-a-distorted-image captchas difficult to figure out but as a percentage of the total effort of posting a comment, it doesn't seem that much of an imposition to me.
David Aubke • Dec 27, 2012 @10:24 am

@Bearman,
I guess that answers my question.. which I suppose is sitting in Moderation right now even though I've posted before.
David Aubke • Dec 27, 2012 @10:26 am

Or maybe my comment was filtered.
I wondered: Why do folks dislike captchas? As a percentage of the total effort involved in posting a comment, it doesn't seem that onerous.
I use the WP plugin named Captcha which makes you answer a simple math problem. I sometimes find the word-as-a-distorted-image ones difficult but still not enough to change my mind about posting a comment.
shg • Dec 27, 2012 @10:30 am

I've never found a truly happy middle ground, but then, eliminating 10,000 spam comments a day was worth it for me.

I find Askimet less of an annoyance than the recaptcha forced down my throat by my program. On the other hand, it has saved me countless hours of deleting and banning spammers.
Ken • Dec 27, 2012 @10:33 am

David's first comment above is an excellent example. I only knew to look for it because of his comment, and it was on the first page of spam, which is the only reason it was practical to look for it.
David Aubke • Dec 27, 2012 @10:37 am

Also, just a note – I've noticed in the past few months MANY malicious emails with links to innocent WP blogs that are unwittingly hosting the attack's payload. At first they were all under the Akismet plugin directory but they seem to be spreading to other plugins now.

Just suggesting other WP users, particularly those with the Akismet plugin, may want to look around in their /wp-content/plugins/ directory for strangely-named directories.
M. • Dec 27, 2012 @10:39 am

@TJIC: I dislike captchas because I have trouble parsing them and usually have to try 2-3 times.
Kirk Taylor • Dec 27, 2012 @10:42 am

My method to keep spam under control is to have a blog that is so boring as to attract very few readers and thus have very few comments. It's easy to filter out the 1 spam comment from the 2 legitimate comments I get each month. I also try to post as infrequently as possible.
David Aubke • Dec 27, 2012 @10:48 am

@Kirk – I can assure you that a boring blog in a niche market is no defense. Within days of allowing completely unfettered access to my comments section, I was inundated with spam throughout the day. I notice yours at least requires some sort of log-in. I'll bet that's blocking more than you think.
cybele • Dec 27, 2012 @11:10 am

I get a lot of spam and use Askimet, but I also keep a pretty good blacklist. So it cuts down on the actual entry/submission of a fair bit as time goes by. I don't close old post to comments, as those are often my most popular gathering places on the site.

But I have to say, I've commented on Popehat at least three other times and I don't think my comments have ever been published. So perhaps the threshold is set too high.

(WordPress driven blogs may be a particularly hard hit sites, check your search logs to see if they're searching for the phrase "powered by WordPress" and perhaps reword that to something a little less common.)
Jake • Dec 27, 2012 @11:23 am

If you are looking to reduce the total amount of spam you are getting, even that which is already being sent to the junk mail folder, try disabling the ability to post comments without javascript turned on. Most real people have javascript turned on (at least on sites they trust), but bots never do. Being a popular site you'll probably still get some spam from real people, but it should be manageable. Also, your users wont need to deal with a Captcha form.

There's a WP plugin that does this (plus a little more rigorous screening based on the same concept) called "WP Captcha Free" – I use it on one of my sites and it seems to work just fine.
Jozef N • Dec 27, 2012 @12:15 pm

The best approach I've found, that's also the most effective, is to outsource the comment moderation to a live human. For a few dollars an hour, you can hire someone in India to have them check the spam logs or even check the comments that make it through.
Go to oDesk.com, make a post, and get someone setup with an acct on the site (only give them permissions for comment moderation and nothing else). It's cheap and effective.
Scott Jacobs • Dec 27, 2012 @12:25 pm

Don't allow comments.

Problem solved.
Chris • Dec 27, 2012 @12:32 pm

Lucia @ TheBlackboard has written a lot of about filtering spam and what she has done that works or doesn't work. Maybe useful, or maybe overly technical.

http://rankexploits.com/protect/
Keith Lee • Dec 27, 2012 @12:51 pm

I use Akismet as well. I limit commenting to 60 days as well but, like others, I still get anywhere from 500-1000 spam comments a day. There is no way to hand moderate that.

After hearing lots of recommendations for it, I have just switched over to the Livefyre comment system. It supposed to help cut down on spam and enable easier commenting. We'll see.
Maybrit • Dec 27, 2012 @1:05 pm

Since spammers want to link, you could perhaps deactivate the ability of new posters to create links?
B • Dec 27, 2012 @1:14 pm

Require all comments to be pony-related. Or at least mention a pony. Or a mule. Well, any sort of ungulate, really.
2012.lot • Dec 27, 2012 @4:41 pm

there is a wordpress plugin that collects the IP addresses from all the comments you deemed spam — it has made a world of difference in keeping repeat offenders away – especially the ones that use multiple emails.
Steve Hall • Dec 27, 2012 @11:07 pm

I agree with the Bad Behavior plug-in, and eschew captchas. Even the least obtrusive will discourage me from commenting: I simply hate wasting the extra time. That's a big reason I rarely comment on Blogger blogs.
Tam • Dec 28, 2012 @4:14 am

I only get a couple hundred a day, so it's a quick scan in the morning and afternoon.

I look for reader comments that wound up in there, as well as particularly evocative Markov chains for my forthcoming book on computer-generated Spam poetry.
TJIC • Dec 28, 2012 @6:26 am

@David Aubke :
> @TJIC and others: Why do you dislike captchas so much?

Imagine that you had a bunch of people over for a dinner party, and the conversations is flowing freely.

…then you declare that before anyone can say anything at all, they have to solve a quadratic equation.

…for each and every comment they want to make.

How much conviviality is left? How well does the conversation flow?

How many people choose to come to your next dinner party?

tl;dr: it kills community.
the other rob • Dec 28, 2012 @7:44 am

Author Charles Stross (antipope.org) reported a massive decrease in comment spam when he required registration prior to commenting.

That still leaves the problem of bots trying to register, but there are tools to help with that (for example http://www.stopforumspam.com/search )
Ted • Dec 28, 2012 @7:51 am

Check out Tim Ferri's wordpress plugins that keep him sane. Might help you as well:

http://www.fourhourworkweek.com/blog/2012/07/26/most-popular-wordpress-plugins/
AlphaCentauri • Dec 28, 2012 @3:09 pm

Cloudflare only presents the captcha if you're from a known bad IP range, and you can choose to completely block some ranges if there are human spammers solving the captchas. It draws its IP blacklist from Project Honey Pot.
Brian Lang • Dec 28, 2012 @9:09 pm

I ditched Akismet when I kept getting error messages, and they were no help in trying to solve the problem. I switched to Antispam Bee, and learned a bit about the types of comments I was getting. Antispam Bee adds a hidden form field. If it gets filled in, BAM, instantly marked as spam. Since most spammers use automated systems to fill in comment forms, this catches the majority of WordPress comment spam I was seeing.
You can find it here: http://wordpress.org/extend/plugins/antispam-bee/
Unfortunately the plugin page is in German, but once you install it in your blog, you get English descriptions. You can of course use Google Translate to translate it to English. I am using Antispam Bee on half a dozen WordPress Blogs.
Anony Mouse • Dec 29, 2012 @1:59 am

@David

Any CAPTCHA that requires more than a moment makes commenting less worthwhile. Longtime commentors will hold their nose and soldier on, but potential new people are more likely to move on. How many times have you decided that you didn't really care about some article when a newspaper site required a login, even a free one?

TinyPic uses one of the most offensive CAPTCHAs I've ever seen (watch a 10 second commercial and the phrase appears at the end of it!) and only their general usability keeps me coming back (and the fact that you only sometimes get the commercial ones).

Also, the "scan of 100 year old blurry book page" style are a nightmare when it's literally impossible to tell what the letters are. Frankly, I'd rather put up with an overzealous spam filter occassionally eating real posts than some headache-inducing CAPTCHA nightmare.
Allycat • Jan 2, 2013 @5:03 am

i think Jozef N has a valid point when he says about paying real people to filter the spam. Now spam is primarily a human problem we need a human solution. It is worth the small payout. I would do anything to avoid the CAPTCHA. it really does greatly reduce your comments. I hate the things so much I even started using software called RUMOLA to read and fill them in for me. I know many people who simply won't comment on blogs with CAPTCHA.
Dan Weber • Jan 2, 2013 @12:25 pm

Call on the community to spam filter:

1. Let it be known that appending ?all=true to the URL will show posts in the queue. Default no one here, especially search engines.

2. Let approved posters vote to permanently kill or save items from the queue, merely for spam. Anything that gets at least 5 votes with at least 2:1 odds is saved/killed, although the poster doesn't necessarily become an approved poster if their post was saved and shown to all.

3. Periodically review who is approving what.
Connie • Jan 2, 2013 @12:43 pm

I hate CAPTCHA mostly because I like to view with my images OFF for work purposes. If it requires I turn them back on, I'm less likely to comment or even hang around that site.
Dan Weber • Jan 2, 2013 @12:59 pm

A reminder that captcha's don't have to be images; and we only need them for unapproved posters.
TPRJones • Jan 4, 2013 @8:43 am

I find it best to just dump moderated comments caught by the spam filter unseen. I figure if whatever they had to say looked that much like spam, then they should find a better way to say it.
Gene • Jan 7, 2013 @11:22 am

Try Growmap Anti Spambot Plugin. It adds a checkbox that bots can't check so only humans get get through.

Works great on our blog.
Adria • Jan 9, 2013 @7:15 pm

People comment-spam in order to insert links into your blog. If you stop allowing people to insert links, then the asshats will stop spamming you.

Seriously, you can do all the captcha/comment filtering/magic-wand-waving you want, but unless you remove the motivation for spamming, the spammers will find their way around any perimeter defense.

Also, on behalf of my industry (SEO), I apologize that 95% of us are asshats.
Seerak • Jan 15, 2013 @7:11 pm

Any CAPTCHA that requires more than a moment makes commenting less worthwhile. Longtime commentors will hold their nose and soldier on, but potential new people are more likely to move on.

I'll deal with CAPTCHAS before I'll register. I'm sick and tired of "opening accounts", I have hundreds of them as it is.

But on the bright side, there's not a single imperative "Ditch Akismet" in here (though there is one "I ditched Akismet"). Time was that every single blog which had issues with an overaggressive spam blocker would turn out to be using Akismet. I'm glad they finally seem to have licked that problem.
Seerak • Jan 15, 2013 @7:13 pm

People comment-spam in order to insert links into your blog. If you stop allowing people to insert links, then the asshats will stop spamming you.

If you do this, especially if you do it by blocking all HTML tags, please let us know near the top of the comment form. I hate it when the tags I used to set off someone else's words from my responses just get vaporized without warning, making it all incoherent.
AlphaCentauri • Jan 23, 2013 @6:30 am

A lot of the comments in this thread would have been blocked as possible spam based on the word content. Humans are better at recognizing spam because we can process phrases without any loss of speed. And they aren't fooled by comments that say, "This is a great blog" with nothing more than a link from the user's name, or comments that copy text from previous comments.

Blocking links doesn't stop spammers since they don't come back to check their work to see what it looks like. And it reduces a lot of the two way community on blogs, where you can follow a link and learn more about someone who posted something you found interesting.
Patrick • Jan 23, 2013 @7:59 am

Blocking links stops me from having to review this sort of comment:

Best free anti-spam software!

http://www.anti-spam.com
http://www.antispam.com
http://www.antispamsupreme.com
http://www.antispamisyou.com
http://www.antispamforever.com
http://www.anti-spam-is-you.com

Etc. We get dozens of those a day. If that means that I lose the occasional twenty link legitimate comment from a first time commenter who hasn't gotten onto our approved list (we get one of those about every two weeks), it's worth the price.

You understand we're not Slate. We're not paid to do this. If we had to manually review every comment, that would be the end of the blog.

(Don't click on any of those links, please.)
Robert Reeves • Jan 24, 2013 @9:52 am

In my opinion, Disqus works quite well. I have not used it on my blog personally, but I have seen many of my colleagues use it and have had great success.