Next Time You Are Unexpectedly Banned

Meta

Dear y'all,

We're using WordPress. From time to time, we have tinkered with caching software. At the moment, we're using none because we enjoy our banning software more, and cache seldom plays nicely with ban. Our server also offers caching for interpreted php modules, but we've turned that off. None of the other plugins we employ makes conspicuous use of caching, and we're presently not (knowingly) relying on a CDN. (The server bills at a flat rate, not by the mile.)

Nevertheless, folks occasionally report that they're encountering a ban notice on the front page or on a given post. Typically, these are not folks we've banned (whether directly or by ip range). So far, the false ban message seems to affect only, say, 5 visitors in 20k.

In every case, we've asked these victims of technology to clear their browser's cache and to revisit the site. In every case, this has worked.

We're currently investigating whether the hosting provider employs a pagespeed or caching module in their preconfigured, managed web server. We're also probing the logs to see whether the false bans happen to follow closely on attempted visits from folks actually banned. Meanwhile, if you suddenly find yourself banned and have no reason to think you deserve it, then go ahead and force a reload (CTRL + F5 on Windows; Apple+R or AzaleaBlossom+R on Mac; F5 or CTRL+F5 or meta+F5 on Linux) and see whether that fixes the problem. More aggressively, go to your browser's settings and explicitly delete your local web cache (Option+AzaleaBlossom E on Mac).

Once we're done troubleshooting and making this annoyance go away, I'll report back. And of course, if you're comfortable with the Dreamhost/Debian/Apache/Wordpress stack, feel free to make troubleshooting suggestions in the comments below!

Last 5 posts by David

38 Comments

38 Comments

  1. Jon  •  May 6, 2013 @2:36 pm

    This happened to me earlier today and I almost cried. F5 didn't do the trick at first (trying both Chrome and IE), but visiting about five minutes later or so did. I guess this isn't very helpful to those currently seeing "banned" though…

  2. Ashley  •  May 6, 2013 @2:50 pm

    I got the banned message a few weeks ago. I ran a quick check on http://www.downforeveryoneorjustme.com/ to learn that Popehat was down, and I immediately stopped weeping. A couple of hours later, BAZINGA! Sweet, sweet access to Popehat.

    I don't know if that's the same issue or a different one, but I thought additional information might help y'all troubleshoot. Good luck!

  3. James  •  May 6, 2013 @2:53 pm

    FYI I had a problem twice (about two weeks ago) where I could visit and submit a comment but the new comments would not show up. There was nothing in the posts anybody would find in the least bit offensive so a spam or naughty word filter would not have been the cause, but the posts went into electron heaven anyway. The next day all was right with the world so I didn't mention it at the time.

    Next time it happens I will have to sue you. Just what a non-frivolous cause of action will be will require some thought but I will take my revenge . . . so there.

  4. CaptainPugwash  •  May 6, 2013 @2:59 pm

    I tried to read this message, but it said I was banned, so I don't know what it said or what I have to do to resolve it!

  5. Stephen  •  May 6, 2013 @3:22 pm

    I got the message this afternoon as well. I cleared my cache (using Chrome) and still didn't get in. I also tried Incognito, since I thought that would use it's own (and presumably empty) cache and didn't get in.

    Waiting 10 minutes and trying again worked, however.

    To give a sense of timing, I got the first "banned" page right after the this tweet

    Has anyone seen any litigation recently on the "this defamation threat letter is copyrighted by me" front?

  6. Kat  •  May 6, 2013 @3:22 pm

    Thanks for the info :) I'll keep this in mind in case of mind-breaking ban messages.

  7. Todd Knarr  •  May 6, 2013 @3:23 pm

    It may not be on Popehat's end. Some ISPs force a caching proxy on their users, forcing all HTTP traffic through it whether the users want it or not. There's not a lot the users can do except force a reload (the cached content may be messed up but most of them honor the "don't cache" header that a forced reload puts in). Ken might be able to do something, but any claim that a cache was infringing on his rights would probably be tenuous at best. Publicity and making it a CS headache for the ISP is probably the only viable option.

  8. hamjudo  •  May 6, 2013 @3:26 pm

    Sometimes, large ISPs will cache content too near the browser end of connections. In particular cable companies and mobile providers.

    There is a protocol to ask if a cached copy of a page is current, or if it needs to be refreshed. In Chrome and Firefox, the F5 key asks if its cached copy is good, and replaces it only if the web server says its out of date. Whereas, F5+shift will throw away the cached copy, and get a fresh version.

    Some ISPs do sort of the same thing, but between customers.

    There are many HTTP headers which influence how content is cached. ISPs don't respect them all equally. If I recall correctly, some ways of marking content as dynamic are pretty much uniformly ignored.

    Undoubtedly, someone else has more current experience than I. If not, is it possible to set up a page that will show what a banned user's browser gets? (I mean, without me writing bad things.)

  9. HoveringHalibut  •  May 6, 2013 @3:36 pm

    Just to prevent the ban page from being cached (hopefully any proxy doesn't just ignore it), you could add something like the following to the code.

    <code>

    </code>

    I've never done this with WordPress or even PHP, but in other languages for similar reasons.

  10. leo marvin  •  May 6, 2013 @3:50 pm

    Admit it, the only people getting this message are people who agree with me.

  11. Luc  •  May 6, 2013 @3:59 pm

    Tip: In many browsers, hitting Shift+Ctrl+F5, Shift+Ctrl+R, or holding Shift while clicking the Reload button, forces the browser to grab a new version of the page. Try it some time on an image-heavy page, if you don't believe me. (Works in Firefox, Chrome, Opera, and version 4+ of Safari.) Source: http://en.wikipedia.org/wiki/Wikipedia:Bypass_your_cache

  12. Joe Pullen  •  May 6, 2013 @4:05 pm

    Hmm we'll see. I had a comment that has gone to electronic heaven somewhere. If this shows up clearing the cache did the trick. If not, then Popehat ISP is having an issue with my VPN.

  13. Nick  •  May 6, 2013 @4:06 pm

    How does someone get worked up enough to banned from Popehat? This is interesting stuff, fun to read, toss in a joke every now and then, but how can you get yourself banned from here? This is like baby time frolics, not knife fights on the waterfront.

  14. Ken White  •  May 6, 2013 @4:25 pm

    Nick: consider the comments policy. Now consider this post.

    Saying "frankly I really think Traveler was a terrible game, far inferior to Star Frontiers" will not get you banned. Responding to that with "Fuck you, Traveler rules" is on the continuum that will lead to you getting banned. Saying "you liking Star Frontiers is just part of the bullshit you've been spouting recently" is on the continuum to getting you banned. Saying "I am sick of all the nerd stuff around here recently. When will Ken write about Prenda again?" is on the continuum towards getting you banned.

    You're in our living room. Disagree with us forcefully. But act accordingly.

  15. J  •  May 6, 2013 @4:29 pm

    >This is like baby time frolics, not knife fights on the waterfront.

    From my understanding, a gun fight might do the trick.

  16. Nobody  •  May 6, 2013 @4:33 pm

    I tried that. It still told me it was banned for a while, then it finally went away. Go figure?

  17. eddie  •  May 6, 2013 @4:42 pm

    "Fuck you, Traveler rules" is on the continuum that will lead to you getting banned.

    Objection! "Traveler rules" is in the vocative case here.

  18. Clark  •  May 6, 2013 @4:52 pm

    @Nick:

    This is like baby time frolics, not knife fights on the waterfront.

    Ah, I see someone is trying to restart the repeatedly-memory-holed "how did the cobloggers meet Ken in the first place?" thread.

    …which is on the continuum towards getting you banned.

  19. Nick  •  May 6, 2013 @4:59 pm

    "This is like baby time frolics" – Totally a Archer Season 1 Episode 1 (16:29) reference.

    :)

  20. Kevin  •  May 6, 2013 @5:02 pm

    How about "Judge Wright's order would have been WAY better if he had gone with Star WARS references"? Is that on the continuum?

  21. David  •  May 6, 2013 @5:04 pm

    It may not be on Popehat's end. Some ISPs force a caching proxy on their users, forcing all HTTP traffic through it whether the users want it or not

    This is the prevailing theory. I think we're being cached involuntarily, but haven't yet learned what combination balancers, reverse proxies, Redis instances, or who knows what all the ISP is employing (as a feature!) in our meta-environment.

  22. Merissa  •  May 6, 2013 @5:06 pm

    Delicious paste!

  23. Pierre  •  May 6, 2013 @5:35 pm

    I've nothing useful to add, but I thought I'd add my symptoms to the list in case they help. I was banned earlier today; I tried reloading, incognito mode (Chrome), a different ISP (VPNed into my home connection), and mobile — the ban seemed to cover the entire spread… although come to think of it, I think that the mobile page was loading for me while the desktop site was still banning me (both from my smartphone).

    For the record, I tried Rogers (business Internet and mobile) and Bell (residential internet) in these machinations.

    Then I emailed Ken, got a couple of nice responses, and shortly after, I was back in!

    Lesson learned: if I get banned, just keep emailing Ken until he lets me back in? ;)

  24. Bear  •  May 6, 2013 @7:58 pm

    @Kevin: "How about "Judge Wright's order would have been WAY better if he had gone with Star WARS references"? Is that on the continuum?"

    No, but declaring that he should have gone with Quark references is probably pushing it.

  25. Andy  •  May 6, 2013 @8:27 pm

    According to Ars Technica, HTTPS traffic is generally not cached by caching proxies. Wonder if having that available for people who think they're banned would help. AFAIK, it is possible to generate an HTTPS certificate that might be suitable for testing purposes (though browsers won't like it–they scream bloody murder).

  26. David  •  May 6, 2013 @8:43 pm

    If we used a self-signed cert, browsers might freak out. A legit cert costs. And we don't really have a use case involving secure traffic to/from the server. So I don't think we'll be going https anytime soon. But you're right; that would defeat caching.

  27. deskmerc  •  May 6, 2013 @8:51 pm

    It probably isn't cache on the webserver. Assuming the Apache cache control variables have not been altered:

    Cache-Control: max-age=3, must-revalidate
    WP-Super-Cache: Served supercache file from PHP

    That cache constrol is more for a browser's benefit than anything else. (force cache revalidation after 3 seconds, content expires quickly) You do not appear to be behind a CDN. The WP SuperCache would affect everyone if that was the source of the problem, as it converts webpage output to a static HTML page to try and make it faster to load.

    Something is grabbing things before the page starts to load, however, looking at things like blank User-Agent strings. I used curl to peek at headers:

    [root@derp ~]# curl -I http://www.popehat.com
    HTTP/1.1 403 Forbidden
    Date: Tue, 07 May 2013 03:47:07 GMT
    Server: Apache
    Vary: Accept-Encoding
    Connection: close
    Content-Type: text/html; charset=iso-8859-1

    Whatever you have, it is parsing incoming headers before it even gets to PHP and WordPress. I'd look there first. (mod_security, perhaps?)

  28. Anony Mouse  •  May 7, 2013 @12:46 am

    Everything could use more Archer. A continuing favorite in my D&D game is "Suck it, Samwise."

  29. Red Tonic  •  May 7, 2013 @6:31 am

    I did the same as Pierre a week+ ago. I (almost always) access Popehat by incognito Chrome because… Browsin' at work! Ken & company were very kind. On my end, the problem seemed to strike only when I accessed the site by typing popehat.com rather than http://www.popehat.com. Odd indeed.

  30. Jack  •  May 7, 2013 @7:34 am

    I think in terms of caching Ken was talking about caching all of the MySQL database calls and all of the high-load PHP scripts using Memcached on the server. Basically, using caching on the server, you can handle at least an order of magnitude more traffic if you implement caching.

    As for the banning plugin and making it work with memcached and the various WordPress caching plugins, basically all you need to do when you ban someone is run the "flush all" command on Memcached and it will clear the entire cache so you won't have to wait for the cache to expire before the person feels the ban.

    I say this since it is the easiest solution without modifying the ban plugin to update or flush the specific place in the cache where the banned person is.

    What is good about this is you can simply create a new button at the top of your admin panel that says "Clear Cache" and when you click it, it automatically issues that command.

    I would be more than happy to send you over the very, very tiny PHP file that does this if it will make your site load faster!

  31. Ed T.  •  May 7, 2013 @11:11 am

    It is possible the visitor's ISP is running a caching server in transparent mode. This helps lessen the load on the network connection when lots of folks access popular content, however it can sometimes leads to false positives on banhammer software.

    ~EdT.

  32. Merissa  •  May 8, 2013 @7:34 am

    I'm banned in IE but not in Chrome. Stupid IE. Wish Firefox would run on this antique.

  33. Stefan  •  May 8, 2013 @9:59 am

    Looking at the page source here I see the super cache footer. We use W3total cache and a few weeks ago I updated and had my settings overwritten, so for a few hours the browser cache was set to a year (vs. 30 min).

    This caused hundreds of users to think we did not post anything new for weeks. Getting average users to clear there cache was/is a pain and some are unable to do it and surf us in different browsers.

    Not saying that's what happened here but cache management has it's surprises some days.
    Late to the party with 2c.

  34. chrisdag  •  May 8, 2013 @2:41 pm

    Just want to reiterate something that someone else mentioned. I'm a reader and not someone who has commented before so I'm probably not banned.

    Today I got the "banned" message across all browsers working from a home office with a static IP that I've used for years so the IP is an unlikely source of bad traffic.

    The difference? I had typed "popehat" into the chrome browser bar and it had sent me to http://popehat.com

    I'm still banned when I go to popehat.com but all is well when I'm hitting up http://www.popehat.com

    Maybe the "popehat.com vs. http://www.popehat.com" should be added to the list of things to check for people who find themselves unexpectedly banned.

  35. Chris  •  May 8, 2013 @2:48 pm

    Just got the banned message. Turns out that google search popehat banned links to this article and will let you back onto the site, and from here popehat works correctly.

  36. neverpostedbefore  •  May 9, 2013 @10:18 am

    I got the "banned" notice just now and wanted to chime in. I got the banned page in Firefox, but not in IE. My problem was exactly the same as chrisdag's. I had typed "popehat.com" into the address bar of Firefox; when I changed it to http://www.popehat.com the page loaded normally. In IE I had gotten to popehat via google (and thus the full web address), so I didn't have the problem.
    Hope that information helps you work out what's going on.

  37. Jim Salter  •  May 11, 2013 @12:01 am

    I'm a mercenary sysadmin; I work with infrastructure for extremely large/high-traffic sites, some WordPress, some not. First thing first: you absolutely should NOT disable PHP caching (most likely, you're referring to php-apc) – it won't ever cause the kinds of problem you're worried about. Second, regular server-side caching need not cause problems with banning either.

    Third, yeah, you're probably seeing issues relating to ISP-wide cache and/or issues with popehat.com vs http://www.popehat.com. You can, as mentioned earlier, defeat the former by using SSL – ISPs not only "generally don't" but *can't* cache SSL traffic.

    I get that there's not a ton of budget in running Popehat, but getting a real cert doesn't have to cost a ridiculous amount of money. Namecheap.com offers them starting at something ridiculous like $8/year. Total drop in the bucket compared to your normal hosting cost.

    Please feel free to hit me up if you want more direct help, pro bono. I love Popehat and I love what you guys do – I can't respond to the Popehat signal when you're looking for *attorneys*, but I can when you're looking for linux sysadmins. =)

  38. AlphaCentauri  •  May 11, 2013 @8:30 am

    Echo what Jim Salter said about certificates — the expensive ones confirm that you are who you say you are (for people wondering if they should provide credit card information). The cheap ones just confirm that transactions are encrypted.