Many US ISPs in epidemic of covert search-hijacking of their customers

The Electronic Frontier Foundation worked with UC Berkeley's International Computer Science Institute to uncover a widespread program of search-hijacking by American ISPs. Many US ISPs run covert proxies that redirect certain lucrative search queries (made by customers who believe that they are searching Google or another search engine) to their preferred suppliers, pocketing an affiliate fee for delivering their customers. Participating ISPs, which include Cavalier, Cogent, Frontier, Fuse, DirecPC, RCN, and Wide Open West (Charter used to do this, but appear to have stopped), did not disclose the practice to their customers, who were meant to believe that they were getting the search results that their preferred search-engines had presented.

EFF and ICSI uncovered the vendor that supplied the hijacking software, a company called Paxfire.

Using EFF's HTTPS Everywhere Firefox extension and a search-engine that permits HTTPS logins (such as Google or DuckDuckGo) will prevent this sort of hijacking.

The published research papers did not identify the controller of the proxy servers that were receiving the traffic, but parallel investigations by the ICSI Networking Group and EFF have since revealed a company called Paxfire as the main actor behind this interception. Paxfire's privacy policy says that it may retain copies of users' "queries", a vague term that could be construed to mean either the domain names that they look up or the searches they conduct, or both. The redirections mostly occur transparently to the user and few if any of the affected ISP customers are likely to have ever heard of Paxfire, let alone consented to this collection of their communications with search engines.
The proxies in question are operated either directly by Paxfire, or by the ISPs using web proxies provided by Paxfire. Major users of the Paxfire system include Cavalier, Cogent, Frontier, Fuse, DirecPC, RCN, and Wide Open West. Charter also used Paxfire in the past, but appears to have discontinued this practice.
Why do they do this?
In short, the purpose appears to be monetization of users' searches. ICSI Networking's investigation has revealed that Paxfire's HTTP proxies selectively siphon search requests out of the proxied traffic flows and redirect them through one or more affiliate marketing programs, presumably resulting in commission payments to Paxfire and the ISPs involved. The affiliate programs involved include Commission Junction, the Google Affiliate Network, LinkShare, and Ask.com. When looking up brand names such as "apple", "dell", "groupon", and "wsj", the affiliate programs direct the queries to the corresponding brands' websites or to search assistance pages instead of providing the intended search engine results page.

Widespread Hijacking of Search Traffic in the United States

(Image: 2005_South Africa_Centurion_DSCF0242, a Creative Commons Attribution Share-Alike (2.0) image from hmvh's photostream)

I write books. My latest is a YA science fiction novel called Homeland (it's the sequel to Little Brother). More books: Rapture of the Nerds (a novel, with Charlie Stross); With a Little Help (short stories); and The Great Big Beautiful Tomorrow (novella and nonfic). I speak all over the place and I tweet and tumble, too.

MORE: Business • fraud • privacy • ripoff • security • sleazy • telcoms • web theory

More at Boing Boing

Eurovision 2013: An American in London

The technology that links taxonomy and Star Trek

http://twitter.com/ambiguator ambiguator

How is this not fraud?
- flosofl
  
  Because it’s buried in your TOS somewhere, I’d bet.
http://twitter.com/wadevondoom Wade Neuwirth

Here in Canada, Shaw does this also. At least they have a way to opt out of this “fabulous service”. If you are on Shaw just go to this site from your home (Shaw connected computer): http://nxr.shaw.ca/optout/
Fedule

I’m not entirely sure I get it. They make it so that for example when you google “apple”, you… get taken straight to apple.com? Or to a site that’s *not* apple.com? Or to what?
- http://twitter.com/wadevondoom Wade Neuwirth
  
  Say you misspelled apple.com. Instead of the usual behaviour where you would go to a Google search page you would instead be sent to a Shaw, (Cavalier, Cogent, Frontier, Fuse, etc., etc) run portal with ads on it. To make it worse it is sometimes occurring with legitimate, active URLs as well.
  - dragonfrog
    
    It’s even worse than that, by the sound of it. From the article:
    
    “When the user initiates searches for specific keywords from the browser’s URL bar or search bar, the proxy no longer relays the query to the intended search engine, but instead redirects the browser’s request through affiliate networks, as the equivalent of a click on advertisements.”
    
    So, it sounds like what Fedule said – if you search for “apple”, for instance, you might get sent straight to the page of the electronic manufacturer, and never get to see the search results page that would certainly have many results that are from the electronic manufacturer’s page, but also results relating to the fruit, maybe the record label, maybe news articles that cast the electronic manufacturer in a negative light, etc.
  - Antinous / Moderator
    
    This started a year or so ago with Time Warner. If I don’t type in .com, .org, etc. when I enter an address, it takes me to a Time Warner search page. It’s extremely annoying.
    - penguinchris
      
      As has been noted, switching to the Google DNS servers fixes this.
      
      Of course by doing that you run into a different set of potential problems. The Google DNS is not necessarily faster than your ISP’s, and in fact is most likely going to be slower (though in CA it’s probably fine, and in any case no matter where in the world I am it works fine for me). Also, content distribution networks may theoretically get confused about your location and choose a server near Google, not near you (whether or not this is actually a problem isn’t clear to me, but it was discussed on slashdot a while back by people who seemed to know what they were talking about).
      
      But most importantly, you don’t know if Google is tracking your queries (why else would they offer this for free?) Not that I trust your typical ISP more than Google.
Will Booth

Rogers does the same as Shaw. There is a work around. Go into your network connection, open the IPV4 properties, and set your DNS server addresses as 8.8.8.8 and 8.8.4.4 respectively. That’s Google’s DNS service. I’ve done this on my system, as well as the SMC gateway/router combo they’ve provided me with. I haven’t seen a stealth redirect again
Scratcheee

Wow…lots of confusion about what is really happening here. The way I understood it was, you don’t get “sent” anywhere at all. What you get are search results that you would assume came from your chosen search engine, but actually came from the vendor’s own software. What I don’t see here is whether the results are really made to appear as if they came from your search engine, with the logo and layout and everything, or if they just give you a generic search page with their selected “results.”
bart

How is this not illegal? It has every property of a “man in the middle” attack. (see http://en.wikipedia.org/wiki/Man-in-the-middle_attack )
http://www.facebook.com/profile.php?id=567759481 Joseph V. Kelly

What are these companies doing differently than the user profiling, targeted marketing, search tracking, and data mining techniques which Google is already performing when you issue a search request?

It seems like the major complaint is that they are covertly siphoning off revenue from Google. I guess you could also make the point that whomever they are selling your data to might not be as reputable/secure as Google.
- http://twitter.com/ambiguator ambiguator
  
  Which is exactly why I would think Google et al would be interested in prosecuting these shenanigans.
  
  Further, just because something is in a ToS doesn’t suddenly legalize illegal behavior. Given this description, it sounds as if ISPs are serving their own content masquerading as another entity — a clear case of fraud.
  
  However, despite my initial reaction and fraud charge, I have experienced this personally, and the implementation is usually slightly different than described:
  
  Every so often, a page request (usually a URL typo) will return some page from a content farm that I will ignore. This is different *enough* from the description, in that the ISP is not pretending to *be* Google, rather the ISP is presumably hoping that users will confuse its paid content with organic search listings.
  - archmagetrexasaurus
    
    The phenomena you’re experiencing is not the one referenced in the article.
    
    Your ISP is redirecting you to advertising content when their DNS service would otherwise report an NXDOMAIN (non-existent domain). This generally happens, as you’ve noticed, when a URL is mistyped (especially the TLD part of it). I’m not aware of an ISP which doesn’t do this, and OpenDNS uses to raise revenue for their free service.
    
    In fact, although the codified standard is to not ever, under any circumstances perform redirection on NXDOMAIN cases, Google’s DNS is the only public or ISP level DNS that I’m aware of which doesn’t perform such redirection.
    
    Admittedly, some folks who are uncomfortable with computers get hysterical every time an error occurs (and they’re notified), so the redirection can keep them calm, which probably reduces the volume of truly hopeless customer service calls, (my internet’s not working/what’s wrong with it?/it’s broken).
- http://www.nathanhornby.com/ Nathan Hornby
  
  Because Google ranks listings based not on revenue, but on the likelihood you’d click on that result. It’s an insanely complex and intelligent system and it’d be foolish of them to insert results for monetary gain, as it would compromise the search engine’s accuracy and lead to losing users.
  
  Sponsored search results appear at the top and right, where you can identify them. My assumption is that the ISP’s are ordering the search results based on affiliate profit, which is very different. I’m also assuming that they’re rebranding the search results (Virgin do that over in the UK, but I use Google DNS so never see them) but even if they’re rebranded they’re still touted as Google search results, and therefore the users assumption is that it;s a standard Google search with a branded header. Where in fact it’s more like BoingBoing’s search results using google (showing BoingBoing results), whereas the results are the open web, but reordered to benefit the ISP, not the user.
- twency
  
  Joseph V. Kelly: “What are these companies doing differently than the user profiling, targeted marketing, search tracking, and data mining techniques which Google is already performing when you issue a search request?”
  
  They aren’t merely profiling, targeting, tracking, or data mining. They are intercepting queries and inserting their own results. Totally different, and exponentially more sleazy.
- Guest
  
  Frankly, I have been wondering if Google does not do the same thing, for example, if one searches for certain topics while in a country other than the U.S (that has been threatening Google with legal action)., that are considered politically sensitive by that country, how would I know if my searches have not already been co-opted to reveal less?
ackpht

I’m glad to read this, because I thought I was losing my mind. I had been typing keywords into the Google homepage and getting directed to sites that had nothing to do with my query. My typing’s bad, but it’s not THAT bad.

I have to imagine that Google will take action, as it represents messing with their bread and butter.
- 10brooks
  
  What you’re describing sounds more like malware to me. My take on this story is that you’ll get reasonable results, just not exactly what your search provider meant for you to get. Sounds like it’s effectively rearranging the results ranking to favor sites that send the ISP a kickback.
  - ackpht
    
    If it was malware, it was somehow associated with the Firefox browser, because Explorer did not exhibit it.
    
    This is why my video editing PC never gets hooked up to the web.
usuallyconfused

Oh no…Charter is still hijacking – happened to me just yesterday
Joshua Ochs

Although in other respects I like WideOpenWest, I’ve run into this and it is *horrid*. Strangely, I only found out about it when I surfed from a third-party browser on my phone, and it got caught in the trap.

The way it worked was this – you go to google, put in a query, and instead of getting Google results back, it hijacks the query and you get their own search results – which are crap. It took forever to locate the opt-out link (first sign something is dishonest – it’s opt-out instead of opt-in).

Since it also persists even when I change DNS servers on my router (and by extension, anything on the WiFi network that’s picking up its DHCP info from it), they’re not just doing DNS redirection, they’re actually doing deep packet inspection and redirecting packets in transit. Evil.
Roderick St John

Cox does this. They call it an “Enhanced Error Results” page.

After theDNS hijack, Cox offers a link to an “opt out” page which does not opt you out. Next time the Cox DNS hijack occurs and you go to the “opt out” page again, you are told you are already opted out.

Deceptive and underhanded
- Bersl
  
  I remember when Cox started doing this. You only get to opt out of the advertising, not out of the NXDOMAIN hijack.
  
  Use another DNS server. I use the dnsmasq on my router flashed with OpenWRT, which in turn queries Hurricane Electric’s DNS server (which I can use because I have an IPv6 tunnel with them). Works just fine.
  - Guest
    
    You could also use opendns.org as your DNS server, which would prevent all of this and is faster.
http://pulse.yahoo.com/_7Q3NPTZ7HBYYOV75JJLH3O7I3E No

This explains something that happened to us a couple of weeks ago. A paid member of our games site contacted me by email to tell me that he wasn’t able to get to our games site. Whether he used his bookmark, or the link on our front page, or the link I sent him in email, he would end up on a Bing page with a search query that had an altered domain name, not ours.

He had spent a long time on the phone with his ISP’s tech support, and they kept telling him that it must be a problem with our server, and that we were re-routing him with a bad link. He was pretty angry by the time he contacted me — after all, he’d paid for access — and so I ended up spending some time on the phone with him as well, giving him some credit to calm him down, and asking him to let me know if he was still having the problem the next day when my sysadmin would be in. I got an email from him the next morning thanking me for “finally fixing it.” Which, of course, I hadn’t.

His ISP is Frontier. And Maggie Wilderotter is going to hear from me about it.
Tim Thompson

This coincides with a number of promotions coming out of the Affiliate Marketing camps, promoting a new tool that “taps into a new source of traffic funneled straight to your website”, something “nobody even knows about”. I fear there are more companies than Paxfire with this technology.
we_the_people324

Fuck frontier. Thank you ipredator.
Apashiol

I was just about to ask this very question as I’ve been using OpenDNS for years and hoped it would secure me from this.
Cheers
http://www.bencollier.info/ Ben Collier

But how long until the ISPs start blocking competing DNS services?
http://carrierlost.com/ Tonweight

Even with OpenDNS, Cavalier still manages to hijack me.

I exposed it while troubleshooting my VPN connection – a Cisco connection worked just fine (different tunnelling methodology, I guess), but another vendor’s (from a different job) crapped up with altogether flukey results. I had seen the “bogus” search results page after mistyping once and had asked the Cavalier techs if they could remove whatever “filter” was in the way of my connection, and they said “what filter.”

When I finally got sick of doing the dance and called bullshit on Cavalier, the techs all kept toeing whatever party line they have of “nope, not us – definitely you.” Jerks. I know they’re doing their jobs, but I would find another job if my employer asked me to lie to people. Moral high road doesn’t pay the bills, but I have so many skills that I’m never without employment.

Anyway – my current job uses a Cisco VPN, so I’m without issue again. Well, except for the piddly 768k DSL connection part. >_< If Verizon would allow static IP with residential accounts, I'd switch in a heartbeat.