Study: Bandwidth hogs aren't responsible for peak network congestion

David Weinberger does a great job summarizing a paywalled report by Benoît Felten and Herman Wagter, who investigated ISP usage patterns in five-minute-increments to see if "bandwidth hogs" were really a problem for ISPs.

They found that there is indeed a set of users who download a whole lot: “The top 1% of data consumers…account for 20% of the overall consumption.” But half of these “Very Heavy consumers” are doing so on plans that give them only 3Mbps, as opposed to the highest tier of this particular ISP, which is 6Mbps. So, even with their heavy consumption, their bandwidth usage is already limited. Further, if you look at who is using the most bandwidth during peak hours, 85.3% of the bandwidth is being used by those are not Very Heavy users.

Here’s the point. ISP assumes that Very Heavy users (= “data hogs” = “people who use the bandwidth they’re paying for”) are responsible for clogging the digital arteries. So, the ISPs measure data consumption in order to preserve bandwidth. But, according to Benoît and Herman’s data, the vast bulk of bandwidth during the times when bandwidth is scarce (= peak hours) is not taken up by the Very Heavy users. Thus, punishing people for downloading too much inhibits the wrong people. Data consumption is not a good measure of critical broadband usage.

Put differently: “42% of all customers (and nearly 48% of active customers) are amongst the top 10% of bandwidth users at one point or another during peak hours.” The problem therefore is not “data hogs.” It’s people going about their normal business of using the Net during the most convenient hours.

Are “data hogs” the problem?

• [Michael Foord]

  So bandwidth hogs use 20% overall, but during peak hours that falls to 15%? That’s still a lot for 1% of users.

  • [Entrope]

    If the top 1% only use 20% of bandwidth, that makes them totally different than the top 1% who make 20% of the income, right?  After all, during a recession (when things are congested and slow all around), that percentage drops too, right?

  • [Michael Foord]

    So the summary of the article is “proportion of bandwidth used by top 1% falls during periods when other people are using it intensively”…  Uhm, captain obvious at the helm?

    The top 1% of users are still using 15x more bandwidth than the average during those periods. This doesn’t seem to support the conclusions that follow.

• [Mari Lwyd]

  ISP assumes that Very Heavy users (= “data hogs” = “people who use the bandwidth they’re paying for”) are responsible for clogging the digital arteries.

  Yes, blame users rather than improve infrastructure. That’s some super-awesome customer service policy right there.

  Can we outsource our CEOs from a country that understands infrastructure, please? U.S. business colleges obviously don’t understand elementary requirements for technological civilizations.

  • [Mordicai]

    &/or can we have huge government investment in infrastructure– not just wires & cables but…heck, water pipes & stuff?

    • [Mari Lwyd]

      Where’s the money for lobbyists in that?

      Are these proprietary water pipes that adapt water cleanliness per rate paid to privatized water companies?

      Don’t tell me you’re advocating socialist water.

• [ComradeQuestions]

  Huh.  It’s almost as if bandwidth caps were a form of price-gouging that had no bearing whatsoever on the actual cost of delivering data at the rates customers are paying for.  Weird.

  • [Mordicai]

    Weird, right?  Like this whole tiered structure proposal was a way to get money & establish precedents of control, & not help my youtube video buffer faster.  I’d almost be suspicious that it was a huge con using digital access as a hostage, if I didn’t trust corporations & lobby groups so much!

• [Another Kevin]

  ‘Put differently: “42% of all customers (and nearly 48% of active customers) are amongst the top 10% of bandwidth users at one point or another during peak hours.” ‘

  So 42% of all customers (and 48% of active customers) are data hogs, and the ISP’s can hit them with punitive surcharges. Sounds like a business opportunity to me!

• [Just_Ok]

  some day they’ll try what the power companies do: charge more for usage during peak periods

  • [Alex Stapleton]

    Virgin Media essentially already do this. All their consumer cable products are subject to throttling restrictions if you use too much bandwidth in a certain period. Except of course for their top-tier 50Mb product. It is actually one of the main reasons I upgraded to it.

    • [Just_Ok]

      Damn. Shoulda patented the idea when I had the chance. 

  • [Ian]

    That’s what my ISP already does. A ‘unit’ of monthly prepaid transfer will buy you 2.5Gb from 9am-6pm, 50Gb from 6pm-2am and 1 Tb from 2am-6am. Yes, that’s not a typo – 1 terrabyte.

• [WaylonWillie]

  “Thus, punishing people for downloading too much inhibits the wrong people.”

  I’m not sure this is really the argument they want to make. The Verizon part of my brain says, “Ok, so lets punish the other 85%.”

• [Paul Renault]

  Y’know, there are some advantages to living in the relative boonies: ISPs’ routers and switches don’t have all the latest geehaws and doodads.
   (Actually, it’s because the province of New Brunswick, under the guidance of Premier Frank McKenna, deployed fiber optic cable all over the province for the phone and Internet backbone, during the early ’90s.)

  So, twenty years later, I’ve got a reasonably priced fibre optic Internet connection from the local phone company, which is providing me 15Mbs both ways.  No ports are blocked, no restrictions.  It’s a completely vanilla Internet connection – except for my ISP’s routers’ NATting.  I can set up a file server at home, no problem.

  I could bump up my connections speeds to 70/30Mbs for an extra $30/month or so, but y’know, just how much porn can one guy download, eh? 

  FYI: 70/30Mbs Internet-only is $100/month.   I have a screen capture from SpeakEasy offering ‘Our Best Value’ of 2/2Mbs at only $229/month!

  But the important point here is: my ISP does not have usage caps.  None.  Nada.  Zip.  Their equipment doesn’t have the capabilities to monitor it.  And they don’t plan on adding this ‘feature’ in the future.  They see the lack of caps as a selling tool.

  Bonus: lobster counts as an ‘eat local’ food item…

  • [Ian]

    “Their equipment doesn’t have the capabilities to monitor it.”

    I can assure you it does, this is very basic stuff and is inherent in the monitoring capabilities of quite antiquated kit. We could do this at my first ISP set up in 1995. Your ISP just chooses not to make commercial use of that information.

• [Stooge]

  Cory, I’m having trouble attributing much value to your endorsement of Weinberger’s summary as “a great job” unless you’ve actually read the report.

  So, did you splash out €750 for this 13 page report just so you could say someone else has distilled its essence fairly and accurately, or are you merely praising Weinberger’s spelling and punctuation?

• [666beast1]

  ISP’s would like you to get used to paying for the internet like a faucet.  It means a lot more cash for them and negates paying for technical improvements. It’s a win/win(for them).

• [screwt]

  I’m sorry, in which country/ century is an ISP’s “Highest tier” stuck at 6Mbps?? Is this one of those quaint USAian things?

• [Lobster]

  So… the summary is, “We’re not using 20%, we’re using 15%, but it’s not nice to call us ‘data hogs?’”

• [Ryan Lenethen]

  What people fail to understand, is that ISP as part of their business model oversell their bandwidth with the knowlege that most users do not use their actual capacity. Just like an airline might oversell seats on a plane. The difference is that a plane might oversell a few seats. An ISP oversells the entire plane several times over. Then when everyone shows up to get on the plane the actually paid for, it is somehow their fault for flying too much!

  Seriously.

  • [Stooge]

    Do you mean people so stupid that they are incapable of understanding the distinction between shared and guaranteed bandwidth? I thought they were just an urban myth.

  • [Ian]

    Providing enough capacity for everyone to use the full capacity of their connections simultaneously would be economically impracticable and moreover pointless as this never happens in practice. Likewise we don’t provide enough roads for everybody to simultaneously drive all their vehicles, or enough electricity generation for everybody to simultaneously use the full current capacity of their hookups to the mains.

    In practice demand for all of the above is bursty in nature and the diversity between different users means that less than 100% capacity can satisfy 100% of actual demand. I can’t speak directly for electricity or traffic management but in the ISP world a user to backbone capacity ratio of 4:1 for office networks (which already concentrate demand from many users onto 1 link) and 20:1 for residential users will still leave you with plenty of capacity headroom (typically 50%) for unusually concentrated bursts of traffic.To put this into perspective, if all the cable and DSL users in the UK were to be able to pull their maximum usage simultaneously the network would have to be able to handle an approximate 23 terrabits per second (based on installed connections and typical maximum speeds). In practice, peak UK traffic is around 4 Tb/s and mean traffic around 2 Tb/s. A network built to your standards would need to be five to ten times bigger than it actually needs to be in practice and, unsurprisingly, would cost five to tens times as much to subscribe to. Also 90 – 95% of the capacity on it would be permanently idle.

• [Jellodyne]

  See, this is why it’s so important to choose a broadband provider with enough infrastructure and bandwidth to actually serve all of it’s customers during peak hours, good customer service and technically skilled support reps. On the other hand, in my neighborhood, I only have one provider who can offer me more then 3MB for less than $100,  so my only real choice is to bend over and take whatever Comcast deigns to give me.

• [crummett]

  There you go again, Cory, using facts to support your ideas. You know that doesn’t work in the modern world.

• [odds]

  How dare you say that “convenience of schedule” is the primary factor driving peak usage, when it is completely supported by the facts.

• [Greg Smith]

  As SSDs (flash memory) become ubiquitous as app servers or simply as fast caches for app servers, these IO issues will be going away, only to be replaced with problems of another sort no doubt. In the meantime, flash as cache is so 2012!

• [Matthew Cunningham]

  “Thus, punishing people for downloading too much inhibits the wrong people.”
   
  You know, I have always wondered why companies are always “punishing” or “inhibiting” thier customers. It seems like it would be just good sense for businesses to ”allow” and “enable” thier customers to use the services they pay for. I would think there is a good amount of money to be made by selling thier product to as many people as are willing to pay for it; likewise, there would be more people willing to pay for something if they feel they are getting some value for thier money. Isn’t that how economics is supposed to work?