Browser/IQ correlation hoax: IQ considered unintelligent

A bunch of sites are reporting that the "study" that claimed to correlate IQ with browser choice was a hoax. The funny thing is, the hoaxy part everyone's up in arms about is that the correlation was faked. But surely's the most damning element of this "study" is that it used IQ as a meaningful proxy for "intelligence." If someone came up with a study that correlated star-signs, biorhythms, Meyers-Briggs types, or auras with browser choice, and then it was revealed that the data was fudged, would the fudging really be the most damning fact about the study's validity?

I write books. My latest is a YA science fiction novel called Homeland (it's the sequel to Little Brother). More books: Rapture of the Nerds (a novel, with Charlie Stross); With a Little Help (short stories); and The Great Big Beautiful Tomorrow (novella and nonfic). I speak all over the place and I tweet and tumble, too.

MORE: iq • web theory • woo

More at Boing Boing

Eurovision 2013: An American in London

The technology that links taxonomy and Star Trek

dculberson

IQ is the closest thing we have to a measurement for intelligence. It’s not perfect but there’s nothing better for now. So if anything is going to be used as a meaningful proxy for intelligence, it’s IQ. Equating IQ with star signs is not reasonable as IQ is at least researched and evidence based.
- Alex3917
  
  “IQ is the closest thing we have to a measurement for intelligence.”
  
  No, it isn’t. Intelligence can’t be measured because it isn’t a quantity. Therefore the concept of ‘closest’ does not apply. C.f. the book Measurement In Psychology:
  
  http://www.amazon.com/Measurement-Psychology-Critical-History-Methodological/dp/0521021510/ref=sr_1_13?ie=UTF8&qid=1312386547&sr=8-13
- http://nefariousnewt.blogspot.com NefariousNewt
  
  IQ is bunk. Intelligence cannot be measured by a single number. Instead of measure raw intelligence, why aren’t we measuring overall mental capacity? Raw intellectual power means nothing if it is not harnessed.
  - SWPL_Bro
    
    You’re right. But if you don’t first have a good command of abstract logical reasoning, then there’s nothing to harness. Both are necessary conditions for achievement. Logical reasoning happens to be much easier to measure in the abstract.
    
    Steven Pinker’s definition of intelligence neatly summarizes my point:
    
    “Intelligence, then, is the ability to attain goals in the face of obstacles by means of decisions based on rational (truth-obeying) rules.”
    
    “How the Mind Works” p. 62
- http://twitter.com/enkiv2 John Ohno
  
  There are a host of more ‘legitimate’ intelligence tests. Ask a shrink if there’s ‘nothing better than an IQ test’, and you’ll get a long list.
  
  That said, those other intelligence tests tend to break down intelligence into different subcategories, and thus produce a whole group of numbers rather than a single aggregate. This is quite reasonable, but reduces their popularity as standards for intelligence amongst laymen who don’t know anything about intelligence tests, because it makes understanding the scores more complicated. This is extremely good for researchers and therapists, but not good at all for someone who is rather dim and wants an easy way to rank himself in the primate hierarchy.
  - dculberson
    
    My wife is a “shrink,” and has done extensive testing for research, therapeutic, and disability determination purposes. The WAIS is still the standard for IQ testing in those areas, and it provides the best approximation we have at this time of intelligence. It’s flawed, yes, but it’s hardly comparable to rolling some bones and reading the tea leaves in the bottom of a cup. It’s not even anywhere near as flawed as, say, a polygraph test. Now there’s some institutionalized woo for you!
http://twitter.com/samdesborough Sam Desborough

IQ tests only show us how good people are at taking IQ tests.
- http://www.facebook.com/russingram Russ Ingram
  
  which correlates with how well they do at living life…
- http://twitter.com/leftnotracks Scott Falkner
  
  Could be. But still, I would bet smarter people are better at taking them than dumb people.
  - firefly the great
    
    Could be. But still, I would bet smarter people are better at taking them than dumb people.
    As a general rule, sure. The same could be said about any math test, reading comprehension test, test of general knowledge, quiz on African geography, memorization test, etc. That doesn’t mean that any one of those things should be considered equivalent to “intelligence”
    - atimoshenko
      
      But that’s a problem with the definition of “intelligence”, rather than a problem with the tests. For my part, I always thought it would be more productive to figure out the things we can reproducibly and empirically ascertain and then give those things names, rather than taking some broad, fuzzy things whose definitions people disagree on and then try to measure them.
      
      The again, taking my approach could well make many people unhappy. While definitions of “smart”, “kind”, and “attractive” (etc.) remain fuzzy, we can all consider ourselves to be at least averagely smart, kind, and attractive, with any evidence to the contrary dismissed as merely a disagreement in definitions. Helps us to defend those positive personal indicators which we (rightly or wrongly) believe ourselves to have…
http://twitter.com/iamreddave David Curran

As Alex Tabarrok said “People who say that IQ can’t be measured have no difficulty understanding that lead paint leads to development problems”
- http://nefariousnewt.blogspot.com NefariousNewt
  
  Because development problems can be measured against a baseline which has nothing to do with intelligence. Development works along a spectrum; gross deficits in the match of an individual’s development against a normal curve indicate there may be issues. The study of physical and mental development is not yet an exact science.
  - http://twitter.com/iamreddave David Curran
    
    I do not see how you can put “nothing to do with” together here with “may” and “not an exact science”. You are saying IQ is not perfect and that it is in no way useful whatsoever. Pick one.
http://jpotisch.zeptosoft.com/ Joel P

Everyone who ever says IQ means anything should read “The Mismeasure of Man” by Stephen Jay Gould. IQ tests are very useful at diagnosing developmental delays, mental retardation, etc. Reducing intelligence to a single number (real IQ tests have several axes) is as useful as assigning a creativity number to an artist.
- https://roamingaroundatrandom.wordpress.com/ Natanael L
  
  I did one once, and I clearly remember that it had at least 5-7 numbers on it. The “final IQ score” was some kind of average.
- SWPL_Bro
  
  And everyone who ever says IQ is meaningless should read “The Bell Curve” by Robert Herrnstein and Charles Murray. If you expect someone you disagree with to devote time to reading your favored author, then it’s only fair that you should reciprocate.
http://twitter.com/samari711 samari711

It’s not that IQ tests are useless, but that the higher the score, the less meaningful the score is. They were originally designed as a way to test for developmental disorders.
http://twitter.com/StuartJRitchie Stuart Ritchie

Just tweeted this to Cory Doctorow:

Could you please explain which parts of this Nature Reviews Neuroscience paper on IQ are ‘woo’? Thanks! http://www.larspenke.eu/pdfs/Deary_Penke_Johnson_2010_-_Neuroscience_of_intelligence_review.pdf

It may be a good idea for everyone to read it and get up to speed with the current scientific understanding of intelligence before continuing this discussion.
Meras Hallan

To claim that IQ is a good measure of intelligence is flawed. To claim that a multi-vectored measure of intelligence that was useful cannot be developed is also flawed. IQ is useful for identification of developmental disorders and such.
http://www.facebook.com/john.r.walter John Walter

Ok, so IQ is just a test, a test that asks a variety of questions that require intelligence to solve correctly. Can people point to examples of absolute morons that do really well on IQ tests? or of people that would generally be regarded as brilliant doing really poorly? We can argue all day about semantics, and I’m not suggesting that IQ scores are fantastic, but I still think they provide a reasonable estimate of the intelligence of a person (perhaps given certain assumptions based on how the test was designed (english speaking, etc, etc)
atimoshenko

Why has it now become so popular to deny our ability to measure ourselves? Sure, we must understand the limitations of such measurements and be able to accept new limitations as we discover them, but how do the limitations or incompleteness of these tests render them inherently invalid?

IQ tests may only measure our ability to complete IQ tests, and 100m sprints may only measure our ability to run 100m, but this is replicable, empirical data we can have about ourselves. More empirical data can never hurt.

The same cannot be said of star signs and auras…
- Keisar Betancourt
  
  If i were to say to you (not you: andrei_timoshenko, the generic “you”) that it was impossible to find ways to tell if someone is intelligent or not, you would rightly tell me that my argument is ridiculous and prima-facia wrong. and you’d be right. there ARE ways to measure whether someone is more intelligent than someone else, and some of the people best qualified to determine what those ways are have put together a thing to measure it. it’s called an IQ test, and it measures exactly what it purports to measure, +/- certain cultural influences. IQ is a Real and Effective measure of intelligence. if you don’t think so, you’re probably not very. it also has a high statistical correlation with any number of other measures of mental capacity and real world success.
Jeddy Khan

It appeared in Mashable - claiming that Internet Explorer 6 users were stupid because they did not choose the latest version of Internet Explorer and did not use Chrome and Mozilla. It was like saying people who liked MS Word 2003 more than the newer versions of MS Word are morons. The newer versions of MS Word are terrible, impossible to understand.
http://andreajames.com Andrea James

Thank you for challenging the the validity of both the IQ crap and the Myers-Briggs crap. Both are highly problematic and based on a lot of woo.
Moriarty

Why do discussions about IQ always seem to devolve into the same false dichotomy?

People:

Intelligence cannot be reduced to a single quantifiable dimension.

AND

IQ is a useful metric for more than just “skill at taking IQ tests.”

Just because people insist on misinterpreting the meaning doesn’t make it “meaningless.”
verafides

When you trash things like Myers-Briggs (and IQ), you’re confusing implementation with theory. The fact that people do crappy things with it, or that tests are not easy to self-administer, etc., does not automatically entail that the logic and theoretical apparatus are bad. A theoretical device can be a useful metric for getting at some aspect of human behaviour and cognition without being universally applicable in all contexts ever dreamed up by some random crackpot (on the internet or otherwise). Theoretical constructs are meant to give us new perspectives – new vantage points – to see from and new things to think about. Insofar as it does that, a theory is successful.

Is IQ ‘real’? Is Myers-Briggs ‘real’? Of course not – but it isn’t meant to be. The average Joe has a whole series of serious misconceptions about what Science does. Science doesn’t give ‘real’ in any sense, and was never intended to. Some very confused people seem to think it does, but the assumptions and methodological apparatus of science was never meant for that, and doesn’t deliver those kinds of results. Sorry to burst your belief bubble, I guess.
- Keisar Betancourt
  
  i’ve personally taken any number of both types of test and they’ve usually come out exactly the same, maybe 5% variance at the most, which means they are measuring a uniquely specific thing.
jwepurchase

It helps to understand how empirical tests, like IQ, are developed. You start with a group of smart people and a group of dumb people and give them a set of candidate questions. You then discard all of the questions that either the smart people can’t answer or the dumb people can. When you apply the questions to an ungrouped person, their responses tells you which group they should be in.

The problem with this is obvious; someone had to put people into those original groups. An empirical test gives you a probability that the test developer would have put you in the group. This can be useful, but you have to understand the test developer’s biases to understand the test results. The Black Intelligence Test of Cultural Homogeneity demonstrated how an “intelligence” test can be biased to culture.

Modern IQ tests are developed with an attempt to de-bias, but they’re still flawed.
Tom Bortels

I swear – there needs to be a Godwin’s law variant for “IQ”, where any debate on intelligence ends up devolving to a discussion of the merits, or lack thereof, of IQ testing.

IQ testing is inherently flawed, granted. It doesn’t measure “intelligence”, granted (and, to be sure, it doesn’t claim to, poorly-chosen-name aside – it’s a quotient of test scores). It’s used and misused in places where it shouldn’t be, or where better means of measurement are available – fine.

But – it’s not useless. And it *is* objective, and when applied with an appreciation of it’s limitations, it can be useful. It certainly doesn’t fall into the same categories as “star signs, biorhythms … or auras” (the Meyers-Briggs types analogy is probably more apt). Do not throw the baby out with the bathwater.

The original study is deeply flawed for many reasons apart from their use of IQ as a means of measuring intelligence (including the possibility it’s simply a hoax). It is probably more useful to attack it on it’s other flaws than it is to devolve down to an argument about the validity of IQ testing.

Sadly – I suspect the conclusions of the study are probably correct, even if their methodology is flawed. It would be nice to see someone duplicate the study with real scientific method and controls…
http://paulbeard.org/wordpress paul beard

I interpreted this “study” as showing that there are users who can and do make choices and some who don’t. I’ll accept that there are some who don’t want to or are afraid to tinker with the Sacred Computer Thingie and they are running IE 6 with Windows 2000 or something equally horrible. It’s easy to say that something using such an old system is dumb but maybe they’re afraid to change anything, insecure in their ability to fix if it breaks, or just don’t care. One could assign the blame to designers/engineers who make things hard to change or imply that “there are no user-serviceable parts inside.”
marvmartian

The Bell Curve can be dismissed out-of-hand because it takes seriously the scores accumulated by group IQ testing. Group testing has been shown to have zero repeat validity; in other words, the “IQ” score from a group test is worthless. Compiling these meaningless scores together into a meta-analysis, as the authors do, is nothing more than garbage-in, garbage-out.

That having been said, however, IQ (as tested in a one-on-one setting by an experienced examiner) is our best and only measure of general intelligence. It correlates with almost every other measure of excellence that we have. The disturbing fact is that most of the population scores so poorly on IQ tests.

This is not necessarily disturbing news to some. As John Stuart Mill said, in a letter to the Conservative MP, John Pakington, in March 1866: “I never meant to say that the Conservatives are generally stupid. I
meant to say that stupid people are generally Conservative. I believe
that is so obviously and universally admitted a principle that I hardly
think any gentleman will deny it.”
Antinous / Moderator

If your IQ score is your sole positive personal indicator, it seems only natural to defend it.
Jonathan Badger

The problem is that for many people outside science, things like IQ and the genetic heritability of intelligence have become political footballs. The Left denies them and the Right embraces them, and both mostly from reading biased popular books like ‘Mismeasure of Man’ or ‘The Bell Curve’ (generally praising whichever one that dovetails with their preconceived notions on the topic and damning the other).

While there are current scientific investigations on both topics, they are essentially completely disjoint from the political battle.
http://twitter.com/JayStephens Jay Stephens

I want to second all those basically saying “Don’t make this a false dichotomy – IQ tests are only a proxy for intelligence, but they’re the best proxy we currently have.”

I find it helpful to turn the question around like this:

1. Are IQ tests socially constructed by people with inherent biases? Yes.
2. Is the society we live in partly socially constructed by people with inherent biases? Yes.
3. Do IQ tests objectively correlate very well with life success (not ending up in an asylum, long happy relationships rather than broken homes etc)? Yes.

In other words, a good chunk of the “bias” in IQ tests is the same “bias” that causes people with certain combinations of talent skill and excellence to fail to thrive in today’s world. Should we be fixing the world so those people thrive better and we all benefit from their talents? Certainly yes. Does the fact that IQ tests are more relevant to the real world than to how the world should be make them more or less useful? I’d say more.
kozmer

Really, the most damning part of it is how widely it was reported and re-reported before it was ever investigated. How is that not amazing and appalling?
http://disqus.com/Kimmoth/ Kimmo

+1 WTF at pooh-poohing our ability to reasonably measure intelligence; yeah – cultural bias, blah blah, whatever. Jay Stephens makes a good point that what it takes to do well on an IQ test correlates pretty well to what it takes to do well in life, so the bias is largely moot.

“Oh, my little Jimmy is so clever despite only scoring 70 on an IQ test,” but he has no grasp of maths, sucks at identifying patterns and solving problems, and can’t string words together to save himself. So precisely how is he smart then? One would imagine it’s possible to demonstrate his blinding intelligence in some way, if that’s indeed the case.

So, to get to the actual topic, granting IQ bears a strong relationship to measurable aspects of intelligence, it stands to reason that browser choice would reflect IQ – choice of browser is a problem to be solved, and the quality of possible solutions varies. A possible complicating factor is the degree of concern any given user may feel is warranted regarding the matter, but it also stands to reason that more intelligent people are more likely to notice why it’s worth the effort to make a good choice.

Certainly it seems a fair call to cast aspersions on IE6 users…
Geoff

When IQ tests are done properly and take cultural differences into account, they can be good tools. All they check is your ability to see patterns; similarities and differences between objects, words or numbers. Pop science has unfortunately made them out to be some kind of indicator of worth or future success, but that’s not what they’re for.

I think the fact that people pick on IQ tests is interesting because there are other tests, such as personality tests that are used to diagnose. Aren’t such subjective traits much harder to measure than one’s ability to see a pattern?

Properly administered IQ tests are insulting to half the people that take them, so of course people like to claim poor external validity. If cultural differences are taken into account they can measure ability to see patterns well, and that’s what they’re for. They might want to rename the IQ test, ‘test of logical reasoning’, or something like that, and use a measure that’s less obviously above or below an average to help save face.
Baldhead

IE 6 and MS Word aren’t in the same ballpark. MS Word at least works, IE 6 was always terrible at it’s designed purpose. People who continue to use broken things when better products are available at no charge are, indeed, stupid. It’s like insisting on using a Chevy Sprint with a wonky steering wheel when offered a new Fiat 500 (Chrome) or at very least a Sprint without a wonky steering wheel (IE 8)