Clinical significance is not the same as statistical significance

A great example of why details and context always, always matter, from the surgeon/blogger at The Skeptical Scalpel:

Twelve patients who served as their own controls wore compression stockings for a week and then no stockings for a week alternating. The stockings lowered the amount of fluid in the neck by 60%, a statistically significant difference. So far, so good.

This resulted in another highly statistically significant finding, which was a 36% reduction in episodes of apnea [cessation of breathing] and hypopnea [inadequate breathing]. Sounds good, right? The problem is that the average number of episodes of apnea/hypopnea decreased from 48 per hour to 31 per hour. Patients experiencing more than 30 episodes of apnea/hypopnea per hour are classified as having severe obstructive sleep apnea. This means that the treatment only put the patients in the low range of severe obstructive sleep apnea. They still would require maximum therapy.

Via Ivan Oransky

Read the rest