Beyond all the diehard UFO believers, conspiracy kooks, and capital "S" skeptics, there is an invisible college of scientists who do see the value in looking at the UFO phenomenon through a scientific lens. No matter whether the UFOs are classified government aircraft, extraterrestrials from other worlds, inter-dimensional explorers, swamp gas, or figments of our collective imagination, objectively analyzing the data we do have about the phenomenon is incredibly challenging. After reading the Pentagon's long-anticipated UFO report—"Preliminary Assessment: Unidentified Aerial Phenomena (UAPs)"—which really said next to nothing, Columbia University's director of astrobiology Caleb A. Scharf identified five of the "greatest hurdles" to scientific analysis of unidentified aerial phenomena. Here are the high-level challenges which he unpacks in a fascinating essay in Scientific American:
Challenge No. 1: All UAP/UFO incidents are nonrepeatable: we can't go back and perform the "experiment" of that exact observation again.
Challenge No. 2: There is nothing systematic in how incidents are recorded or reported. Different camera systems, radar systems, data processing, observers and environmental circumstances mean that each incident is, in effect, an uncontrolled experiment, with few ways to ascertain the real quality and sensitivity of data.
Challenge No. 3: There is no easy way to account for "cherry-picking" of data. We don't know how often pilots or other observers see something unexpected but then, a minute later, figure out what they're witnessing (or at least convince themselves they've done so) and consequently don't report anything. There could be thousands of such incidents, or very few. We don't know, and those "mundane" cases could actually represent all cases.
Challenge No. 4: If any incidents or observations are genuinely associated with something tangible and physical, we don't know whether we're looking at a single underlying phenomenon or many. It's a bit like going into a zoo blindfolded and trying to understand what you're hearing and smelling. If there's only one species you might figure it out, but if there are 100 species, then decoding your experience is going to be very difficult.
Challenge No. 5: The popular association of UAP with hypotheses involving alien technology creates a severe analysis bias. Usually, science tries to move stepwise towards finding support for a given hypothesis or for eliminating hypotheses, and weighs those options as evenly as possible. But in this case a hypothesis that would require extraordinarily robust evidence in order to be supported (as with Carl Sagan's famous dictum "Extraordinary claims require extraordinary evidence"), regardless of what some people say, hangs heavily over any analysis or discussion, and there is a vocal community who feel that the answer is already known. That's a problem.
"O UFOs, Where Art Thou?" by Caleb A. Scharf (Scientific American)
