PART 15. Qualitative criteria: Introduction
Although quite a few lists of “best cases” have been presented by various ufologists and groups, surprisingly few of the relevant ufologists have explained (on a case-by-case basis) why they selected the relevant cases.
Indeed, few of those lists are accompanied by even the briefest of explanations of the criteria (if any) upon which selections were made.
However, several UFO groups and researchres have provided _qualitative_ lists of factors which they have suggested should be present. For example, the most common factors mentioned are:
(1) Multiple witnesses.
(2) Profession of the witnesses (e.g. favouring
scientists/astronomers/pilots).
(3) Incompatibility of description with mundane causes of sightings
(Hynek's strangeness factor).
(4) One or more forms of physical evidence.
(5) The depth of investigation of the sighting.
Less commonly mentioned are a number of factors which I personally find quite useful, including:
(1) the angular size of the object.
(2) The duration of the sighting (the longer the better).
Some of the qualitative criteria that have been put forward by various researchers are considered in the following parts of this article:
PART 16 : Qualitative criteria: Credible witnesses
PART 17: Qualitative criteria: Multiple witnesses
PART 18: Qualitative criteria: Miscellaneous other criteria
It is possible that the general absence of any explanation is due to fuzzy thinking about any relevant selection criteria. In characteristically strident terms, Brad Sparks has commented that : “It is typical of the UFO community to not have its act together,unable to cite the best scientific evidence for UFO's because it does not have the foggiest notion of what ‘best scientific evidence’ looks like or what it means. Typically a UFO enthusiast thinks the ‘best’ cases are simply his favorite pet cases, selected out of emotional sentimental attachment, without any rational scientific basis” [see Footnote 15.01]. He suggested that “if [scientists] were presented a list of scientific criteria for best UFO cases they could assent to them and grasp the strength of the case made based on those criteria.”
However, the general absence of any express discussion of relevant criteria may also possibly be explained by a presumption that the basic criteria for selecting the best cases are obvious and well-known within the UFO community.
Any such presumption may be justified. Two of the major UFO organisations of the time, NICAP and APRO, managed to put their considerable differences aside in 1974 to discuss “establishing guidelines and criteria for evaluating the merits and strengths of reports”. According to a NICAP article about that meeting on 9th February 1974 [see Footnote 15.02], “quick agreement” was reached between APRO's director (Jim Lorenzen) and NICAP's President (John Acuff), as listed below:
“1. Close encounter with a possible structured object is preferable to a mere light in the sky.
2. Physical residue left behind.
3. Photographs of object or some kind of instrumental support. (J.C. i.e. radar, etc.)
4. Behavior of the object suggests it was something other than conventional object”
“Witnesses:
1. Multiple witnesses make the best sightings.
2. Should have a background which indicates reliability.
3. Competency of observation.
4. Conditions for observation.”
The website of one of the most respected UFO organisations of more recent decades, FUFOR, includes a webpage which includes a list of examples of the allegedly “hundreds of ‘classic’ cases, well-documented reports that have stood the test of time and thorough investigation”. FUFOR further contends that these reports generally include many or most of the following features:
(1) multiple witnesses,
(2) a physical evidence component,
(3) typical UFO configuration,
(4) extraordinary performance,
(5) no apparent conventional or mundane explanation, and
(6) documentation of witness testimony and supporting evidence.
A small number of individual ufologists have also given some explanation of the factors which they would take into account when selecting the “best cases”. Relevant useful comments have been made by various researchers, including Ronald Story, Richard Hall, Brad Sparks and Paul Kimball.
In his interesting book “UFOs and the Limits of Science” (1981), Ronald Story set out a list of ten “best” cases. Story stated that “If anyone were to ‘debunk’ or explain in prosaic terms – to my satisfaction – all ten of these cases, I personally, would no longer regard the UFO phenomenon as worthy of serious study, except in the realm of the behavioral sciences” [see Footnote 15.03]. That list is considered in PART 6: Consensus lists : Ronald Story’s poll (1979). In the discussion of that list [see Footnote 15.04], Ronalid Story referred to criteria for selecting the best cases and stated: “My own criteria … are these:
- All are multiple-witness cases; none requires that we take the word of any single individual.
- The caliber of the witness involved seems to be well above average.
- Each case has better-than-average documentation, compared with most other UFO reports.
- The descriptions given of alleged anomalies, are, in each case, clearly incompatible with any known phenomenon – i.e. other than UFOs!”
Paul Kimball’s selection of the top 19 cases which he put forward as part of his online poll during March-April 2006 (see PART 7: Consensus lists : Paul Kimball’s Vox Populi poll (2006)), was “limited by the following minimum criteria”:
“i. Must have multiple witnesses (i.e., corroboration) – this could take the form of a single person who has a sighting that is corroborated by radar, for example, or by communication from an aircraft to ground control;
ii. Anonymous witness testimony is not acceptable;
iii. The objects must have been observed at some point in the air (they are “unidentified flying objects” after all).”
Richard Hall has commented [see Footnote 15.05] that “One defines ‘the strongest cases...’ based on standards of evidence, depth of investigation, and recurring patterns or features” and suggested [see Footnote 15.06] that “The following are among the more important elements that should be factored into the selection process (not that all the elements should necessarily be present in every case):
(1) Multiple independent witnesses; elementary (at least) character information about them, and evidence of their observing skills.
(2) Proximity information based on angular size, environmental effects, and/or observed passage in front of known landmarks.
(3) Geometrical shape and/or structure visible; body lights; protrusions; "portholes"; light beams; etc.
(4) One or more forms of physical evidence (radar, ground traces, E-M effects, heat...)
(5) Onsite investigation and reconstruction; thorough witness interrogations; thorough investigation of any possible mundane explanations”.
In a discussion I had with Richard Hall in 2006 [see Footnote 15.07], he suggested to me that “unless you first establish some criteria for selection” all that is obtained by seeking nominations of the best cases “is peoples’ subjectively favorite cases. Something about the cases resonates with them.” He suggested the following:
“Someone should try a questionnaire asking for selection of ten (or 20) of the best cases that:
(a) have been thoroughly investigated and trivial causes eliminated,
(b) emanate from credible witnesses of good reputation,
(c) involve more than one channel of data [see below],
(d) describe an object or phenomenon that performs or maneuvers in unconventional ways and is observed for more than a few seconds, and
(e) is typical in the sense of demonstrating well-established patterns of appearance and performance. That is, the cases individually and collectively should tend to demonstrate that UFOs are not simply natural phenomena or conventional things that are being misperceived.”
“By more than one channel of data I mean one or more of the following: at a minimum, more than one witness preferably in separate locations; use of binoculars or telescopes for better observation; instrumentation of some kind; photographic images; physical trace evidence; physiological effects; radar (and visual) data; or similar afteereffects that lend themselves to scientific study. I have called the ones that leave some evidence behind UFO fingerprint cases.”
“Now the respondent is forced to give some thought to why he or she is selecting a case for the list, to justify the selection in some way other than totally subjective preferences.”
Somewhat more specific and limiting criteria have been put forward by Brad Sparks [see Footnote 15.08]. He has suggested that: “A rational scientific set of criteria will _prefer_”:
[a] “scientists and engineers as witnesses (_prefer_ not exclude all others)”,
[b] “sightings lasting 1 minute or more”,
[c] “multiple witnesses”,
[d] “angular size Full Moon or greater”,
[e] “and of course have all the damned date-time-location-azimuth-elevation data, too.”
Brad Sparks also commented [see Footnote 15.09] that “For pure visual sightings without photographs or instrumentation such as radar or magnetometers, I use an initial screening criterion that requires sighting report data in a case from both of at least two witnesses observing a UFO for at least 1 (one) full minute in which the object is at least a Full Moon angular diameter (with no gaps or "connecting the dots" to try to make it a Full Moon or 0.5 deg in length, it has to be a "diameter" all way around at minimum).”
One factor which is mentioned in relatively few of the lists of relevant qualitative criteria (e.g. the first of the above lists suggested by Richard Hall), is the quality of any investigation done. As Chris Rutkowski has stressed, "For a case to make the list of the best cases, it's not just the unusual quality of the report but also how well investigated it was and how reliable the source was” [see Footnote 15.10].
Seeking to identify relevant criteria gives rise to various interesting questions, explored in subsequent Parts of this article. Those questions include the relative importance of the various criteria and whether (and, if so, how) it is possible to assign any meaningful quantitative overall value or values to a UFO report.
FOOTNOTES
[15.01] Brad Sparks, UFO UpDates discussion List, December 2004.
http://ufoupdateslist.com/2004/dec/m08-004.shtml
[15.02] The UFO Investigator (NICAP Journal) . Feb. 1974 . pp 3 /4.
Text available online at:
http://www.cohenufo.org/enquir.ufogrps.html
[15.03] Ronald Story in his “UFOs and the Limits of Science” (1981) at page 22 (in the Introduction) of the NEL Hardback edition, at pages 24-25 of the revised Quill softcover edition published under the title “Sightings”.
[15.04] Ronald Story in his “UFOs and the Limits of Science” (1981) at page 22 (in the Introduction) of the NEL Hardback edition, at page 25 of the revised Quill softcover edition published under the title “Sightings”.
[15.05] Richard Hall, UFO UpDates discussion List, March 2001.
http://ufoupdateslist.com/2001/mar/m17-028.shtml
[15.06] Richard Hall, UFO UpDates discussion List, March 2001.
http://ufoupdateslist.com/2001/mar/m18-002.shtml
[15.06] Richard Hall, Current Encounters discussion List, 19 April 2006.
[15.08] Brad Sparks, UFO UpDates discussion List, December 2004.
http://ufoupdateslist.com/2004/dec/m08-004.shtml
[15.09] Brad Sparks, UFO UpDates discussion List, April 1999.
http://ufoupdateslist.com/1999/apr/m04-008.shtml
[15.10] Chris Rutkowski, UFO UpDates discussion List, February 2002.
http://ufoupdateslist.com/2002/feb/m27-001.shtml