According to a National Science Foundation's biennial report (2002), 30% of adult Americans believe that UFO's are the space vehicles of aliens from other planets; 32% believe in lucky numbers; 40% believe that astrology is an established science; 60% believe in extrasensory perception; 70% believe that magnetic therapy is scientific; and 88% accept the validity of alternative medicine.

   However, lack of education cannot be blamed for this regrettable state of affairs since those having more advanced education showed no greater ability to discriminate between confirmable knowledge and that without evidential support. For example, 60% of the general adult population believe in extrasensory perception compared to 65% of high school graduates, while those with a belief in alternative medicine ranked 88% for the general population, 89% for high school graduates, and 92% for college graduates.

   In an attempt to comprehend the kind of logic used by the general public in forming their opinions, a survey was made on the understanding of the scientific method defined in terms of comprehension of probability, experimental methodology, and hypothesis testing. From this survey it was determined that less than 30% of the general population have any real comprehension of the scientific method and only 53% of those with a reasonably high level of science education (9 or more high school and college science/math courses) passed this comprehension test.

   So what is this scientific method? Ideally, it can be subdivided into consecutive phases. The first is to clearly define the nature of the problem to be investigated. Then follows the collecting of available data that bears upon the problem. Next comes the task of formulating various hypotheses that could offer a solution to the problem. The simplest of these hypotheses is then put to the test with experiments designed to verify its truth. If the hypothesis fails the test, it is either modified and re-tested or some other hypothesis is tested. If and when the experimentalists are satisfied they have a solution, they may publish it in an appropriate journal, preferably one having a panel of expert referees. By this means it is exposed to the criticism of the science community. Final confirmation of a theory only comes when the work is repeated and found correct by independent laboratories.

   Probability enters the picture in several ways. Many trials are designed in such a way that the results can be analyzed by rigorous statistical procedures. But often we may be confronted with alternative theories that also provide a reasonable fit to the available evidence.

   An example was Newton's law of gravity and Einstein's general relativity. For a number of years there was no experimental evidence that could be used to decide between them. For many people, it remained true that Newton's laws were both simpler and sufficiently accurate for all practical purposes. But for others, the sheer beauty of general relativity won their vote as being closer to the truth.

   However, with the passage of time, relativity was able to make many quite astounding even unbelievable predictions such as that gravity slows down clocks or the faster you can propel a javelin the shorter it gets--and these principles could be tested experimentally. One of these tests showed that the rates at which two neutron stars spiral towards one another is given by Einstein's theory with an extraordinary accuracy of better than one part in 10¹⁴ (or one hundred million million). Today, Einstein's general relativity has passed all its testing and is accepted as the most accurate theory known to man. But despite that accuracy, it remains incompatible with quantum physics. One or the other will eventually need to be modified.

   In summary, the key components of the scientific method are that its concepts are supported by valid, repeatable experimental testing of all the evidence directly pertaining to the theory. No theory can be accepted as being scientifically 'proven' unless it passes this test of validation by properly designed trials that are reproducible by independent workers.

   That word 'proven' does require qualification. For properly educated scientists, mathematicians, and logicians, it is accepted that nothing can be absolutely proven and that all theories need a subjective probability estimate of their truth assigned to them, expressible either as a percentage figure or as ranging between 0 and 1.
   There are many human problems for which the scientific method is inapplicable. As a help in deciding applicability, the concept of the Black Box may be useful. This arose in world war 1 when a method was required to help solve the problem of how secret enemy equipment actually worked when it was also known that any attempt to open it and peer inside might trigger an explosion.

   The only way to discover how such a device might work is to find out what input signals it could accept then to examine how varying these signals changed the output of the device. From the data so accumulated, experts then attempt to provide a mechanism that would explain how the device actually operated.

   The answer to this problem is important. It is that for all such devices for which a simplest operating mechanism can be devised, there is an infinity of more complex mechanisms that would  give the same results. Thus there is no single, unique solution to these Black Box problems.