Simple Interactive Statistical Analysis
Input.
In the top two boxes proportions should be given; decimal values between zero and one. The 'lower' proportion should be smaller then the 'higher' proportion. Following the alpha and the beta level are given, also proportions. Do not change the default values (.05 and .80) unless you have a good reason. In the bottom box you have to give an integer number. This number is the number of experiments or cases you want to consider. Give a relatively high number.
Alternatively, if you give a value larger than 'one' in one of the first four boxes the programme assumes that you meant to give a true percentage. The programme divides the value by 'one hundred' before processing.
Explanation.
Wald's Sequential Probability Ratio Test (SPRT) is currently the only Bayesian Statistical Procedure in SISA (although one might argue that 'sample size' is also Bayesian). What is required in Bayesian statistics is quite a detailed description of the expectations of the outcome under the model prior to executing the data collection. In Wald's SPRT, if certain conditions are met during the data collection decisions are taken with regard to continuing the data collection and the interpretation of the gathered data.
Wald's procedure is particularly relevant if the data is collected sequentially. For example, in the case of following failures on a production line or throughput in a hospital or relapse in behavioral interventions. Sequential Analysis is different from Classical Hypothesis Testing were the number of cases tested or collected is fixed at the beginning of the experiment. In Classical Hypothesis Testing the data collection is executed without analysis and consideration of the data. After all data is collected the analysis is done and conclusions are drawn. However, in Sequential Analysis every case is analysed directly after being collected, the data collected upto that moment is then compared with certain threshold values, incorporating the new information obtained from the freshly collected case. This approach allows one to draw conclusions during the data collection, and a final conclusion can possibly be reached at a much earlier stage as is the case in Classical Hypothesis Testing. The advantages of Sequential Analysis are easy to see. As data collection can be terminated after fewer cases and decisions taken earlier, the savings in terms of human life and misery, and financial savings, might be considerable.
To make this plan work in the program implemented in SISA it is required that one stipulates both the level above which one would describe the situation as 'more' (whatever) and the level below one would describe the situation as 'less'. One has to define these values prior to the data collection. In the program implemented here this is only possible for percentages. Thus, one has to give a percentage for the number positive on the total number above one would declare the situation as improved (or more), and a percentage below one declares the situation as deteriorated (or less). The program echoes for each additional case collected the number positive on the total number which defines the threshold value, for the sample which is collected upto that moment. If a higher number positive is found in the data the data collection should be terminated and the conclusion should be drawn that the study shows divergence between the hypothesis and the collected data. Similarly for the lower value, for each additional case a number is given on the total number collected upto that moment, if values below this are found the data collection should be stopped because the situation has deteriorated (or gotten less).
Two additional values are required, statistical power and alpha level. Power is the chance that, if a difference exists in the real world, one also observes a difference in the data. In the program the powerlevel is set at 80%. There is an 80% chance to discover a really existing difference in the sample. Alpha is the chance that one would terminate sampling because one would think one has discovered a difference, while in fact this difference does not exist. Alpha is set at 5%, which means that 5%, or one in twenty, of the experiments would have been incorrectly stopped because the data signalling that 'something' existed, while in fact it did not. Powerlevel and alpha level can be changed by filling in different values. However, the values pre-given are the widely accepted 'usual' values. Only change these values if you have a good reason for it.
It is amazing that sequential analysis is not more often used, and that the methodology has not much more developed since the early fifties. The main reason is probably that experimenters love to, once data is collected, try out various models, look at the data in various way, keep constant for certain factors, but not for other factors, and in this way give the best possible representation of the theory under study in the light of the available data. Sequential analysis does not allow for this and is in fact quite inflexible, as it requires a well defined and strictly executed design.
Equally amazing is that sequential analysis is not more often used in audit and quality control. One can use the methodology to set aims and targets for the future and define threshold values at which quality control activities are triggered. In the case of relatively rare events, deaths in a hospital, serious failures on the production line, sequential analysis can be a powerful management tool.
Output.
The output starts with a number of descriptive statistics. Ignore these unless you are interested in the topic and have read the papers below. In the first column you will find the number of cases collected. In the second column you will find the number of cases which define the lower threshold value, if the number of cases with a particular characteristic in the sample is below this number, the data collection has to be stopped because the sample has become statistically significantly different from the lowest value pre-specified for the model in the 'lower proportion' box. Initially this number in the second column will be negative, which means that decisions cannot yet be taken. Data collection has to continue. The percentage box, in the third column therefore shows the sign 'cont', continue collecting data. After a number of experiments you will see that the value in the second column becomes above zero, a small percentage appears in the percentage column. If the number in the sample is zero, or less than the small percentage, data collection has to be terminated, the number found is too small compared with expectations and a definite conclusion to this effect can be drawn. Going further down the output the number in the second value takes on a value above one, zero or one number of the characteristic in the sample denotes statistical significance. As one looks further and further down the output higher numbers appear and similar strategies should be applied.
Fourth and fifth columns do the same, but now you look in your sample for values higher then the number given in the fourth column or the percentage in the fifth column. If you find such a value the sampling must be stopped because there is a statistically significant difference between the data and the percentage you gave in the 'higher proportion' box.
Further Reading.
Bellissant E, Benichou J, Chastang CL. A Microcomputer Program for the Design and Analysis of Phase 11 Cancer Clinical Trials with Two Group Sequential Methods, the Sequential Probability Ratio Test, and the Triangular Test. Computers and Biomedical Research 1994; 27:13-26.
Difford F. Performance Review Using Sequential Sampling and a Practice Computer. Family Practice 1988;5(2):136-141.
Howe HL. Increasing Efficiency in Evaluation Research: The Use of Sequential Analysis. American Journal of Public Health 1982;72(7):690-697.
Whitehead J. The Use of the Sequential Probability Ratio Test for Monitoring the Percentage Germination of Accessions in Seed Banks. Biometrics 1981;37:129-136.
