Evaluating Lineups You Or Others Have Created
The basis for evaluating the fairness of lineups is outlined briefly in Rule 3 of the APLS White Paper:
“The suspect should not stand out in the lineup or photospread as being different from the distractors based on the eyewitness’s previous description of the culprit or based on other factors that would draw extra attention to the suspect.“ (page 24)
This concept was reinforced in the report of the Technical Working Group on Eyewitness Evidence, in the National Institute of Justice:
“Select fillers who generally fit the witness’ description of the perpetrator. When there is a limited/inadequate description of the perpetrator provided by the witness, or when the description of the perpetrator differs significantly from the appearance of the suspect, fillers should resemble the suspect in significant features.”(page 29)
Wells et al. (1998) continue:
“[The] extent to which Rule 3 has been met in a given lineup can be tested using a "mock witness" procedure. …. Mock witnesses are people who have never seen the culprit but are given the eyewitness’s verbal description of the culprit, shown a picture of the lineup or photospread, and asked to select the person they think is the suspect in the case. If Rule 3 has been sufficiently met, a mock witness should not be able to select the suspect at a level that exceeds chance expectations based on the number of choices (number of lineup members) that could have been selected. If mock witnesses can deduce who the suspect is under these circumstances, then a concern is raised about whether an eyewitness’s selection was a product of true recognition memory or was due merely to the same deduction process that the mock witnesses apparently used.”
Since the foundational paper by Doob and Kirshenbaum (1973) disinterested persons have been used as participants in research studies to evaluate eyewitness lineups. The process is described in some detail by Malpass, Tredoux & McQuiston-Surrett (2007). Here is a thumbnail description of the process and an example of some ways to represent the results to make them more amenable to interpretation. The basic form of the evaluation is provided for each of the lineup evaluations displayed on this website, but for pedagogic purposes below we provide an extended example from a recent case. It will become apparent that the consequences of bad fillers are failure of the safeguard and inflation of the innocent suspects risk of false identification.
Thumbnail description of the mock witness procedure.
1. Select your population of participants. It is best to choose participants of the same ethnicity as the original witness(s).
2. Provide the information you have chosen to give the participants.
The choice of information given to participants is important. Doob and Kirshenbaum (1973) gave them the description given by the witness – that the offender was “very good looking”. The most gross departures from fairness will probably be detected without providing any information, simply saying that the lineup contains a police suspect in a case involving the particular offense in question, and asking participants to choose the lineup member who is the police suspect. For a more sensitive evaluation participants can be given the verbal description provided by the witness. Some aspects of appearance are difficult to capture in facial feature descriptions but easy to denote in other language - like “very good looking”. That information also can be included in a verbal description. The single witness in a recent case described the offender as “he looks like a waiter”. Giving lineup evaluation participants only that information allowed them to choose the suspect at an above chance rate, and lineup fillers were chosen substantially below the expected rate.
3. Give whatever information you will provide individually to at least 30 (and preferably 50) participants. Don’t display the lineup and the information you provide at the same time: if you do, the identification process may simply be a feature by feature comparison among faces. Then show participants the lineup and ask them to indicate which lineup member is the suspect. Since this is an informal procedure, simply record their response indicating which lineup member was chosen as one of the numbers 1-6 in the lineup, upper left to lower right.
4. Summarize the findings in the manner shown in the Lineup Evaluation Spreadsheet, which you may download here. What follows refers to the spreadsheet.
There were three kinds of information given the participants in this case, and combined for analysis. The resulting identification frequencies are these, for lineup positions 1-6 (left to right).
2 |
13 |
1 |
14 |
9 |
3 |
The suspect was in position 4 and received the largest number of identifications (14 of 42 = 33%), twice the number expected if only chance factors are influencing paticipant choices. This identification proportion can be submitted to a statistical test for bias by use of a spreadsheet downloadable from the Eyewitness Identification Research Laboratory website. For guidance in its use and interpretation consult someone in your organization familiar with basic statistics, or call the eyewitness laboratory at 915-539-0510. In this example, the probability of an identification proportion of .33 occurring by chance, rather than some systematic process (bias), is very low (.01232), which is statistically reliable. Some systematic process or influence is very likely to be at work. The results can be presented in graphic form, as shown below. There is one "super filler", in position 2, which is identified at a rate nearly equal to the suspect, which is far above the choice frequency expected by chance.
It is obvious that three of the fillers are chosen well below the rate expected if only chance factors are involved. The number of fillers which fulfill their role as safeguards to an innocent suspect can be tested quantitatively through Tredoux' E'. A spreadsheet containing the calculation can be downloaded from the Eyewitness Identification Research Laboratory website. But evaluation of the fillers can most easily be understood by examining them in the order of their identification frequency, (L – R) as shown below.
14 |
13 |
9 |
3 |
2 |
1 |
Note that the 3 least frequently chosen fillers total 6 choices, which taken together do not add up to the identification frequency (7) expected of 1 fully appropriate filler. That's nearly one filler but since it takes three to make it up, there are two whole fillers effectively missing. Count the number of persons in the lineup. We have the 3 most frequently chosen plus about 85% of another. Rather than the 6 person lineup that should have been used, we instead have a lineup containing about 3.85 persons. You may think that this is not particularly serious, but consider this: the risk of a false identification has gone from 1 in 6 (16.7%) to 1 in 3.85 (25.9%). There is no one I know who would prefer to bet their freedom on a 26% risk rather than a 17% risk.
This illustrates two important points for the analysis of lineup fairness:
1. Lineup bias is often produced by inadequate fillers.
2. The effectiveness of the lineup as a safeguard against false identification focuses sharply on the effectiveness of the fillers and the setting of risk of false identification that results.
References
Devlin, Hon Lord Patrick (1976). Report to the Secretary of State for the Home Department of the Departmental Committee on Evidence of Identification in Criminal Cases. HMSO.
Doob, A. N. & Kirshenbaum, H. M. (1973). Bias in police lineups — partial remembering. Journal of Police Science and Administration, 18, 287-293.
Malpass, R. S, Tredoux, C. G. & McQuiston-Surrett, D. E. (2007). Lineup construction and lineup fairness. in R. Lindsay, D. Ross, J. D. Read & M. P. Toglia (Eds.), Handbook of Eyewitness Psychology (Vol. 2): Memory for People. Lawrence Erlbaum & Associates.
Scheck, B., Neufeld, P., & Dwyer, J. (2001). Actual Innocence: When Justice Goes Wrong And How To Make It Right. New York: Doubleday.
Technical Working Group on Eyewitness Evidence (1999). Eyewitness Evidence: A Guide for Law Enforcement. Washington, D.C.: National Institute of Justice (i-x, 1-44).
Wells, G.L., Small, M., Penrod, S., Malpass, R.S., Fulero, S.M., & Brimacombe, C.A.E. (1998). Eyewitness identification procedures: Recommendations for lineups and photospreads. Law and Human Behavior, 23(6) 603-647.