Every so often, a client will ask us about usage of hand-held feedback devices, often called Perception Analyzers or “moment-to-moment” data collection systems, for utilization in jury research projects. Being trained researchers at heart, we have conducted extensive research on these devices and our continued analysis and investigation of these devices has consistently borne out that they are, essentially, “eye candy.” While their usage does apparently afford entertainment value, such usage also has real implications for the most important issue that we, as a true jury research firm, are ultimately concerned with – namely, validity.

In research parlance, the term “validity” has a specific meaning that is well-understood and appreciated by a trained researcher. It refers to the extent to which research findings can actually be generalized to real-world situations. In jury research, then, validity refers to the extent to which the observed research results can be counted on to provide an accurate representation of what will eventually happen in the courtroom or in the deliberation room. Our investigations have led us to the clear conclusion that the use of real-time perception analyzers is an impediment to, detracts from, or undermines validity in jury research. The following is a list of the reasons for why this is true:

1.  Research has established, as published in the National Law Journal, that jurors do not deliberate based on real time courtroom events – they deliberate based on what they have stored and retained in their memories. More importantly, what they have stored and retained in their memories is only a small portion or subset of what they have been given in terms of evidence or visual/auditory stimuli over the course of an extended period of time. There are systematic and lawful reasons governing which stimuli become stored and retained by jurors and which do not, but the important point is that the proper focus of jury research is what jurors retain over time – not what they react to during repeated cross-sections of time, as the Perception Analyzers measure. It is attention to what jurors “keep at the end” that allows valid conclusions as to how they problem solve cases, not attention to their moment-by-moment reactions as evidence is presented, because – remember the issue of validity – what they “keep at the end” is what determines their verdict in the deliberation room.

2. Our focus on validity means that we want to simulate the actual conditions of the courtroom environment as closely and accurately as possible. This is precisely why Courtroom Sciences, Inc. exerts more effort to re-create the courtroom environment for in-person mock trials than any other jury research firm. And for virtual jury research, we take great pains to ensure the virtual experience aligns with the expected venue protocols. The environment does affect behavior. In the courtroom environment, jurors are not asked to enter electronic data points into a hand-held device during attorney presentations every 30 or 60 seconds – they are asked to sit, watch, and listen. We do not want jurors fiddling with an electronic device in their laps during the attorney presentations – we want them looking at the presenter and paying attention. We want jurors to do what real jurors do, every step of the way. After all, we need the most accurate replication of a trial possible to confidently report the results to our clients, many of whom are insurance officers who rely on our damages estimates for establishing their reserves.

3.  As an attorney requesting jury research, you have at your disposal the availability of more potential themes, issues, and arguments --and ideas emanating from them -- in your pending litigation than you can use. One of the most valuable services that a jury research firm can provide, therefore, is information reduction, not information augmentation. Of the dozens and dozens of potential ideas and perceptions that could potentially be viable or dispositive in your case theory, which subset of those are the ones that “make the cut”? Where are the hidden gems? The Perception Analyzer piles on hundreds and hundreds of measurements with no way to know which ones are driving the verdict. This does not help you win your case.

4.  Perception Analyzers are expensive and drive costs up – costs that could be much better utilized in a deeper analysis of the data that you have through other measurement technologies that are more closely connected with the deliberation process and outcomes.

Of course, we realize that to some extent jury research exercises are indeed part entertainment, because they are truly interesting and provide unexpected results. However, with millions of dollars (or even more) at stake, we do not feel it is prudent to become distracted from the overriding goal, namely, finding out: How are jurors making their verdict decisions? And what can be done to optimize the tactical position of your client? We do not want to lose sight of the ultimate goal with “eye candy.”