E-Voting News and Analysis, from the Experts

Sunday October 31, 2004

Human-factors issues in Houston and elsewhere

Filed under: — Wallach @ 6:00 pm PST

I’ve heard variants of this story cropping up around Texas on systems using Hart Intercivic’s eSlate voting system. The general problem goes something like this: a voter selects “straight ticket Democratic Party” and then presses the “Cast Ballot” button. The machine then presents a summary of the voter’s choices, and, to the voter’s horror, they find “Bush/Cheney” selected rather than “Kerry/Edwards". I’ve heard variants on this story elsewhere on different voting machines, but I think the eSlate story is easier to analyze.

What went wrong? Maybe we’re talking about:

  • Human-factors error: the voter pressed the wrong sequence of buttons, perhaps a result of a confusing interface.
  • Software error: a bug in the machine software caused the flip.
  • Hardware error: something physically in the machine broke or misbehaved.
  • Tampering / fraud: a deliberately fradulent modification to the machine caused the flip.

The easiest cases to dismiss are hardware errors (the machines otherwise appear to be working fine) and tampering (if you were smart enough to tamper with the software to change the results, you wouldn’t have the summary screen show the other candidate). This leaves software bugs and human-factors issues. My gut tells me we’re looking at a human-factors problem.

On this particular machine, when you select a “straight ticket” vote, it causes all races that have a candidate for the selected party to be highlighted with a red dot. Other candidates are dimmed into a grey color with lower contrast than you have before you make a selection. How could a voter get confused? The “straight ticket” selection appears as the top “race” on the ballot. You turn the “select” wheel to scroll to the party you want and press the big “enter” button. After this, the machine helpfully advances the current selection to the next race, the presidential race. If a voter were to think that the “enter” button had not done anything yet (perhaps because there is a noticable delay while it is performing the selection) and were to then press the button again, they might indeed end up selecting the top candidate on the list of presidential candidates, who just happens to be George W. Bush.

Is this the only plausible explanation for the behavior some are experiencing? Far from it. Luckily, if this is the problem, then the summary screen is the solution. It requires the voter to make sure that these selections are indeed the ones the voter wanted. And, indeed, some voters caught their mistakes and fixed them. In that sense, the system worked precisely as it was designed.

How can the vendor engineer their system to avoid the problem next time around? For starters, they could reduce the latency after selecting a straight ticket with some performance tuning of their software. They could also use a “Just a minute…” popup window of some kind to make it clear that the machine got the command and is working on processing it. They could even consider ignoring button presses that occured while the machine was busy. However, whatever they try, they need to evaluate it. It’s called the scientific method. You get a bunch of test subject voters, with demographics representing people in the real world rather than just college students, and you can experimentally compare the “control” group with the old system to the “experimental” group with the modified system. If the control group is more accurate, then you go back to the drawing board.

This kind of testing would be of fantastic value for every electronic or paper-based voting system. For example, human-factors testing would have caught the “butterfly ballot” issues in Florida in the 2000 presidential election. Such testing could be conducted by the vendor, on new software releases, and by the customers (counties, states, or other municipalities) for each new election. I suspect that human-factors testing will become a growing and important engineering step for any voting system, regardless of its technology.

(edit: I got the names of some of the buttons wrong – it’s corrected now.)

12 Comments »

The URI to TrackBack this entry is: http://www.evoting-experts.com/wp-login.php/wp-admin/wp-trackback.php/wp-trackback.php/wp-trackback.php/wp-trackback.php/wp-trackback.php/wp-trackback.php/7

  1. New E-voting Blog
    Computer Scientists David Dill, Ed Felten, Joe Hall, Avi Rubin, Barbara Simons, Adam Stubblefield, and Dan Wallach have joined forces at evoting-experts.com to post news and commentary on e-voting issues (just in time for election day). The site has on…

    Trackback by Computing Research Policy Blog — Sunday October 31, 2004 @ 9:57 pm PST

  2. Evoting-Experts.com
    There’s a new blog in town, run by a variety of computer scientists and security experts, on the topic of…

    Trackback by Off the Kuff — Monday November 01, 2004 @ 12:01 pm PST

  3. If the machine works as you describe, doesn’t that mean that Bush is the “default” selection?

    Comment by Grady — Monday November 01, 2004 @ 12:52 pm PST

  4. I voted straight Democratic on one of the Hart machines. Being a bit cocksure that I was tech savy, I didn’t bother to review any instructions for the machine. When presented with the first ballot page after selecting straight Demo, I thought well, those are the people I want to vote for so I hit enter, which is the ususal default button for such actions on a computer or web page. As you describe, it change my vote to Bush/Cheney and it took me a few moments to figure out I had to hit the next page button to get to the cast ballot page. So I switched back my vote to Kerry/Edwards and paged through the ballot. A voter with a similar experience that did not change his president vote back to Kerry/Edwards would get to the cast ballot page with the vote still showing Bush/Cheney.

    Comment by Bill K — Monday November 01, 2004 @ 1:12 pm PST

  5. The notion of a “default” on an electronic ballot is disturbing. There’s always got to be a first candidate on any list, even a paper one, but a system which automatically highlights the first entry—making it immediately eligible for selection (such that, as you hypothesize, pressing the selection button too many times “might indeed end up selecting the top candidate")—is extremely dangerous.

    [For what it’s worth, I voted last week (in Brazoria County, TX, south of Houston’s Harris Co.), and I was pleased to see that the antiquated punch-card ballot still in use there suffered no observable failure when I voted a straight ticket. It appeared to represent my selections right up until I dropped it into the ballot box, which is all any individual voter can reasonably verify at the polling place.]

    Comment by Dan Sandler — Monday November 01, 2004 @ 2:03 pm PST

  6. I tested this on an eSlate in Harris County during early voting. I selected straight party and then changed a vote in one race. A big screen came up with a warning that I was changing a straight-ticket ballot.

    Comment by Rob Booth — Monday November 01, 2004 @ 3:11 pm PST

  7. I wasn’t aware they had a split-ticket warning system built into the system. If it always works as you describe, then that would invalidate my human-factors theory. I’ve asked the students in my security class who are voting tomorrow to try this out and see for themselves. Hopefully, by Wednesday, I’ll have a better idea of what’s causing the problem.

    Comment by Dan Wallach — Monday November 01, 2004 @ 5:09 pm PST

  8. That was pretty much my impression when I read about this problem. I certainly don’t dismiss the concerns about tampering, but the tampering hypothesis just didn’t make sense in this case.

    OT - Speaking of human-factors issues, consider adding a “preview” button to your blog comment form.

    Comment by Mathwiz — Monday November 01, 2004 @ 5:17 pm PST

  9. The fact that all of the problems that are occurring with the voting machines in various states are favoring the Bush ticket seems indicative that the machines are in fact rigged. I think we should stop trying to justify this as machine/software error and face up to reality. If you recall, the owner of Diebold told Bush that he would hand him the presidency. This election is being stolen, vote by vote.

    Comment by Angela — Monday November 01, 2004 @ 6:47 pm PST

  10. EVoting usability
    EVoting usability (via Hkon Styri)…

    Trackback by Anders Jacobsen\’s sideblog — Tuesday November 02, 2004 @ 9:10 am PST

  11. Let’s hear it for graduate students. I asked my students, when they went to the polls today, to experiment with double-clicking and anything else they could think of to see if they could replicate my human-factors theory.

    As far as I can tell, the theory just doesn’t hold water. The latency for a straight ticket selection is actually pretty low. Double clicking, in any circumstance, doesn’t cause a secondary selection. Pretty much no matter how you sliced it (according to reports from two students), you can’t replicate the issue.

    This means that, perhaps, we’re really looking at a software bug or even a poor attempt at fraud. As more reports come in today (from real voters and from my own students), maybe we’ll learn more about this issue.

    Comment by Dan Wallach — Tuesday November 02, 2004 @ 11:48 am PST

  12. Will Heflin challenge his loss to Vo?
    Andy Taylor, lawyer for Talmadge Heflin, is still being cagey regarding strategy to cope with Heflin’s electoral loss to Hubert…

    Trackback by Off the Kuff — Thursday November 11, 2004 @ 4:53 pm PST

RSS feed for comments on this post.

Leave a comment

Line and paragraph breaks automatic, e-mail address never displayed, HTML allowed: <a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>

(required)

(required)


Powered by WordPress