Tuesday, January 06, 2009
Exclusive: Humboldt's Secret Hart Attack
On December 17 the We Do Not Consent blog broke the story that Humboldt, CA Registrar of Voters Carolyn Crnich had announced to the Election Advisory Committee (EAC) the previous evening her intention to replace Diebold/Premier "election" junk with comparable secret vote counting computers from Hart InterCivic. I've posted a few times about the need for a public due diligence process around multiple Diebold alternatives, including a letter published in the Eureka Times-Standard and another that ran in the North Coast Journal. Both letters were based on the recent news of a Diebold programming failure that secretly deleted 197 ballots and led to inaccurate election results being certified here. Both letters were also written prior to learning about Crnich's proposed switch. Efforts to spark such a community dialog are happening today in multiple forms, and on the heels of a major new development.
In a phone call this morning, Crnich acknowledged that in November she discussed the planned changes with a small number of volunteers who were asked to keep it a secret. Parke Bostrom is an EAC and ETP (Election Transparency Project) volunteer who has also worked closely with Crnich as an election day poll worker and overall observer of the Elections Department. Yesterday Bostrom posted The "Spend Pork Wisely" Petition that begins:Petition calling for public discussion of Humboldt County's plan to purchase $600,000 of Hart InterCivic eScan election equipment.
The full petition text appears here and at the bottom of this post. Crnich has now tried twice without success to get the matter on the Supes agenda. As I noted on 12/17, her first attempt was for the December 16 Supes meeting, which would have done the deed even before fully revealing the secret at that night's EAC meeting - at which Crnich also said she had hoped to take delivery of nearly 80 new machines the next day in order to get a $28,000 discount offered by Hart if the deal could be completed by year's end.
WHEREAS, at the November 18th monthly public meeting of the Humboldt Election Advisory Committee, County Clerk, Recorder and Registrar of Voters Carolyn Crnich announced the Elections Office's plan to stop using the county's Diebold/Premier AccuVote/GEMS elections equipment and replace it with similar Hart InterCivic eScan equipment.
And WHEREAS, County Clerk, Recorder and Registrar of Voters Crnich then asked those attending the public meeting to keep this plan secret until at least mid-December, thereby minimizing the opportunity for public discussion of the plan prior to receiving approval for the plan from the Humboldt County Board of Supervisors.
The Humboldt Board of Supervisors met again today without taking this on, leaving us planning for a January 13 agenda item, but taking action now with outreach to media as well as the Supes and various local City Councils, whose members we hope will encourage the Supes to have a public process. Eureka Councilman Larry Glass encouraged such an approach when I spoke with him this past Saturday evening in his Old Town music store. (Also worth noting: Glass recently made news for heroically rescuing a would-be suicide jumper into Humboldt Bay).
My 12/17 exclusive landed as the top story on that day's Daily Voting News, compiled by John Gideon of VotersUnite.org. Two days later, BradBlog quoted me heavily and advanced Gideon's observation that Hart InterCivic had withdrawn from the federal certification process. Bostrom's petition puts a fine point on this:And WHEREAS, the version of the Hart InterCivic eScan equipment the Elections Office is planning to purchase is also an old version of the eScan system, as Hart InterCivic has been unable to receive certification from the California Secretary of State for the most recent and up-to-date version of the eScan system.
The Times-Standard has followed up twice since I broke the story of the proposed switch. In a quote of note in the December 22 article (archive), Crnich says, "This plan that is proposed pre-dates any of the problems that were found to exist in this election."
And WHEREAS, Hart InterCivic has withdrawn from the certification process and is not currently seeking certification for use in California of the most recent and up-to-date version of the eScan system; and consequently, so long as Hart InterCivic remains disengaged from the certification process, it will not be legally possible to apply any software upgrades that may be necessary to prevent future invisible failures of the old version of the eScan system
The same article also reports Congressman Mike Thompson sent a letter to "federal elections officials," about his concern over the invisible failure. I wonder if he knew then or even knows now that some votes for him were among those deleted (as Bostrom notes in the petition). I am awaiting a call back from Thompson's office in Eureka.
The T-S followed up again on December 29 (archive) reporting Thompson's letter got the attention of Election Assistance Commission Chairwoman Rosemary Rodriguez, who claims her horribly ineffectual board "doesn't have the authority or capacity to launch independent investigations." She does pledge, and seem to want a gold star for, her intention to "disseminate the contents of the Humboldt County report to elections officials from coast to coast [to] prevent similar problems from occurring elsewhere." What a good idea. If only it weren't the repeatedly unfulfilled yet HAVA-mandated purpose for her existence.
Also check out the first half of hour three of the December 19 Peter B. Collins show when I called in to discuss all this with PBC, Brad Friedman, and Harvey Wasserman.
Happenings in the Humboldt Elections Department have been making national news for a while now. Generally speaking, Crnich has earned lots of fans for her willingness to work with the great team of citizen volunteers that have made the ETP happen. In addition to Bostrom, credit, praise and thanks also rightfully go to Kevin Collins, Mitch Trachtenberg, and Tom Pinto.
With such involved interactions and access to Crnich, these guys have gained her trust and confidence, resulting in candid sharing of information. However, Crnich's request for secrecy, and the volunteers' granting of this request, is not the kind of transparency we deserve from our local government, particularly in the context of a so-called Election Transparency Project.
Asked to comment on Bostrom's petition, Crnich told me she is "ready to move forward," and the idea of slowing down for a public due diligence process would be a "serious impediment to progress" because it puts needed training on hold. While Crnich calls the lateral move from Diebold to Hart "progress," I see it as a false alternative.
I have been unable to reach Collins and Trachtenberg for comment on the petition and secrecy issue, however, Pinto provided the following in an e-mail early this morning:In my opinion, citizens are being offered a disappointing menu of choices with regards to election systems. I really hope the CA Sec. of State will offer us the option of using an election system that incorporates open source technology. However, that option is not being offered at this time.
I concede that I missed several consecutive EAC meetings at which Hart equipment was apparently discussed and even demonstrated (Crnich even chided me for this when I arrived at the December meeting). However, I do not think that these small and largely undocumented meetings, of a committee with no formal mandate from the county, constitute a sufficient public process. Pinto notes that "interested persons" have been given an opportunity to weigh in, but not enough has been done to make the community at large aware of the proposed switch to Hart, let alone interested.
I think that CC [Carolyn Crnich] has made an acceptable decision to purchase Hart eScan based upon what is being offered by the State and based upon the age and problems of our existing software. I think CC has researched it sufficiently and she expects it to save [Elections Manager] Kelly [Sanders] a HUGE amount of time. I'm glad the Humboldt County Election Transparency Project is in place to the catch poll worker errors and software bugs (regardless of which proprietary system we're using). I hope that more citizen volunteers will participate in the next audit.
I think CC has done a first rate job of reaching out to citizens to participate. She has given interested persons, such as the ones who attend the EAC meetings, sufficient opportunity to voice any concerns about the eScan. She has also invited the public to inspect these machines. I respectfully question whether there is sufficient need for creating a formal time period for additional public participation. I also doubt that the conclusion of such a comment period would result in any difference the current plan. However, I do not object to the creation of formal public comment period.
Bostrom's petition at least seeks to bridge this gap, also noting that the contract for the proposed switch to Hart has not even been made public yet. That aspect alone deserves rigorous public scrutiny. I'll have more on this developing story in the days ahead.
* * *
Petition calling for public discussion of Humboldt County's plan to purchase $600,000 of Hart InterCivic eScan election equipment.
WHEREAS, at the November 18th monthly public meeting of the Humboldt Election Advisory Committee, County Clerk, Recorder and Registrar of Voters Carolyn Crnich announced the Elections Office's plan to stop using the county's Diebold/Premier AccuVote/GEMS elections equipment and replace it with similar Hart InterCivic eScan equipment.
And WHEREAS, County Clerk, Recorder and Registrar of Voters Crnich then asked those attending the public meeting to keep this plan secret until at least mid-December, thereby minimizing the opportunity for public discussion of the plan prior to receiving approval for the plan from the Humboldt County Board of Supervisors.
And WHEREAS, on the evening of December 16th, at the next monthly public meeting of the Humboldt Election Advisory Committee, after 197 ballots had been invisibly deleted by an invisible failure of the AccuVote/GEMS equipment, when asked what the planned response to the invisible AccuVote/GEMS failure would be, and having been bumped from the overfull agenda of the Board of Supervisors meeting that very morning, County Clerk, Recorder and Registrar of Voters Crnich publicly announced the plan to purchase the eScan equipment.
And WHEREAS, the invisible failure of the AccuVote/GEMS system deleted 114 votes for Representative Mike Thompson; 102 votes for Assembly Member Wes Chesbro; 76 and 148 votes for, respectively, City of Eureka Councilpersons Linda Atkins and Frank Jager; 110 and 84 votes, respectively, for and against California's Proposition 8; 83 and 113 votes, respectively, for and against City of Eureka Measure J.
And WHEREAS, the invisible failure of the AccuVote/GEMS system is believed to be due to a bug in the old version of the GEMS equipment that is claimed to be fixed in the most recent version of GEMS available for use in California.
And WHEREAS, the version of the Hart InterCivic eScan equipment the Elections Office is planning to purchase is also an old version of the eScan system, as Hart InterCivic has been unable to receive certification from the California Secretary of State for the most recent and up-to-date version of the eScan system.
And WHEREAS, Hart InterCivic has withdrawn from the certification process and is not currently seeking certification for use in California of the most recent and up-to-date version of the eScan system; and consequently, so long as Hart InterCivic remains disengaged from the certification process, it will not be legally possible to apply any software upgrades that may be necessary to prevent future invisible failures of the old version of the eScan system.
And WHEREAS, the cost to purchase the old version of the eScan system will be in excess of $600,000 of taxpayer money.
And WHEREAS, as of January 5, 2009, the proposed contract with Hart InterCivic has not been publicly disclosed, and consequently there has not been any open public discussion of the specific plan to purchase the old version of the eScan system.
And WHEREAS, open public discussion of the plan to purchase the old version the eScan system is a reasonable step to take to reduce the likelihood of future invisible failures of election systems.
THEREFORE BE IT RESOLVED that we, the undersigned concerned citizens call upon County Clerk, Recorder and Registrar of Voters Carolyn Crnich, and also upon the Humboldt County Board of Supervisors to take the following steps to promote public confidence in the outcome of future elections in Humboldt County:
1. The county SHALL issue a press release containing the proposed contract with Hart InterCivic and inviting the public to submit written questions, comments and concerns regarding the planned purchase and use of the old version of the Hart InterCivic eScan system.
2. Following the publication of the press release, the county SHALL give the public at least 2 weeks to submit such written questions, comments, and concerns.
3. At the end of the submission period, the county SHALL publish, in an electronic format, all the questions, comments and concerns submitted by the public.
4. Within a reasonable period of time thereafter, the county SHALL prepare written responses to all the public's questions, comments, and concerns. In preparing said responses, the county may, if it so wishes, consult with Hart InterCivic and/or any other parties.
5. When the county finishes preparing written responses, the county SHALL publish, in an electronic format, the public's questions, comments and concerns together with the county's responses.
6. Following the publication of the county's responses, there SHALL be a period of reflection. The period of reflection of SHALL be at least one week long.
7. Prior to the conclusion of the period of reflection, the Board of Supervisors SHALL NOT approve the purchase of the old version of the eScan system.
8. After the conclusion of the period of reflection, the Board of Supervisors may create an agenda item for an open public hearing of the request to purchase the old version of the eScan system.
9. Additionally, if the Board of Supervisors is interested in considering, in a thoughtful and deliberate manner, alternatives other than the purchase of the old version of the eScan system, we recommend to the board that now, prior to the purchase of the old version of the eScan system, is a very good time to consider any such alternatives.
Sign the Petition
Permalink:
http://wedonotconsent.blogspot.com/2009/01/exclusive-humboldts-secret-hart-attack.html
Labels: Carolyn Crnich, Diebold, Election Advisory Committee, eScan, Hart Intercivic, Kevin Collins, Larry Glass, Mike Thompson, Mitch Trachtenberg, Parke Bostrom, petition, T-S, The Journal, Tom Pinto
Wednesday, December 17, 2008
Humboldt's False Alternative to Diebold
After missing several consecutive monthly meetings of the ad hoc volunteer Election Advisory Committee (EAC), convened by Humboldt County, CA Registrar of Voters Carolyn Crnich, on Tuesday night I rejoined the group to learn Crnich has at last given up on Diebold/Premier's optical scanners and GEMS central tabulator as our official vote counting method. After years of opposing Diebold, this might seem like a dream come true. But that's only the beginning of the story.
Crnich informed the group that we will continue to use paper ballots and precinct based scanning with Hart InterCivic's eScan system (complementing Hart's eSlate machines already in use for HAVA compliance). A distinction was made between Diebold's optical scanners and Hart's so-called digital scanners, though a similarity worth noting is that rather than counting the actual ballots both technically count the images of the ballots internally created within black box technology.
Whatever the supposed relative merits may be for Hart compared to Diebold, this is the epitome of a false alternative, the appearance of choice in a no-win situation. We will still be using an accuracy-challenged proprietary and secret system found vulnerable to undetectable manipulation by California's Top To Bottom Review, Ohio's EVEREST study, and Colorado's Secretary of State.
Humboldt County has made news lately for its Election Transparency Project, which revealed a failure in GEMS that caused Crnich to certify inaccurate results from last month's election. Crnich later told Wired.com:Crnich told Threat Level the issue has made her question her confidence in the voting system because, even though the company provided officials with a workaround, the problem indicated a fundamental flaw in the company's programming. She said she'd heard a lot of stories from other election officials about problems with voting machines, but never thought they applied to California.
Crnich has been widely praised for working with citizen volunteers to create the audit mechanism that identified the problem, which was entirely the fault of Diebold and not at all of Crnich's doing. Certainly some credit is due there, though as I explained last week, this has been exaggerated in an unhelpful way that now will likely make it more difficult to challenge her decision to switch to eScan. Indeed, at the EAC meeting I was accused of "casting aspersions" for raising questions of timing and public input. Yet check out how this almost went down...
"I've always sort of listened to those anecdotal incidents with a jaundiced ear because California has some very stringent requirements of election systems that are in use here as well as some very strict security procedures and I didn't think those things affected us here," she said. "But this has sort of put a cloud over any confidence that I had in the Premier equipment that's been in this department since 1995."
At Tuesday's EAC meeting, Crnich said she has the sole authority to choose our vote counting method and had already done so. In fact, she had hoped to take delivery of nearly 80 new machines on Wednesday in order to get a $28,000 discount offered by Hart if the deal could be completed by year's end. The total cost is estimated to exceed $600,000 and would be paid entirely out of the County's unused HAVA and Proposition 41 funds. Procedurally, Crnich said she needs the County Board of Supervisors to approve her plan, and then approval of the CA Secretary of State for the use of funds.
Crnich said she attempted to get a spot on the agenda at Tuesday's Supes' meeting. An unusually long list of business items prevented that and has likely created a reprieve until January 6. However, that isn't guaranteed as Supervisor Jimmy Smith, in attendance at Tuesday's EAC meeting, suggested the possibility of a short-noticed special Supes meeting that could possibly occur this year if it would still allow the $28k savings.
Crnich told the EAC the next scheduled election is not until November 2009, though there is talk of a possible statewide special election as early as March. In that regard, she did not express urgency for the Hart transition to begin. If anything, she said she looked forward to having the time to evaluate candidates for an open position within the elections department. She lamented the delay between the announced opening in July and her recent receipt of the qualified applicant list.
The intent here is not to attack Crnich. These are the things she has done and said, which fairly raise the following questions.
There is an element of deja vu here. In 2005 I wrote about efforts to prevent Crnich's flirtation with going touchscreen, though it certainly wasn't presented as an argument for the status quo. Crnich periodically receives heaps of praise for nixing the switch, typically without reference to the public resistance.
I can only think to call this a bittersweet irony. After years of urging the immediate abandonment of Diebold equipment, now Humboldt County can't drop the hot potato fast enough. The glow of the Transparency Project is currently blinding and seems to shield the sense of embarrassment for not only certifying inaccurate results but also defending the continued use of equipment known to be flawed all along. Who else will make clear that one laudable achievement does not mean complete deference on important questions of public policy?
* * *p.11
Here are some key observations from Harri Hursti and others about Ohio's EVEREST study:
The Red Team, working in close conjunction with the 2007 TTBR Hart Source Code Team, discovered that the Hart EMS software implicitly trusts all communication coming from devices appearing to be Hart-branded and neither authenticates the devices nor performs adequate input validation on data transmitted to it by the devices. This allows for the possibility that a compromised device, such as an eScan that had been tampered with at a polling station, could infect the EMS systems. In particular, the Source Code Team discovered a weakness in the code that would allow an eScan to perform a buffer overflow attack and execute arbitrary code on the computer running SERVO.
...
The team was also able to access device-level menus that should be locked with passwords but were not. This access could allow an attacker a vector for altering configuration settings and/or executing a denial of service on the eScan.
Some of the findings from previous studies on precinct count optical scanners were replicated on the eScan, and they allowed the Red Team to maliciously alter vote totals with the potential to affect the outcome of an election. These attacks were low-tech and required tools that could be found in a typical office.
The Red Team implemented an attack devised by the 2007 TTBR Hart Source Code Team that was able to extract election-sensitive information from the eScan and issue administrative commands to the eScan. The leaked information would allow an attacker the ability to execute further attacks, while administrative commands issued to the eScan could erase electronic vote totals and audit records from an eScan while putting it out of service for the remainder of the Election Day. For more details on these attacks, please see the 2007 TTBR Hart Source Code Team report.3.2 Insider Defenses
For more see this 2006 report from VotersUnite.org, the same site's Election Problem Log 2004 - Present, as well as BradBlog.com.
Attack Class 6: eScan Manipulation – We were able to exploit a number of vulnerabilities in the eScan that could give election insiders the ability to compromise election results and voter privacy. Some of these were a result of a lack of physical security. We were able to replace the eScan's internal flash memory card containing the eScan executable and configuration file with only a screwdriver in about 2 minutes. After replacing the card, we were able to boot the eScan into the Linux operating system. This simple attack gives a single poll worker with a few minutes of unobserved access to the eScan the ability to undermine all votes cast at a precinct (EVEREST 20.3.1).
While opening the eScan to replace the memory card, we broke three tamper evident seals. While such seals may prove that a machine was opened, a preventative measure is preferable. A poll worker may intentionally break these seals in order to cast doubt on election results. It has also been shown that tamper evident seals do not always correctly show that tampering occurred [14].
Insiders may also wish to use their access to ballots to determine voter choice. This can be done with the eScan due to the design of its ballot box (EVEREST 20.3.4). The eScan's scanner sits on top of its ballot box, which is essentially a plastic tub. When a ballot is scanned, it is subsequently dropped into the box. No measures are taken to disturb the order in which ballots fall, allowing a malicious poll worker to note the position in which certain votes are cast and then relay these positions to an election official with access to the ballots. We observed ten numbered ballots as they were cast with the eScan, and verified that the vote order was preserved.
...
Attack Class 14: Open Audit Interfaces – Both the Hart JBC and eScan have open interfaces that allow for the erasure of votes and audit log records. As detailed in Issue 3 of the CA TTBR, the eScan is managed through an accessible Ethernet port that listens for connections on TCP port 4600. This port is normally used for sending and receiving commands from SERVO, such as file transmission and reading images of the eScan's memory. No cryptographic tokens are required for these operations to occur.
We discovered that with a handheld device such as a Palm computer, an attacker with an Ethernet cable can mimic the actions of SERVO to the eScan during a live election, and cause the vote records and audit logs to be erased from both the eScan's internal memory and the MBB inserted into it (EVEREST 20.3.7). Any voting that had occurred on the eScan to that point would be erased, necessitating a manual recount.
...
Attack Class 19: Autovoting – A final example of unsafe features intentionally added to the Hart systems is the Ballot Now’s "Autovote" feature (EVEREST 20.7.2). Autovote allows for the creation of pre-filled-in paper ballots. Once again, this feature is enabled through Windows registry entries. Once these entries are enabled, Ballot Now displays the Autovote menu option when started.
The Autovote menu allows the Ballot Now user to choose the number of pre-filled-in ballots to print. The user has no control over the selected filled in entry for each contest, however, the selected entries are uniformly distributed. This allows an arbitrary number of ballots with the desired results to be printed with the overhead of some ballots with undesired results that may simply be discarded.
Paper ballots generated by Autovote initially say "Autovote" on the front and back, making them conspicuous and easy to detect in an audit or recount. We were able to overcome this by installing a PNG printer driver on the Ballot Now machine. This driver allows ballots to be printed to PNG image files as opposed to paper. We could then open the files in an image editor, remove the Autovote label and print them. Aside from the label, Autovote ballots are identical to regular ballots. We conducted a normal election and an election with Autovote ballots, and could not identify any differences in the eScan unofficial printout, the audit logs, or the cast vote records on the eScan's MBB.
Autovote could be used in tandem with the eScan's duplicate ballot feature to perform a ballot stuffing attack. Using Autovote ballots is advantageous over using photocopies, as each Autovote ballot has a unique serial number, and thus cannot be differentiated from legitimate votes in an audit.
Permalink:
http://wedonotconsent.blogspot.com/2008/12/humboldts-false-alternative-to-diebold.html
Labels: Carolyn Crnich, Diebold, Election Advisory Committee, eScan, EVEREST, false alternative, Hart Intercivic, Jimmy Smith, Top To Bottom Review, Transparency Project
