New TAR Case Affirms Sedona Principle Six, Approves Search Term Pre-Culling

A new technology-assisted review (TAR) case—Livingston v. City of Chicago—provides instructive guidance on any number of key issues surrounding the use of TAR. From affirming the notion of Sedona Principle Six and approving the use of search terms to pre-cull a data set to emphasizing the importance of not holding TAR to a higher standard than other search and review methods, Livingston provides additional clarity on issues sometimes clouded by conflicting TAR case law.

 The Parties’ Initial Dispute Over Collection and Identification of Relevant Documents

 In Livingston, the parties had been at an impasse for well over a year regarding the methods that defendant, the City of Chicago (City), should use to identify and search for emails responsive to plaintiffs’ discovery requests.[1] Plaintiffs had sought the production of relevant emails from the City in an effort to substantiate their claim that the Chicago Fire Department discriminated against women who applied for positions within the Department’s paramedic division.

In response to plaintiffs’ discovery, the City proposed using its “Microsoft Tool” to search for and identify responsive emails. Plaintiffs disagreed with this approach and suggested that the City instead engage an eDiscovery vendor to export all potentially relevant emails. Plaintiffs insisted that the City run search terms against the collected universe of documents and then produce the search hits without performing a relevance or privilege review. When the City refused to accede to these demands, plaintiffs sought relief from the court.

Channeling Solomon in the resolution of the dispute, the court granted half of plaintiffs’ requested relief. The City would be obligated to engage a discovery vendor to collect the relevant emails and then run search terms against the data set that plaintiffs developed. However, the court rejected plaintiffs’ blanket production request and deferred to the City on how to conduct a responsiveness review.

The Parties’ Subsequent Dispute Over the Use of TAR

After collecting potentially relevant documents and running plaintiffs’ search terms, the City was left with a subset of 192,000 emails to review for relevance and privilege. The City informed the court during a subsequent status conference that it intended to review the emails using TAR. The City identified Relativity’s “Active Learning” tool as its designated TAR technology and indicated it would run TAR now that it had used plaintiffs’ search terms to identify potentially relevant emails. The City also explained it would use various validation measures including reviewing samples of documents designated nonresponsive by the TAR technology.

Plaintiffs objected to the City’s TAR approach and argued that the use of TAR would violate the court’s earlier discovery order. In motion practice on this issue, plaintiffs took the position that the prior discovery order limited the City to using search terms and mandated the production of all search hits after the City conducted a privilege (but not a relevance) review.

Plaintiffs also argued that if the City were allowed to use TAR, it should do so pursuant to plaintiffs’ proposed TAR protocol. That protocol would require the City to forego using the search terms plaintiffs originally developed to pre-cull the data set. Instead, the City would be required to run TAR against the entire collected set of approximately 1.5 million documents—not the 192,000 email subset—to alleviate plaintiffs’ concern that the search terms would leave out “large amounts of potentially relevant ESI.”

The Court Allows the City to Use TAR and Rejects Plaintiffs’ TAR Protocol

In response, the court discarded its Solomon-like “split the baby” stance from its previous order and unequivocally rejected plaintiffs’ requested relief. As an initial matter, the court observed that its prior discovery order did not confine the City to using search terms or otherwise dictate what methodology “the City must use to identify responsive ESI.” Nor did the order restrict the City to merely conducting a privilege review on the search hits. Instead, the City could adopt whatever method it deemed appropriate to conduct its “responsiveness review.”

With the City permitted to use TAR, the court held it would not allow plaintiffs to dictate how the City would develop its TAR workflow. Citing Sedona Principle Six, which provides that the party responding to discovery is best situated to select the methods for accomplishing its document production, the court acknowledged that the City should be free from interference from plaintiffs in developing and running its TAR workflow:

. . . the court agrees with the City that as the responding party it is best situated to decide how to search for and produce emails responsive to Plaintiffs’ discovery requests. . . . The City has disclosed the TAR software—Relativity’s AL—it intends to use and how it intends to validate the review results, which in this case is sufficient information to make the production transparent. Plaintiffs’ insistence that the City must collaborate with them to establish a review protocol and validation process has no foothold in the federal rules governing discovery. (emphasis added)

As part of its holding, the court specifically rejected plaintiffs’ insistence that the City run TAR against the entire set of 1.5 million documents rather than the subset of 192,000 emails. The court reasoned that running TAR against the entire data set would not alleviate plaintiffs’ concerns any more than running search terms given “the low richness [under 15%] of the ESI collection.” Forcing the City to run TAR against the entire data set based on conjecture—and thus winding back the document review process several months—would ultimately be “wasteful and unduly burdensome.”

Lastly, the court disregarded plaintiffs’ concern that the City’s document review team would “improperly train the TAR tool by making incorrect responsiveness determinations or prematurely ending the review.” Heeding Magistrate Judge Andrew Peck’s warning from Rio Tinto v. Vale that “it is inappropriate to hold TAR to a higher standard than keywords or manual review,” the court explained that “these concerns are present no matter which methodology is employed.” Moreover, the court felt the quality and nature of the City’s validation measures would sufficiently ameliorate plaintiffs’ concerns.

Livingston’s Guidance on Key TAR Issues

TAR case law—including Livingston—often lacks bright-line rules for establishing precedent on TAR use issues given the fact-intensive nature of court decisions. And yet, Livingston does provide some general guidance on four TAR use issues that should be instructive for courts as they are called on to resolve disputes over TAR.

First, Livingston once again clarifies the primacy of Sedona Principle Six and that responding parties should be free to select the methods they believe will best facilitate reasonable and proportional productions of relevant information.

Second, Livingston emphasizes the importance of transparency in using TAR. While not obligating the City to divulge significant aspects of the TAR workflow like some earlier cases, Livingston did acknowledge the importance of disclosing the responding party’s use of TAR, the identity of its TAR tool, and the general nature of its validation measures.

Third, Livingston approved a responding party’s use of search terms to pre-cull a data set before running a data subset through the TAR workflow, finding it was both “reasonable” and “proportional to the needs of this case.”

Fourth, Livingston spotlighted the importance of not holding TAR to a higher standard than other review methods such as search terms or manual review.