Court Orders NYC to Provide Sample TAR Documents to Plaintiffs and to Supplement TAR Review with Plaintiff Search Terms
In Winfield v. City of New York, No. 15-cv-05236, Dist. Court (S.D.N.Y., 2017), a lengthy dispute between residents and the city regarding the availability of affordable housing led to contentious discovery.
Defendant New York City’s current goal is to build or preserve 200,000 below-market-rate apartments by 2024 as part of its affordable housing programs. This case involves a challenge to the Community Preference Policy. New York City residents who want affordable housing and otherwise meet the qualifications can apply for certain affordable housing units through a lottery system. Plaintiffs, three African-American residents of New York City, want affordable housing and have applied for housing through lotteries, but have not yet been selected to be interviewed for affordable housing developments. They claim that the Community Preference Policy has a larger impact on African-American and Latino applicants in “neighborhoods of opportunity,” which they assert are predominantly white.
The issues presently before the Court relate to the City’s production of electronic documents. Plaintiffs have sought wide-ranging discovery, which the City has resisted vigorously. Plaintiffs believe the City’s Technology Assisted Review (“TAR”) training and review process was faulty, as the City i) failed to use appropriate search terms for the review population, resulting in a deficient potential review corpus; ii) searched for documents that were responsive to only a subset of Plaintiffs’ document requests; and iii) over-designated documents as non-responsive when in fact they were responsive to their document requests.
Federal Rule of Civil Procedure 34 governs the production of documents and electronically stored information (“ESI”). Rule 34 provides that a party may serve on another party a request to produce documents and ESI within the party’s possession, so long as the documents and information are relevant to the party’s claim and proportional to the needs of the case, and whether the burden or expense of the proposed discovery outweighs its likely benefit.
Courts understand that keyword searches used for review must be carefully crafted. Here, Plaintiffs have provided 665 additional search terms to be applied. The City has stated that the supplemental search would require review of 90,000 additional documents at a cost of approximately $248,000. However, the City also stated that it is willing to use all of Plaintiffs’ proposed search terms and use TAR. Plaintiffs object to the City’s proposal because they believe that the City’s TAR processes are flawed because Plaintiffs claim it results in the over-designation of documents as non-responsive because the City’s human document reviewers over-designated documents as non-responsive. As a result, Plaintiffs claim, the TAR software is unable to recognize and properly categorize responsive documents. As courts have noted, the producing party is in the best position to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information. Courts have traditionally not micromanaged attorneys, because as officers of the court, they are expected to comply with Rules 26 and 34 in connection with their search, collection, review and production of documents, including ESI. Additionally, internal attorney ESI work processes may reveal work product, litigation tactics, and trial strategy. Finally, perfection in ESI discovery is not required: the producing party must take reasonable steps to identify and produce relevant documents.
While Plaintiffs here do not have clear insight into the City’s training, this Court has required the City to provide in camera submissions addressing these subjects which reveal that the City appropriately trained and utilized its TAR system. The City provided detailed training to its document review team as to the issues in the case and they were provided with all of Plaintiffs’ document requests to use in connection with their review and designation of documents as responsive and the City has produced a sample of 80 of these supposedly privileged documents for in camera review. This Court has reviewed these documents in camera and determined that nearly all of them are responsive to Plaintiffs’ document requests to some degree, though most are only of marginal relevance. Because the City incorrectly categorized a handful of documents as non-responsive, this Court does not question the accuracy and reliability of the City’s TAR process as a whole.
While the Court disagrees with Plaintiffs’ assertions that the TAR process is defective, it finds that Plaintiffs have presented sufficient evidence to justify their request for sample sets of non-privileged documents.
Thus, this Court granted Plaintiffs’ request in part as follows: 1) the City is directed to provide to Plaintiffs a sample of 300 non-privileged documents. These documents should be randomly pulled from the corpus of non-responsive documents; 2) the City shall also apply the Plaintiffs’ search terms to the review population. These documents shall be provided to Plaintiffs’ counsel on an attorneys’ and experts’ eyes-only basis.