On June 18, 2019, the Court entered the parties’ ESI protocol, which provided, in part:
“II. Search Terms for Electronic Documents
The parties agree that they will cooperate in good faith regarding the disclosure and formulation of appropriate search methodology, search terms and protocols, and any TAR/predictive coding prior to using any such technology to narrow the pool of collected documents to a set to undergo review for possible production. The parties agree to meet and confer as early as possible to discuss, inter alia:
• Search methodology(ies) to be utilized (including but not limited to Boolean searches and technology assisted review/predictive coding).”
The ESI Protocol also included a provision requiring the parties to meet and confer regarding a dispute as to whether they should produce documents in conformity with the Protocol, and if the parties cannot reach agreement, the issue would be brought to the Court’s attention.
As noted by the Court, the parties negotiated for months about search terms and custodians. However, about a year after the ESI protocol was entered and about a year after the parties stated negotiating the search terms, and only two weeks before Teva’s first rolling production was due, Teva informed Plaintiffs for the first time that Teva was using a continuous multimodal learning (“CMML”) platform to assist Teva’s review and production of ESI.
Plaintiffs objected to Teva’s revelation, arguing that had they known that Teva contemplated the use of TAR they would not have agreed to limit the review of custodian documents to only those that contained the negotiated search terms nor would they have wasted time negotiating over search terms or agreed to layer a TAR review with search terms. Teva in response asked the Court to confirm that Teva was authorized to use CMML. The parties then subsequently attempted to reach an agreement regarding TAR including an agreed upon CMML protocol. However, Teva resisted resolution of the matter over two issues – that the CMML protocol not be put on the record and that Plaintiffs could not review 5000 alleged non-responsive documents to evaluate and validate Teva’s CMML process.
Unbeknownst to Plaintiff, Teva also attempted to self-validate its CMML process. Teva claimed, among others, that out of 15,000 documents that the CMML platform indicated was non-responsive, after a manual review, only 109 or 0.73% were deemed by the reviewing attorneys to be responsive, and that the review of the 15k documents involved 330.6 hours of attorney review time. Based on its self-validation, Teva argued that it would be grossly disproportionate to require it to review documents designated non-responsive by its CMML platform and that if force to do so, its review cost should be shifted to Plaintiffs.
As part of its analysis, the Court identified several things that were NOT at issue and that the Court did not have to decide. Not at issue was the question of whether TAR is an appropriate discovery tool or whether Teva’s CMML program was effective. However, what was at issue was whether Teva violated the parties’ ESI protocol.
As noted by the Court, while Teva was not barred from using TAR, the ESI protocol required that before TAR is used, the parties were to cooperate in good faith regarding TAR’s disclosure and formulation. In view of its express language, the Court found that ESI Protocol required Teva to timely disclose it use or possible use of TAR when Teva objectively knew or reasonably should have known that it might use TAR to reduce the universe of documents to review, and that the ESI Protocol also required Teva to timely collaborate with Plaintiffs. As stated by the Court:
“The Court will not countenance a situation where for months plaintiffs were led to believe and proceeded under the reasonable assumption that Teva would do a manual search term review of its ESI and then, after a substantial effort was expended in this direction, Teva changed course and unilaterally adopted and used its unilaterally adopted CMML platform.”
Teva’s argument that Plaintiffs were attempting to control Teva’s method of ESI review was also rejected by the Court. As noted, Plaintiffs were instead attempting to compel compliance of the ESI protocol which required timely transparency and collaboration.
While the Court denied Teva’s application, the Court ultimately permitted Teva to do a TAR review using the CMML protocol that was negotiated by the parties. However, that permission included the two conditions that Teva objected to – that the CMML protocol be entered as a Court Order, and that Plaintiffs would have the right to review at the end of Teva’s production, 5000 alleged non-responsive documents Plaintiffs designate at their own choosing for review as part of a validation process.