What Large-Scale Discovery Reveals About Analytics & Managed Review

Large document review becomes difficult not simply because of volume, but because the work itself is constantly moving. New data arrives, priorities change, and review teams have to maintain consistent decisions even as the circumstances around them evolve. 

One recent matter reflected that complexity on an unusual scale. What began with roughly a dozen reviewers grew to approximately 500 attorneys working over the course of a year across about 20 related matters and 10 review platforms. More than five million documents were ultimately reviewed, requiring the team to maintain consistency while reviewers, data, and workflows shifted throughout the matter. 

The experience reinforced an important principle for large-scale discovery: analytics and managed review work best as part of the same operational strategy. Technology can reduce the universe that requires human attention, while strong review management helps ensure the work remains focused, consistent, and defensible. The goal is not simply to put fewer documents in front of people, but to make better use of human judgment where it matters most.

Scaling More Than Reviewers 

Adding reviewers can increase capacity. It does not necessarily create scalability. 

In this matter, the review team expanded rapidly, at one point ramping from a relatively small group toward approximately 500 reviewers in about two weeks. At the same time, reviewers were not working sequentially through one database. They were moving among multiple workspaces hosted by different providers, often returning to earlier environments as new collections or review populations became available. 

Each platform presented information somewhat differently, from how fields and interfaces were structured to the patterns and anomalies within the underlying data. 

That made standardization essential. The goal was not to force every technology environment to look identical. It was to remove unnecessary operational variation so attorneys could focus on the substance of the review rather than repeatedly relearning how to perform it. 

 Review protocols, field mappings, workflows, and training materials were structured, so the experience remained as familiar as possible when reviewers moved from one environment to another. When a platform required something to be handled differently, reviewers were shown what had changed and why. 

This distinction matters well beyond unusually large matters. A scalable review process does not depend on everyone working in the same database under the same conditions. It establishes enough consistency for the process to withstand changes in technology, volume, staffing, and timing.

Analytics Should Shape the Review, Not Simply Precede It 

Analytics is sometimes approached as an early-stage exercise: use technology to reduce the initial population, hand the resulting documents to reviewers, and begin the review. 

That can leave substantial value on the table. Analytics can continue to reveal patterns throughout a matter, and effective managed review should respond to what those patterns show. 

On this matter, analytics continued throughout the review. Active learning, sampling, document-family analysis, and other techniques were applied as new data appeared, and the team learned more about what it contained. The effect was significant. Although more than five million documents were ultimately reviewed, that represented approximately half of the potential review universe. 

In another phase, a new population of roughly 50,000 documents appeared ready to move directly into review. Before assigning hundreds of reviewers to it, the team examined the data, sampled large document families, and identified substantial amounts of noise. After discussing those patterns with the case team, the review population was reduced by approximately 90 percent. 

The important lesson is not the specific percentage. It is that review populations should continue to be questioned. 

Rather than treating the workflow as something to establish once and execute, teams should continually ask whether the data is pointing to a better approach. A defensible decision made at the beginning of a matter does not mean the same workflow should continue unchanged months later. As new data arrives and the team develops a deeper understanding of the matter, the review strategy should be able to evolve with it.

Remove Noise So People Can Exercise Judgment 

Contract attorneys entering a review rarely begin with the same knowledge as the lawyers who have lived with a matter for months or years. Even experienced reviewers must learn about the company, the facts, the relevant individuals, the review protocol, and potentially an unfamiliar area of law. 

Their role is not to replace the subject matter expertise of the case team. A well-designed workflow instead creates a progression of judgment, allowing different levels of review to focus on the decisions appropriate to them. First-level reviewers address more straightforward coding decisions, quality control identifies patterns and inconsistencies, and more nuanced or strategically important documents reach the attorneys with the context necessary to evaluate them. 

Redactions provide a useful example. The matter included a substantial redaction component, but asking reviewers to independently redact repeated copies of identical documents would have added work without adding judgment. By using MD5 hash values—the digital fingerprints associated with files—the team could identify exact duplicates and apply redactions consistently across matching documents rather than asking attorneys to repeat the same task. 

The benefit goes beyond time saved. Repetitive work creates more opportunities for inconsistent treatment while consuming attention that could be directed elsewhere. Removing that noise allows reviewers and case teams to focus on documents where differences, context, and nuance actually matter. 

The objective is not simply to reduce the amount of human review. It is to structure the work so human judgment is being used where it contributes the most value.

Quality and Efficiency Reinforce Each Other 

A 500-person review creates an obvious challenge: how do you maintain consistency across hundreds of individual decision-makers without simply reviewing everything again? 

On this matter, quality control helped the team identify where additional attention was needed and where reviewers were already performing consistently. That allowed the review to focus more deliberately rather than applying the same level of scrutiny everywhere. 

Just as important was the speed of feedback. Reviewers were working with unfamiliar information, moving among different workspaces, and developing their understanding of the matter as the review progressed. Questions, trends, and corrections therefore needed to move back into the team while the work was still happening. 

Near-real-time feedback helped reviewers adjust before isolated misunderstandings became broader issues. It also gave them a better understanding of what they were seeing, which led to better questions and allowed ambiguities in the review protocol to surface earlier. 

That engagement has a direct effect on efficiency. Reviewers who understand the matter more deeply are better positioned to recognize meaningful documents and identify issues that require escalation. The alternative is discovering an important question late in the review and realizing that it may affect decisions made much earlier. 

For large matters in particular, quality and efficiency are not competing objectives. A review that identifies issues early, communicates them quickly, and keeps attorneys focused on meaningful work can reduce unnecessary effort while improving consistency. It can also limit avoidable overtime and help preserve attention for the periods when deadlines genuinely require it.

Technology Variability Requires Process Discipline 

Working across approximately 10 review environments created another layer of complexity. No single platform could serve as the operational center of the matter, and the technologies were largely determined by the different hosting providers already involved. 

The challenge was therefore not choosing one technology for the review. It was developing a consistent method that allowed reviewers to move among environments without losing the knowledge and practices developed elsewhere. When the team identified an approach that improved the review in one environment, the goal was to determine whether it could be applied across others and work with the respective providers to implement it consistently. 

That required more than proficiency with different platforms. It required a review process that could accommodate differences in technology while preserving the underlying approach to the work. 

For complex matters involving multiple technologies and providers, that process discipline can be what holds the review together. The platforms may differ, but the people working across them still need a consistent way to apply what they have learned and move the matter forward.

Measuring Review Success Beyond Activity 

Traditional service metrics tend to emphasize activity: documents reviewed, hours worked, utilization, or the size of the team deployed. Those numbers describe effort, but they do not always describe value. 

For clients, a large review will still require substantial investment. The more important question is whether that investment is being directed toward the work most likely to advance the matter. Reducing a 50,000-document population by 90 percent, for example, does more than lower review hours. It changes what attorneys can focus on instead. 

A broader view of success asks different questions: 

• How much unnecessary first-level review was avoided? 
• How quickly were reviewer questions and emerging issues resolved? 
• How much repetitive work was eliminated while maintaining consistency? 
• How effectively was attorney time redirected toward documents and decisions with greater strategic importance? 

Those questions provide a more complete picture of review performance because they look beyond how much work was completed to consider how intelligently the work was structured. 

That is where analytics and managed review become more than separate discovery services. Together, they provide a way to continually direct people, technology, and resources toward the work that can have the greatest impact on the matter. 

Build for Complexity, Not Just Volume 

The most important lesson from this matter is not that a review team can grow to 500 attorneys or work through millions of documents. It is that scale becomes manageable when the review is designed to absorb complexity rather than simply react to it. 

Large matters will change. New data will arrive. Technologies may differ. Review teams will expand and contract. Questions will emerge that could not have been anticipated at the beginning. Success depends on having enough structure to maintain consistency while still allowing the strategy to evolve as the team learns more. 

That means treating analytics, review management, quality control, and attorney engagement as connected parts of the same operation. Technology should help narrow and organize the work. Review processes should help people apply judgment consistently. Feedback should move quickly enough to improve the work while it is still happening. And success should be measured not only by how much was reviewed or how much was spent, but by whether resources were directed toward the decisions that mattered most. 

Those principles are not limited to unusually large matters. They are what allow a review of any size to scale when complexity increases. The objective is not to make complex document review simple. It is to build a process disciplined enough to manage complexity without losing sight of the legal work the review is meant to support.