The short answer is no. Although of course you would expect a document automation blog to say this.
The question, however, does deserve a more nuanced answer. Anyone who’s tried ChatGPT knows how good it can be at drafting content—or at least sounding like it knows what it’s doing. If you can ask ChatGpt to write a contract for you, who needs document automation?
After a few months of tinkering, testing, and sighing at hallucinated clauses, we’ve concluded that drafting contracts isn’t GenAI strongest skill.
However, there are some specific scenarios where GenAI can assist in drafting legal documents – and this isn’t a stance you often hear among document automation purists.
We also believe the real strength of GenAI isn’t in drafting—it’s in enhancing a myriad of micro-processes and steps within document automation itself.
GenAI won't replace document automation. It will amplify its capabilities.
This is the article outline:
Avvoka’s GenAI and document automation philosophy
GenAI for drafting contract clauses or templates
When GenAI could help in drafting legal documents
GenAI’s role within Document automation platforms
Conclusion: the future of GenAI in document automation
Avvoka’s GenAI and document automation philosophy
If you know us by now, you’ll notice we stay open to innovation while maintaining a practical, grounded, and realistic approach.
The truth is, not all AI brings meaningful enhancements in the domain of document drafting and automation. We take a threefold approach to separate improvements from hype.
Research the actual capabilities of GenAI – What can LLMs do well right now in terms of general functions? For example, clustering, tagging, identifying errors, and recognising patterns.
Identify processes where AI can add value – What processes in document automation platforms or legal workflows can GenAI improve? They don’t have to be large processes – they could be small steps –i.e. adding files to a folder
Add small but progressive improvements – Incorporating GenAI into a solution doesn’t need to revolutionise how users interact with it. Most of the time, it’s the small, incremental, improvements that add the most value.
With these caveats in mind, where can GenAI add value in practice when it comes to document automation?
GenAI for Drafting contracts clauses or templates
Probably the most discussed application of GenAI to legal is drafting contracts from scratch, starting from a prompt: "Draft me a shareholder agreement”.
It does seem easier than the hassle of preparing and setting up a template, then answering a questionnaire in a document automation solution. And it seems to be a compelling use case for those who believe that GenAI could replace document automation.
Here’s why we believe drafting isn’t GenAI’s strongest capability.
Why GenAI is not very good at drafting legal content
There are a few disadvantages of using GenAI to generate legal content from scratch.
Different outputs
LLMs are designed to generate slightly different outputs each time, even with the same, carefully honed, prompt. Reducing the "temperature" helps, but it doesn't fully solve the problem.
The legal industry is unique in that nuances in form and vocabulary are crucial and can make a significant difference. Literature is another field where this holds true.
But the difference is that consistency in literature is a weakness, whereas it’s a virtue in legal, as Alex Hamilton pointed out.
A couple of issues with the lack of consistency that GenAI causes:
Potential for more review
If the changes risk affecting not only the form, but the substance, lawyers need to review the draft carefully (more carefully than they would review a document automation generated document, but more on that later).
Lack of branding force
Two lawyers within the same firm and practice area who produce contracts that differ in wording and style can end up puzzling and frustrating clients. Form is a strong branding element. Consistency is perceived as predictability and stability.
If you’re in the story-telling business, consistency is a weakness. It’s boring. If you’re in the contracting business, consistency is a virtue.
Alex Hamilton, CEO Radiant Law
Verbose
GenAI outputs tend to be overly verbose. AI-generated content has low fact/high word count ratio. In other words, it contains minimal factual information and a lot of unnecessary text.
And contracts should aim for brevity – being “short, clear, reasonable and relevant” an advantage.
Hallucinations
There is, of course, the well-known issue of hallucinations—content that is entirely fabricated, delivered with the same smoothness as a five-year-old telling a white lie.
Studies show that GenAI still makes mistakes quite often, with hallucinations occurring 17% to 34% of the time when using AI-powered legal research tools.
While this is a bigger issue in legal research, it can also cause problems when drafting contracts – for example, if AI references a law in the contract that doesn't actually exist.
Plausibility not correctness
AI-generated content may sound plausible because it functions as a next-word predictor.
What it delivers is very sophisticated “probabilistic guesswork”, not an interpretation of "legislation, regulation, and case law, in order to deliver an informed legal solution”.
You can ask GenAI to draft a Loan Agreement from scratch. And it could do a plausible job – but the way it functions means that it could decide to “add the word <<not>> in an errant spot where it may be statistically very probable”, not necessarily generate a legally and commercially consistent contract.
In other words, you can’t be sure – until you check it - if the AI-generated contract is:
- Internally coherent – it doesn’t contain clauses that contradict each other;
- Legally sound – it doesn't include provisions that are unlawful. For example, if a subsidiary guarantees the obligations of its parent, this might constitute unlawful financial assistance that clashes with the UK Company Act 2006;
- Commercially sound – it aligns with my objectives, does not introduce unnecessary issues that could lead to lengthy negotiations, and takes into account the outcomes of previous negotiations or general market trends.
The Result: More time – and energies - spent reviewing
Lack of consistency, verbosity, the potential for hallucinations, and the need to check if content is legally and commercially correct means that lawyers need to spend more time reviewing AI-generated content – compared to reviewing a contract drafted from a template.
Also, as humans, we are not particularly good at checking “walls of text” , especially when they sound plausible. Plausibility dulls our critical thinking, as much as that polished, confident tone ChatGPT uses.
As a lawyer from the European tradition used to working from curated templates, it even scares me to have a first draft produced by a machine where I cannot understand its provenance, cannot trust that it doesn't conflict itself and cannot determine much around context
Jack Shepherd, Senior Principal Business Consultant, iManage
Advantages of document automation
Document automation is template-based. The template serves as the foundation for creating contracts, that can be refined or tailored later.
This approach comes with a few advantages.
Consistency and Safety
The final output of document automation is always consistent.
Templates are drafted and checked by lawyers, often by multiple professionals.
This reduces the risk of deviating from what is acceptable both legally and commercially. In a way, document automation is safer by design.
Traditional document automation tools will give you precise a pre-approved language every time
Ryan McLead, CEO and Founder Sente Advisors
Better outputs
There is value in refining a specific version of a contract over time, rather than starting from scratch. Templates are built progressively—they represent a "best of" collection drawn from tested precedents and compiled into a new artifact. To put it a bit more dramatically, these templates have “withstood the test of time”. They have benefited from feedback both within the firm and from counterparties. Over time, this process strengthens the template, even though market positions may evolve and change.
Faster Review
Reviewing documents generated by a document automation platform is faster than reviewing those created by GenAI, for the reasons outlined above. Since the template has already been thoroughly vetted, the review focuses primarily on the specific details that have been entered into the questionnaire.
Standardisation
Standardisation means to establish a single version of truth for an agreement—one that is widely accepted within the market. This brings a number of benefits: reduced negotiation costs and lower risks of non-compliance. We’ve explored the value of standardisation in legal in more detail here. Having a template is a conditio sine qua non of standardisation – in other words you can’t have standardisation if, with GenAI, the content changes every time.
AI-generated legal content undermines the very essence of standardisation.
When GenAI could help in drafting legal documents
Is GenAI entirely unsuitable for drafting legal documents? We don’t think so, and our perspective on this is somewhat unconventional (though it might align with the team at Radiant Law).
There are scenarios where document automation templates fall short. For instance when they don’t address a very specific use case. If no precedents (in the UK sense, meaning previous deal documents) are available to rummage through, novel content is required.
Typically, lawyers handle such cases by drafting from scratch. So, why not explore the potential of GenAI in these instances?
When creative input is necessary, GenAI can prove valuable, especially when used within the framework of Retrieval-Augmented Generation (RAG) (see our box) —a technique designed to reduce the risk of errors like hallucinations.
What is Retrieval Augmented Generation
Described as a “mysterious emerging behaviour”, this consists in training the model with a series of prompts while providing it with a database of information from which it can fetch answers.
In other words, you can give the LLM a list of questions with answers for it to learn from. You also provide access to a contextual database where to get more information (for instance, a legal database).
The output still requires careful review, but it is significantly less likely to hallucinate compared to simply inputting a prompt into a tool like ChatGPT.
GenAI Drafting Use Cases
Here are a few practical examples where GenAI can provide valuable assistance in creating legal documents from scratch.
Custom Indemnification Clauses
Indemnification clauses may need to be customised to cover very specific scenarios.
The creative and brainstorming abilities of GenAI can be quite useful in this context, for instance by suggesting alternative risk-sharing models. Of course it remains the lawyer’s responsibility to review these suggestions, but somehow the creative input GenAI provides sort of mitigate the risk and reduce the time spent on review.
Risk Factors in Private Placement Memorandums
Risk factors in PPM need to be tailored to a specific company. GenAI can provide a useful starting point by generating a list of:
- Generic risk factors: these are quite standard and could lend themselves well to templating (i.e. litigation exposure or insurance coverage)
- Industry-specific risks: these are the risks where GenAI creative input could be useful as they depend on the context in where the company operates. (i.e. environmental or supply chain risks)
This is an area where precision in wording is less critical compared to contracts – so the risk of losing the nuances is less strong.
Settlement Agreements
Settlement agreements more than other contracts require a creative approach to balance the parties interests and needs.
GenAI can come up with alternative financial arrangements, or dispute resolution mechanisms.
Compliance and Regulatory Documents
Compliance and regulatory documents can generally be handled by document automation.
There are two cases where GenAI help may come in handy.
No available template or precedent – policy is so specific that the firm does not have previous documents available - so starting from scratch is the path of least resistance.
Need for industry-specific or creative input - For instance, ESG, DEI policies – they need to be drafted by looking at legal requirements that are more conceptual or high-level rather than prescriptive (e.g., frameworks or guidelines).
Again, this is a domain where precise language is less of a concern than in contracts.
Caveats
Obviously hallucinations remain an issue, even when mitigated by RAG techniques. Lawyers must still review GenAI created content and ensure that is legally and commercially sound.
In summary, we believe:
- Document Automation beats GenAI when templates are available.
- GenAI beats starting from scratch when templates or precedents aren’t an option, provided its outputs are treated as suggestions rather than a finished product.
GenAI’s role within Document automation platforms
There are other processes and use cases where GenAI will not replace but enhance document automation. Of course, these might be seen as ‘boring’ use cases! And that’s exactly where we think AI delivers the most value. Yet they are often overlooked. “Probably because people jump to the most complex things instead of trying to fix the mundane...”(Jack Sheperd)
Automating Template Setup
GenAI excels at recognising patterns, which makes it particularly useful for automating templates in document automation platforms.
GenAI can streamline this process by:
- Identifying variables within a document, for instance name of the parties, or conditions that have to appear
- Transferring variables into the platform: Automatically mapping identified variables and conditions into the document automation platform
- Linking variables to a questionnaire: Associating the identified variables with relevant questions
Some platforms are already leveraging GenAI to perform these tasks, making template setup more efficient and accessible – our SmartAutomation feature does just that, for instance.
Consolidating templates
AI is also quite effective at identifying similarities and differences between documents. One use of this in document automation is leveraging GenAI to consolidate more versions of the same template.
Take two templates of an employment agreement, one for US and one for UK, that look the same with the exception that the first contains an “At will” employment clause and also a different Governing law and jurisdiction clause. It would make sense, from a knowledge management point of view, to consolidate these two templates into a single one.
GenAI can simplify this process by:
- Merging the templates;
- Automating them within the platform;
- Linking alternative clauses to specific questions in the associated questionnaire (where is the employee based?)
Answering questions
AI is great at answering questions by retrieving information from a defined database.
This ability opens up several use cases. Within the domain of document automation, we found two:
Use Case 1: Chatbot Support
GenAI can act as a support chatbot, looking for answers within a contained database like a knowledge base, FAQs, or policy documents.
At Avvoka, for example, we created an Operation Builder—a tool that helps users build operations and formulas within Avvoka by simply describing what they want to achieve.
Use Case 2: Auto-answering Questionnaires
GenAI can help with document automation by answering questionnaires using data extracted from related documents.
For example, it can fill out a Loan Agreement questionnaire by pulling information like party names and loan amounts from a Term Sheet.
At Avvoka, our Draft Assist feature does exactly that, streamlining the process and saving time.
Finding mistakes
AI is also quite effective at spotting deviations within a given context – essentially identifying "mistakes" (a task also lawyers are known to excel at, of course).
At a basic level, AI can detect typos, but the potential goes beyond that. For example, in mass document generation, when hundreds of documents are created by uploading an Excel file with data, AI can quickly identify deviations from a pattern within the dataset. If 100 employees are each assigned 100 shares and one employee is given only 10, AI can highlight this inconsistency.
The best solutions we’re seeing right now are additive. They’re ones that are drawing on different types of machine learning, as well as other types of technology to produce something that is robust and genuinely useful.
Nicola Shaver, CEO Legal technology Hub
AI for labelling and categorising
AI is very good at labelling. This use in document review isn't new. In document automation, automated labelling could play a key role to get structured, data-rich, and easily searchable documents.
For instance, AI can automatically label contract clauses, by identifying the type of clause (e.g., “this is an indemnity clause”) or by evaluating its content (e.g., “this is a contentious clause”).
The potential uses of this are extensive. We’ve identified two key scenarios:
Knowledge Management
If documents are correctly labelled at the clause or document level, it becomes much easier to organise and retrieve information; documents are labelled “at source” within the document automation platform and can flow into – an integrated- document management system in a structured way. For more on the uses of AI in knowledge management, see this excellent post.
A curated library of contracts and clauses reduces the need to rely on GenAI to draft content from scratch – it should be always possible to find, if not a template, a precedent or a clause to refer to.
At the same time, while GenAI can become instrumental for Knowledge Management -helping with labeling and categorising – good knowledge management could improve how GenAI works.
Quote: Legal KM is more important than ever in the era of gen AI because legal professionals rely on curated data sets to train gen-AI algorithms. Michael Korn, Legal Knowledge Manager McKinsey
Negotiation
GenAI could also be leveraged for negotiation purposes. A couple of use cases:
- Clause recommendation — Labeling clauses could provide the training needed for AI to suggest alternative clauses, for instance fishing from a curated internal database
- Flag contentious clauses – By learning what constitutes a contentious clause—based on previous negotiations—AI could automatically flag problematic clauses in a contract submitted by another party.
Conclusion: The Future of GenAI in Document Automation
GenAI isn’t here to replace document automation—it’s here to make it better. To put it like Nicola Shaver, CEO at Legal Technology Hub, the best uses of GenAI are “additive”, they enhance and build on the power of existing solutions – including document automation platforms.
This post ends here, but it’s far from the end of the story. AI keeps growing and evolving as we figure out new and exciting ways to use it to enhance document automation.
