In the rush to embed AI into their practice, most law firms have not realised how disruptive this could be to the way in which the business of law has traditionally operated. If we replace junior lawyers with AI that can do their work, how will we train the senior lawyers we need?

This is a slightly edited, link-enhanced version of a post that first appeared in The Mint. You can read the original here.

Last week, I was speaking at a conference in Singapore on the impact of artificial intelligence (AI) on the practice of law. On my panel were lawyers from four different continents and it was clear that we were all falling over ourselves to incorporate AI into our respective law firms.

Since every firm was already using AI for drafting (from simple memos and client communication to basic contract drafting), much of the discussion revolved around how to do that effectively: the prompts we were using and the workflows we’d designed. Some had invested in due diligence products, using AI to review documents so that it could quickly surface risks in merger and acquisition transactions that lawyers could then address during negotiations and contract drafting. And we were all using AI for research, whether it was to quickly generate summaries of current positions in law or just identify issues that could apply given the particular facts of the case.

Improving Efficiency

For the most part, this drive to incorporate AI into as many aspects of the business of law as possible comes from the pressure we all face from clients to improve the efficiency of our services. But at the same time, these very same clients impose constraints on the ways in which we can use AI, prohibiting us from training it on any information they give us and often going so far as to say that even the queries we ask of AI should not contain any sensitive client information.

Whether it is because of these restrictions or other reasons entirely, it is fair to say that all my co-panellists were mildly disappointed with how AI was performing. In our cumulative experience, AI is still no better than a novice—requiring constant supervision and repeated instruction before it is capable of produce anything even halfway close to the quality that our clients demand of us. If this is the technology that is going to take our jobs, it is off to a pretty poor start.

When it was my turn to speak, I was blunt. I said that I’d been hoping AI would perform at the level of at least a 4th or 5th year qualified lawyer, but so far it was doing no better than a first-year associate. In order to get it to give me output I could send directly to clients, I had to spend time carefully crafting prompts—sometimes again and again—to coax out of it results that were useable. To be honest, its come to the point where it is quicker for me to just dictate the answers I need.

Knowledge not Intelligence

The ‘intelligence’ of current AI models comes from their ability to compress vast amounts of information into vector databases that are then used to predict the most appropriate answers to the questions posed to them. As a result, everything AI says comes from information that already exists. While this is helpful it does not encompass all that lawyers are expected to do.

When clients come to me, it is usually with questions that have never been asked before. They come to me with new lines of business they are looking to initiate to see whether they can proceed under existing legal frameworks. They want me to advise them on how existing laws apply to new circumstances for which there often is no precedent, and what, if anything, needs to be done to operate while still remaining compliant. These are the sorts of questions that any bright young associate will, with a little bit of guidance, be able to answer—but which AI struggles to get right.

Having said that, even at this stage, when AI is frustratingly underperforming its potential, I am convinced that it will inevitably improve to the point where it will replace junior associates—at least for some of the tasks we rely on them to perform. And when that happens, we will have an entirely different type of problem to deal with.

A Different Kind of Problem

In an earlier article in this column, I had pointed to how the practice of law is quaintly artisanal even to this day:

Law is practised today in much the same way as it has for centuries. Modern law firms may be large multinational organizations but when you strip away the veneer, even the most advanced law firm in the world is built around an inherently artisanal framework. At the heart of this structure is the concept that your legal education is not done until you have successfully completed an apprenticeship. Law firms leverage this by filling up the lower ranks of their organization with an army of paralegals, trainees and first-year associates.

These lawyers perform relatively mundane but important tasks of research and documentation in exchange for the opportunity to hone their skills on live client matters. For its part, the law firm benefits by having trained lawyers to perform simple jobs at a lower cost. This symbiotic relationship has existed for years and is fundamental to the way in which law firms are organized.

When it comes to nurturing good lawyers, the only way we know how to train them is on the job - offering them opportunities to develop, through observation, develop the skills they need. As a result, the first 3-4 years of a young lawyer’s life is organized so that she can spend time in the company of more experienced seniors in order to absorb, effectively by osmosis, an appreciation of the craft of the law. We work to provide her frameworks within which she can practice on real-life situations so that she hones her skills to the point where we are confident she can deal directly with clients.

Once AI gets to the point where it performs at the level of a 4th-year qualified lawyer, there will no longer be any need for junior lawyers. Firms will only need to hire lawyers who have the experience to supervise the output generated by AI to make sure their clients’ needs are addressed. But if we know of no other effective way to create senior lawyers other than by training them on the job, when AI replaces our junior lawyers, there will be no junior lawyers in the law firm to train.

A Tidal Wave of Disruption

As law firms around the world rush to incorporate AI into their business, I worry that they may not have properly thought through the long-term consequences of this. Since our current approach to training lawyers has stood us in good stead for centuries, its hard to think that it will no longer be relevant. If we do that, we will join the long line of industries caught completely unawares by the tidal wave of technology disruption.

We need to reimagine how we train our legal workforce so that even as we incorporate AI into different aspects of our practice, we continue to train our lawyers to do the sorts of things AI will never be able to.