Subscribe
Share
Share
Embed
In this episode, Aurelia and Chris talk about the Harvey-Reddit debacle, whether it is really that big a deal, and more broadly about cultural change in law firms.
Links:
- The Reddit post
- Watch on YouTube
- (00:00) - Introduction
- (01:36) - Harvey's value proposition
- (05:03) - Cultural Shifts in Legal Tech Adoption
- (08:44) - Adoption and retention: is it really that bad?
- (12:13) - Change management
- (15:02) - How important is founder legal experience?
- (17:20) - Communication between product and lawyer
- (19:26) - The importance of being able to empathize with users
- (22:23) - Will / should law firms be building their own tools?
Links:
- The Reddit post
- Watch on YouTube
Creators and Guests
What is Too Legal; Didn't Read?
In this podcast, we talk about what is hot and what is not in legal tech, and our journey building one of the first AI-native legal services in the UK. Hosted by Tacit Legal co-founders, Aurelia Butler-Ball and Chris Bridges.
Hello. Welcome to episode four of TLDR with me, Aurelia, and Chris from Tacit Legal. Chris, on this beautiful sunny autumnal day, Tell us, what did we talk about in today's episode?
Chris:So we talked about the Harvey Reddit debacle that's probably taken over everyone's LinkedIn feeds. Whether it's really that big a deal and the role Harvey, Legora, others like them serve in creating the right environment for change in both culture and ways of working in law firms.
Aurelia:I absolutely love this discussion. So let's get it started. LinkedIn absolutely blew up last week over a damning Reddit post made by a seemingly disgruntled former Harvey employee. In this episode, we want to talk a bit about that, but really more broadly about the state of the market and how those remarks made fit in and whether they're a big deal or not so much. Now the actual post, the key points it made was that Harvey really had zero value add.
Aurelia:It was just a wrapper on top of GPT. Its adoption rates are poor, its client retention rates are poor, and just the co founders of Harvey don't have sufficient experience in legal. Now these really hurt Harvey. There was a response back from the co founder, but let's have a look at really what those points raised mean. Chris, talk to us a bit about Harvey.
Aurelia:It's a platform. What is the value of a legal specific AI platform versus just using GPT? What's the value add?
Chris:Yeah, so really, if you go back like a year, eighteen months or so, Harvey originally set out saying that they were going to kind of fine tune their own models, they were going to build a model on top of other models, but much more the process you go through is a lot more of a technical process effectively retraining the model on legal data sets. Back then there was a whole debate about whether that was the right thing to do or not. Turned out the industry generally decided that wasn't the right thing to do and actually you got worse results a lot of the time by doing that. So the alternative route is you basically just build on top of what's already there. And if you treat GPT or Anthropic Claude or whatever as a language engine, it's something that understands language and can write language, you can do what you want on top of that.
Chris:Like, you can build up your own kind of domain knowledge layer. You can put like There's loads of value adds you can do on top effectively that are legal specific, whether that's kind of a knowledge angle or how lawyers like to work angle. So yeah I mean the the claim about zero value add. Nothing zero value add. Otherwise no one would be buying it.
Chris:But I guess what he's getting at is, you know, it is just just, in inverted commas, the latter, they've got a UI on top of it. They've got maybe got some knowledge on it, some custom kind of system prompts and instructions. But that's not necessarily a bad thing. And actually, most legal tech is that now. So that in itself isn't a bad thing.
Chris:Yeah, I guess think Harvey, Ligora, etc, that they are platforms in my head they are platforms for R and D and they're a platform for culture change. So it's not necessarily about huge efficiency gains, it's about the hearts and minds of the lawyers, getting them used to working with AI, doing that in a safe environment that you know the law firm's comfortable, it's secure and they can govern it and they can put their own safety net around it. That doesn't sound like zero value add to me. It's just not the value add that that poster understands and obviously has never worked extensively with law firms and the way they like to work and the way they like to buy and the things they look for. Yeah, I mean, only have to look at like NetDocs and iManage as an example.
Chris:Like they are wrappers in the same way around file storage, like big deal, like you wouldn't think that is something that you would ever be able to have complete market control of which iManage and NetDocuments kind of do. In the same way people don't just use a shared folder or you know Microsoft SharePoint, they want a legal specific thing that works the way they like it to work and that's not a zero value add, that's just a different value that that poster is not necessarily understanding, I think.
Aurelia:I think that cultural point is really interesting because it makes sense that, of course, the first tool which is used most widely in legal is going to be a tool which most people are using in all of their other different parts of their lives as well, a chat tool. We're all using that in lots of different ways in our everyday life. And so that shift from using a tool like that into our work is not a very big one. And so this then is Harvey has got huge widespread use and attention. But then there is this gap because a lot of the hype around AI is around it, I don't know, replacing actual jobs that we do or replacing us as lawyers.
Aurelia:But Harvey's a million miles from that and has never claimed to be replacing lawyers or work that we do. No.
Chris:Yeah, exactly. Law firms aren't like, they're not stupid. They're not spending hundreds of thousands of pounds and half in Ligura for no reason. Most of the people I spoke to, they understand that it's not going to massively massively change the way they work. They understand it's not going to suddenly drive huge efficiency gains and change pricing models and it's not.
Chris:It's a path to that. It's about bringing lawyers along for the ride. And I think one of the claims made in the post was that it's only first year associates. Obviously the post is American because I've used first year associates. It's only first year associates, effectively year one trainees in The UK that are using it.
Chris:And that's not really that surprising, is it? Like the adoption would be higher in younger people in the same way with any tech generally speaking, younger people will be quicker to adopt it. And they're probably using it already in their personal lives. They're like, great, I'll just use it in my work life as well. So that's not that surprising really and I don't think that's that big a deal.
Chris:I think the true test for Harvey and Legora and others and it's not just their test on them, on the law firms that are implementing them. How do you take that enthusiasm in the younger population for new tech and how do you get your senior associates, your managing associates, your partners thinking in the same way? You know, then they are not going to suddenly just jump to whole workflows being automated overnight or very few will. Mean we kind of have but we are a bit different in our mindset. But most firms, you can't just say, right, 3,000 people, right, we're going to automate all your workflows.
Chris:You've got to bring them on the journey.
Aurelia:It's got to be a phased journey. Now I remember starting when I started as a trainee, and all the senior partners at the law firm at that time, they all had legal secretaries they were dictating and there was still a room of legal secretaries typing out all the attendance notes and letters and everything else. But there was recognition at that point that those efficiencies had to be made. And so us juniors weren't given that legal support. We were given, you know, computers and the internet and keyboards and things like that.
Aurelia:You could So, Revolutionary. See I know, it felt quite revolutionary at the time. But that was the phased approach. And I guess this is exactly what be happening is that as those juniors make their way through the law firm, so will the tech be going with them. But what do you think that real kicker point around adoption and retention is?
Aurelia:Why is it so slow? It can't just be because, I don't know, older people aren't that used to using tech because they are. Is there some wider resistance, do you think, about adopting these tools?
Chris:35% adoption for a new tech solution. So that was the claim made in the post. It's actually not that bad in law firms, like for something that's optional in the sense that you don't have to log on to it to do your job. As in time recording, you're going to get 100% adoption. And even then you're not going to get 100% adoption.
Chris:I'll come back to that in sec. You're not. Or document storage, like those kind of things you would expect much higher adoption. But something like Harvey or Legora is not necessarily useful for everything and a lot of the time it will slow you down. It genuinely will slow you down.
Chris:A lot of the time it will slow you down because you are changing the way that you want to work or you are learning something new at the same time. A lot of the time you could have just done the job anyway, unless you kind of built workflows and whatnot, but that's way down the line. But I think adoption rates, for, like I said, time recording, it's quite funny. Winston Weinberg, the CEO of Harvey, responded with, oh, we've got a 77% seat utilization, which I think is rubbish as well. 35% probably is true, but I don't think it's that bad.
Chris:77% seat utilization, that means 77% of people have logged on. That's not adoption. We've got a new shiny tool that everyone's talking about. I'm going log on and see what the hype is all about. What really matters is what happens after that point.
Chris:Know, if someone's just logged on, that's not adoption. But, you know, you wouldn't even expect real with a decent metric, I don't think you would expect anything like month to month beyond like 90%. So there's always people out the office. There's people on mat leave. There's people on holiday.
Chris:Like you don't have 100% adoption on anything all the time. And even things like time recording, you've still got time recorders that are writing their time down and giving it to their secretary. That still happens in law firms. So even time recording doesn't have 100% adoption. It should, like for law firm, know, three line whip, it should have 100% adoption, but it doesn't.
Chris:But yeah, I don't know. I just think it's one of those things that happens over time.
Aurelia:I think that curiosity point is super interesting where, yeah, someone might have logged on, tried it, dropped it. But there is the curiosity around AI and I think we find that with Tilda, we have lots of conversations with in house teams. They're just curious about what we're up to and what's out there. And that having those conversations and that education point still now. I mean, for us, it's really exciting because we, you know, it's great to have those conversations and bring people on that journey and be, ensure that they're excited as we are about everything.
Aurelia:But how do tools make sure people keep using them and not just try them and drop off? Do you think there are ways that we can do that?
Chris:Yeah. And I mean, again, this is nothing new in the tech world. And yeah, if anything, I just think it shows that the post has not been in tech or doesn't have a lot of experience in enterprise tech. So, you know, adoption in a small firm like Tass is easy. You know, I I'll go out, see our own buyer tools, say, right.
Chris:This is what we're gonna use. It's huge time saving. We'll spend an hour on a call going through how it works, benefits of it, yada yada yada, and we'll all use it the next day. Easy. When you've got 3,000 people to bring along for the ride, mean there's a whole profession that exists for this purpose.
Chris:Change management isn't just a made up profession. Law firms employ tens, if not hundreds of people to implement change. And I think a lot of people don't realize that when they see change management, they think that's just project management or just doing the technical side of it, but it's not. So much of implementing tech is about the people and that's why alongside most enterprise tech implementation programs, you'll have a whole change management program which is about how do you get it from 0% adoption, you know 0% of people using it in their day to day jobs to 40%. Then how do you get it from 40% to 60%?
Chris:And it huge it s a culture thing. It s a hearts and minds thing. It s a training thing. It s about giving people the right tools, the right training, the right understanding to figure out how to use it in their jobs. That's where, you know, earlier I talked about R and D and I think that's where the real interest is really in law firms at the moment.
Chris:Most law firms don't have a huge big picture plan of how LLMs are gonna or AI more generally is gonna play into their practices. But by getting lawyers using it, getting excited about it, going, oh my god, I just managed to get that thing that used to take me three hours done in five minutes. That's a use case, right? Now let's build something that's more structured around that, that other people can use. And yeah, so I think it's an iterative thing.
Chris:It's like a hearts and minds propaganda, whatever you want to call it, getting people using it, getting people excited about using it for the right reasons and then using their experience to inform what happens next.
Aurelia:Now another thing that the post said was about the background of the founders of Harvey. What do you think founders in legal tech, do they need to have a deep legal experience or is actually kind of tech first better for these types of products coming into legal?
Chris:I'm going to give you the classic Gloria answer. It depends. It does depend. I think it comes down to the claims you're making around domain knowledge. Take Tilda for example.
Chris:It is a very, very domain specific product. We are claiming we're making big claims about its domain knowledge. It's in playbooks included. We've got fifty years experience between us and that's how we've built these playbooks. So I think experience does matter then because you are making claims about domain knowledge.
Chris:I don't think it matters as much or if at all for things like Harvey or Legora. They're building a platform for lawyers to use and experiment on. They're not building like a point end to end solution. They're not claiming to solve a specific problem. You don't necessarily need domain knowledge to build a platform.
Chris:All you need to do is or I say all you need to do is quite hard. All you need to do is be able to have those conversations with lawyers. And it's just good product management. Is user interviews, understanding what users, how they want to use it, why they do not like ChatGPT, how can we make our thing better than ChatGPT for lawyers. There is much more of a iterative, I say extractative, I don't know if that's the right word really, that makes it sound like it's quite dark.
Chris:But it's about going on a journey and learning how the user wants to use it, not about building in ten plus years of partner knowledge or whatever. So yeah that's a long winded way of saying it depends.
Aurelia:I guess if then they don't have a legal background then yeah that relationship with lawyers and getting that feedback is going to be really important and I guess that's often where we see a bit of an issue. It's thought lawyers are quite inherently a bit too harsh on kind of outsiders coming into the legal industry to innovate, and don't really give those companies, developers the feedback that they need, they don't invest that time. Do you think that's true? Do you think that often we kind of, we see there's a bit of a gap between the tech first products and the lawyers and, you know, not enough information is going back between them?
Chris:It's an unfortunate reality that the profession is quite like that. They will instantly, even if you've only got two years of experience, believe your belt. A partner will take you more seriously because that's just the way it is. It's unfortunate it shouldn't be that way but it kind of is. I made a bit of a career out of that at our last firm and I was the bridge between finance and Fianos.
Chris:I was the bridge between IT and Fianos and marketing and Fianos a lot of the time and being able to break down those barriers a bit as a lawyer and as someone that understands those other domains. Honestly, I think that's a bad thing about the profession. We should be more open to talking about non lawyers. So me saying, oh, actually it is helpful to have two or three years on the about as a lawyer is more damning about the profession than it is about anything else, I think. Yeah, I do think unfortunately in legal tech it's important to have at least some legal experience as a leader just so you can go into a room and you've already earned some credibility.
Aurelia:I mean we have the wonderful Maeve who works with us on Tilda, qualified lawyer and handles prospective clients and client success journeys. I guess we had a we really understood that having a lawyer in that position was important and I guess it was to appreciate and understand the pain points that lawyers are feeling and in some way those issues which they can't articulate, but we have some understanding of. Do you think that kind of helps to bridge that gap?
Chris:Yeah, absolutely. I think it's really hard work trying to and this isn't just a legal thing it's really hard work trying to explain how you work to someone that's never worked the way you work. And it requires quite a lot of patience. Again, as lawyers we're probably not the most patient people and I think we turn off quite quickly if we don't get like an easy ride. I think the people who are I know you don't have to have legal experience for this, but you do have a bit of a head start.
Chris:But being able to riff off what people say is massively important in that kind of environment. So lawyer says this, you have to read between the lines. They've not given you the whole picture, they never will, they've oversimplified it because that's the way in their brain it's very simple. You have to be able to riff off it and go oh you just said X and Y, Right. So, does that mean you do A, B, and C?
Chris:Right. Yes, I get it. So, actually, you know, B, G, and E are an issue. Like, I get that. And then, it becomes a conversation and not like an interview, like a police interview.
Chris:Yeah. After you you you did your link to post the other day about starting an interview room, didn't you? But
Aurelia:Yeah. I did. Yeah.
Chris:I I think it'd be it it the dynamic is as as a lawyer, by the way, the audience, not as a not as a criminal. But it changes the dynamic of that process instead of being like what can feel a bit confrontational and why do you do it that way? And lots of questions. It's more of a conversation and riffing off each other. But you don't have to be a lawyer to do that.
Chris:You could just, you know, be one of those people that can understand what people are saying very, very quickly. And we had an experience like that recently with the consultant we used, didn't we? Yes. We played it to them and they're not a lawyer. Had some experience in legal, they've never done our jobs but they just got it straight away and it was like wow that's unique.
Chris:But yeah I think being a lawyer does give you a bit of a head start there because you just have a bit more understanding on you can riff off stuff a bit easier. But it's not impossible to do without legal experience as he demonstrated the other day.
Aurelia:Yeah and I think so looking big picture, do you think then that these tech legal tech companies founded by non lawyers and that always throws up some of the issues that we've discussed around trust and understanding and knowledge. Do you think we're going to see more law firms start to develop their own tools for clients to try and get some leverage on the issues that we've discussed. They're trusted, they understand the issues. Do you think we're going see more of that?
Chris:I think we've got to see some we've got to see more of it. Don't think law firms do generally want to go in that direction. A chat window is never in itself going to change the world. It's very flexible. It allows lawyers to not really have to change the way they work too much other than like: oh hang on actually writing up this email or summarizing this thing I could probably use now that.
Chris:It's lawyer first but they reach out for a tool because they know it'll be quicker. You're never going to get big efficiency gains that are measurable, that you can actually feed into pricing and stuff without taking that knowledge and then putting it into a workflow. Whether that's a tool that's client facing or internal facing, up for debate but those tools will be created and are being created to take this task, how do we make that task more streamlined, how do we allow the work someone else has done playing with Harvey to be you know leveraged at higher scale by more people? That's got to happen. It will happen.
Chris:It's happening but it's going to happen over time.
Aurelia:Yeah. I mean, feels really harsh, doesn't it? That Harvey who has, I think, taken the hearts and minds of this profession a huge way forward that we are now, many of us are using a tool in our everyday working lives, which I think is incredible in such a short amount of time. But now quite quickly, lawyers are turning around and going, well, is that all it does? Is that, you know, is it just a chat talk?
Aurelia:So there's appetite, hopefully, I think, for more tools. Mean, we're looking we with Tilda, look at contract review. But where do you think the market now is heading if people are looking for some more efficiencies to be gained in their legal work? What do you think is going to really kind of head the kind of best tools out there?
Chris:I personally, I think the best I've just gone with controversial things though perhaps, I don't know. I think the best tools that will be created will be created by law firms. I think it's the buy versus build debate and I say that because you can never get the same level of integration, slickness, of like client designed client experience the way exactly that you want it by buying in a tool. Like you've got to work the way the tool works. You've got to you know you might be able to change some colors but you won't be able to like full on brand it.
Chris:You won't necessarily be able to integrate in exactly the way you want with the other workflows you've got going on. So I personally think the best tools will be built in house. I think whether law firms currently have the capability to do that is a different question, the best tools will be built in house. But it may not be as quick or as easy as that. I think I do yeah eventually I do think some of the best tools will be built in house.
Chris:And then maybe some of them will get spun out as products in themselves, but I do think a lot of them will originate from taking the firm's own way of doing things and really doubling down on that.
Aurelia:And is that kind of external, say, workflow tools that are out there but then work very closely with the law firms and maybe upskill them or lend them some skills in order to tweak
Chris:So those bespoke workflow tool, off the shelf workflow tools absolutely will I don't think it's one or the other. You're only going to build something if you know it's going to scale and it's got the volume there to be worth it. That isn't a lot of things like people like to think it is but it's not actually that many things. There will be some use cases the way you can do that but a lot of the time you need a bit more flexibility than that. If a client's coming to you, like every client even on say a corporate job, every client's going to have a slightly different way of doing things.
Chris:They want stuff in a slightly different way. You can't build your own tool to cater for all of those scenarios. So I think those workflow tools, off the shelf workflow tools where it's kind of like a low no code type thing, they're always going to serve a purpose because I'd say a client comes to you with a half million pound project but actually their budgets only $300. It's kind of worth, well it's not kind of worth, it is worth looking at how can you automate some of that work in a semi throwaway way and doing that with a low slash no code or a workflow product makes complete sense because you don't even have to be a developer to do it. You just have to be a kind of power user designer workflow, bish bash bosh.
Chris:You have automated that part of that project, and at the end of the project, if it is useless, it does not really matter. You could throw it away. So I think those products still serve a purpose for that type of thing. And they're also great research ground. Know, got this idea, could we do it with a workflow?
Chris:What should the workflow look like? You can use that to create like a proof of concept or a MVP very easily. Might see you may end up building up one of those tools and then go actually we've got as far as we can with that. We really want to take it to the next level we're going have to build our own version of it. So yeah, I think they go hand in hand and it's not a one or the other thing.
Chris:I think most firms will use both.
Aurelia:We ourselves really lived the pain of trying to develop a tool inside a law firm when we were our big national law firm and that was one of the main drivers of us spinning out and starting TACIT in order to give us the kind of agility and speed to develop a contract review tool for clients that we have and we now speak to lots of law firms and I think it still surprises me how slow and difficult it is for them to come to the market with their own tool. There's a real appetite for it, isn't there? We see that, we have all those conversations. But the amount of times they say to us, my God, you're so far forward then. And we see some of the big law firms there kind of spinning out smaller subsidiaries to try and drive innovation and acquire some knowledge and existing tools out there to try and speed up the innovation from themselves.
Aurelia:Do you think we'll see more of that to try and for big law firms to try and innovate in this space?
Chris:Thinking carefully before I answer this one. Think the best innovation that so maybe not the best, but the quickest innovation will always happen outside law firms. Innovation labs have been tried before in law firms, but it just never really works. Unless the law firm is really willing to go complete arm's length, put them completely outside the fold, they can do what they want, You just don't get, you can't get that level of responsiveness and iterativeness and all of that good stuff. Like can you see a world in a big law firm where to start starting in a big law firm I'm talking about not saying necessarily not bought in later but starting in a big law firm would I have been able to develop version one of our product myself?
Chris:Absolutely not.
Aurelia:Hard no.
Chris:Hard no. Just wouldn't even be a starter and that's what's allowed us to move so quickly. But really because we can iterate so quick because we've got the technical expertise and the domain expertise together so we can just go. And if client comes to us with like a new feature request and we love it, we can have it shipped by the end of the week because I can build version one of it. I might hand it off to someone else to finish.
Chris:But you just reduce that feedback loop of what's the problem, understanding it, getting it into code, getting it into UI to hours instead of a week of a design sprint or whatever, which would be the kind of traditional way of doing it. But yes, do that. I think the quickest innovation won't happen in law firms. That doesn't necessarily mean the best innovation won't happen in law firms.
Aurelia:Very diplomatically said, Christopher. Look, we're out of time. This has been a great chat. Chris,
Chris:thanks Thanks for your
Aurelia:everyone for joining us. And despite the disgruntled employee post about Harvey, I don't think we need to worry about them, do we? I think they're doing all right. Apparently Harvey is, I think they've got dollar or 2, so it's
Chris:Yeah, and I don't think any of this is news to them. I'm sure they're fully aware of problem and the work they need to do and whatnot. It's a joint thing between the solution provider Harvey Legora and the law firm to kind of properly change, manage and win hearts and minds on using LLMs.
Aurelia:Yeah. Chris, thanks again. Thanks everyone for joining us. We'll see you soon. Take care.
Chris:Bye.