Subscribe
Share
Share
Embed
In the Original Thinking Podcast, experts and academic colleagues discuss their latest research and original thinking at Alliance MBS.
For a list of our latest webinars, news and useful business content please visit ambs.ac.uk
>>:
I'm now joined by, uh, John Toon from, from Beever and Struthers and Leonid Sokolovsky from the business school here. Um, first of all, just introduce yourselves and then we'll talk further about this subject.
>>:
Yeah, sure. Um, so thanks very much. Thanks for having us on. I've worked at Beaver and Struthers for just over a decade now. joined the practice as an audit manager. But, um, over the last two and a half years, my role has completely pivoted and changed. So I now have a job title, which is Tech Strategy Lead, which is one of those nice, fancy made up titles that marketing people come up with. But essentially what that means is I'm really responsible for looking after, like, where do we go forward as a practice, and how do we make sure that we make our staff's lives as easy and simple as possible when they're engaging with technology and delivering service to clients? And also, how do we do that for our clients as well? So there's an opportunity there for us to provide a service and a consultancy service to them, to help them to improve the way that they run and operate their businesses as well.
>>:
Thanks, John and Leo.
>>:
Jim, thank you so much. It's fantastic to be here. I'm a lecturer in accounting and finance here at the Alliance Manchester Business School, and I'm also employed as a research associate on the research project that looks into the impact of generative AI on professional judgment. So University works together with Beever and Struthers, um, on this case. And it is sponsored by ICAS, Institute of Chartered Accountants of Scotland as well. So my role within this project is an associate. I spent probably 2 or 3 days a week within the firm talking to people, um, looking into the adoption cases, conducting interviews, um, observations, ethnographic type of work.
>>:
Just tell us then, without going into too much detail, what actually what you're finding. I suppose at the moment in, in this project.
>>:
I think it is very interesting because, as we say, generative AI, if we look, you know, for the past 2 or 3 years, this is probably one of the biggest technologies that has come out. It's a big bang, I think, for professional service firms, which is about this application of highly specialised sort of solutions to client problems. I think for them it is very interesting. So in terms of, I think, findings, Jim, the impact on professional judgment. First of all, how this technology is used. Some people use it maybe more as an advanced search engine to retrieve knowledge. It's very good at that sort of, you know, advanced Google search. But another way to use it as well. And this is very interesting what we're picking up from some of the advanced users. You can use it as a sparring partner. So let's say I come up with a set of assumptions or solutions for the client or suggestions, and then I can bring generative AI to further enhance that or to see whether there are any blind spots.
So I think for professionals and this is you know, we're talking, of course, accounting and finance auditing. But I think for professionals in other areas as well. So you can on the one hand, you can use generative AI to retrieve some of that knowledge, build up the foundation, come up with a more informed judgment. But on the other hand, you can almost do the other way around. Once you have the judgment, you can use it well as a sparring partner, I suppose to really challenge your assumptions, you know, second opinion type of scenario. So I think that's really interesting and challenging.
>>:
Assumptions is really strong, as you said there. You know, you're looking all the time at how the firm you can embrace digitalisation in most best way possible for your firm. And yeah tell us about the day to day. This must be a huge issue for your firm.
>>:
I joke with people because, you know, like I say, I took on this new role about two and a half years ago, and of course, ChatGPT kind of came into our world just over two and a half years ago. And prior to taking on this role, I had to write a strategy for the firm to talk about where do I think we're going to be going from a technology point of view. What kind of vendors or things should we be focusing in on? What should be, um, you know, what should be the key parts of what we're going to focus in on? And there was no mention of like, large language models at all when I wrote this, because I wrote it about three years ago. And then, of course, I got into the job and pretty much ripped up the strategy on day one and sort of like start again, because we've got this thing called ChatGPT, which is like making waves. I think what's really interesting around ChatGPT, in particular of the large language models, now that we're starting to see in the market, is that historically, as accountancy firms in particular, we've typically been led down our technology paths by the vendors that have been selling the technology to us.
That's not always the case, because, I mean, there is some stratification there in terms of the big four, firms have typically invested and built their own technologies. Um, but for the vast majority of the accounting segment, um, we've been led by what we can buy from the people that we can buy from. Um, and so it's been really fascinating to see how ChatGPT is having an impact on that, on that, because it's requiring a bit of a change of methodology, and it's requiring us to, um, think a bit more carefully about, uh, what the art of the possible is, which we're not necessarily very good at because we're not particularly creative people sometimes. Um, but also about how do we then apply it, you know, and Leo was talking about, you know, you can use ChatGPT almost like as a sparring partner. I would say in some cases that's quite an advanced use case still, because we're still finding people really struggling to identify absolute killer use cases for a technology like this.
Um, aside from kind of maybe that kind of advanced Google search type thing that we're starting to see. I guess because we are sort of two and a half years or so in from, from ChatGPT launching. We are also in that kind of like Gartner curve of sort of seeing people, you know, place a bit of resistance against this technology now as well and sort of say, actually, I've done this and it's rubbish. It doesn't work. And actually, when you kind of dig into the detail of what they've actually done, it's actually a skill issue for them. Not actually the technology not working.
>>:
How can academic knowledge help you?
>>:
I think for me, what's really interesting, I mean, again, we joke. We joke sometimes with the researchers is, you know, not just in this project but with the KTP is that they really drill into the detail of absolutely everything. And you kind of expect like accountants and auditors to do the same. But, but I think because we're commercially driven, you know, we can't, we can't invest a ton of time and sort of saying, oh, is that comma in that sentence in the right place? And, and should it be a comma or a semicolon or whatever. And so they get they get hyper specific about the detail, which is really great because that allows us to kind of step back a little bit and maybe think a bit more about where we want to go and be a bit more creative, which, as I said, is sometimes a challenge. The other thing that's really great is that they can come to us with this broad range of experience and this knowledge and things that they're picking up from, from other places. And, um, and then sort of come to us and challenge us around that and sort of say, look, we've seen this research piece from, I don't know, I think Leo picked up one from about 1983 or something, which was sort of talking about not generative AI, obviously, but AI and the use of that, but how that can then be interpreted into our use case currently within our industry.
And, um, and that's just something that in our world, we just don't have that, that that broad breadth of ability to kind of go off and research and come back and present some findings and then present some challenges for us to kind of overcome.
>>:
And, you know, given the speed at which AI is developing, it's interesting. Its almost taken us to take a step back a second to see, you know, it's not okay. You know, you're trying to say technology is racing ahead. There's a sense in which the principles or values of academic rigor still have a real place here, don't they? Despite the speed at which this is happening. Maybe trying to explain myself here, but you know, in terms of the benefits you can bring.
>>:
Yeah. Yeah, absolutely. Jim, this is wonderful question. And just to add to what John was saying, I think the value of an academic, let's say, coming inside of an organisation, professional service firms, which is about, you know, the work that these people do, it is in the public interest. Audit is a public interest function. And it of course, as John said, it is also a commercial organisation. So things like, you know, costs matter, efficiency matter. You know, you want to have clear link to the bottom line based on all these projects that are conducted. And I think as an academic, as John said, one thing that you come in, you're not really a consultant, but maybe one thing you can bring in, you start thinking differently about maybe some of these issues or have you thought about that broader implications type of thing. So for instance, when we talk about generative AI. One question we could ask does it impact the way people interact with each other? You know, social interactions, a big part of professional work. We know that.
So, you know, let's say as a junior colleague just starting. Maybe you don't feel very confident. Sometimes you're afraid to ask a question. So maybe rather than asking your senior manager or a friend, you know, you go to ChatGPT and then it gives you this a very convincing answer, which sounds really expert based. And then maybe you are likely to use it, but then at the expense of other wider social interactions. Jim, in terms of what I think, I absolutely agree with you because in some way the technology is developing so quickly. I mean, in a few months from now, we might be looking at a very different landscape in terms of use cases, in terms of applications, in terms of limitations. You know, things like people would say, oh, ChatGPT hallucinates a lot or produces wrong answers, but that might not be the case in a few months time. So I think then the value of academic work is we, um, you know, and that's maybe where the theory is important because you try to build something which endures and something that whether it's a research report or an article, something that people can pick up 10 or 20 years from now and say, well, this was interesting. This was exciting. So a little bit like John said, you know, even work that has been done even decades ago, you can say about a different type of technology, you can read it today and it is still relevant because ultimately, you know, there are a lot of commonalities there.
>>:
I, for one, think, you know what, really what you need to do is in industries, you need to drive knowledge sharing and experience forward. And in particular now, because we seem to be in this position where we're not being vendor led, what the industry is crying out for is like, show me how to use this technology, give me some use cases that I can take away and I can take into my business because I don't have the time to think about these, or to experiment or to create these things. And I think again, you know, we talked about this. This is the commercial challenge of being in an organisation that is there to ultimately make profit. Um, you know, and if you are engaging with a, um, with a consultant, you know, they would come in with a set of objectives which are, you know, let's get to here. Whereas when you're engaging with the academic team, it kind of pivots on its head because they don't really have a set of objectives other than to get some academic research out of this. They're not bothered really, if it fails.
You know, and we completely come out the other side of this with nothing. Or if we're incredibly successful, because what they get is they get to document that, that sort of journey almost and say, here's, here's the impact that we've seen in this, in this, you know, almost like this petri dish of accounting that we've had, you know, in here in Manchester where we've had experimenting on stuff.
>>:
I was struck by your use of the word, the art of the possible. You hear a lot, everything's possible with AI. It's almost limitless in some regards if you start to really think about it.
>>:I was at an AI conference last couple of days down in London, and, um, you're one of the presentations I saw, and I haven't, I haven't fact checked this, but one of the things they were saying was that with the with the incredible competition that there is between the various, um, people involved in this, whether it's OpenAI or Google or, you know, um, you know, DeepSeek from Chinese or, and a whole bunch of other things that are out there is that these models now are so competitive that that, you know, the model that is the most effective is only the most effective for about two and a half days, and then it's replaced by another one, um, from, from someone else. And that's the pace of change that we're in.
>>:
How do you how do you see the relationship developing between the firm and business school now?
>>:
So I think especially for a place like Manchester, you know, the Manchester method, I think we've always been about this close collaboration with industry, with firms interested in what's going on.
>>:
You know, if you go back the school’s 60th anniversary this year. You go back in the records. 60 years ago, we were doing a lot of work with Manchester businesses, industrial businesses, you know, going into those firms. Today you're doing the same thing, but it's yeah, you know, and you're going into a business talking about AI and technology. So it's, you know, the same principle if you like.
>>:
And I think, Jim, we also used to have as academics, for instance, you know, you would write in terms of research output, you would have one research output for the academic audience. You would have another one for practitioners, you know, what does it matter for me if I'm working in practice, I'm employing. How does it enhance my work? And then you would have another output for students as well. So, you know, teaching is important because you will have a lot of stresses, a lot of members of the firm, a lot of colleagues are very young people. They are, you know, they've joined this profession and they want to be, um, a part of the profession which is vibrant, which, as John said, has a future, which has this, you know, art of the possible. And that's important as well. I think we need to equip our students as well.
>>:
Well, I'm glad you mentioned students because that's another part here, isn't it, that we of course we have. We're sitting in a university with thousands and thousands of students, and we can learn from them as well how they are using adopting these technologies as well. I guess that's part of this for sure. Collaboration.
>>:
Well, I think even we look within the firm in terms of what kind of people are more likely to use ChatGPT. You will have young.
>>:
>>:
Yeah. Again I this AI summit for the last couple of days and um, you know, one of the things that came up is, you know, we're talking a little bit about skills and, um, you know, I think, I think one of the things that we're probably going to start to see, and I mean, we must admit, we haven't really thought about this conceptually, you know, as an organisation properly yet. But I think one of the things you're going to start to see is like when we are recruiting you. um, your students, whether it's from universities or from colleges or from schools and stuff, um, you know, is that we're going to want to understand what their familiarity and capability is with a large language model, you know, and are they able to prompt it in the same way that we I guess in the good old days, we just assumed that people could use Google, right? Um, but maybe we're going to have to start to test that, because it is going to become a really fundamental part of this. And, and I think, um, you know, one of the objectives of what we're doing with, with Leo and the team is to understand what's the impact of using large language models on professional judgment, you know, and whether that either enhances it or erodes it.
Um, and I think, you know, again, one of one of the interesting things that we've already started to see, I don't think we're kind of giving away anything that hasn't been sort of said or published yet, but is that is that, you know, professional judgment is almost like graded in different levels, and that comes with the level of experience, you know, and so, so a student coming to us who doesn't have a huge amount of experience, they may have done an accounting degree, which gives them a great sort of bedrock of and the foundation of understanding of certain things. Probably doesn't have that lived experience of working in an accounting practice, dealing with clients day to day. When you build that up over time, that then starts to have an impact on the way that you make judgments, the way that you understand the impact of the other challenges that we have in our space.
>>:
The point I'm making judgment is absolutely crucial. And as the technology becomes available, it's the importance of making those judgments. You're going to have to come back in a couple of years time and tell us what we're up to.
>>:
Well, it won't be us. It'll just be robots.
>>:
Well, I'm conscious of the time that I want to thank you both for coming in today. And fascinating topic. Uh, and, uh, look forward to hearing further about the, the collaboration between the firm and AMBS in the future. Thanks both.