This episode explores mathematical models of how people make decisions with Dr Gabriel Tillman. Gabriel describes his research into why people are worse at driving when someone is talking to them, as well as the parts of speech sounds people use to understand what they are hearing.
Starns, J. J., Cataldo, A. M., Rotello, C. M., Annis, J., Aschenbrenner, A., Bröder, A., Cox, G., Criss, A., Curl, R. A., Dobbins, I. G., Dunn, J., Enam, T., Evans, N. J., Farrell, S., Fraundorf, S. H., Gronlund, S. D., Heathcote, A., Heck, D. W., Hicks, J. L., … Wilson, J. (2019). Assessing theoretical conclusions with blinded inference to investigate a potential inference crisis. Advances in Methods and Practices in Psychological Science, 2 (4), 335–349. DOI: 10.1177/2515245919869583 |
The transcript for this episode was developed by Eugenie Dale.
What is Psych Attack?
Psych Attack focuses on the diversity of the domain of psychology. Join us for a relaxed conversation with experts discussing the topics they are passionate about in psychological research and/or practice. The aim is to better understanding the spectrum of human experience, the methods used in psychology, and the people attracted to working within it. The conversations will be of interest and accessible to novice and experienced psychology listeners alike.
Hosted by Dr Jasmine B. MacDonald (jasminebmacdonald.com.au).
Dr. Jasmine B. MacDonald (00:08):
Hello, and welcome to this episode of psych attack. I'm Dr. Jasmine B MacDonald. Today, Dr. Gabriel Tillman, and I explore how mathematical models can be used to understand how people make decisions. I hope you're going well and have settled in with a warm cup of tea. Gabe, thanks for joining me on the show today. How's everything going?
Dr. Gabriel Tillman (00:34):
No worries. Jazz. Thanks for having me, um, glad to be on your show. It's going well, you know, getting research teaching and everything else done
Dr. Jasmine B. MacDonald (00:42):
Excellent that time of semester, so I was thinking maybe we could start by you introducing yourself and telling us about what you do.
Dr. Gabriel Tillman (00:50):
Yeah well, I'm, well, Gabriel Tillman at Federation university in Australia. Um, I guess what I do here is I work in the psychology department. I teach into stats and cognitive psychology. Um, my research is in cognitive psychology, but more specifically in decision making and things like that. Um, I guess the, the main thing that you would say is that the decision making sort of a very special kind, like it's not, you know, whether you're gonna get married or what bus you're gonna catch. You know, this is one thing that comes up a lot with students is I'm like let's do decision making and they get all excited and then I'm saying, let's ask people if dots are moving left and right. And then they go, what is all that about? Yeah. So , I have a very niche area in cognitive psychology and you would probably call it perceptual decision making.
Dr. Jasmine B. MacDonald (01:39):
Okay. Nice one mm-hmm okay. So set the scene for me. Um, how did you get interested in this area?
Dr. Gabriel Tillman (01:47):
Dr. Jasmine B. MacDonald (01:47):
And, uh, what psych training do you have?
Dr. Gabriel Tillman (01:50):
So how did I get to looking, making people look at dots on a screen let's see. Well, I actually started, I don't think you know this, but I started my psychology training in Coffs Harbour. And, um, when I was there, my main goal was to I'm a clinician. So I was really into psychology as a degree to become a clinical psychologist. And so I did two years there and at Coffs Harbour, the Southern cross university there, the main thing that they have to supplement their psychology courses is counseling courses. So I actually did a lot of counseling and, you know, practical stuff. So their assessments were all practical and, you know, I was doing volunteering for on phone counseling and that kind of thing. And so I'm two years in to a psychology degree and I'm well on my way to thinking I'm gonna become a clinical psychologist, but what happened was just things that just kind of came up in life.
Dr. Gabriel Tillman (02:41):
I ended up moving back to, uh, the university in Newcastle and which is around where my hometown is. And basically there was a really strong cognitive and statistics department. Or you could probably just say lab there's, you know, some really good international researchers, uh, well known re um, internationally for cog psych. And, you know, I guess mainly some bayesian statistics stuff, which you could talk about later, but that stuff I wasn't really exposed to yet in my first two years of training. So I was, you know, learning about how to work with people, doing a lot of counseling and then all of a sudden I'm doing how the mind works and, you know, lots of hardcore stats and getting taught by some of the, the best researchers in the world in this area. And really what ended up happening was I started to really like it and, um, not, I don't know how I would've went as a clinic clinician, but I definitely had doubts about, you know, what, whether it was for me after doing counseling sessions.
Dr. Gabriel Tillman (03:42):
So there would be sessions that I'd have with people when I was like, Hmm, how, how am I as a counselor or as a clinician. But then when I started getting into this cog psych and stat stuff, I really enjoyed it. And I started, you know, coding in my free time and learning programming languages to get better at some of this stuff. And, and then really, it just took off from there and what happens, I guess, at the end of the psych degree is you choose a research project. And so the research project was, uh, in cognitive psychology. And I ended up going into that lab that was at Newcastle. And then I guess the rest is history. I just stayed on that path from there. The, the next thing that guess I happened that happened was after you do your honours year, I started a PhD and I had to choose whether I wanted to go and like join the real world, I guess, or do a PhD in that lab. And, you know, I ended up getting the funding for it. And the funding kind of said, well, that's money and no one else has offered me money elsewhere in the world. So I started my PhD in the same area that I was already interested in. So it wasn't, I guess, the main driver, but it was definitely a driver. And then I started my PhD in the area of mathematical psychology, uh, specifically in decision making. And so that's how I got there really all the way from clinician to mathematical psychologist.
Dr. Jasmine B. MacDonald (05:02):
That is an awesome story. I didn't know any of that. And it makes me very happy .
Dr. Gabriel Tillman (05:07):
Dr. Jasmine B. MacDonald (05:07):
So I'm really only gonna learn about mathematical psychology today. It's not something that I've come across in my own work. Um, but I kind of make this assumption that, um, that area of psychology is almost at the, the other end of the spectrum from counseling psychology.
Dr. Gabriel Tillman (05:24):
Dr. Jasmine B. MacDonald (05:25):
Do you think they're complimentary, like, did you have these skills from your counseling experiences where you're like, yes, this is really useful for the research I'm doing?
Dr. Gabriel Tillman (05:33):
Well, what was interesting for me was that, you know, I probably wasn't a very present person in, you know, high school and growing up and that kind of thing. And, you know, I didn't really have many of those skills that the counseling degree kind of taught you, you know, listening carefully and, you know, making sure you're reflecting on back to them, what they're saying and then, and, and kind of, you know, actively listening as well, letting people know that you're there with that kind of thing and those kind of skills I didn't have. And so interestingly enough, the two years that I did training for that, uh, have come in handy just being a person. So obviously a big part of academia is networking and talking to other people. And so it doesn't have necessarily a direct relationship with math psych, but being, I guess, a person in the world it's impacted every aspect of what I do in the sense that hopefully I can listen. Um, and I think a lot of those skills were learnt in those two years. And I'm sorry to everyone I spoke to prior to that counseling.
Dr. Jasmine B. MacDonald (06:39):
Um, yeah, I would imagine it's gonna be really useful in terms of teaching and, and connecting with students as well. Um, and research supervision. Yeah. That's really cool. Very complimentary skills.
Dr. Gabriel Tillman (06:52):
Yeah. Well, with teaching it really, I see myself sometimes like being able to pay attention and listen in, in like, obviously not in the, at the point, but if I reflect on it now, really being able to listen and focus and give a hundred percent attention was a sort of side byproduct of learning counseling skills
Dr. Jasmine B. MacDonald (07:11):
Yeah awesome. Okay. So, um, I think if we start, um, honing in on math psych.
Dr. Gabriel Tillman (07:19):
Dr. Jasmine B. MacDonald (07:19):
And your research around decision making, you've already kind of given the clues of where this fits more broadly in psychology. Like if we think about, you know, first year psych and a nice little topics where we are definitely in col psych in cognition. Mm. So, um, unpack that a little bit for me. What what's kind of the, of background knowledge and ideas that you bring into your research?
Dr. Gabriel Tillman (07:43):
Well, I guess if you told someone, if you ask someone, you know, do you know what cognitive psychology is? They wouldn't know. They would kind of be like, unless they're close to the field, have heard it before. It's not really an intuitive term, but it kind of dates back really to, you know, when computers were really kicking off and cuz, and because before that, before we had computers, obviously people might have known, we had a mind, you know, they, they were conscious and they could experience the world and they knew there was something going on that wasn't just the environment and behavior. But prior to that, you know, we had the behaviorists like Skinner in that, that basically said, you know, you can't study the mind it's, you can't observe it. You can't smell it. You can't taste it, touch it, you can't study it. And yes, it may exist, but we need to study things that we can that are malleable.
Dr. Jasmine B. MacDonald (08:33):
Right. It's just a black box gate. We can't know
Dr. Gabriel Tillman (08:36):
Exactly it's this black box idea. And so just put it aside, we can't study it can't even talk about it. It's difficult. And so what ended up happening was, you know, there was this sort of, this kind of a couple of things that developed, uh, in the middle of the last century and one of them is this development of information theory. Uh, and so Shannon sort of developed this idea and sort of in short, really, it's kind of what we see every day. You know, why is it that I can have this podcast discussion with you and I speak and I vibrate the, the particles in the air and then that those particles hit the microphone and what's happened is that whatever pattern those particles were moving in, they keep that pattern when they go into the microphone. And then when the microphone sends its electrical signals, and then when there's light signals going through the fiber optic cables, you know, going all the way across to wherever you are at now, there's so many different ma mediums that it in and those things are different, but the pattern remains the same.
Dr. Gabriel Tillman (09:39):
And so that idea hadn't really been thought about before. And what started to happen was we could realise that there were things in the world that were from the pattern of matter rather than the matter itself. And that's sort of this concept of information. So that's one idea that kind of got developed in the middle of the last century was this thing called information. And from there, there was, uh, some developments from touring, basically with computers and really a model of how humans think, uh, in the sense that we could start to use some of these terms that are associated with IT and computer science and that kind of thing, that people were starting to use, like memory and storage and processing and, and recall. And all these terms we sort of use now have this strong relationship with computers and computer science. And even though we are not a computer per se, but we don't have like a single hard drive and Ram and that kind of thing.
Dr. Gabriel Tillman (10:40):
And we don't work like that. It gave us these metaphors in computer science gave us a way to talk about the mind. And so that's how we got outta that problem from behaviorists where Skinner would talk about, we can't study it because we can't see it or touch it, then information theory and the metaphor of a computer and things like that allowed us to say, well, there's memory and there's storage and there's processing. And then we use these words to describe how our mind works. And then, um, we know that those things can exist because of information theory, it's perhaps a pattern of behaviour in the neurons. We suspect it might be that, but that's really the foundation is that we, we started to be able to talk about the mind and that was the birth of cognitive psychology, which was, is the study of the mind.
Dr. Gabriel Tillman (11:27):
And one big approach in cognitive psychology is to try and come up with a model of the mind. And so you might hear the word model in lots of places and, and really it's very similar, like a statistical model. You might hear that or in my field, we have mathematical models. And really, you can just think of these as sort of like a map. You know, if you come up with a map of a train station, you know, the train station is really there. And so what I could do is I could draw a map of the train station, label it with net words and then put colours on the train tracks on my drawing. And that would be a model of the train track, but it's not exactly the train track. It, isn't what it is. It's a different thing. And hopefully it's simpler. So, so that way you can look at the purple track and say, oh, that's the one that goes to Melbourne. However, when I go down to the train station, I'm assuming there's not gonna be a big purple track there.
Dr. Jasmine B. MacDonald (12:22):
That would be awesome though.
Dr. Gabriel Tillman (12:24):
Yeah. If there was a big purple track there, that would be awesome. Yeah. And so cognitive psychology got on, got on this pathway of creating models which are kind of like these maps of the human mind and it just spans attention, perception, memory, uh, and all that kind of thing. And I started honing in, on decision making and basically got into the field of developing models of decision making.
Dr. Jasmine B. MacDonald (12:51):
Right. Um, what, when you describe to people what you do or you, you introduce it, what do people often assume that you do they hear the term mathematical psychology? Because I know in just prep for having a discussion with you today, I've talked to other people academics and, and people who don't have a background in psychology. And I say, oh, one of the next episodes I'm working on is with Dr. Gabriel Tillman. And we're gonna talk about mathematical psychology, and I can tell you, most of them have no idea what I'm talking about. They might have worked in psychology for a long time. And I one, I love that. Um, but two, I then assume people must make all interesting assumptions when they meet you at a party or when you first introduce yourself to people.
Dr. Gabriel Tillman (13:35):
Well, the, I guess what I've learned to do. So the there's kind of two answers to that. The first is that what I actually say to people is I avoid the term mathematical psychologist completely. And I kind of avoid the term psychologist because the, you know, obviously a lot of conversations can come up and people think that I help people. Yeah. But I'm, I, you know, I don't know how many people will I help by developing models of the mind, but what I would say, if someone were to ask me what mathematical psychology is, it's sort of related to this idea that you can come up with a mathematical model of something. And so you need to kind of define that in reference to what we normally do, which is a verbal model. So if I had told you that, you know, the way the memory works is we have short-term memory working memory and long-term memory.
Dr. Gabriel Tillman (14:27):
And then, you know, a sensory information stored in short-term memory, a sensory memory, I should say for a very, very short term, it's stored in short term or working memory and a little bit of a longer amount of time. And then it's really stored for a long amount of time and long term memory. I'm kind of telling you about the world or modeling it with words it's linguistic it's, it's the, you have to understand the definitions of those words to know what's going on with my model and to understand what's happening in the world. Mm. A mathematical psychologist would say that, yes, we can use words to describe what's happening. But if we assign exact values to those things, then that will produce exact outcomes. And so one example might be, is I have a model of say, decision making, where I can assign specific values to that model N words so that, you know, people can't just look at the numbers and so that there would be numbers and words associated with this model that I've created.
Dr. Gabriel Tillman (15:31):
And it would produce actual behavior, like something like timing of a decision or the, the decision that is made. And so really the, the difference is normal normally, or in most fields you have what's called verbal models where you just linguistically or use language to describe what's happening. And mathmatical psychology sort of adds this dimension where words that are assigned to it also have numbers. And then by doing those, by having those numbers, which relate to each other in some way, exact predictions occur. So, you know, you will get an exact timing that a decision is made and an exact decision that is made. And if the data don't support the, that it's much more easier to falsify it. And it's much more easier to, to say that wasn't correct versus a verbal model where you would say, oh, you know what, the data don't support this, but I didn't actually mean that by that word, you know, that's not what I meant by short term memory I actually meant something else. And so you run into this problem of people being able to interpret words in infinite amount of ways. You know, it's very postmodern to say that, but it's what happens sometimes three different people, the same words and three different interpretations.
Dr. Jasmine B. MacDonald (16:47):
Yeah, absolutely. So is your role then, or, um, part of what you're doing, um, building those parameters and, and, and the math behind the models, or could you be working in this area and, you know, work from, um, algorithms or, you know, models that other people have developed and test that without kind of needing to have, um, experience with programming or like be super mathematical in nature?
Dr. Gabriel Tillman (17:15):
Yeah. So the main thing I would say I do is the programming aspect of it. And so I guess there's a, the co I would, in fact, one of my recent papers that I collaborated with some people from, um, the United States, we, I put forward some math and know someone in the field emailed me and was like, you know what? This is, uh, incorrect. A lot of this math that you've done, and this is the correct solution to the, the, like the expression. And then I remember thinking, well, I'm a mathematical psychologist, and this is, uh, a bit embarrassing, but there's also like another side to it. Like, like you said, there's algorithms and coding to be done. And so a lot of the stuff that I do would, I'd probably take someone else's equations and then I would code them up. And when I do that, I can use a lot of the, the, the mathematical models that are out there.
Dr. Gabriel Tillman (18:05):
But in saying that I can still develop them. So people would've done all the hard yards to get 'em to where they are now. And then I would just do slight tweaks and test what happens. Um, and it's this kind of testing. That was a big part of my PhD. So, you know, uh, and some of the, a lot of the stuff I'm doing now is testing what's needed. You know, you can make things too complicated, adding all these things to the models, and that might not be the best thing to do, because it might not generalize outside of your current experiment. And so there's a lot of things that I do there. And obviously there's the whole other side of this, where yes, we spend a quite a bit of time developing it and making sure it's a good method and it works, but then there's the whole aspect of applying it and, you know, going out of the sort of math Buzzle bubble and saying, Hey, does this actually answer any questions in the real world?
Dr. Jasmine B. MacDonald (18:56):
Dr. Gabriel Tillman (18:56):
And so that's when you start applying it to things like those driving paradigms that I was sending you, or vow perception and, or even clinical psychology, they're starting to break in there.
Dr. Jasmine B. MacDonald (19:06):
I think that's an awesome segue into having a chat about those study is as examples. So which whichever one you wanna start with, I, I sat back in having a reading, having a read of them and just thoroughly enjoyed it. I did, if I'm gonna be honest, have to skip through some of the results .
Dr. Gabriel Tillman (19:25):
Dr. Jasmine B. MacDonald (19:27):
Um, but the, the application of mathematical models to think about response times and decisions people make in a, in a really, um, serious situation like in driving. Mm that's super fascinating.
Dr. Gabriel Tillman (19:42):
Yeah. Well, the driving example was interesting cause a lot of the work has been done. So, and what I mean by work is the experiments. And a lot of the cool findings is so like David Australia has been doing this for quite some time now, and he's made some of the most, you know, landmark discoveries with driver, um, distraction. And so like basically this paradigm is that you would be in a car, but the car would be in a, a kind of fake built one in a lab, but it ha it's pretty real, it's, it's, it's trying to be ecologically valid in the sense that there's a steering wheel, you're in a cabby, uh, at the front and you're in a normal car seat and all that kind of stuff. The, the, the thing that's not like the real world is, it's a computer screen, that's showing other cars and the environment and that kind of thing.
Dr. Jasmine B. MacDonald (20:28):
Dr. Gabriel Tillman (20:28):
And so basically in that paradigm, you can have someone drive and you can see how often they miss red lights, or if the car in front of them breaks, how quickly can they respond? Um, lots of things that involve decisions, hopefully people were making decisions on the road and not just closing their eyes and hoping for the best . So what we
Dr. Jasmine B. MacDonald (20:49):
We do have that experience where you drive somewhere and you don't know how you got there. You, you just like have not consciously been driving .
Dr. Gabriel Tillman (20:57):
Yeah. It's sort of like a paradox that that's like this automatic processing where the better you get at something, the less you actually need to process it.
Dr. Jasmine B. MacDonald (21:06):
Shouldn't admit that that happens, but it does awkward
Dr. Gabriel Tillman (21:09):
, but who knows you, you might, you might actually be better. like, it's paying attention. That makes you stuff up sometimes. Right?
Dr. Jasmine B. MacDonald (21:16):
Interesting. Yeah. Okay. So sorry, tangent there.
Dr. Gabriel Tillman (21:21):
No, that, that that's good. The, the driving aspect was to get these, so you, you some, how long did it take you to pump your brakes when the car in front, and you would obviously time stamp that. So you would know when the car in front put their brakes on and you would know when the red light went red, um, and that sort of thing. And you know, the main findings before we got involved with were quite interesting already. So if you're driving and you talking on the phone, you're much slower to pump your brakes and you miss many more red lights. If there's someone in the back and that's, if you're talking on the phone, there's someone in your backseat and you're talking, uh, you're gonna pump your brakes slower, and you're gonna miss more red lights. There's one weird finding, actually that if you have someone in the front seat with you talking, you actually improve your driving performance relative to when you're by yourself.
Dr. Jasmine B. MacDonald (22:12):
What is that about?
Dr. Gabriel Tillman (22:13):
So I'm assuming that the person in front is, you know, they're reacting to your driving. So if the car you, it's kinda like it's two eyes on the road, instead of one, you know, you can hear their gasp, an audible gasp is happened. And you can say that person's scared, I better slow down. Or so you kind of get two sets of eyes on the road. And so in, when they're talking to you, they might be slight fluctuations in their voice. Um, that kind of thing, Uhhuh.
Dr. Jasmine B. MacDonald (22:41):
Yeah. Right. Interesting. All these additional cues that will help us work out if there's a risk in the environment.
Dr. Gabriel Tillman (22:46):
Yeah, exactly. So it's more information, those, someone sitting behind you or someone talking on the phone, they're probably not looking at the, the window, so they're not giving you any information except for, you know, what they ate on the weekend or whatever, and that's not helpful for, for the driving. And so, yeah, so that's the kind of data they got and then what they wanted to know is they wanted to know, you know, why is this happening from a, you know, how can we get a theory to explain why this is happening? And so obviously we have our theories that we use, which is what we would call evidence accumulation models. And so the idea was, can we apply them to this paradigm? And so really an, an evidence accumulation model, like as we mentioned before there's maths behind it. And there's also words, and I guess the main parts of it is we assume you are looking out into the world and you take in information at some rate.
Dr. Gabriel Tillman (23:42):
And so we call that a drift rate and that's the word we assign to it, but you also assign a number to it, cuz it's a mathematical model. And then you need a certain amount of evidence to make a decision. And we would call that a threshold and that would get a number assigned to it as well. And those, those are the key parts of it. And basically we assume you accumulate evidence at some rate and when it gets to the threshold amount you decide. And there's lots of other things that can be involved in there, but that's the crux of the theory. And there's an equation behind all that. And if you type in any number, it'll pump out a timing of your response and a decision that you made and that kind of thing, and you can have, um, some other things in there as well, but that that's all you really need to know about.
Dr. Gabriel Tillman (24:27):
Cause they're the key things. And so what we wanted to do is we wanted to figure out, well, are people going slow because they're changing how much evidence they need when people are in the car or talking to 'em on the phone or are people actually processing things slower. And so what you would do is you would run them through this normal experiment, and we've actually got this device that it's a little light that just, uh, it's called a DRT and it basically displays a light and when you see that light, um, you have to respond to it and we get this timing data. So when did you see the light and how quickly did it take you to respond?
Dr. Jasmine B. MacDonald (25:04):
Dr. Gabriel Tillman (25:13):
And so what you end up getting is a bunch of decisions and how long it took. And so let's say you have 500 of those. Our next job would be to, uh, take this math model, change its values of the rate and its values of the threshold and see what numbers it gives. It will give you 500 response times and then you compare them. You say, what does that look like compared to what really happened? And usually it's like, oh, it looks crap. They don't match it all. And so then you keep iterating through that. You keep changing the values. And obviously that would be painful if you did it as a human. So there's, luckily there's really cool algorithms that can do it for you. And they basically just spend, you know, it might take a month or a week just going through all the numbers that could possibly happen and it'll come up with the best one and it'll say, yep, this is the best number that produces the data.
Dr. Gabriel Tillman (26:00):
That looks exactly like what that human was doing. Sort of like a, a AI or.
Dr. Jasmine B. MacDonald (26:06):
some random bot
Dr. Gabriel Tillman (26:09):
Some random bot. So basically this bot has produced data that looks exactly like what the person really did. And so why would you care about that? Well, you can get those values. You can get the value of their rate and you can get the value of their thresholds. And if you do more complicated things, there might be other parameters that are in there. But basically you can say, look, this is how fast the bot was processing. And this is how fast this is their level of evidence they needed. But obviously it's, you are assuming that that might be what the human was doing because it matches so well with their data. And then you can start to make conclusions. And basically what we'll find is your processing speed to clients.
Dr. Gabriel Tillman (26:49):
When you get someone talking to you, you don't process things as effectively, probably because of the cognitive load. And because of that, um, your, we now have a, you know, more theoretical under of what's going on. Someone talks to you, you process things much slower probably because of the cognitive load. And that was all derived from this mathematical modeling. And so, yeah, that's basically the, in a nutshell, what ends up happening and you can apply this to many different paradigms and you end up getting, you know, answers that were not observable. So we didn't observe someone's processing speed and we didn't observe someone's threshold, but after all this kind of fancy math, we've got it. We've got what your processing speed is. We've got what your threshold of evidence is and that sort of thing.
Dr. Jasmine B. MacDonald (27:37):
Yeah. Awesome. So then from an experimental standpoint, your conditions are not having someone speak to you while you make these decisions, having someone in their front seat with you talking or having someone on the cell phone, and then you compare across those three different conditions.
Dr. Gabriel Tillman (27:57):
Yeah. So that was out. That was the setup. Basically you could have someone on the phone, someone sitting next to you or no, um, no phone or person at all. And it matches up with the, well funnily enough, the first paper I did that, I think I sent you, we were, we came to the conclusion that maybe people were changing how much evidence they needed across those three conditions. And so it's funny that one of my first papers published that's, you know, a lot of people are citing it now, but it's, we, we were wrong. uh, we were, we it's very difficult to tease the tease, the answers apart. And so we ended up, they ended up doing follow up studies where they had people make, you know, a couple of choices. Is it a red light or is it a green light?
Dr. Gabriel Tillman (28:43):
And it was that a constraint where you were making a choice between two rather than is the light on that actually helped the researchers discern that it was people's rates that were going down.
Dr. Jasmine B. MacDonald (28:55):
Dr. Gabriel Tillman (28:56):
And yeah, but that, yeah, that's basically the condition, um, that you run through is a couple of experimental conditions. And you wanna see if things like rates and if things like thresholds change across them, and you've sort of got at, you've gotten more from your data before you would just be able to say, yeah, you were this accurate. Now you can say things like across the conditions you're processing speeds are declining. And not only that you're actually requiring less evidence to make decisions for some reason, you're setting yourself lower criteria for decision making than, than you did in the first condition.
Dr. Jasmine B. MacDonald (29:32):
Mm-hmm . So in your work, um, have you, can you quantify it in terms of seconds of how delayed people are while they have someone in the car next to them, or while they're on the cell phone? Like what, what kind of delay are we talking about?
Dr. Gabriel Tillman (29:46):
Yeah. Well, it depends on which conditions you go through, but you can get effects like yes, seconds, but it's hundreds of milliseconds. And so, um, that might not seem like a lot, but obviously when it's between life or death situations with red lights and that kind of thing, and pumping breaks
Dr. Jasmine B. MacDonald (30:02):
Someone or something suddenly coming across the road, like a kangaroo or a like, you know, a person.
Dr. Gabriel Tillman (30:09):
Yeah, exactly. So you, yeah, that extra time is, is useful if, if you need to break. And, and I guess the, the, the main thing to conclude is that, you know, David stray came up with a lot of these, um, experiments that kind of hinted at what was happening. You got slower, as people are talking to you and you made more mistakes and that's fine, and it's empirical and it's useful. People can look at that and say, wow, that's cool. It's not cool. It's, it's very useful to know that you get slower and make more mistakes when people are talking to you. But then there's sort of this other side to science where it's, we have a little bit of why, you know, it's not just, um, you're slower and you're making more mistakes and that's useful practically because that's an empirical finding that has direct relevance to the world. But there's also this why aspect that we've added, you are processing speeds, declined, and that's why you're making more mistakes. And that's why you are slowing down.
Dr. Jasmine B. MacDonald (31:08):
Mm-hmm, kind of giving you the capacity then to think about, um, like how you can address that issue or improve a situation.
Dr. Gabriel Tillman (31:14):
Yeah, definitely. You can start, well, you can start, I guess, theory allows you to then extrapolate it to other scenarios. So obviously there's an empirical finding in driving of a car, but now that we have a theory in terms of decision making and evidence accumulation models, we can then take that into other contexts. There's also like this aspect. I kind, I dunno how you feel about this, but when I kind of do a lot of science and research, there is a big push to have an application, you know, here's your research, what's the application and who does it help kind of thing. And I think that's, you know, that's a commendable reason to do research, but there's also like a more selfish reason. That's kind of more like art and music and creating things where, you know, a researcher spends all this time doing experiments and doing analyses and writing it up.
Dr. Gabriel Tillman (32:05):
And then in that writeup is a new bit of knowledge that we didn't know previously.
Dr. Jasmine B. MacDonald (32:11):
Dr. Gabriel Tillman (32:11):
Um, just like a piece of art we didn't have previously, or a piece of music or something like that. And maybe not everyone would find research and knowledge as appealing, but to me it's certainly appealing. Like, and so that's what that extra getting to the why kind of speaks to that as well, which is another reason why I do this kind of thing is I, I really like not only the application of science, but also the fact that we're just contributing to the why, you know, why are people and why are we doing things and how the mind works and that kind of thing, kind of like a cultural comput, uh, contribution.
Dr. Jasmine B. MacDonald (32:46):
The, the more that we say the mind and why I go to the very first Simpsons episode where Homer freaks Bart out and he is like, what is mind?
Dr. Gabriel Tillman (32:54):
What is mind?
Dr. Jasmine B. MacDonald (32:55):
Never matter. What is matter? Nevermind
Dr. Gabriel Tillman (32:55):
Dr. Jasmine B. MacDonald (32:59):
Um, um, so I, I, I, I think it's really interesting what you're saying around, I guess I would think about it then in terms of these incremental additions, to science and knowledge about the mind and brain more broadly and discussing this is really interesting because I, my work tends to be really applied. Yeah. Um, where I'll take this broader knowledge that we have, and look at a specific occupational group, and then look at what's unique about their experience, but, um, what what's kind of cool in listening to what you're describing there is almost like you, you're developing this little plugin that could go into in multiple different contexts. It's not context specific, and it's not focused on application, but that actually kind of means that people could then use that in whatever way they want in their research, in whatever area they're working in. That's pretty cool.
Dr. Gabriel Tillman (33:53):
Well, the funny thing is, is you've probably just touched on the strategy of many mathematical psychologists, and that's why you, you end up seeing like, you know, the diffusion model of vow perception or the diffusion model of how chickens lay eggs. Like it's just, it's just getting applied absolutely everywhere with that same, uh, sort of argument that you've stated is, oh, this is a cool theory that works. Let's apply it everywhere. And you know, that can be, that can be, um, bad in the sense that it's everywhere and not falsifiable And like, it just works all the time. We need it to fail sometimes to show that it is something that can be tested, but a lot of the time you get more out of your data when you apply these things than you could previously, like there's example, like one thing that I'm trying to kind of work work into, but it's a lot more difficult than I imagined.
Dr. Gabriel Tillman (34:43):
And I'm, I've quite, I've spent a little bit of time on it and haven't made too much progress, but it's, it's ongoing, but it's this application to clinical work. And so I know that I've kind of talked a lot about how theory and the why comes out of these papers. And it's really cool that you got more out of that, but there's the other ways that they help, like in clinical psychology, for example, there's demonstrations where you can get someone to do a task, like some cognitive task, get a group of people to diagnosed with major depression disorder and healthy controls. And sometimes, and you won't see differences between them in the task, like the accuracy relatively the same on average or their, uh, response times, uh, different on average, you know, there is some tasks that they show differences, obviously, but then there's this situation of, ah, I didn't see differences, um, in my measures and these models that I'm talking about are actually more sensitive to that. And so you can apply them to the very same data and you'll see differences. And so there's this aspect where you can actually apply them and get more out of your data. That's not just theoretical, it's actually a measurement thing. And it's empirical, um, previously response time and accuracy showed nothing between groups transform them into the, these parameters of the model and the differences are present. And maybe you could use that to predict, um, group class or things like that. And so, yeah, there's other aspects that, of their usefulness.
Dr. Jasmine B. MacDonald (36:10):
That is super interesting. The idea of, um, yeah, I guess, uh, tapping into statistics and the math behind what we're doing, um, at a deeper level, which I, I don't think occurs to a lot of people. They might just see, um, their data and go, well, there's not a difference here. We take into account the error and the method that we've used around how well this represents the population that we're trying to measure in the situation we're trying to, um, you know, like simulate or whatever. Um, but I, I think what I'm getting out of this is I need to, uh, collaborate with, um, mathematical psychologists.
Dr. Gabriel Tillman (36:49):
Yeah. Well, I guess the, it's definitely a good idea. Lucky, you know, one. Exactly.
Dr. Gabriel Tillman (36:57):
Yeah. Well, I, I think there's a lot more in some of the most basic data people use. Like if you ever get mean response time or something like that, a lot of people just use that and then that's it, but there's the whole distribution. Like someone might do a 600 trial experiment and then they turn that into a mean, and then they say, oh, then those means don't differ between groups, but then you have 600 sort of probings into that, that participant 600 times, they've given you information from their mind and you've got 600 windows into the mind and you sort of collapse that into one number. Whereas these things that I'm talking about, use that whole set of information to, to try and conclude what's going on. That's partly why they're senst- more sensitive is because they use all of those trials of responses rather than just the mean.
Dr. Jasmine B. MacDonald (37:43):
Hmm, sure. Um, I guess as we're talking, I'm thinking as well about, um, like what constitutes a decision because when you say 500 decisions or data points or 600, um, that probably sounds like it's really overwhelming. It takes a really long time, but we actually make a lot of like lots of small decisions all the time. So if we, I'm just thinking about people listening, if you, if you go back to your, um, driving paradigm study, how long would've people been in the simulator, driving the car to get, say 500, um, responses
Dr. Gabriel Tillman (38:21):
Well, 500 trials. So the simulator actually took a little bit longer. If you look at how long cause that's basically the way that that works is you would have someone driving and then the stimulus would come on and then they respond. And so technically you could have done say three minutes of driving or something like that. And then a stimulus comes on, but then that, that one trial is only that one little slight of time. But how it works typically is you would have someone sit down and you would have 500 trials of, let's say, uh, you ask them to just decide whether a square is red or blue. Like it's, you can get more than that. That's obviously sounds deadening, but let's just use that as an example. And you decide with buttons on a keyboard. So the square comes up and you say, uh, blue or red blue with a button on your right and red with a button on your, left and you do that a hundred times and sequentially.
Dr. Gabriel Tillman (39:19):
So to maybe a cross would come up for 500 milliseconds, then the stimulus would appear, then you would decide and then it would go away and then a new one would come up almost immediately. Um, and so you can see cause they're sequentially, you pump through a hundred, you in three or four minutes and then it takes you 20 minutes to do, um, 500 or something like that. There's breaks you obviously you've gotta give everyone breaks. Sometimes people don't do it though. Like there's been experiments that, um, in labs that I've been in, in the past where people, they come in, they get the course credit and then they leave like, you know, you bust in on them and say, Hey, like, so I, what are you doing? The data doesn't seem to be looking good. And they're eating Cheetos and watching big bang theory on their phone. just like mashing the buttons. Like that's that happens that's
Dr. Jasmine B. MacDonald (40:09):
Oh, data collection,
Dr. Gabriel Tillman (40:11):
data collection. Yeah.
Dr. Jasmine B. MacDonald (40:12):
Is data collection with humans.
Dr. Gabriel Tillman (40:14):
Dr. Jasmine B. MacDonald (40:15):
How do we know if this is, is a, um, a valid response, they're actually paying attention and doing the task
Dr. Gabriel Tillman (40:21):
Mm-hmm and that's a big area that, that I spend a lot of time in. Um, and it's sort of, it's not credited that you do this. So, you know, a lot of times when you do a paper and you look through the data and you spend so much time on this, is that real person like, or do I try and account for this? And that's our like days and days and days of work to do that. But obviously it's probably a sentence and a publication that how you've cleaned up that. Yeah.
Dr. Jasmine B. MacDonald (40:49):
Ah, it hurts 'cause it's true.
Dr. Gabriel Tillman (40:52):
It hurts. Yeah. If only someone cared about, well obviously there's the whole idea that you need to be more thorough in your description, but then if you send it to a journal that looks like less of that please and more of the shiny stuff.
Dr. Jasmine B. MacDonald (41:07):
Yeah. That's it. I, I think it's, um, a fascinating thing 'cause it starts to overlap with other areas of cognition, like participant motivation.
Dr. Gabriel Tillman (41:14):
Yeah, Exactly. Like, am I doing a study on participant motivation or decision making? I can't tell anymore. maybe both.
Dr. Jasmine B. MacDonald (41:24):
Um, so, um, what about the, uh, phoneme study, um, as another example of the work that you've done?
Dr. Gabriel Tillman (41:34):
Yeah. So this was something that this was actually one of my first projects, but it kind of took quite a while to do so. The idea was very similar to the driving one, basically there was a well known paradigm and it's actually quite old the paradigm where you could present people with a sound and like it was something as simple ahh or ah, and so there, that's how we that's now, you know, the kind of simplicity of these experiments, but what you could do, if you get, you know, we had some good collaborators on there that could, um, manipulate these phonemes what you do is you can stretch them, make 'em longer in time, or you can play with the frequencies that are in there. Cause all sounds are made up of, um, a couple of different, um, spectral frequencies and that kind of thing.
Dr. Gabriel Tillman (42:14):
And so you have these two dimensions of a sound it's frequencies and how it long it is and basically what they do without us. So the, the experiment is they look at, you know, native speakers of Dutch versus English, uh, native speakers. And so that would be someone who speaks Dutch and then someone who speaks, uh, English, but can also speak Dutch. So that way they can do the experiment, which is all in Dutch. Let's say that's for my, for this experiment, we're talking about it was all in, in Dutch. But the, the main thing here is that they kind of knew that the native speakers of Dutch would attend to the frequencies of the sound. They would say, oh, there's a, uh, basically if you manipulate that, it changes how they respond. But for some reason when you're not a native speaker, but you know, Dutch, you attend to the length of time.
Dr. Gabriel Tillman (43:07):
So maybe it's a bit more difficult to pull out the spectral information when you're not. So like, you know, familiar with it, you would have to probably talk to the linguists on the paper to get all the, that information. But we kind of come in where they collect this data, they got timing data and they've got the choice that they made. Did they hear ahh, or did they hear Rah and they're trying to decide how important spectral frequency and duration, and that is. And so they're, they're thinking, how can we do that? We've kind of milked the data for as much as, as we can. And there's just not much we can get out of it. What you, what we ended up doing was we developed a model of how people perceive phonemes. We basically it's the similar one to we described before. There's some sort of accumulation rate.
Dr. Gabriel Tillman (43:52):
Um, there was some sort of threshold. And when you reach that threshold, you decided to, that you heard, ah, however, there's now a race. There was another, exactly the same thing. It's called an accumulator and it has a rate and it has a threshold and if it wins. So now it's a race between these two sort of, um, accumulating evidence accumulation models. And if that one wins then you make the decision. Ahh, anyway, so that's the model in a nutshell, and we can apply it to the data. And like before that model will produce choices, ahh, and ah, and it'll produce timing data. But what we did, which was interesting was we made the rates and we made the parts of the model dependent on the frequencies of the sounds and the, and the durations of the sounds. But we, we didn't sort of give the information to the model.
Dr. Gabriel Tillman (44:45):
We just had these conditions. There was like 10 different frequencies and 10 different durations. And what we could conclude was basically that the, you know, the native speakers had these really sharp change, really strong changes in their rate of processing with frequencies. And, um, you know, it, it wasn't the same for, for the English speakers. They didn't use frequency as much. and so it kind of mapped onto this processing speed thing that they could process the spectral frequency really fast. And if you change the frequency, they were really in tune with that. Um, but people that were not native speakers didn't, and so it kind of contributed to the field in the sense that we now had an understanding of what might be happening and why natives speakers, um, not only are faster, but they can make differentiations and they rely on a different kind of information from the sound like you think everybody would just take the same info from the sound to hear it. But native speakers are using the frequencies. Non-native speakers are using the, the length and duration of the sound.
Dr. Jasmine B. MacDonald (45:49):
Dr. Gabriel Tillman (45:50):
Dr. Jasmine B. MacDonald (45:50):
Whoa. So I I'm learning Spanish. I've actually been trying to learn Spanish for a while, but I dabble in and out of it. And you, I think you don't appreciate all those small decisions that we.
Dr. Gabriel Tillman (46:05):
Dr. Jasmine B. MacDonald (46:06):
That we make in trying to, um, kind of understand, you know, like it, the sounds within a word or phonemes within a word. I think when you're learning, you think more around recognising a specific word and how that's different to other words, rather than each of the, um, parts of that word. So I, it's very fascinating.
Dr. Gabriel Tillman (46:23):
Well, it's the build. Yeah. It's the building block of everything that we say like, and the weird thing is, is that I don't know how it's really difficult to that we can even hear because when you look at a sound or a speech signal and you, you plot it, it's just one big blur of frequencies and, and how we pull that apart is amazing. Like obviously the we've it's kind of well understood how we do that. Yeah. The fact that we've developed that ability is, is quite amazing.
Dr. Jasmine B. MacDonald (46:54):
yeah. I'm like nodding vigorously, as you say that because, um, of the editing I'm doing for the podcast, and I've not worked with audio before and just looking at, you know, knowing where one word ends and another starts, it's not there. Yeah. Because we don't speak like this
Dr. Gabriel Tillman (47:13):
Exactly. Yeah. Maybe after a few whiskeys, I did a bit, but not right now, but exactly. And yeah, so I think you're probably seeing a pattern here where we get collaborators in a field and they know a lot about the topic area and they're kind of at a point where they want to get data, uh, get more out of their data and that's where we come along and help. And so there is sort of this aspect that we're kind of like a gun for hire. Uh, but you know, I think the, the backbone of it all is that, you know, we really develop these models and we, the, we try and apply them to different areas, but the, the model itself is a theory and it is, you know, it kind of sounds like a tool that I'm using and it is, but it's it also, if you, you know, read a paper on it, it describes what's happening as a process. And so it's actually an explanation as well.
Dr. Jasmine B. MacDonald (48:05):
Dr. Gabriel Tillman (48:06):
And so we work on that as well. So we kind have our own fields, you know, math like we do. And, you know, I'm only talking about decision making models, there's model, mathematical models on all the things that are cognition. And they're just as, um, amazing. And just as thorough in sort of explaining phenomena,
Dr. Jasmine B. MacDonald (48:27):
right, What keeps you working in this area?
Dr. Gabriel Tillman (48:30):
Well, the, the, the thing that, uh, keeps me working kind of goes back to that idea that, you know, I have a lot of weight on discovering and the why of things. So I think before I started to realize how much these models could add to certain paradigms and that kind of thing I was actually interested in the implementation of it. So, you know, in honors year and in my PhD, I really enjoyed coding and I really enjoyed, uh, the research involving using these sort of techniques and, and, and coding it up and making these algorithms work. And, you know, obviously that can be frustrating, but there's sort of a, like to the actual practice itself, liking the process. That's one way to say it, but as a, at a more conceptual level, there is something sort of really rewarding about this whole cultural contribution aspect of it, you know?
Dr. Gabriel Tillman (49:25):
Yes. We're trying to apply things where we're trying to learn more about how drivers are getting worse with distractions and how do we process phonemes and even the clinical psychology example, you know, being able to predict maybe treatment response, they're all good applications. And that's one aspect that keeps me motivated in it. But the other part is that every time you get a project, it's a bit exciting to know that doing this will get more from the data and give you that why and give you that, that further, uh, explanation that is, is quite satisfying.
Dr. Jasmine B. MacDonald (49:59):
Mm. And I mean, really it's, um, as a way of looking at it as a kind of a, a necessary and ethical thing to do, if you are spending time collecting data from people, um, to make the most of that data and think about the ways that it could be used, it's like, like necessary, right?
Dr. Gabriel Tillman (50:19):
Yeah, exactly. It's like, 'cause if data's not, didn't come to any conclusion, but it could, you might think that it's ethically your obligation to, to, to use that data and come to the conclusion that it can obviously there's, there's downsides to it. Um, you know, like there was that, there's one thing that I was a part of recently, uh, theoretical inference paper, basically they got sort of, they would call a memory experts. I would, a lot of the stuff I do is with models that were traditionally made in memory. And so that's why, I guess I might have been invited onto this and, you know, experts all around the world were given the same data set and they were all analyzed it with whatever model they felt like you could use evidence accumulation model, signal detection theory model, whatever, all these different models that exist. And one of the conclusions was everyone came to different conclusions. And so
Dr. Jasmine B. MacDonald (51:14):
This makes me feel anxious of the controversy that could come out of this
Dr. Gabriel Tillman (51:17):
Yeah. And so we had this paper that kind of basically said, Hey, we gave everyone the same data and they could use whatever model they could use. The, the, like, you know, the most , uh, old, the oldest model that there is, or you could use what's at the forefront of mathematical psychology or whatever. And interestingly, there's a lot of differences in what people would conclude. So you could imagine if everyone wrote their own paper, there would be quite a few differences to, to what they would conclude.
Dr. Jasmine B. MacDonald (51:44):
Yeah. Right. I mean, I guess in a way it's similar to theoretical approaches where you're gonna have maybe different explanations of what's what's happening and then trying to make a decision in what context is the best way to do it. But yeah, I don't know. I feel like maybe asking that question of how do you work out? What is the best model? Is that a, how long is a piece of string kind of question?
Dr. Gabriel Tillman (52:08):
There's actually a big part of our, so a lot of my training in my PhD was how do you choose the best model? And there's obviously a rule of there's there's statistical approaches that basically quantify this rule of thumb. And the rule of thumb is, you know, don't be more complex than you need to be. And so what you can do is you can get two models and you say accounting for the fact that one might be really, really, really complex. And one's more simple. How well do they predict the data, which is sort of that aspect of making them produce data. And if it matches you get a score of how good that matches, you know, like the likelihood I guess, and you get that and you compare it to the other model and if they're really close, you would end up penalizing the one that's really complex and say, you know, it pretty much does just as good a job as the simple one, but it's got the extra bells and whistles, so I don't think they're needed.
Dr. Gabriel Tillman (53:07):
And so we kind of use that approach, this principle of parsiomony, first we check how good they are and if they're very similar, we, we penalize for complexity and see, um, if it's even worthwhile going with it and then you end up getting the model, that's just good enough to predict the data and it's not too complex. So it'll generalise to other scenarios, cause usually when you've add things on to models, whether it's statistical or theoretical models, that just means they won't generalize because you added them on because of a very specific trend in the data of one experiment rather than more generally,
Dr. Jasmine B. MacDonald (53:43):
I've really enjoyed this discussion because it's, you know, has me thinking about psychology, um, and, and approaches to research in a way that's, it is really different to my own. And I think that that's really cool. Being able to have a discussion like this. So, um, thanks for your time.
Dr. Gabriel Tillman (54:01):
That's fine. Jazz. Yeah. If you ever wanna learn more about mathematical psychology hopefully, it has, it's less of a scary term now and really all it is is just putting numbers to words in a way. Um, and so I think that will clear things up if anything , but no, it's been great.
Dr. Jasmine B. MacDonald (54:18):
That's probably a good time to say if people, other researchers or, or listeners in general wanted to reach out to you, or they wanted to find out more about the work that you've done, what's a good way to do that.
Dr. Gabriel Tillman (54:29):
So a lot of the stuff that I have on is in two places. So I have my Twitter at Gabe underscore Tillman. And from there you can actually get to my Google scholar link as well. My, I had a webpage going there, but I'm, I'm working on that again. Now. I think it's, it's go it's under the development. So I wouldn't, I wouldn't plug that now.
Dr. Jasmine B. MacDonald (54:48):
Okay. Deal. Well, I will link to that in the show notes that, um, Twitter and Google scholar, so people can, can find it easily.
Dr. Gabriel Tillman (54:56):
That sounds good.
Dr. Jasmine B. MacDonald (54:57):
Um, is there anything in particular wanna kind of spruik that you are, that you've currently been working on or that's coming up?
Dr. Gabriel Tillman (55:05):
Well, a lot of the stuff that I'm worth working on is, you know, submitted and becoming pub journal publication. So there's like no book or anything like that. I guess that would be more for a lay audience. So if, if you are interested in looking at that, that'll be just up on the Google scholar page, few conferences coming up, but
Dr. Jasmine B. MacDonald (55:22):
in person or will they be online,
Dr. Gabriel Tillman (55:22):
Hopefully in person it's gonna be in person. I still haven't done the work though. So I'll lot of it is, so it's finding the time to do the work. What you do is you commit to something and then you have to do it because it's coming up in a week.
Dr. Jasmine B. MacDonald (55:35):
This is so true. I, I laughed a lot on Twitter last year when people were doing, um, online conferences and they were freaking out because they were like, there's no plane trip for me to actually finalise my talk. It's not gonna get done.
Dr. Gabriel Tillman (55:50):
yeah, exactly. We have to adapt. We just need to build a fake plane cockpit with seats that we can go sit in for an hour.
Dr. Jasmine B. MacDonald (56:01):
Yes. And we're back to simulation. Exactly. Cool. what are the plans for the weekend?
Dr. Gabriel Tillman (56:08):
It's actually my daughter's birthday. So we're taking her to the, a local, the it's like the sovereign hill theme park and yeah, a lot of other things that are going on that accompany, a young girl's birthday.
Dr. Jasmine B. MacDonald (56:21):
Fantastic. How old is she?
Dr. Gabriel Tillman (56:23):
Dr. Jasmine B. MacDonald (56:24):
Seven. Mm-hmm that's gone way too fast.
Dr. Gabriel Tillman (56:27):
Dr. Jasmine B. MacDonald (56:28):
Do you like when people who don't parent your children tell you how quickly they've aged you? Like ,
Dr. Gabriel Tillman (56:36):
Dr. Jasmine B. MacDonald (56:41):
Well, that's fantastic. Tell her happy birthday for me. Yeah. And, um, and yeah, I really appreciate your time, Gabe. It was super interesting and it's just awesome to catch up.
Dr. Gabriel Tillman (56:49):
Yeah. Anytime, jazz. It's been great. Thanks
Dr. Jasmine B. MacDonald (56:54):
For those of you at home. That's all for today. Show notes for the episode can be found www.psychattack.com. If you've enjoyed, listen to psych attack, please rate it on your favorite podcast platform and share this episode to help other people find the show. If you have questions or feedback, you can reach out on Twitter at psych attack cast. Thanks for listening. And we'll catch up with you again. Next time.
Speaker 3 (57:25):