Owl Have You Know

How is AI enhancing our creativity and problem solving and how does that compare to other digital tools we use everyday, like search engines? 

Through controlled lab experiments and field studies, William S. Mackey Jr. Distinguished Assistant Professor Jaeyeon (Jae) Chung spends her time at Rice Business researching questions just like that. Her work in the field of marketing focuses on the groundbreaking innovations and psychological implications of AI’s growing role in our lives. 

Jae joins host Maya Pomroy ’22 to chat about her journey from psychology to AI, her recent study on generative AI’s effectiveness in assisting human creativity, and her research that delves into how misinformation is spread through platforms like YouTube. 

Episode Guide:
01:35 Journey from Psychology to AI
04:20 Academic and Professional Achievements
06:44 Why Rice University?
08:40 Teaching and Research at Rice
12:47 Exploring ChatGPT's Creativity
17:43 AI's Role in Empathy and Innovation
21:42 Concerns and Regulations Around AI
27:50 YouTube and Misinformation
34:05 Future Research and Projects

Owl Have You Know is a production of Rice Business and is produced by University FM.


Episode Quotes:

Exploring Jaeyeon's recent study on AI and creativity
13:21: [Maya] You did a recent study about ChatGPT as a powerful tool for enhancing everyday creativity and problem-solving. So can you walk me through that experiment that you ran and through your study and your findings on that?

13:59: [Jaeyeon Chung] So, my curiosity stemmed from the fact that GPT may perform better than Google or human in subjective tasks. It seemed quite obvious that it does a great role, a great, great job in solving math problems or writing code, but would it be superior when it does a very subjective test, which is perceived as completely human? And that was the creativity test that a lot of people have been saying is a unique territory of humans that no machines can replace.

14:17: [Maya] With empathy. Empathy was one of the emotions that you studied, yes?
 
14:21: [Jaeyeon Chung] Right, right. And I was trying to challenge that assumption, and that's how I started to run a different set of experiments.

How Rice supports junior faculty with research funding
07:06: Rice has a great opportunity that the school is providing to junior faculty members for sure. I mean, as a person who has always been looking for, like, the best, Rice offers one of the best packages that all the scholars prefer to get in terms of the research funding. It's the research budget that they're really fully supportive of. So, for me, I'm running a lot of experiments; let's say that I pay $2 for a 10-minute survey. It's a very small amount of money, but if you are getting 2,000 participants to write a paper, that becomes a whole lot of money that I can't just like get from my own pocket. And the school needs to support that, but a lot of other schools required for a high-standard research without providing the adequate support, and Rice is the school that really provides all these detailed steps, and they really protect the junior faculty members to focus on what they're interested in.

Why mastering AI tools matters more than fearing replacement
27:06: I think understanding a broader aspect of how the industry works is critical for humans to fully have control over GPT and to use it as a tool to develop their careers further rather than replacing it here or there. So you shouldn't be worried about yourself being replaced by AI, but you should be worried about losing a job because of a person who better uses AI.


Show Links: 
Guest Profiles:

What is Owl Have You Know?

Owl Have You Know is Rice Business’ podcast created to share the experiences of alumni, faculty, students and other members of our business community – real stories of belonging, failing, rebounding and, ultimately, succeeding. During meaningful conversations, we dive deep into how each guest has built success through troubles and triumphs before, during and after they set foot in McNair Hall.

The Owl Have You Know Podcast is a production of the business school at Rice University (Rice Business) and is produced by University FM.

[00:00] Maya: Welcome to Owl Have You Know, a podcast from Rice Business. This episode is part of our Up Next Series, where faculty, researchers, and alumni weigh in on the trends currently shaping the world of business.

AI, ChatGPT, social media: the risks, the rewards, the evolution, and its intersection on our lives is on the forefront of everyone's mind, especially one Rice Business professor who is making it her life's work to study these groundbreaking innovations and their psychological implications.

Professor Jae Chung is our guest today on Owl Have You Know. Born and raised in South Korea, Dr. Chung has always been drawn and fascinated by human behavior, which is now being transformed by artificial intelligence. She talks to us about her most recent experimental research, the unexpected results and her insights about how these powerful tools have exponential potential for the greater good.

So, welcome, Professor Chung. We are thrilled to have you here today.

[01:02] Jaeyeon: Thank you, Maya. It's a very nice introduction.

[01:05] Maya: Well, I'm really excited to learn more because, right now, AI is on the forefront of everyone's mind and, you know, ChatGPT and the interworkings of all of it. And you've really dug in to the research and the psychology of it. You started in psychology. So, can you tell me about how your start in psychology has, sort of, dovetailed you into this kind of work?

[01:31] Jaeyeon: Well, I did my bachelor's in psychology at U of Michigan. But before that, I actually did, like, a one-year college in Korea and I did econ. Like, everyone was telling me that you need to be a banker and you need to have a major in business and econ. And that's how I was, like, forced to do it as, like, an 18 years old not knowing what to do. And after one year, getting a B in my Econ 101 class, I learned that this is not my thing.

[01:58] Maya: Oh, that's okay. Lots of people get Bs and go on to have illustrious economic careers. [crosstalk 02:02].

[02:02] Jaeyeon: You know, in an Asian standard, that's not a good sign, especially in 101.

[02:06] Maya: I understand.

[02:07] Jaeyeon: So, I just wanted to explore a bit of the world out there. And Korea, South Korea is very innovative, but it's a small country. And that's why I decided to come to the States and I started attending U of Michigan. And I was, of course, undeclared, and Psych was one of the best departments they had there. And I was so glad that I just got to meet a good set of faculty members there, started to research. And I think I've always been interested in applied aspect of psychology because a lot of psych theories are so rigid and, like, so complicated that nobody understands outside in the world. So, I always wanted something that's more applied. And when I was trying to write a thesis, I emailed a bunch of professors in the Psych department. Nobody answered me, nobody except one person. And that person was an adjunct professor who was in the marketing and also in Psych. Her name is Caroline Yoon. And thank you, Caroline, if you're ever listening to this.

And that's how I started writing a thesis about nationalism and antagonism. It was completely unrelated to consumer behavior. It was more psychology. And my desire of understanding how the society operates, how this, like, groups, that goes beyond an individual set a social agenda, and how that all impacts us. Social behavior is what I was interested in.

So, that's how I started my thesis. And, of course, my advisor was a marketing faculty. And I got to get involved in a lot of experiments, meaning that I bring in participants, I ask them to respond to a whole list of survey questionnaires, pay them. And all this undergrad RA work was what I was first involved in. And that's how I slowly transitioned to the consumer behavior of marketing, which is now the field that I'm currently in.

[04:05] Maya: And you went on from psychology to not get, one, but two master's degrees and then a Ph.D. You have a master's of philosophy in marketing and a master of science, both from Columbia, and along the Ph.D. in business from Columbia as well. So, what sparked that interest in you in philosophy, in marketing, and then to go on and pursue your Ph.D. in business?

[04:28] Jaeyeon: Yeah, I mean, just like a lot of undergrads who just… or those who graduated, I just had multiple interests. And human rights was something that I was deeply into it. I mean, South Korea — I was born in 1980s — has gone through a lot of political changes. Before I was born, there was some level of dictatorship. Right above us is North Korea, which is still under a big dictatorship. And there's so many, so many North Koreans that are risking their lives to come to South Korea and to meet their family members. Even my grandfather and my grandma, as well, who has passed away a few years ago, came to South Korea during Korean War. And, after that, the country was split into half. And my grandpa still misses North Korea, not the dictatorship, but all the family members and all the… the hometown that he has never gone before.

And he was an educator. He became a principal in an elementary school in South Korea. He really built everything from scratch. And all his kids, four of them, became doctors and pharmacists, which is my mom. And everyone got their professional degree. But there was such a disconnection from the heritage in the family that we were missing.

And because there were so many families that were torn apart and so many of them were around us, it just came naturally as something that I was interested in. So, when I went to U of Michigan, of course, I looked for this RA work opportunities. And there was this refugee center that Professor James operated. And that was basically helping different refugees from around the world to get the right kind of legal resources. So, my role was to, kind of, organize and collect all this international refugee law cases, so that all these lawyers and legal parties can access and to make a more informed decision.

[06:28] Maya: So, let's talk about how you found your way to Rice, because, you know, coming from New York, Columbia to Rice, why did you decide to come all the way here to Texas?

[06:39] Jaeyeon: Well, because Rice offered me a job.

[06:45] Maya: That’s a good reason.

[06:48] Jaeyeon: Yeah. I mean, and Rice has a great opportunity that the school is providing to junior faculty members, for sure. I mean, as a person who has always been looking for, like, the best, Rice offers one of the best packages that all the scholars prefer to get.

In terms of the research funding, I'm not talking about how much I get paid monthly. It's the research budget that they're really fully supportive of. So, for me, I'm running a lot of experiments, meaning that, let's say that I pay $2 for a 10-minute survey. It's a very small amount of money, but if you are getting 2,000 participants to write a paper, that becomes a whole lot of money that I can't just, like, get from my own pocket pocket. And the school needs to support that, but a lot of other schools require for a high-standard research without providing the adequate support. And Rice is the school that really provides all these detailed steps. And they really protect the junior faculty members to focus on what they're interested in. It's not like you need to do a certain thing.

So, the school looks good. The priority is on the faculty members, that they really foster you, so that you can dive into the topic that you are naturally inclined to research. And that very safe environment, I'm not getting threatened by, you know, my job security. I can really explore as a child being on the playground. I can explore different things. And the school is a safe environment for me to do what I really want to do.

[08:23] Maya: So, tell me about the classes that you teach at Rice and about your students and the kind of experiments that you do and the work that you do collaboratively at Rice.

[08:33] Jaeyeon: I teach two classes. One is an undergrad. So, there are two sections for my undergrad. It's the most basic marketing class, which teaches, like, segmentation, targeting, conjoint analysis or case studies. Not many schools across the states have the business program for undergrad and Rice has started its own. And as a result, a lot of really smart students are coming in, flooding in to the school to learn about the basics of marketing or business, more in general. And mine is a very fundamental required course that teaches every little bit aspect, whether that be digital marketing, price, promotion, distribution, and so on, so that they can complete a market research project by combining all the skills and tool sets that they've learned through the class.

And sometimes, I talk about my research with my students. For example, I have a project on Instagram and I try to examine what types of influencers tend to be the most popular.

[09:34] Maya: Effective?

[09:34] Jaeyeon: Yeah. I get more engagement from their audience. And because a lot of my students have their own Instagram accounts and some of them are even micro-influencers, that gives us a really good conversation on how to make it better for a lot of their followers to respond to their posts. And, for example, some people say that you need to use a lot of hashtags, or some other people say that you need to post at the right timing, like, on weekends, for example. And those are all true.

And what I do is to get a list of 1,000 top influencers from around the world. And some are, like, traditional celebrities, like Anne Hathaway, like Kardashian family, but some are, like, truly bottom-up influencers, YouTube influencers, and so on.

And what I do is to scrape all their posts for the past 6 months. That leads to several, almost more than, I think, 70,000 posts. And I do the image analysis. I do all the caption text analysis to see which one has the strongest indicator that predicts the engagement likelihood, which is a measure through the number of likings. And what I find out is that, well, a lot of things play. For example, expressing emotion helps. Happy emotion helps.

[11:01] Maya: To connect with others.

[11:03] Jaeyeon: Yeah. But more importantly, negative emotion has almost three or more times effect than the positive emotion. Like, you are feeling depressed, you are so upset, things like that.

[11:16] Maya: And that resonates more than a positive emotion.

[11:20] Jaeyeon: Exactly, exactly.

[11:23] Maya: Why do you think that is? That's interesting to me.

[11:26] Jaeyeon: I think that this American culture is so accustomed to talking about all the great things that's happening to their lives. When you talk about something that's negative, that feels much more authentic. Everybody goes through a difficult time, but nobody really talks about it. But when you are, like, opening up, that resonates more with people, as my aunt just said. So, all this sense of self-disclosure tends to play an important role.

[11:57] Maya: Transparency.

[11:58] Jaeyeon: Yeah, yeah. And you need to… as an influencer, you need to be in the fine line between, you know, sharing your public life, but also some of your private lives, so that you are, kind of, playing a…

[12:09] Maya: A real person.

[12:10] Jaeyeon: Yeah, yeah, yeah, so that people can relate.

[12:12] Maya: It's all of your dimensions because, you know, being honest, I think that key aspect of being honest and authentic and real, you know, rather than the Photoshopped images that everybody puts up and you're like, “Okay, whatever, that's not… I can't relate to that person.”

So, I want to talk about a recent study, specifically about ChatGPT. And that was another question that I had for you. Because I have young daughters that are in school and ChatGPT, specifically, has been promoted in schools to be used, which I was fascinated by when my daughter came home and she's like, “I'm supposed to use ChatGPT to help me with this English paper.” And I'm like, “Wait, what? That doesn't make any sort of sense to me,” like... and then when I dug a little bit deeper, I understood what the teacher was trying to do and to use it as a tool rather than, you know, have ChatGPT write the whole paper.

But you did a recent study about ChatGPT as a powerful tool for enhancing everyday creativity and problem-solving. So, can you walk me through that experiment that you ran through your study and your findings on that?

[13:17] Jaeyeon: Yeah. So, I ran these sets of experiments with my co-author at EF Houston. And that was about the time when everyone was talking about the rise of GPT and how Microsoft is only bringing this OpenAI, and everyone was so excited about it. And that's when people started to talk about whether the GPT will really replace Google. So, my curiosity stemmed from the fact that GPT may perform better than Google or humans in subjective tasks. It seemed quite obvious that it does a great role, a great, great job in solving math problems or writing a code, but would it be superior when it does a very subjective test, which is perceived as completely human? And that was the creativity test that a lot of people have been saying that it's a unique territory of humans that no machines can replace.

[14:17] Maya: With empathy. Empathy was one of the emotions that you studied, yes?

[14:21] Jaeyeon: Right, right. And I was trying to challenge that assumption, and that's how I started to run a different set of experiments. So, I ran, like, five experiments, which is in my paper in nature, human behavior. But the basic paradigm is like this. It's two stage. In the first stage, I bring in people and randomly assign them into ChatGPT condition, web search condition, which is a Google condition, or depending on the experiments, it has a human-only condition where humans do not use any technological assistance. Or it's also GPT-only condition. So, you provide an input, you get the output, but you don't make any modification. So, it's GPT only. And then I give them a different set of creativity tests. So, for example, in one experiment, I asked them to come up with an innovative dining table that doesn't exist in the world.

[15:11] Maya: A dining table.

[15:12] Jaeyeon: A dining table. And in other cases, yeah, I ask people to come up with creative gift for a teenage girl.

[15:23] Maya: Oh.

[15:24] Jaeyeon: Teenage girls are really hard to satisfy, you know.

[15:26] Maya: So hard to shop for, I agree. There's no right answer. That's why it's like, “I'll just give you money and you go get what you want because I can't figure it out. You're not going to like it.”

[15:37] Jaeyeon: Right, right. So, depending on people's conditions, they either get the help of GPT, Google, or figure it out by themselves, or they just use GPT but do not make any more modifications. So, I get this whole list of ideas. And what I do is to remove all the conditions that are attached to the responses, and I shuffle all those and give it to a different set of participants who are called expert judges, or lay judges, lay person. And expert judges are hired from our Jones School. They were MBA students. There were three of them who had five years or more experiences in consulting and product management. But I also had a bigger set of lay consumers from the same population set. And I just provided smaller chunks of ideas that I got from this whole list of ideas and asked them to rate how creative these are.

And then, now, this is two stages. First, I got this idea. Second, I got this external judges who evaluate these ideas. And later, I matched this with the conditions that the participants from the first stage were assigned to and then statistically examined whether the scores from GPT condition is higher than the other condition. So, surprisingly, GPT was always better than Google.

[17:00] Maya: Huh.

[17:01] Jaeyeon: GPT-assisted condition was better than human-only condition. And there's this last condition, which is a GPT-only condition, meaning that there's no more human modification that's made after you got the initial output from GPT. That was even better than Google. So, using GPT was always better than not using it. And this effect held even in cases where a task involved some level of empathy.

So, just to talk a little bit more about that empathy task, I asked participants to read a scenario, which was that there were, like, two products that a mom owned and it was used by a daughter who's no longer using it. And she's now a grown-up. And these two products are something that's really cherished. And this person, rather than throwing these products away, they wanted to, or she wanted to, reuse it by creating something new. And I never wrote something that's very emotional in the text. I'm just describing here. But even with this context, the ideas that were generated by GPT was better than the one that was created without using it, with just using Google. So, it seems quite of a strong effect that technology, which was perceived as being incapable of understanding subjective human emotions, can now even do so and read really between the lines to find out the answer that the humans are seeking for.

[18:39] Maya: And that's just… correct me if I'm wrong, but that's machine learning. Is that part, like, is that what it's doing, it's just learning?

[18:46] Jaeyeon: So, it's really pulling different set of ideas or concepts. Let's say that there's different words that tend to be used more frequently. And what this AI does is to understand the common concept that tends to be connected together to come out with another idea that's pretty new.

[19:07] Maya: So, it's statistical probability.

[19:09] Jaeyeon: Yeah, yeah. So, what was really interesting here was that, maybe, because of that aspect of AI creating its own responses, the ideas, when rated by expert judges, these ideas were perceived as incrementally new. And what I mean by that is…

[19:28] Maya: Yes, it's not radical. It's incremental. That's another part of this research, is that it's incrementally better. So, let's expound on that.

[19:36] Jaeyeon: Mm-hmm. So, in the creativity literature, or in our field more, in general, innovation has been always examined for several centuries. And there are two types of innovations, which is an incremental innovation. You make a slight modification to what's already existing out there versus something that's radically new. And so, now, I had this list of ideas and I had this expert judges from our MBA program. Asked them to rate how innovative, the degree of innovativeness. And perhaps, because of the tendency of AI, just pooling the existing concepts together to create something new, led it to be scoring high on the incremental innovativeness. But it wasn't any higher in the radical idea aspect.

But this doesn't also mean that humans are so much better in radical idea generation, because statistically, they were equal. And it was, kind of, saddening, right? Because as a human, you want some aspect of yourself that cannot be penetrated by-

[20:39] Maya: Uniqueness.

[20:41] Jaeyeon: Yeah,

[20:41] Maya: To not be replicated. It’s… you don't want yourself replicated.

[20:45] Jaeyeon: Yeah. Mm-hmm. And so, I looked a bit farther into this aspect, and there is a lot of researchers that are now working on AI. And I also found another paper, which is also in this journal of nature, that found similar findings, but the idea was that it's not that AI is better than human in radical idea generation. They also find that it's equal. And they, kind of, hinted in a way that there is a hope that human can be better, but not that AI is better.

[21:18] Maya: Well, because there’s also that danger that people are worried about with AI, that it's going to take over the world. And, you know, that is a bit frightening when you have people that are, like, you know, this needs to be regulated, there needs to be some guardrails put around this. What are your thoughts on that?

[21:32] Jaeyeon: I mean, I think that's definitely true. A lot of legal institutions are trying to make a law around that. So, I also have been examining AI art, how AI creates such wonderful, you know, masterpieces without any soul.

[21:48] Maya: Art without soul. I like it. No-soul art.

[21:54] Jaeyeon: Right. And art is all about, like, being authentic and, you know, expressing who you are. And because of that, like, people feel betrayed when they learn that this artwork is AI-generated. But, you know, what's interesting is that there's also AI artists on social media, like Instagram that… those people label themselves as AI artists. And they are not positioning themselves as someone who's stealing ideas from the others. Rather, they're saying that AI really provides them an opportunity to expand the limitations that they have. Many do not have all the money to buy materials to create their own stuff.

So, there's this positive side and negative side. Of course, there's stealing of the ideas is the biggest issue. So, nowadays, a lot of our legal institutions or language AI training, companies are trying to allow these artists to license their work so that they get, like, a small fee, like, Spotify letting musicians get a small fee whenever people listen to their music on Spotify. Similar term applies to these artists when they are providing their artworks to train their models. They get a small fee in return. Or, they can also exclude their products or their artworks from being utilized on the training of the model so that their artworks tend to stay authentic without giving other people a chance to, like, copy that.

So, there is a lot of movements that's going on and especially Europe, there is slightly faster movement than in America. But we'll have to see because I think the development of AI is quicker than how the legal system is moving forward.

[23:43] Maya: Yes, it takes time, legal issues. So, yeah, there's definitely that issue in terms of trying to catch up. You know, AI is, like, every single day, it's something new, right? And with ChatGPT, which I am new to, because I'm a late adopter, but now that I know that I can ask you what to get my daughter for her birthday, I'm going to, “That's my next question,” is I'm going to get on and be like, “What does a 13-year-old want for her birthday.”

But then, but the issue though with that is, and I want to know what answers your experiment came up with, is that ChatGPT doesn't know my 13-year-old and doesn't know her likes and her dislikes and whether, you know, she's on social media, which she's not. Just need to put that in there. But, you know, in terms of the things that she's drawn to… so, what did your experiment say that, in terms of gift-giving, what is it that the teens want?

[24:37] Jaeyeon: Yeah. And, you know, as you said, teenagers are very hard to satisfy. And because of that, we gave many restrictions, sometimes little, sometimes a lot. Like, you have to think about the educational aspect or the child really cares about aesthetics or being environment friendly because we have to be, you know, sustainable in every aspect that's possible.

We sometimes gave, like, three restrictions, sometimes, like, six restrictions, talking about, like, it should be about technology. And just to make it more real, every person got different set of restrictions. So, some people got restriction criteria on sustainability, some people more aesthetic, some about technology or, you know, with music related. One thing that I remember, and I'm not sure if this exists, is a solar battery charged portable charger that has an engraving of her name. This is, like, one those.

[25:30] Maya: So that it's hers and her sister or brother can't take it.

[25:33] Jaeyeon: Yeah, and it's a solar panel.

[25:35] Maya: That's fascinating. I would have never thought that that is something that would be of interest to teenage girls. So, AI is reshaping just about every single industry. And what direction do you think that it's going to go into?

[25:51] Jaeyeon: That's a very interesting question because I've been watching seminars and being into classes just to find out that answer as an audience and now you're asking me.

[26:02] Maya: I am.

[26:04] Jaeyeon: So, when people are asking whether GPT will fully replace these professional workers, well, in pre-confined, it only has a certain limit on what it can say, because it may be good in providing an average-level response, but if a person is really professional, the person who understands the legal term of how to create a dining table, what's allowed and what's not allowed in the industry, whether a manufacturer facilities can actually make it or not, those are knowledge, it’s not something that a GPT may have a full, you know, access to.

And I think understanding a broader aspect of how the industry works is critical for humans to fully have a control on GPT and to use it as a tool to develop their career further rather than, like, replacing here or there. So, you shouldn't be worried about yourself being replaced by AI, but you should be worried about you losing a job because of a person who better uses an AI.

[27:09] Maya: Yeah, that, I haven't heard that. That's really profound. Like, you're not going to lose your job because of AI, but you may be replaced by somebody that has the capability to better utilize AI.

So, let's move on to YouTube and misinformation because I don't want to miss this part because this is also something that you've spent quite a bit of your time exploring. And YouTube is a different kind of social media in that it's been around for forever. I mean, I think YouTube's been around longer than Facebook, right?

[27:42] Jaeyeon: Yeah, yeah.

[27:43] Maya: But YouTube is a social media platform that is also known to propagate a lot of misinformation that goes unchallenged. And so, tell me about your research about YouTube and misinformation.

[27:56] Jaeyeon: Yeah. As you said, YouTube has been there for a longer period of time. To my knowledge, it really started to take off from, like, 2012 or ‘13. So, it has been more than 10 years. But during that time, not many people used that for serious information. YouTube was mostly for entertainment, fun videos. The first video on YouTube was about zoo animals, so it was completely [crosstalk 28:20].

[28:20] Maya: I remember that.

[28:22] Jaeyeon: Oh, you do?

[28:23] Maya: I do. I do.

[28:25] Jaeyeon: Yeah. So, now, like, everybody's watching YouTube, like, not many people watch TV. YouTube has become, you know, part of our daily lives. What really led to the success of YouTube is that it really does a good job providing you with personalized content. So, it's really good. It's, like, one of the best platforms that I've seen and on different social media platforms that really understands your watch history, what people similar to your interests watch. And it does, like, recommend videos that you may like. And if you start watching, it just provides more and more. And this algorithm is a success factor, because the more you watch, the greater engagement, the more advertisement you show, and then that's how they make money.

[29:10] Maya: The more money YouTube makes.

[29:11] Jaeyeon: Right.

[29:14] Maya: It's economics.

[29:16] Jaeyeon: Yeah. But the problem here is that, because so many people are relying on the algorithm to find what they want, they are now getting exposed to all types of information that they'll not even search for. So, on the homepage of YouTube, you see some of the videos that are from your subscribed channels, but a lot of the channels are pretty new, but related content. So, let's say that you have an illness and you watch certain health video from a doctor, but then there's this related video about the same illness from all these influencers, and you start to watch all these related videos.

[29:51] Maya: They're not doctors. They're influencers.

[29:56] Jaeyeon: Right. The problem is that people don’t care about it.

[29:58] Maya: They're not paying attention.

[30:00] Jaeyeon: Right. So, the basic finding is that there is a degree of people who tends to rely more on the recommendation algorithm versus who stick to what they only watch. And these people who are more open to algorithm are more susceptible to misinformation. So, how I do this is to measure our recommendation reliance, or sometimes we manipulate it by assigning people into high reliance or low reliance. And then we show them different sets of videos. For example, keto diet has been widely popular. It works for some people, but it doesn't apply to every single one who has all the existing, you know, health conditions. But this doctor, like, self-claimed doctor… by the way, not all doctors who have the doctor's license are, like, telling the truth on YouTube.

[30:46] Maya: Right, right, right.

[30:46] Jaeyeon: And then this person is really arguing that everybody really should adopt keto diet. And the more people rely on algorithm are more open to this argument and they're willing to buy keto product and they say that they believe in it more and they're more likely to subscribe to this channel. And that can lead to all the harmful consequences that people have to bear on their own.

[31:10] Maya: So, what do you do about that?

[31:13] Jaeyeon: It's a hard question because YouTube has been saying that they're trying to fight for misinformation. And the problem to their approach is that they ask people to flag information that seems incorrect. But people who rely on recommendations are believing in it before they even judge it to be true.

[31:32] Maya: There's a bias in that.

[31:33] Jaeyeon: Right. And a lot of things are in the shady area and people never report it as being wrong. And there's, obviously, news that are, like, completely fake, but there are a lot of, like, shady news that are partly true, but not applies to every single human being, or it's only showing the partial truth, but not the full scope of truth. So, that's a big problem. And nobody has actually come up with it. In my experiment with my co-authors, we tried to implement it within our experiments, which worked. And let me just talk about it a little, which is that, we tell participants that AI, which is now so popular these days, is evaluating all the auto-captured captions on each video and fact-check it. And whenever it's fully fact-checked, it gets the green check mark.

[32:23] Maya: Verified. It's verified.

[32:25] Jaeyeon: Yeah. With the one that, that doesn't. And human can't verify all information, but the appliance of AI can, perhaps, help that. And when we show them that it's not verified yet, of course, they are much less likely to… they, kind of, put a brake on their believability and, like, try to assess whether that's true or false. So, that's one way of fighting for misinformation, which has not been adopted in the industry level, but let's hope that can be used, I think.

[32:53] Maya: Not yet, yeah. That's AI to benefit all of humanity, to verify truth or fact or fiction. I mean, it’s not biased as humans are. And hopefully, that remains the same. That's the fear, I think, is that you can manipulate AI. So, that's the part that needs to stay neutral, right? And that can actually benefit humanity. So, Jae, tell me about some upcoming studies that you have in the works at Rice and what you're the most excited about for the future.

[33:24] Jaeyeon: There’s a lot of projects that I'm doing currently. By the way, the project on misinformation is still work in progress. So, it's currently being revised for a deadline that's upcoming. Let's hope that I'm finishing that before the deadline. I also have a project on NFT and what kind of NFT art tends to get more popular over time. So, this is also a project that's currently being revised. You know, NFT art is, like, digital art, like, board…

[33:54] Maya: Non-fungible token, correct?

[33:56] Jaeyeon: Right, right, exactly. So, like, Bored Ape, a lot of people know, CryptoPunks are pretty ugly pictures, but they're also quite, you know, good-looking digital images that are sold. And NFTs are, basically, like, blockchain-based code. It’s the strings of letters and numbers that are attached to each digital work.

[34:17] Maya: [crosstalk 34:17].

[34:17] Jaeyeon: So, whenever you… yeah, you pay it, you get it, you transfer ownership. And everything is fully in the history in the digital world. So, you can claim for the ownership of your digital art. And because art is not only physically bounded, but it's really expanding to the digital world, I'm trying to see which digital artwork tends to get more popular over time.

[34:42] Maya: Like baseball cards.

[34:44] Jaeyeon: Yes, yes, exactly. And some of the artworks are really, like, ugly. Within the same collection, not everything gets popular. One thing gets even more popular than the others. And people have really no idea.

[34:57] Maya: Something tips and something doesn’t.

[34:59] Jaeyeon: Yeah.

[34:59] Maya: Well, it's like Malcolm Gladwell's tipping point about Crocs. You know, like, why did they tip? And what was it about them? And why certain things have this gravitational pull towards and other things, that you would think that they would, don’t.

[35:13] Jaeyeon: Right, right.

[35:14] Maya: And that’s psychology.

[35:15] Jaeyeon: Yes, yes. So, I try to examine what leads people to choose one artwork over the other. To do so, I go to OpenSea, which is, like, an Amazon version of digital artwork. All the NFT artworks are there. So, I go to Bored Ape, which is the most popular, the collection that has the most cryptocurrency invested on it, and examine, or actually download the number of likes that each artwork, each of the 10,000 artwork gets each day for a duration of 150 days, which is around five months. And I try to examine which one gets a steeper slope of like increase than the others, and then try to examine all the aesthetic aspects and all the minting time, which is the date in which it was uploaded, and et cetera, and find out which one is more popular than the others.

[36:06] Maya: Well, that is something that we will certainly look forward to, of some of your upcoming research, along with the misinformation that you're still working on. So, Jae, it has been a pleasure to have an opportunity to talk with you and learn about the really transformative research that you're doing at Rice. And we can't wait to see what you have up next. So, thank you for being a part of Owl Have You Know.

[36:29] Jaeyeon: Thank you, Maya. It was a true pleasure. It's my first time doing a podcast, and you made it so conversational I just enjoyed this time so much.

[36:41] Maya: Thanks for listening. This has been Owl Have You Know, a production of Rice Business. You can find more information about our guests, hosts, and announcements on our website, business.rice.edu. Please subscribe and leave a rating wherever you find your favorite podcasts. We'd love to hear what you think.

The hosts of Owl Have You Know are myself, Maya Pomroy, and Scott Gale.