Joseph Rueter: [00:00:00] So aside from not wanting cash grabs and that people find everything which you're working on after a good run, 5, 000 plus games, do you have life lessons, like takeaways from building games, how has it impacted you, how do you talk to aspiring creators about their game efforts?
Andrew Bartle: for me, it's a cliche, but I think you've got to make sure it's something that you have some passion for to make it successful.
If you're just going into it because you think, Oh, if I put this out there, the people will come and then I'll get ad revenue. And then
Nate Kadlac: Welcome to the Hey, Good Game podcast, where we chat with the creators of your favorite games that you secretly play in the cracks of your day. I'm Nate Cadillac and I'm here with my cohost, Joseph Reuter. And today we're excited to speak with Andrew Bartle, the creator of QuizWise, the quiz game for serious trivia enthusiasts.
QuizWise. QuizWise. com. [00:01:00] Andrew is a veteran software developer with over 25 years of experience as a Microsoft. NET principal developer. He founded PeerWeb Limited, the software and website development company that created QuizWise. and co founded Storby, an online shop builder tailored for independent retailers, especially for vet practices and pharmacies.
QuizWise is a web game created by Andrew alongside Marjorie McKee and the late Barry Lakeman. Filled to the brim with all kinds of general knowledge, quizzes, you can also enjoy seven different quiz categories and even create a personalized quiz that fits your subject perfectly. Andrew, we're so glad you're here.
Andrew Bartle: Thank you. It's a pleasure to be here. I'm looking forward to talking to you.
Nate Kadlac: We usually kick things off, Andrew, with what's your favorite game to play these days?
Andrew Bartle: Favorite game to play? we got caught up in our family and, in the Wordle craze. My brother created a Wordle group, which is a lot of fun.
I think what I enjoy the most about that is just connecting with family and close friends [00:02:00] in that group each day. Just that little connection there. And then of course we've expanded that. Over the last few years, there's a few other games that we play in there. I think, the New York Times gets overrepresented in that group.
Certainly my favorite game that we regularly play in there is Connections. I really enjoy it. I love it. All those sort of aha moments when you click to a clue and it's got a bit of risk taking. It's very satisfying when you get it. And I think the clues themselves are very clever. And the little red herrings that they put in there.
So Connections is a favorite on there for sure. And Strands I do, regularly do with my 13 year old son. He's very he's very good at those sorts of games. So that's a, another one on there I particularly enjoy. Yes, those would be the main daily games that I play and share in that group.
It's a really enjoyable little part of my day.
Joseph Rueter: That's fantastic. We've got a certain set of us that are good at certain games and not. I [00:03:00] wonder if we can run across somebody somewhere that's done some research on which parts of the brain are firing for certain games or something like, you're going to be this kind of human in the future, right?
They're all in grade school. It's that one's not a scientist. Yeah. Yeah.
Andrew Bartle: Yeah. Yeah. Yeah, it's true. Different sorts of brains work best for different sorts of games. So yeah, I'm saying, yeah, that's true. Different types of, synapses they have to fire and connections need to be made to bring together the different.
The different elements of a game to solve it.
Joseph Rueter: Yeah, but you've been around this, decades, right? Late nineties, got into games.
Andrew Bartle: Yeah. Games generally. absolutely. Yeah. I was very lucky to grow up in a household that always had computers. I think I was about four or five years old when my father arrived home with a ZX81.
And that was, my brothers were probably able to take better advantage of that point than I was, but I certainly remember The challenges [00:04:00] of loading up, loading a game off a tape deck and you're just crossing your fingers, hoping it's going to work and typing programs in and out of a book was quite common, you'd have a book with the printed code in it and you'd be typing it in and of course, I'd make all sorts of errors that I didn't understand and once again, my brothers would have to come to the rescue and point out that I've mistyped a few characters and that's why it won't run.
Then sort of Apple IIe's and and then onto PCs. So yeah, I was very lucky to have that exposure with my brothers interested in programming and to always have computers in the house. So, I'd always be tinkering and creating little programs and then little games for, myself. That's always been a part of my life.
Joseph Rueter: That's awesome. So you were Pair coding in the nineties,
Andrew Bartle: I don't know if it was so much coding. I think actually when I worked my best, that's when I just in my own little world, that hyper focus you can get where you're something's [00:05:00] inspiring you and exciting you, and you're just doing it. Getting into it, of course, the challenge with any of those sorts of things is seeing it through to completion quiz wise was one of those examples, rare examples where I, pushed on and actually put it out there rather than did the fun 80 percent and then, realized that maybe I had some other more important, remember my other important priorities that I was, what I was supposed to be working on when, I'd get to that last 20 percent where actually, you've got to polish everything up and actually get it out there.
Got a lot of those.
Nate Kadlac: So you, your brothers and you, are you all in development these days or is it's a fun shared experience growing up, I'm sure.
Andrew Bartle: Yeah, that's a good question. They're quite a bit older than me, seven years and nine years older than me. My oldest brother, he's over in the UK.
That's Philip. He, so he is very much scripting. He's working with big data, if you like, across medical trials. [00:06:00] Very much programming as a part of his job, but quite different from what my other brother, Shane, and I do. So Shane and I founded Storby, you mentioned at the beginning there, an e commerce platform that was originally targeted at very It was small people wanting to run a little business online from home, but ultimately found it's a niche, particularly as lots of competitors popped up in that space as they do. And we've now fallen into this pharmaceutical healthcare niche and which we've been successful with here in New Zealand and Australia.
And now just starting to get a bit of traction in the US, which is very exciting for us. So Shane very much a developer and certainly helped me, develop my skills in the area. My first professional experience was, working with him on contracts. Yeah, so both of us have a very wide, we're in New Zealand.
It's a smaller market here. I think people, are [00:07:00] often less specialized here. And so a real sort of jack of all trades get in there and, make it work. Infrastructure, the database, the web, the, and the bit in the middle, that's the background there. when it comes to creating things like quiz wise, I'm not afraid of, the technology at all.
I'm confident in my ability to build it. The challenges come from elsewhere, of course, the ideas, it's not hard to have ideas. I have a lot of ideas. I've got a notebook that I, use as an outlet. I'm sure we all do use an outlet, jot down anything that pops into the head. But of course, actually, deciding which one to do and executing on that and seeing it right through.
That's, the hard part. I tend to, being self employed, I'm very reactive. I'm always dealing to all of the things coming from clients. Or customers, and I don't prioritize enough doing the fun things that are in that notebook, I think.
Nate Kadlac: I think we [00:08:00] can all feel very familiar in the, idea phase.
Yeah. There's a lot, else to do. So when, is quiz wise the first game that you worked on or is there something before that? Good
Andrew Bartle: question. I always, as I say, as a kid, I was always making little games. Not really, just for myself really, And then when Doom came out, when I was a bit older, it was, making doom maps to play with my friends.
But actually the first thing that I had that kind of went viral was, you remember the old MSN messenger, so Microsoft's instant messenger of the day, and I can't quite remember what inspired me, but I managed to, it didn't have any kind of open sort of API or official way to communicate with it, but then actually.
Down with the protocol level, it wasn't too difficult to, get in there and pretend to be a client. And I wrote an early chatbot. I wrote an early chatbot called DrApeBall that hooked into it. [00:09:00] So you'd type something and it would respond with answers. And I didn't know anything about how to actually for a start, the technology obviously was much younger, but to actually use any kind of deep learning AI kind of techniques to create it, I just created a, basically how I did it was, the concept of an eight ball, right?
So yeah, you answer the question, you shake something. And it would come back with a response. And so how it worked was that it would come back with, it would do some really basic matching on what you put in there. And so if you started with a question of what blah blah blah, then it might come back with a different random answer out of a list than if you said who blah blah blah.
then it might come back with just random names. And built it up from there, and what I found is that people generally asked it the same sort of stuff. And so by just expanding it, so they were starting to ask, what time? Then I could guess they were asking something about the time, and just expanded [00:10:00] from there.
And actually got a lot of people using it. And things they would say to it were quite outrageous, to be honest. They would invite it into their group chats with their friends. And yeah, that was spreading like wildfire. And I would go take the logs and I would look through and every time I saw that it said something I didn't like I'd go and expand it to, give a better answer next time it was asked that.
It's Achilles heel with spelling because I didn't have anything to do with the fact that people spell stuff terribly and I didn't really know how to manage that at the time. But yeah, so it wasn't quite a game, but it was, something entertaining that I put out there and then it was just incredible to watch it take off.
Yeah. And I remember my boss at the time, cause I did it at work, got very excited about the possibilities of it, but then priorities moved elsewhere as they do. And the MSN is no more.
Nate Kadlac: did that just die on the vine or did that, go anywhere after all that traffic?
It did die
Andrew Bartle: on the vine. I think, I left that company and it was [00:11:00] entirely my own sort of piece of work. And so I don't, think anyone else picked it up or anything. And I felt like I couldn't really use the technology because I had done it there. yeah, but of course, as I say, as the message has gone down and now, Chat GPT and, the likes as well, and truly, taking that space, it was only so long that technique was going to improve people now that we have the extraordinary AI tools that we do.
Yeah, truly.
Nate Kadlac: I'm curious, what's the story behind QuizWise? When did you, start working on that and, I'd love to hear a little bit of how that came to be.
Andrew Bartle: Sure. the motivation for creating QuizWise originally, it's very much evolved over time. But I was a web developer, people were asking me all the time about marketing, search engine optimization, how do we get our site out there?
I was interested myself, both for, yeah, for my own work and for the C Commerce platform we have. And [00:12:00] one of the problems I had, New Zealand's quite a small market, sites that I were building were often quite niche. If we take Storby, for example, we were using Google ads to promote Storby and we created a range of different ads and we would be checking the numbers to try and work out well, which ad is the most successful, which one's getting clicked on the most, which one's converting the most because we were all geeks, two developers and a designer who started that marketing was really quite a foreign concept to us.
And of course, we always were trying to look for ways to fall into what we were comfortable with and setting up Google ads and then playing with the statistics and the numbers with something that us geeks could cope with. But again, the numbers were so small, it was hard for them to be significant.
We'd suddenly get excited because we think that we were getting more conversions on Wednesday afternoons, that we should pump up our ads on Wednesday afternoons. But the chances are it's actually just a blip. And so I thought [00:13:00] it would be really fantastic to create a website with a wide appeal and then to be able to hopefully build up a little bit of a general audience and be able to put out, do these little A B tests or try something out and actually see enough significant change to know whether or not it was working.
So that was the original inspiration. I then thought, what has a wide appeal? Okay, a quiz website. And how hard can it be to write just seven questions a day, What's the capital of Argentina? Oh, that was obviously very naive. To do that in a sustained way is actually extremely hard.
And so then I think I dabbled with, okay, I'll just buy a database of questions. I'll just buy a database of questions. The site doesn't have any traffic. It doesn't have any traffic. really matter the, site doesn't have to have its own voice yet. And so I dabbled with that, but I think I was pretty unhappy with the sort of diversity of the questions and the quality of the questions.
And [00:14:00] that's when I had an idea of asking my dear friend Barry. So Barry and I, we met a mutual friend and Barry were looking for a third actor to play a part in David Mamet American Buffalo for a local theater. And Yep. So I entered that project with him playing Bobby. And since then I've been involved in many projects with, Barry and his wife, Marjorie, who has similar interests.
Barry, it's one of those remarkable people who. Always had this enthusiasm for fun projects and then would draw his friends into these fun projects and I think he also loved it when it was reciprocated, because he was often the one kind of getting these things going. Somebody came to him with a cool idea, he was often, would show a lot of enthusiasm and really help make those ideas work.
And I knew that Barry, he created a cryptic crossword for a local newsletter. [00:15:00] And I knew he was a member of a pub quiz trivia group, so I knew he, he had interests in those area. And I thought, Oh, maybe I'll ask Barry just in case he's interested. And he was, I couldn't believe it. he really threw himself into it.
there's no guarantee that people were gonna, come to the site and use it. But no, he did. He really threw into it, himself into it. And sadly, Barry died in 2011, pancreatic cancer. Which obviously was very devastating for all of us that knew him. And Barry was very methodical about the way that he produced the questions for QuizWise.
He would work in batches in particular areas. he might spend an entire day writing questions about the Olympics or the periodic table, and he would, fill out a spreadsheet with those. But what that meant, of course, nobody wants to be just doing questions about the Olympics for a month.[00:16:00]
you He would only include a few of those questions into each batch. And what that meant was that when he died, he still had a huge number of unpublished questions in these spreadsheets. And I'm very grateful to his wife, Marjorie. She agreed to take over from Barry in terms of producing these batches of questions.
A really key motivator for her was wanting to make sure that all of Barry's work was published. But of course, the questions were sparser in some areas than others. So it did mean that she needed to do a lot of work herself to research and write questions for the site to fill those gaps. And various questions, they're all well and truly published now, but again, I'm very grateful that Marjorie has agreed, has continued to, in that role to produce.
The questions for quiz wise, yeah,
Joseph Rueter: you're 5, [00:17:00] 743, it's been a big run,
Andrew Bartle: sorry, 5, the number of quizzes.
Joseph Rueter: I've never thought of
Andrew Bartle: the number that way. I think about the years. Yeah. Yeah. That's a lot of quizzes. There's a lot of content there for sure. Yeah, no, that's right. For Marjorie it's, it is challenging.
She's not, she. There are days where, finding inspiration for new topics is not easy for you, for sure. But in the early days, right through the time that Barry was writing the questions, And for a while afterwards, it took a long time for the traffic to build up to the site. And so it was only, I'd get together with Barry for, for coffee or, whatever.
And, we'd just joke that QuizWise was covering the cost of the, coffee. And sorry, are you getting those beeps? [00:18:00] So I was just getting some beeps of mine. No, you're okay. I'll say that again. Yeah. Yeah, for a long time, right through the time that Barry was writing the questions and for some time afterwards the ad revenue we were getting from the site, which was just me running with Google ad network myself, it would barely pay for the, we'd get together for coffee and, we'd joke that.
That it was paying for our coffee and so it was very much a labor of, love for a very long time. And, it's only in more recent years that the revenue has built up to a point that certainly not enough for me to give up my day job yet as much as I would love that as much as I'd love to be able to really make it, make Quizwise and other projects like Quizwise a really huge part of what I do to something that is contributing and rewarding us a bit for the time that we invest into the site.
Nate Kadlac: QuizWise was started in, was that mid 2000s? Early?
Andrew Bartle: I think, yeah, 2000, I think we've got [00:19:00] quizzes published on there back to 2010. Okay. I think it was perhaps 2008, 2009. I had a, maybe a little bit of a reset. It was actually on a different domain originally, freequiztrivia. com, I think it was.
And then I decided that, yeah, I would have chosen something with keywords again, in many ways, but I started it. It was a little bit of a sort of experiment about ECO and those sorts of things. But that quickly evolved, and changed to quiz wise, we wanted something a little bit, that sounded a little bit more serious, because at that point we were starting to take it more seriously.
Not just as a piece of entertainment, but something that's always been important as the educational value of the site. I know that remains the case for Marjorie. She, really thinks a lot about not just what's a good, challenging question to ask, but also what will people learn from the site?
What information is from, [00:20:00] interesting and educational. It's
Joseph Rueter: fantastic. There's one today where I was like, ah, certainly I know this one. Where did princess Diana's, I'm like, Oh, Paris. So I just picked the first one. And of course it's not. And that's where I stopped and went wait a second. This is multiple choice, like standardized tests.
This is, high school. Why am I doing this voluntarily?
Andrew Bartle: Yeah.
Joseph Rueter: What kind of reaction do you have among your peers? Consumer base, your players, your education themes, your, who do you know that plays this regularly and what kind of feedback do you get?
Andrew Bartle: that's a good question. My mum plays it regularly, so I'm grateful to hear.
because, that's one person who can point out we've made a terrible mistake. But yeah, if we get feedback, of course people are much more likely to tell you when they think something's wrong than when they, that's wrong. yeah. Yep. That's wrong. We do occasionally. we take all the feedback that we receive very seriously.[00:21:00]
We certainly read each. Each one and give it a good thought. But yeah, sometimes you get the ones that just say, Hey, love you. Love your, site. I do it every day. That's really fantastic. Yeah. The most common, feedback we'd get would be about a particular question challenging if it's correct or pointing out typos, if they've slipped through.
Yeah, a lot of time, of course, they haven't read the question properly, or there's some misunderstanding there, but even, if there's a misunderstanding, that can be a prompt for us to go, how do we tweak this? I'm a bit of a believer that if one person has misunderstood a question or had some issue with a quiz that there's probably thousands of people who haven't reached out.
So certainly we'll always look at that. The quizzes, because they've been published for so long. Of course, sometimes somebody will point out that something is, no longer true. So one of the key decisions right at the beginning was that I really didn't want it to be filled with content that would quickly go out of [00:22:00] date.
publish a question about what happened last week that nobody in five years could possibly remember. So it was always a goal of the site to try and write questions that were at least a little bit timeless. But of course, it's impossible to write a good quiz. Without the possibility that some things are going to change, the queen might die, then the queen's council might become the king's council.
And sometimes people will point out in old quizzes, but we always clear, when you're looking at a quiz, if you're going back and doing the archive or a random quiz, it'll show the date. So if you're doing one from 2012, you'll want to bear in mind that it was published in 2012 and that's where your head needs to be for the answer.
But having said that, we will go and retrospectively correct a question, but whenever we do that, there'll be a clear note saying this question was updated in October, 2018, so that you always know if you're answering a question where your head has to be [00:23:00] at in terms of time. But again, we try and avoid that to a degree by not.
We'll certainly never say what happened last year, we'll try to have questions that will remain generally accurate wherever possible.
Nate Kadlac: Yeah. So your daily puzzles, daily games have been around for a long time, but Wordle obviously impacted kind of the game economy, at least browser based games.
Did you see any impact once Wordle started taking off just in terms of daily users around that time? Maybe that was the pandemic too, but curious, like what happened to QuizWise during that time? Absolutely.
Andrew Bartle: That's an excellent question, and I wish that I had some numbers to look at that, because if it did, I didn't particularly notice.
For me, it was very much something, I haven't been someone who's been looking at the numbers every day. I tend to go through little pockets of time of looking at it. So I didn't particularly notice with Wordle as such that there was that growth, but [00:24:00] obviously there has been growth through the years.
But yeah, I wouldn't say that I've particularly noticed key moments of growth with Wordle. I've been quite, I tend to be quite reactive in the work I do again, because of things, that are always coming at me. at me and being quite busy. And so I tend to work on QuizWise, there'll just be those moments of that feeling of passion of, I really want to implement this.
I really want to do that. Often, it's on the day because right now I'm excited about this idea and I'll leap in and I'll, do that. And so I haven't particularly made changes to QuizWise in relation to what we saw with Weirdle, but I probably should. And right now I'm feeling inspired.
Maybe we'll hang up and I'll go and do it. because obviously there's a lot of potential there around like a glaring omission that I really should add would be a button to click to share it to your friends, on WhatsApp, like all of those games do rather than, I've seen some sort of Facebook groups, they'll just go and they'll just [00:25:00] post, I got six out of seven, I should make that easier, shouldn't I?
And. Being able to actually track your progress over time, I'm sure something that people would like. People don't tend to send a lot of feature requests, but I do get emails of people saying they want to subscribe. Please subscribe me to your quiz. And I'm always thinking, that's a good idea, but I don't currently, I haven't really thought that through yet.
Opportunity that I haven't taken advantage of the year.
Joseph Rueter: I'm pretty sure I wouldn't want to share my results today. I only got 43%. they're not easy quizzes. So the
Andrew Bartle: way that the difficulty is why
Joseph Rueter: I thought maybe I was in high school taking a standardized.
Andrew Bartle: That's right. They're not easy. How the difficulty works is, so every day there's seven questions and as you mentioned at the start, there are seven categories like for your pursuit, so you've got science and the entertainment categories, et cetera.
And so there's one [00:26:00] question in each category each day. And what that means is that every week there's also a roll up of questions from that each category. So on a particular day of the week will be a science quiz that's released that has the seven science questions from the week before. So you can treat it as a weekly quiz if there's just a particular category that you're interested in.
And each day, We try and have a mix of difficulty. So Marjorie obviously just has to make a guess at how difficult the questions are. it's not always just about how hard the question itself is, but you have a fair bit of control when it comes to the options, the multi choice options and how ridiculous you make them versus how close to the true answer you get.
And she'll have a go at guessing this is how difficult I think it will be. But of course, everyone's in their bubble, and what she or we might find, think everybody knows around the world is not actually the case, [00:27:00] and vice versa. But that's okay, because that just means you're going to have some quizzes that are going to be easier than others.
You do want that. I do run something that actually calculates what was the true difficulty of that question, but that doesn't affect, of course, what quizzes it was published in. But that would affect, like you mentioned, that there's the ability to actually go and say, I want to generate a random quiz with these categories and these difficulties.
They'll generally work off actually what was the calculated difficulty. So that works, quite well, I think. Yeah, and I'd like to actually, yeah, dive into how to use that information that we have.
Joseph Rueter: Yeah, fire away. is that target user a teacher of history or something of that sort?
I could see myself wanting some help if I was teaching a load of humans. like 23 curious minds and seven of them are paying attention. How do I, get them to focus? [00:28:00] Yeah. So a tool like this, if you're describing that, are you saying I go here and I say I want X and I get X?
Andrew Bartle: Yeah, you could go and say, yeah, I want a history quiz with, seven, seven, easy questions and that's what it will give you.
Yeah. it'd be great if a teacher is using that up in front of their class. Yep.
Joseph Rueter: I want a medium sport quiz, go. Oh, dope. Wow. How do you get people to come and play? that's a killer feature. How many people know about that? I don't know.
Andrew Bartle: Who knows? I don't actually even really track the statistics of how used that is.
they'll be there. But, I just tend to be focused on the, on those daily quizzes and number of visitors. Yeah, there's a lot of questions there now.
Joseph Rueter: What's the technical term for measles? Measles? this is a medium question.
Nate Kadlac: I'm not about to, [00:29:00]
Joseph Rueter: Come on, we can stream this later on, the YouTube machine.
Nate Kadlac: I am curious, do you have an idea of how many people are playing your games every day?
Andrew Bartle: yeah, sure. So I think, we get about 250, 000 page views a month. it looks like in terms of completed quizzes, We get about 140, 000 completed quizzes each month.
Nate Kadlac: Yeah, that's impressive.
Joseph Rueter: Out of seven?
Yeah.
Nate Kadlac: Especially because it sounds like you're not doing any marketing or, it's mostly word of mouth, I'm guessing at this point. That's right.
Andrew Bartle: Yeah. Word of mouth and, search engines. Yeah. Yeah. that's it. Certainly there's no. Yeah. Yeah. Yeah. No marketing of any form taking place and no paid advertising.
Played with, that very briefly in the early days more again as an experiment around, yeah, what sort of marketing. Works, if [00:30:00] my experience certainly at that time was paying for ads to bring somebody to a site where, which is funded by ads was not, was never going to really, was never really going to bring a return.
Unless what they landed on was super sticky. And of course, we do have users who not only visit every day, but work their way back through the archive. But yeah, I'm sure the majority of people that land there are going to go, Oh, this is cool, do a quiz or two and then close their browser tab. So again, that was more for experimentation, but no, I haven't made any efforts to market it.
And certainly haven't invested the time, a huge amount of time, even in optimizing it. Again, a lot of sort of ideas about how we might be able to do that, create kind of specific quizzes to target particular keywords, those sorts of things. I think we did publish a Christmas quiz once. I haven't checked.
Does that rank? I'm not sure. [00:31:00] So yeah, so some ideas there to think about for the future.
Joseph Rueter: But on those numbers, almost 5, 000 people finished a quiz today. you're in tomorrow already, but 5, 000 people hanging out and finishing is fantastic.
Andrew Bartle: Yeah, it's pretty cool. I forget myself and I see the numbers and go, wow, this is cool.
It's a lot of people. Getting enjoyment. You don't want
Joseph Rueter: that kind of line outside your boulangerie in the morning, right? Like it'd be a long time waiting for coffee if there's 5, 000 people in front of you.
Andrew Bartle: This is true. And what I, it'd be interesting to know too, just what potential that gives to, if I produce other, another game or other ideas, how much that can be leveraged.
To help her get exposure to those. I'm not sure. I did listen to a couple of, had a chance to listen to a couple of your other interviews and certainly, yeah, some of the others, those, jealous of those [00:32:00] who have managed to make this their full time job and start working on other games.
I'll be interested in hearing more about some of their experiences of doing that. I did find one word search through that. That was the guy that created waffle, I think. Yeah. And that was quite fun. so my son and I have been doing that one a bit together. Yeah. But it also sounded like perhaps it's not just as easy as putting another game out there and putting a link and saying, try this new game that trying to recapture that magic of what really does grow and, capture people's.
attention and get shared and have that go viral magic that we're all looking for with these things.
Nate Kadlac: going back, it sounds like there wasn't really any kind of meteoric inflection points in the game's history, but do you remember like getting your first thousand users? Do you remember any early milestones and what you did to get there?
Gosh, I want to [00:33:00] say, yes, I've got a terrible memory for these sorts of
Andrew Bartle: things. It was probably a really big deal at the time. I was probably jumping around the room, but I actually don't remember. Honestly, don't, probably changing ad providers would be a time when it's most significantly see some kind of change.
I went from years ago, I went from just having my own Google ad, ad sets on the other. And I get emails just. Every day, all the time now, from providers like that, trying to actually know what to pay attention to. They're,
Nate Kadlac: they're growing in numbers too, I believe, it just feels like they're,
Joseph Rueter: I think AI is, my inbox is just set to filter if the word ad is used.
Andrew Bartle: To be honest, I very almost missed your invitation because, I just, thank goodness I actually, yeah, if I pay attention to it. Yeah, I feel a bit bad because, I've given up thinking about replying to emails. [00:34:00] To the so many offers of of people wanting to publish articles on the site or add links to the site or take over managing ads for the site.
But yeah, I switched to, Freestar and years ago and that, yeah, that was a successful switch. I had to use it, move away from them more recently. So I've experimented with a couple of ad providers, ended up on Raptive now, who I'm very happy with. I don't know if you guys have had this experience. The really stressful things about changing ad provider is there seems to be this little period when you first switch of kind of that little teething period where it's very hard to really know when you're about to change ad provider, exactly what the experience is going to be.
They can set up on a test site, but without the real advertisers properly in place, that it's hard to, once it actually goes out there, suddenly there seems to be a lot more ads, they're more in your face, they're popping up where you don't expect. And you can get on top of that, you can working with their providers, [00:35:00] they're generally very good.
we need to pull this back or, and you don't necessarily see people in different markets are seeing different things, different quantities. And so both the changes that I made recently, there was this kind of really horrible couple of days where I was, cause I've got all of these loyal visitors who are coming.
To the site every day and the last thing I want is them arriving and just being like, Oh, they've sold us out now. It's just a cash grab. And they've got ads up in their face and yeah, so it's it's quite challenging. And then you get things settled down to what you hope are, to what you hope is reasonable compromise.
Because of course, nobody wants ads. I don't, I'd love to be ad free on the site, but that's how the bills are paid. So always trying to find that balance.
Nate Kadlac: Yeah. we are no strangers to that experience and or people uploading like incorrect creative and it just like breaks. The bounds of what you've set, [00:36:00] and yeah, it just, it can be, we've gone through a few providers ourselves.
What was the impetus for, what were you looking for in an ad provider when you were making this last switch?
Andrew Bartle: certainly, revenue, obviously, it does seem to vary. I think the mistake, we've got a very large portion of our visitors are from the UK. In the UK, they really do seem to like the, their puzzle games and their pub quizzes as we do here in New Zealand.
And I think that was a key thing is working with, the bigger ad publishers tend to be. U. S. based and the one that we had switched to, that's where I really saw a huge fall off was in the revenue coming from the UK. They did not seem so well set up for that and switching to the new one that seems to be much better.
But a big thing for me is trying to find that compromise that it is important to me that people, the last thing I want is any content going over the top of the quiz, blocking you [00:37:00] from being able to read them. There's always going to be a certain distraction element, but I want people to just be able to do the quiz in a, in an uncompromised way.
And so knowing that they would work with me on that. And another part of it for me is, again, I am quite reactive. I'll have periods where I'll focus on it. But a lot of the time I've got things coming at me from lots of other directions, so it needed to be one, a place that would be there and responsive when I needed them to be, but at the same time could just keep things, I could trust that they would keep things ticking away in the background when my attention was somewhere else.
Joseph Rueter: Yeah, I did have an idea next time we change ad providers, we should just throw a model and just fire them like this is a cash grab. That's what it's going to be right across the
Andrew Bartle: top.
Joseph Rueter: You remember 1 million web page or whatever? [00:38:00] Now I want cash grab, I'm
Andrew Bartle: sure I'm sure these big boxes, right?
Joseph Rueter: Not wanting cash grabs and that people find everything bad with what you're, working on. Do you have after a good run? 5, 000 plus games, do you have life lessons like takeaways from building games? How has it impacted you? How do you talk to aspiring creators about their game efforts?
Andrew Bartle: for me, it's a cliche, but I think you've got to make sure it's something that you have some passion for to make it successful. If you're just going into it because you think, Oh, if I put this out there, the people will come and then I'll get ad revenue and then be able to lie back on the beach.
It's not going to work that way. That way you really, it's just going to feel like a day job that doesn't earn you much money. You [00:39:00] have to enjoy it. And when I have an opportunity to actually really work on the site and add something new to the site, I get excited about that and I really enjoy it.
And I enjoy thinking, even though I have a lot of ideas that I don't have a chance to execute for new games or other ways we could present the questions and things. Daydreaming about that spot, even if you don't actually do it. Yeah, I'd say it's going to be interesting to see what happens with the landscape over the coming years.
Now with AI being what it is, our questions are not written by AI and that we're certainly not planning to start using AI to write our questions. I'm expecting that there's going to, we have been accused of it. We've had feedback from somebody who didn't like, like one of our questions and they said, stop using AI.
You won't be able to reply back. We're not using AI. There's going to be an explosion of content out there, and I think AI could, can be used to produce good quality content, but that in itself is hard work. if you just ask AI, give [00:40:00] me a list of a thousand trivia questions that I can put on my website, it's going to, there's going to be errors in there, probably they're all going to be the same kind of things that will be repetitive.
You're not going to actually get a good quality experience from that. The only way you possibly could with our generations to be really methodical and careful about, and actually be coming up with the kind of categories and all that kind of stuff I mentioned, it'd be a lot of work. And so I'm sure there'll be somebody who will do that, but I expect there'll be a lot of people who are just going to do the what are a thousand quiz questions, shove it out onto the internet, publish random ones every day and try and compete that way.
So we'll certainly want to be and stay different in that way, which isn't to say that I don't use AI in ways that feel Authentic, that still feel authentic to me and one thing that I have started doing quite recently with some success is to, so now whenever Marjorie supplies a [00:41:00] batch of questions, I'm running it through ChatGPT's API.
With a bunch of questions about the questions, is this question accurate? Are there any typos? Are the incorrect answers of the multi choice actually incorrect answers? we haven't inadvertently put another true answer in there. All these questions, to get back its feedback, just as an additional, quality control step for anything that we may have missed, because it's the worst feeling in the world to put a question out there that is wrong.
It's embarrassing. And if we're mis educating people, that's a really horrible feeling. And it comes back at the moment, I think it's probably a prompt kind of, it can, It'll improve, it's probably more a prompt issue than anything else. But it comes back with a lot of feedback that is useless to us.
it doesn't always get what makes a good quiz question. So it might suggest, we think this question needs more clarification. It'll suggest how to clarify the question. That just gives you [00:42:00] the answer, takes away any of the quirkiness or humor from a question. So poor Marjorie has to troll through a lot of rubbish, but each time there are a few things that it pops up, which we're really pleased that it popped up because it would, whether it's an actual error or just pointing out something, might be hard to understand or something.
So there's an opportunity before it gets seen by the public for Marjorie to, Yeah, to tweak the questions. So that's an example of how AI is, benefiting quiz wise, but, at the same time, the questions are authentic human generated questions done the hard way.
Nate Kadlac: I am, before we wrap up, I am curious about that, like the creation of a quiz question.
Like what goes into that? What makes a good quiz question in your mind?
Andrew Bartle: That's a great question. I don't want to put, words in Marjorie's mouth. she would be obviously the best person to [00:43:00] answer that. I've already mentioned that I think that for her, it has to be something that, that she herself finds interesting.
She doesn't want to put out low quality questions that are just soaking up somebody's time. There's a lot of, this ties to, and to just the addictive qualities that there are of a lot of these sites. We want, one bit of gamification that I welcome into my life is with Duolingo.
We had a German homestay student last year, kicked me off, doing German and Duolingo, which I enjoy. and it's always reminding you, trying to push you to make sure that you do your daily streak, etc. And I'm not really interested, I ignore streaks of all the games I play, even those that I really do try to do every day.
But that's what I encourage, because I'm learning from it. And so that's very much a goal with QuizWise, and I think that's important to Marjorie. It was important for Marjorie that Barry's legacy gets continued, and in terms of her question writing, it's [00:44:00] important that if people are using the site, that it's not just a way for them to pass the hours and the day, but they're actually getting something from it.
Yeah. So she's looking around such, I don't really know her exact process of where she gets those sparks from, when she finds something that interests her, a topic that will interest her. She'll look to find a quick question that will, fit with that topic and give the people doing the quiz that same aha kind of moment.
Nate Kadlac: Love that. So Andrew, if people are to find you online, if you want them to reach out, if they have questions about. Creating games. Where might they find you?
Andrew Bartle: they'll find quizwise. com. if you type in quizwise into Google, it should be there at the top. And yeah, there's a link on there to send us an email if you'd like to send an email with any feedback.
So I say we get a lot of feedback about individual questions, but we don't get a lot of feedback [00:45:00] about what people would like to see on the site, any changes they would like. So I'd certainly welcome that one change that we made. So we used to only just have links there to send an email. One change that I did make it, which has been very valuable is that when you answer a question now it slides down and it gives you the opportunity to thumbs up or thumbs down the question or submit a comment about the question.
And that's been really great because that has encouraged more people to reach out and let us know the sort of questions that they like or dislike or any issues they have with the questions. Also an opportunity there, if you do go to submit a question, I'm sorry, a comment, it'll list out a bunch of, little things to read first.
Have you actually double checked your facts? Have you re read the question? Sometimes we'll, it's always a, we're looking to, sometimes we might leave something out for brevity, right? I think the example we have on there is, if we were asked which is the largest city, We mean the [00:46:00] largest of the four options, not the largest city in the world, because inevitably you'll get people going, oh, none of these are the largest city.
But you don't want, nobody wants to read a novel before they answer a question, So that's always a big challenge in writing the questions, is how to phrase them succinctly. While still getting the information across. So yeah, so there's a few ways that they can find us at quizwise.
com and welcome any feedback about features that you would like to see added.
Joseph Rueter: That's fantastic. Yeah. When I saw your modal fold out and there's a giant submit feedback, I was like, Oh, what? Like you, you're going to ask for that. That's crazy. And then when I read through the bullets, I was like, Oh, you hit the tutorial.
Like you're teaching me how to do this. Yeah, that's an interesting point.
Andrew Bartle: Maybe we do need a little bit more of a help button.
Joseph Rueter: Oh no, the help button is under, under, so you think you want to [00:47:00] submit feedback. You don't know until you're going
Andrew Bartle: to submit it. That's right.
Joseph Rueter: Yeah, I think it's great. Yeah, it's better than an FAQ.
Andrew Bartle: of course, sometimes, the most frustrating feedback to get, you just ignore it, but you'll just get somebody to just go, what a stupid question or that's wrong or something and you're like, okay, what, about it did you not like? And particularly when they say that's wrong.
So then you're sitting there going, in what way is it wrong? So actually, yeah, since adding that little bit, we've had, oh, and you notice the last one, the last line there is, is a little bit of politeness goes a long way in encouraging us to consider your point of view. And, I think that's helped a little bit too, because, sometimes these things are a little subjective, especially if they're asking for more information.
Quite often though, I find that interesting. People will ask for, they want more information to slide out after you answer the question. And Marjorie very diligently will often go and add more information to that section. But of [00:48:00] course, we're always thinking, you could open another browser tab and Google it, of course, but.
But that's fine if they're interested, we're happy to get the feedback because we'll go at it for other, uses. But yeah, definitely, we, if somebody is rude in the way they tell us something, it makes us less inclined to see things from their point of view.
Nate Kadlac: Yeah, we were talking to, James at Waffle and the Knuckle guys, they said that after you complete a game, they do provide a lot of the definitions and.
That eats up most of their time, they said, that in the game creation, the content of doing that every day is quite arduous.
Andrew Bartle: Absolutely. And you know what? That I heard that bit and I was like, really? Because we play waffle a lot. And I went to waffle and I looked for it and I saw it, yeah, I didn't actually realize it was there.
I hadn't noticed it. So yeah, that was cool. And I enjoyed that little hidden feature.
Joseph Rueter: There's a UX [00:49:00] challenge on your hands. Yeah, that's right.
Andrew Bartle: Yeah.
Nate Kadlac: it was great having you here, Andrew, and, we're excited for people to check out QuizWise if you haven't already. I am now slightly addicted, enjoy it and thanks for being on the pod.
Andrew Bartle: Oh, absolute pleasure. Yeah, it's really great to meet you. Thank you for inviting me on.
Nate Kadlac: Thank you.
Andrew Bartle: Our pleasure.
