Antisyphon Training Anticasts

What if you could build tools, automate tasks, and write real code without ever having called yourself a developer?

Join us for a free one-hour training session with instructor Carrie Roberts, Developer and InfoSec Engineer, as she teaches you how GitHub Copilot is changing what it means to "know how to code" and show you how to put it to work in your browser right now, with no setup required.

You'll learn how to use Copilot to generate, debug, and refactor code through hands-on demos, including how to assign tasks directly from GitHub without ever opening an editor, so you can start building things that actually work faster than you ever thought possible.

📚 Train with Carrie Roberts
https://www.antisyphontraining.com/product/powershell-for-infosec-what-you-need-to-know-with-carrie-roberts/

Chapters
  • (00:00) - Introduction & What is GitHub Copilot
  • (05:17) - How Copilot Works (Concept + Setup)
  • (11:27) - Prompting Strategies for Better Results
  • (15:49) - Turning Ideas into Code (Workflow)
  • (23:53) - Debugging, Limitations & Security
  • (36:11) - Q&A

Credits
Chat with your fellow attendees in the BHIS Discord server:
https://discord.gg/bhis
in the #🔴live-chat channel

🔗 Register for FREE Infosec Webcasts, Anti-casts & Summits –
https://poweredbybhis.com

Click here to watch a video of this episode.


Brought to you by:
Black Hills Information Security 
https://www.blackhillsinfosec.com

Antisyphon Training
https://www.antisyphontraining.com/

Active Countermeasures
https://www.activecountermeasures.com

Wild West Hackin Fest
https://wildwesthackinfest.com

Creators and Guests

Guest
Carrie Roberts
Carrie Roberts is a programmer, turned pentester, turned red teamer, turned blueish purple. She is currently on the Red Team at Walmart. She loves to learn and give back to the community. She is one of the primary Atomic Red Team project maintainers and developers and has developed many of her own open-source tools. She holds master’s Degrees in both Computer Science and Information Security Engineering.
RP
Producer
Ryan Poirier

What is Antisyphon Training Anticasts?

Podcast audio-only versions of weekly webcasts from Antisyphon Training

Ryan Poirier:

Welcome everybody to today's Antisyphon Training Anticasts. We've got Carrie Roberts here. She's gonna tell us about Copilot in GitHub and how to do fun things with it. I'm going to hand it right off to Carrie and we'll get right to it. Carrie, it's all you.

Carrie Roberts:

Alright. Let's get started. So I'm Carrie Roberts and I was a developer, primarily web app developer before I got interested in cybersecurity when my web application failed a pen test. And then I got real interested in cybersecurity and I went and studied stuff and so, and I do things listed there, but also I am just giving my own opinions and not representing my employer right now. So let's skip ahead and get into the real meat of this.

Carrie Roberts:

So I also included some of the fun blogs I've done that are non technical that people might find interesting, especially this from high school to cyber ninja. A lot of people ask me how, they might get into information security and I provide a lot of advice about how to go from, you know, basically zero to knowing a lot about cyber and being able to work in cyber. And that's for mostly free or, you know, I have a lot of free resources listed there that are good and and some nearly free resources that are excellent. And so you can go there. I developed that content for a couple of my daughters who are getting into cyber and using that resource.

Carrie Roberts:

So you can check those out if you're interested. So let's talk about GitHub Copilot, an AI you can use with GitHub and also about GitHub. But to start, we're gonna switch over to GitHub and I'll just give a little intro about GitHub for those who aren't familiar. So GitHub, it's a free online resource you can use. There's some paid versions you can use for the enterprise as well.

Carrie Roberts:

But it's a place where you can put the code that you write. So the reason why you might do that is to have a backup of your code. But also, it's really helpful when you're working as a team so that, multiple people can be working on the same code and you don't overwrite each other's changes. So I start adding some lines of code and you start adding some lines of code, and then if you save yours after I save mine, normally, that would delete my code. But in this case, it gets all merged together through the magic of GitHub and source control.

Carrie Roberts:

So you'll see a lot of open source projects published on GitHub. And I actually have a tool that we're gonna look at as an example of using GitHub Copilot on here. It's great to have open source tools on GitHub so that other people can contribute to the same code so you kinda have this community of developers. But also even if others don't want to develop, they can still post and say, hey, there's a problem with this code. Could you get it fixed via posting an issue?

Carrie Roberts:

So we'll look at all that. So in GitHub, they have this concept of repositories, which is just a folder that contains each of your projects. So for my repositories, have a bunch of different folders here of projects I've done. So we're gonna look at one specifically called DPAT, which is short for domain password audit tool. This is a tool that I wrote when I was a pen tester at Black Hills to help audit, password, use within an organization.

Carrie Roberts:

It has a it has all the code here, but it also has a read me that describes what the tool does install and use it and so anybody can install and use it. And then across the top here, some important things are issues and pull requests. So, pull requests, I think it's it's named a little weird. That's never been a terminology that really worked for me. But what a pull request is is if somebody wants to contribute to this code, it's a open source project, so they could create a copy of this code and they could fix something in it or add a feature to it.

Carrie Roberts:

And then if they wanted that to be part of the main Deepak tool, they could basically say, hey, I I I made an addition. Would you be willing to accept this into your main code and make it part of your project? To do that, then make a pull request. And so here we have, under pull request, somebody actually, I didn't didn't notice this was here before, but, somebody says, hey, I've got a update here. I can click on this and read about what they've changed, then I can go look and say, oh, well, there's two things they changed.

Carrie Roberts:

They took out this stuff and they put in this stuff, and if I like it, I can approve it and and merge that in. So that's what a pull request is, is somebody asking, can you make these changes part of your code? But also there's this issues. So issues, it doesn't really always have to be a problem. It could be a feature request.

Carrie Roberts:

But over here, somebody's saying, hey, could you add a flag to show enabled accounts or do this or add more statistics? So, these are the currently open issues, but there's some closed issues. I wanna show you one of these closed issues cause this is what got me really excited about GitHub Copilot. So back in, let's see, October 1. So somebody posted on October 1, it'd be really nice to have a light Dart mode toggle for the report.

Carrie Roberts:

And so I read that and I my job has changed since I created DBAT and I haven't used it for several years, but I'm still the maintainer of it. And so when I saw this, was like, oh, yeah. It would be nice to have a light dark, but I don't know that I wanna spend a few hours doing it. I haven't used dBAT in a while. Am I gonna keep maintaining this?

Carrie Roberts:

Well, when I came in here and saw this request, I saw this assigned to agent, at the time it was called assigned to Copilot. And and then it kinda triggered, I'm like, hey, I'm feeling a little burnt out on maintaining this tool. What if I just assign this to Copilot and let AI implement, the light dark mode because I don't really wanna Google that and figure it out. And so I did that and and I happen to see this, notification come in right when it happened. So about ten minutes later, GitHub Copilot had made the change, created a pull request, put it on a separate branch which is like the version of your code without messing up your main code.

Carrie Roberts:

And so I said, hey, can you test out this branch which Copilot had just done for me in about ten minutes and try out the dark and light mode toggle and let me know what you think. And he said, hey, that was really fast. And he says like, oh, thank you. But it's really GitHub Copilot. I hadn't even used it.

Carrie Roberts:

And he says, it works great but when I when I display it, I get it added this moon moon symbol. So if you click on it, the moon changes from light to dark. And then he says when I do it, it kinda overlays on the top of the table of the report. So that's kinda weird. I had a larger screen so I hadn't noticed.

Carrie Roberts:

And so, you know, maybe you could use Bootstrap Bootstrap five Navbar and I've heard of Bootstrap and I know it's pop possib I'm trying to say. Oh, popular popular, but, you know, I would have to look up how to use it again. And, you know, if there was a navigation bar at the top, that'd be super cool and then keep it from overlapping. And, you know, while you're at it, if you're using that, you could utilize this, feature of Bootstrap that takes tables like this and makes them, searchable. So you got a little search bar at the top.

Carrie Roberts:

And so some of these reports like when you're listing the details of the password hashes are hundreds I mean, thousands of passwords all on one page. And what this Bootstrap five will automatically do for you is paginate those so you know, it'll do a 100 per page and you can say next next next. So it just takes care of all that for you. And also gives you a search bar. So if you're searching for a specific user, you just type that in and you find the user.

Carrie Roberts:

So I'm like, oh, yeah, that does sound really great. I'll just have AI do that. So then again, I just literally copy out this whole thing right here and give it to Copilot and say, you know, that worked great but this overlaps and how about Bootstrap five? And then it just did it and I I'll show you how you can go and and see how it works before you know, so you can get confident in this change it makes. But then I'm like, I merged it in.

Carrie Roberts:

Thanks. And so that was super cool with very little effort of mine. I got to, get that feature done and, you know, be a good maintainer of this tool without a lot of effort on my side. So now I clicked on this pull request that that really GitHub Copilot had implemented for me and it had made a branch called, dark mode toggle. And so it added all these notes about what it did.

Carrie Roberts:

And then if I wanted to see what was changed before I approved it and made it part of my project, I could go to files changed. And so in order to do these features, this is what it had to do. It added a line. That one line here, it did a little HTML here. Oops.

Carrie Roberts:

And and then it added a few lines to add in Bootstrap. Uh-oh. That's a little longer than I thought. My screen's smaller than I'm used to. But oh, let's see.

Carrie Roberts:

It says at the top oh, it added a 196 lines of code, but a bunch of that was the HTML to support the search and the pagination with Bootstrap five. Anyways, in the end, let me show you the difference. And all this took about ten minutes of my time. Where's my presentation here? So in the end, we had no toggle mode button.

Carrie Roberts:

And after the first try, we had to toggle mode, but it overlaid the top. And then by the end, we had a nice navigation bar at the top with the toggle here. We could paginate our results with a certain number of entries per page. We could search for a specific user name here. And so that was super cool.

Carrie Roberts:

That so really, it was last September that I got super excited about coding with AI, especially with GitHub Copilot because it's integrated right into GitHub, which I use GitHub all the time. So I wanna show you some, let's see, some more examples, some more ideas of what you could do with GitHub. Okay. So on a personal note, I I've been playing this game called Queen's Master on my phone. And so let's see.

Carrie Roberts:

Let me play this. Okay. So in this game, you get this grid of colors and you have to figure out where what the you know, where a queen can go, where you only get one queen per row and per column and per color. So there's a certain way you can set the queens in here so that you don't end up with more than one queen per color per row per column. So for whatever reason, our brains, some people's brains like to solve things like this.

Carrie Roberts:

And so we were having fun and we were having a family competition about playing this game. And I decided, oh, I wonder how AI could do solving this game. So I I said, hey, AI. And the first time I went about this, I said, hey, AI. Here's here's this game.

Carrie Roberts:

Let's talk about how you could write a solver for it. So it it we talked back and forth and it said, you can install an emulator on your computer. That way, I can interact directly with the game and click the buttons and and play the game for you. And so we did that and it made this, web page over here on the left where it kinda tells tells me what it's up to. And I can put it in continuous solve mode and it would solve the levels for me.

Carrie Roberts:

And so I have a little video of it playing the game. So over here, you'll see it clicking the button to put the queens in one per row. And over here, you see see it's taking a screenshot, it's figuring out the solution, it's clicking the buttons. And this was just just to see what AI could do. And it's pretty pretty impressive and it's pretty complex if you think about what it's having to do here.

Carrie Roberts:

So, really the hardest part about this was dealing with the ads because all sorts of different ads can't come up and you have to dismiss them in different ways. So I had to teach it if it if it looks like this type of an ad, click down here. If it looks like this type of an ad, if it's an offer for this, you know, and plus at the end that you had to press the next level button. And and so, I learned a lot of things when I attempted to do this. I actually gave up the first time I tried to do it with AI because I I literally just said, you know, here's the game, here's the rules, write a solver.

Carrie Roberts:

And I did it and it would solve a couple mazes, but these these grids get different sizes. You know, they start out small and then they get more and more squares and the colors change and it it would struggle as the grid size got different size. And I would fight with it and I would say, hey, you're missing, you know, I'd say show me show me what grid you're calculating and I would draw the grid on there and it would be like halfway cut off on the bottom. I said your bottom row is cut off halfway. And so it'd fix it and it worked for a while.

Carrie Roberts:

Well, when I finally took the time to look closer, it was just like, when it said when I told it it was cutting off half the bottom row, it just like, doubled the the y coordinate and hard coded it to that. So it worked until the grid size changed again. So I got frustrated when I realized that the whole the whole way I'd gone about it was kind of hacky. And I came back around and decided not to just launch AI at this huge task all by itself. So when I came back around the second time to do it, I said, okay, I just have this grid that I that changes sizes and I want you to take a screenshot of it and figure out where the grid is and then draw the grid on there so I can make sure you're doing it right.

Carrie Roberts:

So we went through that and I got trained really well on finding the grid. So as the grid size changed, it kept finding the right grid lines and taught it really well on that. And then we moved to identifying the colors and, you know, I had to I had to teach it that there's shades of blue and that it has to be able to tell the difference between shades. And so once I did that, I was able to get AI to play this whole game. And so that that was fun and just an example of things that you can do with AI.

Carrie Roberts:

So next, I went in to GitHub online and I made a new repository and I said create a browser create browser based retro games. And this was just on Monday night, I was preparing this demo. And what it came up with was something that looked like this and it is kinda retro looking but it's not it wasn't really what I was thinking. But I mean, I didn't give it much. I said creep browser based retro games.

Carrie Roberts:

So, when I played these games that it made for me, they were half broken. So a snake game was it was just doing weird things when I played palm. The the ball would hit the paddle and then, like tenth of the time, the ball would hit the paddle, slide up the paddle and then bounce off. So that was super off work. And so I decided to go back and start over.

Carrie Roberts:

So and do a little research before. So I have a little demo I'll walk you through. I prerecorded it because I just, was nervous about how demos would go here. So I went into Gemini and I said, what would be a good technology to use to do these games? So I'm just looking for ways to give it a little better guidance about how to, write this app so I can get what I want when when I really don't know what I want.

Carrie Roberts:

But that's the beauty with AI is like, I I I didn't really know what I want. I asked it for something and that wasn't it. Well, was great learning like, yeah, that's not exactly what I want. I didn't realize it before but now that I see it, I know I don't want that. So now I'm saying, what would be a good technology for developing some easy to code browser based games like Pong and Snake?

Carrie Roberts:

And so it says, well, you know, there's this option and that option and that option. So I scrolled down, to the bottom and the last one, it was called phaser three. Said, well, phaser three is really good because it says right here, it has a built in like, are two things colliding? And since we the first app had a bug about that, I was like, well, if it's already got the boring stuff like it says figured out, phaser three would probably be good. And I never heard of phaser three before Monday.

Carrie Roberts:

So I'm like, okay, I'll I'll go back and I'll give it some guidance. Let's use phaser three for this and and see how it works out. So now we're gonna go back in to get him. Let me put a drink here. Feel like this is moving like turtle speed here.

Carrie Roberts:

So go into GitHub and I can I see my repositories here which is are just folders for projects? So I say create a new repository and I'm gonna call it games phaser three. And I actually am using I I didn't talk about this. Let me let me process. I actually am using the paid version of GitHub Copilot which is $10 a month.

Carrie Roberts:

So when I decided I wanted to do this webcast at the end of last year, you could use GitHub Copilot for free with a free account. But they took that away at the beginning of the year. And then they had But there's still pieces of it you can use, you just have to get get to it in less convenient ways than you can right here that I'm showing you. And also they had a free thirty day trial of GitHub Copilot up until the tenth of this month. So things things are changing really fast.

Carrie Roberts:

But in any case, if if you're gonna be doing much coding, you know, it'd be it saves you so much time and energy that, you know, $10 seems really affordable in my opinion. But I am showing you, you'll only see this this option down here when you go to create a repository with a prompt, with the paid version right now. So here I'm gonna say create create the games using and let's use the phaser three framework. So giving it just a little more guidance to help us through. But like I said, the beauty of AI is, you don't you used to have to really come up with your requirements really well beforehand before you went to all the trouble to develop it.

Carrie Roberts:

But now this is all shifting and and we don't have to do like all the thinking beforehand to figure out what we want. We're just like, give us something. Yeah. Not quite that and tweak this and do that because now it's so easy just to redo the code with AI. So we tell it create this new game and all we get in our repository here is a single file read me and all it says in it is phaser three.

Carrie Roberts:

So we have no code. But under pull requests, we see that our AI agent is actively working on it. So this WIP work in progress. Copilot is off implementing this game, this feature for us, this game on a different branch, not our main code and doesn't wanna mess that up. But it's gonna implement it if we like it, we can merge it into our main code.

Carrie Roberts:

So it says, hey, I'm in the middle of it said, thank you for asking me to do this. And we can let's see what I did here. Okay. So now let me pause this. We're going to make a code space.

Carrie Roberts:

So this is another concept in, GitHub. A code space is just a virtual machine, a computer in the cloud, a computer on the internet that we can use that GitHub hosts for us. So we can just click start up a new computer, and it starts up it's a a Linux computer, and it starts up with your code already loaded inside of Visual Studio. So you don't even have to install Visual Studio on your computer to do development, or install anything. You can do everything from the browser.

Carrie Roberts:

So here we are saying, give me the code space. So basically, a computer in the cloud, so it's opening this up for it. It's gonna open up with our code in it. In Visual Studio, and, you know, all this is in the browser, and we see all this stuff on the left under the explorer is already a bunch of code that it's working on. And it's already developed the game.

Carrie Roberts:

So we can tell it run the game and it's gotta connect to our GitHub, so we'll say connect with GitHub. Okay. So we got all connected to GitHub. Now it's gonna start up, our game on a local web server. Open that tab.

Carrie Roberts:

So, this this thing on the right here let me pause this. This thing on the right is GitHub Copilot chat. And so you get to type your instructions down here what you want it to do. And then it does it but if it has to do something like run a command, it'll just check with you, do you wanna allow me to like run an install command on this computer, which is just this cloud computer they've created for us. So we have to allow certain things.

Carrie Roberts:

And so we go ahead and then allow that. And then it's gonna start up our games in the browser for us to look at, see how it goes. So we can say open in the browser. And so here we have our games. I think I like the look of this one more and then the black and green one.

Carrie Roberts:

But I mean, that's just look. So let's let's play Pong. So I go in here and to play Pong. Little slow. So I start playing Pong, I'm the guy on the left and I'm not moving.

Carrie Roberts:

So I see down here it says, use w's or s or the up and down keys to move while I'm pushing w and s and it's not moving. So I'm like, oh, man. This palm game is broke too, that stinks. So I go try the snake game, see if that worked any better. In this snake game, you're this little snake and you need to run around and eat these red dots and every dot you eat, you your tail gets longer.

Carrie Roberts:

And if you end up eating your tail or eating into your body, you die. So so the arrow keys are working on this and we can play this old fashioned style game and see see how we do and it's got a score and so here I I went back on myself and ate myself and and died and so that that all works, but we have a broken pong game. So we go back into our cloud computer and and we just tell it, hey, my my paddle doesn't move. I'm pushing s and w and it's not moving. And then you just have a look into it.

Carrie Roberts:

So my my paddle won't move when I press w or s or the arrow keys, and we let it go. And it shows us a little status of what it's doing along. I found the root cause in the update loop, the paddle body is being reset every frame. So it's trying to move by getting reset back, let me fix it. And it says here.

Carrie Roberts:

I'm gonna pause this. It says I edited pong dot j s. I added 16 lines and removed nine. And so I'm gonna click here on one file change. So it lists all the files that's changed since you've accepted the changes.

Carrie Roberts:

So we expand this, and we see the file. We can click on each file and we'll see over here in the middle, we'll see green for what it added and red for what it removed. So it had to add these lines so we can say, hey, yeah, that looks reasonable what you did. We can press keep to keep just this little piece that it changed or if we click one of these keeps at the bottom, it would keep everything. Everything in this file or everything in all the files that changed, but it only changed one here.

Carrie Roberts:

So if we if we select keep, then it quits highlighting it for us and basically we we accepted the change. And so now it says now you want me to, I'm gonna keep it and run it. So I gotta allow it to run. So we run it again. And it says, okay, I made the change now.

Carrie Roberts:

Run it again. Now refresh the page. Try reloading the page. I'm gonna refresh the page. Okay.

Carrie Roberts:

And now our paddle's moving. So so whatever we needed to do or whatever features we want, we can can just ask it for it. And this is all on a different I think I I go over in the demo here and show you. But this is all it's doing this it's not doing this in our main feature in our main code area, it's doing it off on a branch which is just like a separate copy of the code so it doesn't allow anything. So I just told it commit and push and this is GitHub terminology for, basically, putting a tag on this version of the code, you know, where we could refer to it by a certain number.

Carrie Roberts:

And then also putting a copy of it online. So now that I've pushed it, I can come into this branch of the code and see everything that would be changed if I accepted this branch of the code. So we have these all these new files, everything's new because there was no code before. So now now I'm gonna say I'm happy with this this feature that you created for me, this pull request. And I wanna make it part part of my code.

Carrie Roberts:

So when people come to my main code page, instead of seeing an empty project with a read me, which we see here. That's all there is, is this read me that says games phaser three. I wanna tell it, hey, I want what you just did for me to be my code. So that's called, you merge this branch that it made for us, this version of code into your main code. And so I tell it here, merge this code into the main branch.

Carrie Roberts:

So it's great because you don't need to use, you don't need to learn the git command line things, which is a little cryptic. And you can start using natural language to tell it how to manage your code. So you know you know what you want it to do, you just don't know the exact syntax for the command line and then it just does it for you. So we approve that. And it says I'm gonna merge this in, do you wanna allow this?

Carrie Roberts:

And we click allow. So that that's the code right there that you saw, the g h and all that stuff. That's hard to remember. It's not hard to remember how to say merge this into main. So, I really like that.

Carrie Roberts:

I'm gonna get rusty with my git commands but I'm happy to have less things I have to remember at this point. Okay. So now we go back into go back to our main code. So now our main code, instead of having just an empty read me, it has all that stuff from that branch that we just played with and we like the gains and the gains are working. Now that branch and our main branch that you get to when you come to this page are the same at this point.

Carrie Roberts:

And we could go in and use this issues tab to we could say, hey, there's a bug with this, fix it, or we want this new feature. So I'm demoing here how to get a new feature. So, I'm just saying, this is a feature request and the hangman. Yay. And this is just a way for me to give more instructions to Copilot or another developer or myself could could implement this feature.

Carrie Roberts:

But here I'm gonna assign it to the agent. And we could give additional instructions if we want, but we're just we'll just say, go ahead and assign this task to the agent. Well, I guess I skipped ahead. Yeah. I I think I edited that video too much, but, it created a new poll request where it says that, you know, thank you for helping me work.

Carrie Roberts:

For asking me to implement the Hangman game. I've created this feature branch. You can kinda see a note of it here. I created a branch to add the Hangman game and here's the code and I implemented it. And now I'm over in my code space and I'm asking it, hey, run run the run the game now.

Carrie Roberts:

And so now I have my main code that doesn't have the Hangman game in it and then I have this branch of my code that has the Hangman game, which is the one I'm interacting with now. So when I run my code now, I'm gonna have the Hangman game. But anybody who's using my main game wouldn't see Hangman until I approve this. So here I have this new Hangman game. I come in here and I I thought this was an interactive keyboard so I'm sitting here trying to click the letters of the game, but it's not.

Carrie Roberts:

And I realized I'm supposed to type. So I start typing letters and I I play the game and I lose. Just see how it works. And it says you lost, press the space bar to play again. And so I start playing again, see what it looks like when I win.

Carrie Roberts:

So it wins, I test it out and maybe I'm happy with it, maybe I'm not. So let's pause this. So maybe I thought this wasn't very intuitive that I should be able to click. I could go into GitHub and I could create an issue that says, the Hangman game shows the keyboard on the screen that looks like it should be clickable and I want it to be clickable so that users don't have to type, could just click the letters on the screen. And then I could have AI implement that.

Carrie Roberts:

I could also tell it about the layout like this whole thing, the keyboard is on top of the base of the hangman and, you know, I want that to look better or I want the graphics better or, you know, I want more than the basic stick figure. I want five fingers and five toes on each hand, whatever you want it to do or I want different colors. And now as a developer, this has transitioned you from, kind of a tedious implementer of let me go Google, how do I change the back color again? The background color again or do this kind of a, interaction to the high level designer. You know?

Carrie Roberts:

Like, this is what I'm going for. This is a structure. This is, you know, you're more of a product developer at that point and a architect. And so you get to focus as a human on those high level skills and all the little stuff that that you're just having to constantly go back and and Google, what's the syntax for that? How do you do that?

Carrie Roberts:

You let you let the AI handle that. So that's what's really great. It brings brings you up brings you up a level and lets you do that really novel architecting stuff, designing stuff, without, like, kinda getting exhausted at some of these things in the weeds, that are so there's so much minutiae with it. Let's see. Was there anything else you're just showing that?

Carrie Roberts:

Okay. I think that's the end of that demo. Oh, well, that that was my last demo. I don't have I mean, I have some other projects I can walk you through, but I wanna pause here for questions. I I can't present and read chats at the same time, so I don't even have Discord or the chat open.

Carrie Roberts:

But can somebody are are there questions that have been asked or do people have questions that we could talk about?

Ryan Poirier:

I heard you asked for questions, so I am jumping in. If anybody has any any questions for Carrie at this point, ask them in the Zoom q and a for us or in the Discord. Ping us again if you've already asked your question. Ping us again so it it bumps down to the bottom of Discord and we'll get to it. So I do get a couple here in Zoom.

Ryan Poirier:

Can you choose which AI model Copilot is using?

Carrie Roberts:

Okay. Yeah. Let's go into that. The answer is yes. So let's go actually, let's find this game that I made on Monday.

Carrie Roberts:

So go into my repositories, and we'll go into the phaser three games. And I can just say, open open a code space for me for this game, and then I'll show you. Okay. And it's it's still oh, it's still loading. Okay.

Carrie Roberts:

So over here on the right is your chat. And if that's closed, you you can get back to it with this toggle chat here. So you have this toggle chat. And this chat could be GitHub Copilot or it could be Claude or it could be some other things. But by default in GitHub, the extension for GitHub chat is here.

Carrie Roberts:

So down here, we have our agent selection. We can pick other agents. No. Actually, maybe in this online code space, you can. But I mostly don't use Go to Spaces when I develop because I do a lot of developing and it makes sense for me to go to the trouble to install Visual Studio Code locally and to not use up cloud resources.

Carrie Roberts:

And so in actual Visual Studio Code installed under my agents menu, I or no. Let's see manage models. I guess it looks a little different on here. Hold on. So to answer your question, I don't know if you can in here pick it as well as that.

Carrie Roberts:

I'm opening up another project over here that I can show you. Okay. Visual Studio Code. So when you have installed on your computer, you have the

Ryan Poirier:

Come on, demo gods.

Carrie Roberts:

Come on. This is why I wanted to record the it it switched to Claude code. I have Claude code too so. They you know, they're changing stuff all the time. But anyways, over here to the right of my agent, I get to choose what I wanna use.

Carrie Roberts:

So I could do Claude Opus 4.6, and you see this little, thing on the side, one x three x, and a 10% discount. So you have a certain amount of quota you can use. And if you use Claude Opus 4.6, it's gonna use three times as much as your quota as if you use one of these other ones. And if you use auto, which auto is they're gonna decide what they think is best for you to use slash saves resources. And so if you agree to use auto, you you save a little bit, but maybe they're just using like the little less functional one all the time.

Carrie Roberts:

And so these are probably just ones I've used recently, but there's other models. So you can go through and here's a point three three times one if you use Haiku. So, yes, you can pick your model. And this is still GitHub Copilot here. So

Ryan Poirier:

That could be Agent Okay. Go ahead and finish.

Carrie Roberts:

Sorry. You broke up a little.

Ryan Poirier:

Go ahead and and finish your thought.

Carrie Roberts:

Oh, okay. And then under agent, you've got different agent types. You can be in agent mode. Agent mode has most power where it can actually create and edit files within your project here, which is what we were in when it was creating code for us. But there's also a plan mode where you're like, I don't want you to write files so the AI gets it's really anxious to just just write code.

Carrie Roberts:

I mean, it's like, it works going to disable, it wants to write code. So you can go into plan mode and say, I don't want you go in and change a bunch of my code before we have a solid plan and I've agreed to it because it really can just take off and before you know it, it's like completely redone everything you did, which of course you can tell it, hey, undo all that. But sometimes it's frustrating like, holy cow, let's talk a little before we do this. So you can go into plan mode, where it has access to your files or just ask which is more like you're just in a browser with with no other context. So yeah, that's how it typically works.

Carrie Roberts:

So I also have the Cloud Code extension. I'm subscribed to Cloud Code and I've been comparing Cursor AI to Cloud Code to GitHub Copilot. And so I also have the cloud code extension in here, which works a lot of the same ways. And and you know, people probably interested in what I think of each. I mean, they're using, the same LLMs behind the behind the scenes, you know.

Carrie Roberts:

You've got CloudSonnet that you could access from all, you know, three of those. And they just have some additional wrap around it, like some things that kind of taught taught the system to act in certain ways to help developers and you may or may not like one better than the other. But, as far as I'm concerned, they're they're pretty comparable. Cool. Are there other questions?

Ryan Poirier:

They there are so many questions. They just rolled right in as you were answering that last one. So there's sort of, I guess, related to what you're going through. Question is, when do you hit limits on using Copilot?

Carrie Roberts:

For for the $10 plan, there are limits. And I generally don't hit the limits unless I am really pinging it hard for, like, three hours. Do this. Do this. Do this.

Carrie Roberts:

Do this. Do this. So when it does hit a limit, what it does right now, forget Hub Copilot, it it'll say you've you've hit your limit. You can't do anymore for another hour. And so I have to wait an hour.

Carrie Roberts:

So then you come back in an hour and then you can do some more but not as much as you did before before you hit the limit again. And then it'll say, you can't use it again and for eight hours. And then you can't use it again till tomorrow. You know, I and I'm kind of a binge coder, like I get on and I hit it really hard really hard really hard, but then, you know, I don't touch again for two weeks. So that's a little little hard for me because I just I wanna use my whole month's quota today.

Carrie Roberts:

I wanna get it done, get out. And so that's that's a little hard. But, I've also got right now, because I'm interested in comparing, I've got those other I've got Cursor and Claude that I can switch over and just pick up right where I left off. So that that's how I I get around that limit.

Ryan Poirier:

Have you tried to use have you tried to use different models to create the same code to see how they vary the output.

Carrie Roberts:

Yeah. So I was talking some in the pre pre show banner about for my PowerShell for AI class. I worked in some AI stuff because of course we're we're learning how to use PowerShell and it it just makes sense to have AI help you because at the end of end of the class when the when I'm not there anymore, the instructor's not there anymore, AI is gonna be your instructor and they're gonna be there. And you're gonna be asking AI, you know, what does this loop do and how do I loop through something and how do I do this stuff? And so in class, I give an assignment that's to write information gathering script for the current computer and do a bunch of things and have this option to read from a CSV file and output to a CSV file and all this stuff.

Carrie Roberts:

So very explicit assignment that they're supposed to do. And I gave this assignment to the four different AIs that I am subscribed to. So I gave it to GitHub Copilot, Claude Code, Gemini, and Cursor AI. And I had it write the solution according to the assignment. Then I gave it other instructions other than assignment, but the assignment's pretty verbose.

Carrie Roberts:

And so it made it solutions. And then I named the solutions a b c and d, and then I had AI grade the four solutions. I said, here's the assignment. Here's four different solutions. I want you to grade them, tell me which one's the best and why.

Carrie Roberts:

And I want you to score them on, maintainability and how well they met the requirements and how well it's scalable. And then it went through and it it, it graded them. And three of them got like a's and b's and one of them got a d. And it surprised me because, the one that got a d was Gemini. And at this point, I had found that I was pretty happy with Gemini.

Carrie Roberts:

I haven't used Gemini like within a code, within Visual Studio code like this that takes an added subscription, but I have a subscription just to the browser one and it had done a lot of great things for me. And so I looked at why the report said that Gemini had done so bad. It's like, yeah, this this did terrible because it didn't implement this and it was supposed to do this and it didn't do this. I went back and looked at, my conversation with Gemini about this assignment. And, you know, I was doing things quick quick and I didn't pay enough attention.

Carrie Roberts:

But when I went back and reviewed the Gemini conversation and I said, hey, write this script to gather reconnaissance information from a local computer, I said, oh, that seems like something that seems like a hacker tool and I'm not allowed to write hacker tools. But I'll just instead, I'll write, you know, just a general gather some information about a computer script. So I completely ignored my original instructions and did some other script which explains why I failed the assignment. So I I went back in and I said, hey, Gemini. This is for a coding class and I'm not a hacker.

Carrie Roberts:

Please do it. Oh, okay. So then it did it and I went back. And it ended up getting second place according to, you know, the AI that created them. But they were all pretty pretty close, pretty comparable.

Carrie Roberts:

And, the assignment had a bonus section at the bottom that's like, bonus to try to, you know, put these whistles and bells on the assignment. And as it turned out, one of the AIs went ahead, you know, and decided to implement the bonus features, and the other ones didn't. And so it ended up getting the best grade. And but, you know, I'm kinda torn on that because if I would have explicitly told each day, do this assignment including the bonus, then they all would have done the bonus and then who would have got better grade? So, you know, there wasn't a clear winner.

Carrie Roberts:

The each each AI is configured to kinda use a different default whether they're gonna, you know, go full bore and write tests and do whistles and bells and add a features or if they're just gonna keep it simple and get the job done. But if you're explicit with it and tell it which of those you wanna do, then it it can still do it. So that's been my experience.

Ryan Poirier:

What about for reviewing code or reviewing each other's well, another AI's code output? You got down that road?

Carrie Roberts:

Yeah. Well, you should just use them all, you know. Have all of them review all of them and see what

Ryan Poirier:

you What about existing code?

Carrie Roberts:

What's that?

Ryan Poirier:

Existing code, like code that AI didn't write or maybe you wrote or somebody else wrote it and you wanna check it.

Carrie Roberts:

Yeah. I don't I don't explicitly have experience to say that one does better than the other. But I'm a big advocate for using more than one tool, because you do get different answers like, oh, this one noticed this and this one didn't. Or or, you know, you get three tools saying that the same thing and you start to get a little confidence around it. Because if you haven't played with AI much, it it does make stuff up sometimes.

Carrie Roberts:

Well, probably probably more more than you would expect. Gosh. I don't know if I had to estimate, but, you know, if I'm just guessing, like 10% of the time it just goes off in its own little world. You know, most of the time it's really solid. And then other times, it it'll be like, oh yeah, just use the dash, do it this way flag And you try that and it errors and you're like, where did you come up with that?

Carrie Roberts:

There's no such flag. Oh, I'm sorry. I should start assuming there's just flags based on convention. And you're like, yeah. Could we just always do that?

Carrie Roberts:

So it gets frustrating.

Ryan Poirier:

The question, can you install your own local LLM?

Carrie Roberts:

I don't know if you can point Visual Studio Code to your local LLM, but you can definitely install your own LLM and use it from the command line. I've done that with LLM Studio. And it starts up a local web server where you can call the API, over local host, which is super cool. But I the models this was like six months ago. And when I did that, I wasn't really impressed with its capabilities, you know.

Carrie Roberts:

But of course, I don't have a giant AI server either. So real studio integration to your local.

Ryan Poirier:

Looking through the questions. Here's a fun one. Have you had any success using AI to develop Atomics and Atomic Red Team?

Carrie Roberts:

Damn. My best friend leading into Atomic Red Team. I've done a lot with Atomic Red Team, was maintainer.

Ryan Poirier:

We must know you.

Carrie Roberts:

Red Team for a long time. So and I also have a a coworker who is a maintainer of Atomic Red team. And he's done great great work with this exact thing, making atomics writable with AI. So what he did is he developed a MCP server, which is a model context provider. So you install this m p an MCP server into Visual Studio Code, which basically says, hey, if you're trying to do things with, Atomics, you know, that you should talk to me.

Carrie Roberts:

You know, it's telling the AI. So you go you say in here, you say, hey, here's a blog post, about this attack procedure, develop atomics from it. And then the AI is like, oh, I know that if I'm developing atomics, I should reach out to this MCP tool. And and then this MCP tool knows the exact, format of atomic tests, how they're written in YAML format and what the required parameters are and how they need prerequisites and, and, what it should return if the test worked or if it didn't, it knows all that stuff and it reads the blog post and it implements the atomics. And then you can just say, hey, those look great.

Carrie Roberts:

Can you submit a pull request to the Atomic Red Team repo with these atomics? And it does all that for you and it's super amazing. Then he's got some blog posts and presentations about that MCP server he wrote. So, yes, definitely, you can use AI for that, and it's super valuable for that. Thanks for asking.

Ryan Poirier:

Yeah. No problem. In terms of uploading, like, your own code or existing code, what about the concern with these models using what you upload to train itself? Is there a big concern? Is that on your mind, you work with this stuff?

Carrie Roberts:

Well, definitely on an enterprise level, that's concerning if what your coding is. If your the privacy of that is important. So definitely it's something to be thinking of. So if we think about some of the things I demoed, so they're open source tools or they're just, you know, projects I've been doing for fun. So I'm not I'm not concerned if AI learns from its own attempt to solve this game we played or its own attempt to create retro games on the web or to do a password audit tool because that's all open source anyways.

Carrie Roberts:

Atomic Red Team's open source. But when it comes to the enterprise, it it depends. So, yeah. So so then there's times to be concerned about that. So in that case, if there is something I'm working on at work that I don't want to be potentially revealed through AI learning, then I can just take pieces of what I'm trying to do that aren't sensitive and have AI implement them.

Carrie Roberts:

Like, I could, you know, have it say, I could go and say, write a tool that reads a list of things from a file and then calls an API using REST and then puts the results on a web page. So I could make that generic and get that working and then bring it in and hook up the real API or something, maybe something sensitive that I didn't want out. So that's one way to work around that. But also, enterprises can make agreements with AI companies about, and and get licensing where the companies agree not to use it for training. And so that's something that the organization has to handle at, you know, that higher level and make that available to their employees.

Ryan Poirier:

You had issues with underdeveloped underdeveloped code or even overdeveloped code like useless code?

Carrie Roberts:

Yeah. You do have to be the babysitter of what they're up to and what the AI is up to. And keep an eye on it because, you know, overall it's just really amazing and impressive what it does. But the more the more you the longer you work with a code base and you start having it like, you come back another day and say, add this feature or do this, the more it has lost the context of what this code even does and it does try to take shortcuts like it's not gonna automatically read all the code when you ask it to do something and think of the best way. It's gonna kinda shortcut like, I assume I probably should read this file, this file, this file.

Carrie Roberts:

And then it won't know that there's like a helper function that it should've used and it'll create a duplicate way of doing it. And now you've got three ways in your code to do it and you find a bug and you fix it in the one place, but it's still the bug in the other two places. So as the architect, you still have to keep in mind, what it's doing. So you watch what it does and you're like, it looks like you just implemented a feature to cache the result, but we already had this whole library where we cache results, use that instead. So that we have maintainable code where we only had to fix the bug in one place.

Carrie Roberts:

So I've had to do a lot of that, like, no, don't don't make a copy of that code. Don't do this. That's that's a crazy way to do it. And so yes. Like, you know, it still takes that architect.

Carrie Roberts:

And, you know, and that's kinda good news from the perspective of, you know, we all hope to our jobs through through this whole AI phase. But it lets us step up and be instead of the grunt doing the grunt work too, we we just get to use our mind and be the architect and and keep an eye on, you know, our our highly skilled intern here that that still needs a little guidance.

Ryan Poirier:

An anonymous attendee asking how did you get the or how, yeah, how did you get the agent to interact with the Queen's phone game?

Carrie Roberts:

I installed an emulator and it used ADB deep debug interface to ask for a screenshot and it would find the grid and then it would calculate the x and y position and send the click command over ADB to have it click where the queens

Ryan Poirier:

Sort of a repeat. There's a couple of people asking this sort of question. So as a person who's not a developer, I would include myself in that category. Where do you start? Where do you begin?

Ryan Poirier:

How hard is this to start learning and dive into?

Carrie Roberts:

Well, really it's easy because you just get on the terminal where you get to talk to AI and it has the answers, you just need to guide it. So you could you could say, well, let's say you don't know code. If you came in and you're like, here's this queen's game, write a solver. And that's all you told it, it'll run off and it'll try to do that and it might tell you to do a few things. But it's probably just gonna go haywire.

Carrie Roberts:

And so instead you just talk to us like, I'm not a developer but I like to write a solver for this Queen's game. Tell me tell me what I should know. What would be a good way to go about this? Why would I use this emulator versus that emulator? Actually, I did that too.

Carrie Roberts:

Like, what's the advantage of this? Because it suggested a different emulator than I used before and I was like, why would I use that one instead? And it had some good points. And then, you know, and you just talk through this whole plan and and then you say, well, I wanna make sure this code is maintainable and I wanna be able to double check your work, so how could I go about that? So then maybe you'd say, well, I could show you intermediate steps.

Carrie Roberts:

Like I could show you the screenshot I take and then I can show you the grid I calculated and then I could show you the colors that I detected and you could check if I'm detecting, you know, down to the right shading level. And so basically what I'm saying is, AI is the teacher and the doer. So you just have have it teach you like and you could say, you just implemented this, piece of code in this file, you know, line function on four twelve and I don't understand it. Why'd you do that? Why'd you do it that way?

Carrie Roberts:

Is there a better way to do it? And just by asking, you can make it realize there's better ways to do it. And you can say, review all the code in here. Do you see any redundant code usage? I've done that to it before.

Carrie Roberts:

Say review all the code. Is there a code reuse that makes this not very maintainable? And I was like, yeah, actually I realized I I reimplemented this over here and if I I took these three pieces and made them a reusable function, it would be a lot more maintainable. So really this is just an interactive learning thing that you can do with AI, to get to the point where you know how to guide it even though AI told you how to you know, taught you how to guide it in the first place. But you can maintain that context and where it it has what it calls this context window where it it only remembers so much about what you've ever told it to do.

Carrie Roberts:

And when you get too much in there, it has to bump some other stuff out. So this whole thing about when when you write code for me, make sure it's maintainable and you don't repeat code. Well, that's gonna drop off at some point. And so you as the architect have to keep bringing that back in. Like, now's a good time for us to go over, you know, some of the things that are important to me, which is maintainability and not and not having the same code written two different ways, double check it.

Carrie Roberts:

And so then you get to be that person who brings that context.

Ryan Poirier:

Well, is past the top of the hour, so we're gonna probably wrap things up here. Couple more questions, couple maybe fun ones. I saw someone in Discord comment that Dee Kennedy thinks Claude code has sucked in the recent weeks. Do you agree or disagree?

Carrie Roberts:

I haven't used Claude code in the recent weeks. So can't see.

Ryan Poirier:

Can't answer that one. Alright. What are your thoughts about it says, Cursor? I don't know if that's supposed to be cursor. Cursor.

Ryan Poirier:

Yeah. What are your thoughts about Cursor overall?

Carrie Roberts:

I like it. I don't I don't develop full time. I develop, you know, something, you know, maybe 10 to 20% at work, and then I do a lot of projects for fun. And so for me, the added benefit the cursor gives, it it kinda like it was a it was a integration around AI built by developers. So, you know, they have all these ideas of what would make life just the easiest, smoothest, slickest for, you know, a developer.

Carrie Roberts:

And then it's pretty smooth when you go in there and use that as a developer. But that's also $20 a month where GitHub Copilot is $10 a month. So that's a big thing. And GitHub Copilot is integrated right into GitHub. So I get somebody submitting a request for my open source tool and I can just say, hey, GitHub, go do this.

Carrie Roberts:

I could do that from my phone. So I could be fixing bugs and merging pull requests from my phone without ever writing code or use it being at my computer and that's just the an integration I don't get with Cursor AI. But if I was, full time developer and all those kinda whistles and bells around smoothness and convenience in the in the IDE, you know, I'd be tempted for the cursor AI. But yeah, I I do like it.

Ryan Poirier:

Cool. Alright. We're gonna wrap up. I'm throwing some links in the Discord chat once again. As we wrap up, we got Carrie's class coming up at Deadwood for Wild West Hacking Fest later on this year.

Ryan Poirier:

I've shared the link to our Threat Hunter Anticasts Anticasts Training Summit that's coming up in June. And one more that I'm gonna grab is for a workshop we got Friday with Doc Blackburn, how to think like a cyber security defender. A great beginners course for people just starting out in cyber security and want to know how to get into that mindset. Carrie, thanks again for joining us and sharing your knowledge and experience with us. Exciting new frontier of AI that is developing very quickly and getting almost frighteningly better as time goes on very quickly.

Ryan Poirier:

So Yeah. Thanks again for joining us. We'll see you on the next p t siphon eatcast or the next p h i's webcast. We've got another webcast tomorrow with with Matthew, who's gonna talk about Microsoft things. That one could be fun.

Ryan Poirier:

So before Be sure to check that out. You can see all of our upcoming webcasts as we publish them on be powered by bhis.com website. That's gonna get you to our security c and which has all of our live webcasts that you can sign up for, including the threat hunting summit that's also on there. And that's it for today. Any any closing thoughts, Carrie?

Ryan Poirier:

I see you've just pushed your microphone away. You're like, I'm out of here.

Carrie Roberts:

Then I asked yeah. You Definitely One more. I think I'd just encourage anybody who, you know, has been hesitant. I I actually was a slow adopter for AI. I could have been making use of it a lot sooner than I did for coding or for anything in general.

Carrie Roberts:

And you know, my advice is to just get over that and start using it. It is it can be overwhelming but you're gonna be left behind if you don't if you don't jump on. It is so powerful, so helpful. I learned so much. I probably spend a couple hours every day interacting with AI, whether it's coding or not, just just as a sounding board, as, you know, a learning a learning a teacher for technology.

Carrie Roberts:

And so that's my advice is just get in there and and start using it.

Ryan Poirier:

Awesome. Alright. Thanks again, everybody. It is time to kill it with fire, Megan. Kill it with fire.