Oxide and Friends

Time is the bane of programmers and the villain for far too many operator horror stories. Oxide and Friends discuss time, timezones, standard time, daylight time, leap seconds, and how it all can go so so SO wrong.

Show Notes

Oxide and Friends Twitter Space: March 28th, 2022

Time, Timezones, Metric Time, Losing and Saving
We’ve been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it’s not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for March 28th, 2022.
In addition to Bryan Cantrill and Adam Leventhal, speakers on March 28th included Tom Lyon, jasonbking, Matt Campbell, Akshay Kumar, Aaron Goldman and Simeon Miteff. (Did we miss your name and/or get it wrong? Drop a PR!)
Some of the topics we hit on, in the order that we hit them:
  • [@8:07](https://youtu.be/BHtfqleSHAs?t=487) Y2K, leap years 
  • [@15:28](https://youtu.be/BHtfqleSHAs?t=928) Matt’s stories 
  • [@23:29](https://youtu.be/BHtfqleSHAs?t=1409) Jason: daylight saving time in Indiana 
    • “Time in Indiana” wiki
  • [@26:31](https://youtu.be/BHtfqleSHAs?t=1591) Time zone database 
    • John Bemelmans Marciano (2014) Whatever Happened to the Metric System? How America Kept Its Feet book
    • Geopolitical aspects of time
    • Eastman plan calendar
  • [@32:23](https://youtu.be/BHtfqleSHAs?t=1943) Aaron’s stories, setting clocks back, Leap Day
  • [@35:54](https://youtu.be/BHtfqleSHAs?t=2154) Akshay: Ken Thompson’s six day work week?
  • Leap seconds 
    • Time of day hardware bug
  • [@48:54](https://youtu.be/BHtfqleSHAs?t=2934) 2038 - the end of time 
    • Y2K problems
    • GPS week number rollover wiki
  • [@57:58](https://youtu.be/BHtfqleSHAs?t=3478) Matt: Cory Doctorow’s “Epoch” short story podcast commissioned by Mark Shuttleworth
  • [@1:00:28](https://youtu.be/BHtfqleSHAs?t=3628) Ultimate, penultimate, antepenultimate
  • Oxide and Friends podcast!! 
    • transistor.fm launch point, has links to Spotify, Google, Amazon etc players
  • Laura Abbott (23 March 2022) Another vulnerability in the LPC55S69 ROM write up
If we got something wrong or missed something, please file a PR! Our next Twitter space will likely be on Monday at 5p Pacific Time; stay tuned to our Twitter feeds for details. We’d love to have you join us, as we always love to hear from new speakers!

Creators & Guests

Host
Adam Leventhal
Host
Bryan Cantrill

What is Oxide and Friends?

Oxide hosts a weekly Discord show where we discuss a wide range of topics: computer history, startups, Oxide hardware bringup, and other topics du jour. These are the recordings in podcast form.
Join us live (usually Mondays at 5pm PT) https://discord.gg/gcQxNHAKCB
Subscribe to our calendar: https://sesh.fyi/api/calendar/v2/iMdFbuFRupMwuTiwvXswNU.ics

Speaker 1:

Yeah. I mean, it's brutal where, like, everyone else on the call just thinks you're crazy because you're talking about people sounding garbled when, you

Speaker 2:

know, as

Speaker 1:

far as I can tell, everyone everyone sounds fine.

Speaker 2:

Everyone sounds fine. And I thought especially with Andre, week before last where I mean, obviously, it's such a weighty subject. It was really just and we were having him, you know, stand on his head and everything to sound less garbled when I think he was, like

Speaker 1:

Right. Turn the Wi Fi off. Turn it off. Right.

Speaker 2:

I know. The I we, so we have actually to revisit just a a brief old favorite. Have you been watching any of the dropout? We we've talked a little about this.

Speaker 1:

No. No. No. I know I know I need to, but I have not.

Speaker 2:

God, it's so good. It's so good. I it is so good. And I know parts of it are fictionalized, but they have been fictionalized so deliciously and so plausibly that it's just it's great. You've gotta watch the dropout.

Speaker 2:

But the dropout is on Hulu. Hulu has a lot of problems. And much to my and my 14 year old who whom I'm watching the dropouts chagrin, many of the Hulu problems can be solved by power cycling television. Yeah. Exactly.

Speaker 2:

And so and, of course, in the modern day, turning the television off does not power cycle the television. That mere way, like, turns off the screen. Right? Because it's like No.

Speaker 1:

Right. You gotta go to the corner to the pole. Totally. Totally.

Speaker 2:

Exactly. So and, of course, you do this and then and, it's it's it's kinda disgusting that all this works. Anyway, I wish you had the power cycle so many things. I I'm I'm embarrassed. We haven't arrived yet.

Speaker 1:

No. No. I mean, I I I've even got it worse with the unforgiving 4 year old, you know, when the audio cuts out from Netflix inexplicably, and you're like, let me just go to the breaker to turn off the TV. Not not a lot of forgiveness there.

Speaker 2:

But what are we teaching our children? We're teaching our children to reboot it. We're not teaching our children to debug it because I can't. It's super

Speaker 1:

Yeah. Disturbing lessons.

Speaker 2:

Disturbing lessons. But so we I mean, like with many things DST, we got on this when we were very hot on this subject. And now and now, like, time has faded. You're like, oh, what's that a couple weeks ago? Well, right.

Speaker 2:

But I think we got here because I was, having a hard time waking up my teenagers post DST. So daylight savings time started, what, 2 weeks ago now in the US. And in 2,007, daylight savings time was moved to start earlier and end later, which if you are a parent of children, I think this is I think this is brutal. I think I think the the week ago, where it was, 2 weeks ago, I think that's the hardest week of the year for a parent. But, Adam, do you not agree with that?

Speaker 2:

Did you have a hard time waking your kids up?

Speaker 1:

No. I I mean, I did have a hard time, I mean, just getting our act together for, for t ball and then for high school the next day, yes, hard. But I when I saw the, the you know, legislation passed in the senate, I got excited not because I am a DST fan or a standard time fan, but just a, you know, pick 1 and stick with Pick

Speaker 2:

1 and stick with the fan. Okay. So so your view on this is just like, I wanna stop changing every year. That's right. And I am willing for the sun to get up at basically, like, 9 in the morning, in exchange for not having to change time?

Speaker 1:

Yeah. Pretty much. And I guess I I value afternoon, evening suns. I always find, like, the the winter to be pretty bleak. Like, I remember many many times, like, you know, stomping the home on, public transit, you know, not that late in the evening and having a field pitch blackout.

Speaker 1:

So yeah.

Speaker 2:

Okay. But now now See, I am

Speaker 1:

a DST fan.

Speaker 2:

You are. You're a DST maximalist. I mean, it but it's gonna be I mean, it's one thing you'd be like, oh, you know, I was never so into mornings to begin with, so it'll be fine. But, like, this is gonna be like, the sun will not get out. Even where we are, we're not that far north.

Speaker 2:

So the San Francisco area, whatever word, 37 degrees, whatever it is. I mean, the sun won't get up until 8:30 in the morning on I mean, that is

Speaker 1:

No. I I no. I I I'm I get it, and I'm with it. And and, I think I think it might have been Tom Lyon actually who I see here on the call who who had tweeted out something like, hey, didn't we try this in the seventies? And they tried it for 1 year and then everyone lost their minds and got sick.

Speaker 1:

Yeah. And and I could clearly imagine how that's happening. Like, a bunch of DST fans are turned into seasonal effective.

Speaker 2:

Well and so I am half convinced or maybe even 3 quarters convinced that most of the country thinks that daylight savings time creates daylight.

Speaker 1:

Okay. That's

Speaker 2:

I think that people are thinking like, no. Like, the legislation that the senate is considering is is, should we have year round summer? That is what we're actually considering. And I think

Speaker 1:

Should we do the opposite to fight global warming? I don't know.

Speaker 2:

I think it would be lovely to have the sunset at 8:30 PM on Christmas day. And it's like, that's not what it is. That's not where we are on the sphere, folks. I mean, I I I so that I find, I I don't think people are kinda, like, thinking it through. That's just what's driving me nuts.

Speaker 1:

That's that seems like I count myself among them.

Speaker 3:

That seems like the only explanation as to why why not go for standard time. Right, is this idea that you're creating daylight. Like, that I don't understand. Unless it's just like a desire to be different from the rest of the world. Why not just go for standard time?

Speaker 2:

Right. Exactly. Because what you're basically saying is, look, when time zones were created, it was just like an off by one ear. They got everything and it's like, I I don't really buy that, you know. I I don't so

Speaker 1:

And and I'm with you also, like, that solar noon is at 1. Like like, I get it.

Speaker 2:

I'm not I'm not

Speaker 1:

that into that. That's fine.

Speaker 2:

So okay. So and I like to I get it. Like, changing it kinda sucks. I think that standard time and I get that, you know, standard time the knock on standard time is that in northern latitudes, the sun is gonna get up, like, hella hella early. I mean, it is gonna be a very, very early sunrise.

Speaker 2:

Right? It'll be a whatever it'll be. I think so. Right now, the sun gets up at, like, 5:30 in the morning where we are on the summer solstice. So, okay, at 4:30 AM sunrise, I'm with you.

Speaker 2:

I kinda think that, like, shifting the clocks is not a bad compromise, unfortunately. Hold on. I know.

Speaker 1:

I mean, but but even just like the the, you know, from the software perspective. Like, how how nightmarish is daylight savings time? I think it's crazy.

Speaker 2:

Okay. But from the software perspectives, aren't we isn't the best change from the software perspective no change? Isn't I mean, we did this because and I I would actually love to hear from people who are on the front lines in, 2007 when DST was the DST changes were deployed. Because I know that broke a lot of shit. And I feel like I was kinda off the front lines at that point.

Speaker 2:

I think you and I really had to deal with that.

Speaker 1:

No. We we both were sort of, like, glad it's not me, I think, on this. In that, all this antique software that that, isn't being updated or whatever suddenly needs to be updated if anyone's running it anywhere, you know, or else you're you're manually changing the time or whatever.

Speaker 2:

Wait. So I think the DST was worse than y two k. So I do think it's worth talking a little bit about y two k because I think that that was a, because Adam, were you at Sun for no. You were not at Sun for y two k.

Speaker 1:

No. No. No. No. No.

Speaker 1:

I was so I was still in college for y two k. And I gotta say from the perspective of a computer science student who thought he knew everything like many computer science students do. I thought, you know, what's the big deal here? Like, 2038 is gonna be a big deal for sure. And I still think

Speaker 2:

that Yes.

Speaker 1:

But, but I don't know. I was like, how much software is storing, dates in 2 digit decimal? And and, of course, the answer is

Speaker 2:

a lot. A decent amount. Although, you know, our problems are y two k problems. Our most acute y two k problem was not a 2 digit problem. It was a leap year problem.

Speaker 1:

Oh, it miscalculated the the staff of Ra kind of like, you know, add 1, subtract 1, unless it's this.

Speaker 2:

You know, it's not it it is not the staff of raw. I didn't know if you it is that it is that complicated. But the so the rule is that a we have a leap year. I mean, and the fundamental fucking problem is that that we I the the amount of time it takes our little sphere to take a lap around the sun is not an even multiple of the number of of laps we take and the the number of times we we this this sphere spins around, which is very problematic. It is really, really, really, really annoying.

Speaker 3:

Okay. But it's worse than that, Brian. It it's not constant either.

Speaker 2:

Okay. We leap seconds? We are coming for you later, leap seconds. So don't worry. We are coming for leap seconds.

Speaker 2:

But the that's right. It's not constant either. But so this is a very, like, this is a very pesky and annoying problem. Every 4 years, we inject a day to we we inject February 29th to kinda make up for it. But as it turns out, that's not totally correct.

Speaker 2:

And what it's it's the the that's the Julian calendar. The Gregorian calendar is the observation that, like, no. No. That's not totally correct. And it's almost correct, but not quite correct.

Speaker 2:

And every 100 years, you've got to take that day back. So every 4 years is a leap year, except for every 100, which is not a leap year. Okay? Except for every 400, which is a leap year. So so 1612,021,000 are leap years.

Speaker 2:

They have February 29th. And we had a lot of software that was keyed off of 1900. So it incorrectly said that 2,000 was not a leap year. Which I don't know. Is that surprising?

Speaker 2:

I'm not sure if that I it was surprising to me that, like, how would you encode the kind of, like, the, like, the first rule and then, like, the kinda, like, the subtle rule, but then not the other the third.

Speaker 1:

You're like, blah blah blah. I'm sure that's enough. So what

Speaker 2:

what is this document keep on going? It's like, okay. Wow. I got it's like, no. No.

Speaker 2:

There was another page. Did you gotta go to like, what do you need another page?

Speaker 1:

Whatever. Whatever. It's quitting time.

Speaker 2:

Right. It's it's like you're getting out of the exam. Like, what did you get for problem 6? Like, there wasn't a problem 6. Like, no.

Speaker 2:

That's a little bit. That's right. So yeah. And so in particular, have you ever used the at command?

Speaker 1:

Yeah. For sure.

Speaker 2:

Okay. Someone was asking earlier, like, what's the oldest piece of software you use? Do you actually wait a minute. Do you actually use the at command?

Speaker 1:

No. Have I used it?

Speaker 2:

Okay. That's better. Do do I use it?

Speaker 1:

No. Because I feel like I know enough to get directions away from it. Like, I feel like every every document, like, describing at is, like, you you should not use this. Like, this is unsafe at any speed. If you're considering it, consider something different.

Speaker 1:

And and I've read on and then not used it in Niagara. I'm like, oh, what's this?

Speaker 2:

Well, that is you're you're very wise because the at command in particular, the at command, like, tries to be friendly in some strange ways. So at is a command that allows you to run a job. They say at a specific time, but it's like, it's not a specific time because you've got all these ways of expressing time relatively. So you can say, I would like to run this a month from now. It's like, well, alright.

Speaker 2:

What does a month from now mean if you're on the 31st day of a month? Like, what does that mean? What you you is a month from now, is that on the the there is no 31st day of the next or if there is no 31st day of the next month. If it's January, what does that mean for February? And in particular, we had a y two k problem that if the the the y two k bug only service in one command, the at command, if you use this weird plus what n month syntax, and you ran that on the 29th day of one of the days in 1999 so January 29th, February not from 29th.

Speaker 2:

There's a February 29th. March 29th, and so on. What are those 11 days? And your target day was February 29th 2000, which did exist because it is a leap year. The it would run it on March 1st.

Speaker 2:

Like, that was the bug.

Speaker 1:

Like, irrespective of when you like, whether it was November or

Speaker 2:

No. No. No. No. No.

Speaker 2:

No. No. That that might be a legitimate buck. This is like you are on it is it is only if you have the intent of running it on so in other words, if you you have to be running this on December 29th, running at plus 2 month to denote like, I wanna run this in 2 months, your at job. Instead of running on the correct date, February 29th, it would run it on March 1st.

Speaker 2:

Gotcha. Gotcha. Which to me I tried to explain, this feels I mean, does that feel likely for someone to hit that?

Speaker 1:

Pretty remote.

Speaker 2:

Yes. I would say exceedingly remote. I'm like, you've gotta use this, like, weird syntax that you've gotta have that has, like, already semantics that are, like, murky. And as you say, like, I'm already got the sense that I shouldn't be using at at all, you're using it at the end of the month, which you should be doing. You're doing it on the 29th day, a day that doesn't exist in all months.

Speaker 2:

You are targeting this in this so anyway, I thought it was super weird. But everyone was so fucking terrified of y two k that we destroyed all the CDs we've made for Solar 7 over this. That's

Speaker 1:

and this was the release that you were the

Speaker 2:

the technical Yes. Right. Exactly. So I so I feel it is, like, my job to inform the rational decision, which I did not perform. I I, like, I I I failed because I'm like I and I have this, like, long email explaining just how impossible it is for effectively someone.

Speaker 2:

And because, again, you have to run this on one of these these 11 dates in 1999. And we were already at that point, like, past a couple of them. So it's like, they're only, like, the 4 days left that you have to run another brand new album. I lost they decided they were too scared, and they destroyed all the CDs. Several $100,000 for the CDs were destroyed, and then we fixed the bug and they repressed them.

Speaker 2:

So that is and so that is the peril I think of, like, you sir and that's not with the calendar changing. That is just and I think that that we only saw this because people were so dialed in the y two k, which then effectively fizzled. I think when the calendar starts changing, and I'm I'm hoping that somebody's got some, Yeah. Matt, I know you can't Jason, just hand up. Matt, I don't Matt, do do you have a a DST story?

Speaker 2:

I'm sure there there I I know there were a bunch of them out there.

Speaker 4:

I I do. I also actually have a y two k

Speaker 2:

Oh, good. Go for it.

Speaker 4:

So, I'll do the y two k anecdote first. In January of 2000, I was, emailing back and forth with someone who, for whatever reason, was using the Elm email client. And, I I I don't know if he was even using the latest version of Elm at that time, or he might have been using the version that was available on his shell account at his Internet service provider. But, this this email client would, format the date header with a 2 digit year. And the way that it did it was by subtracting 1900 from the actual year.

Speaker 4:

So when year 2000 came around, I got to see year 100 in the date header of this guy's messages.

Speaker 2:

And was this discovered before y two k, or was this only on January 1, 2000? That this is

Speaker 4:

I only found out about well, I I I never researched. I I never looked at, like, well, the this being 2,000. I I don't know if Elm had a bug tracker, but I didn't look at mailing list archives or anything. I just I just observed it and said, well, if this is the worst that y two k brings us, we did pretty well.

Speaker 2:

That's an actual y two k bug in the wild post 2,000, which I feel is like, that's pretty hard. That's that's

Speaker 5:

pretty hard.

Speaker 4:

Like I like I said, he might have he or his his ISP might have been using an old version of the client to begin with. Now now I do wanna research that bug and see. I wonder if they've still got archives of of old I'm sure somebody still has archives of old versions of Elm, and we could probably we could if if I really wanted to, I could bisect and see when, you know, when the bug was fixed if it ever was.

Speaker 2:

I will tell you that Elm y two k problem definitely has his. You could definitely see this problem and a lot of folks hitting it. So yeah.

Speaker 4:

Now, do you do you want me to go ahead and get into my, DST

Speaker 2:

story? Please. Yeah. Yeah. Definitely.

Speaker 4:

Okay. So, this this was a few years ago, so I don't remember all of the details now. I remember fixing it late on the Saturday night. Well, I remember that looking at it at least and and and, you know, working around it late on the Saturday night that, that that we have transitioned off of Daylight Saving Time. So, I was respond I had developed a, a web application that, connected to a third party service, on behalf of the user and authenticated with that service using OAuth.

Speaker 4:

And we we had a cron job that would periodically connect to, into the service to to pull new pull new data. And, as you may know, OAuth, at least in some implementations, has an access token and a refresh token where the access token expires something like, every hour and the refresh token lasts much longer. And when you when when the access token expires, you have to use the refresh token to get a new access token. Well, so, the logic I had implemented for finding out when the access token expired, went like this. I've when when whenever we got an access token, the the OAuth server would return how many seconds until the token expired.

Speaker 4:

And then I would yeah. My code would add that to the current time and then store it store the expiration time in a in a table in a MySQL database. And the MySQL in MySQL, the date time type is basically year year, month, day, hour, minute, second, no time zone. And I was using the Python date time type to add the the duration in seconds to the current time. And I was using the version of the Python date time type that also had no time zone.

Speaker 4:

So, Saturday night, we're getting ready to get go off of daylight saving time. And, that that that's, I mean, that's the time when 1 am occurs twice. Right? So, because you, you fall back an hour. So, so, yeah, the the Cron job is getting new access tokens, and let's say that the access to you know, that that the time is, you know, 12:53 AM when we get an access.

Speaker 4:

Yeah. 12:50 and oh, and I was and all of this was in I I had our servers set to eastern time, so all of it I I was doing all the in local time just because, well, I started doing that when I was 21 and didn't know better. Youthful discretion. Yes. Got, tying back in fact to the theme of last week.

Speaker 2:

Yeah. Right. Right.

Speaker 4:

So, so yeah. So let's say it's 12:53 AM, just to pick a random time. And when we're, yeah, when we're pulling a new access token, we we add add an hour to that. Yeah. Let's say that access token expires in an hour.

Speaker 4:

We add that, and then, yep, store the expiration time as 153 AM in the database. And then we go off of daylight I I don't remember the exact detail, but you can probably see where I'm going with this. Did you roll roll back? Yeah. It it Yeah.

Speaker 2:

Did did folks end up having to reoff? I mean, so what what was the

Speaker 4:

So well so so I started getting I think I started getting emails about internal about errors in the grant job because it didn't know that it was supposed to pull a new access token. So I think I I think my work around, my immediate work around was to to just, force, you know, manipulate the database so it would force a, a refresh. But, yeah. I'd probably probably the the the way to avoid this whole thing would have just been to work with Unix time stamps all the way through.

Speaker 2:

Yeah. You really don't wanna store anything in local time. The local time is a huge problem. UTC because you UTC,

Speaker 4:

you think UTC would have been fine. Right.

Speaker 2:

UTC would have been fine, until we get to leave nuggets, which we can

Speaker 4:

Ah, okay. I I will look forward to hearing all about that because I I have not been bitten by that yet.

Speaker 2:

Oh, god. I will so before you leave seconds, I'd be curious if if anyone because it sounds like the now the issue that you hit was a just a the DSP the the change from standard time to daylight time that that happens every year and storing things in local time, As you say, you put in indiscretion. I would be curious if folks because the the thing that was, I think, was really disruptive was the actual change to when we went on daylight savings time and went off daylight savings time in, I believe, 2007. Right? I mean, it was it the that was the the what, made that we changed that time, and that caused a lot of software to break, because now you got software that is actually in the wrong time zone for for several weeks if it's not upgraded.

Speaker 2:

Yeah. Jason, go ahead. I know I got your hand up for a long time.

Speaker 1:

So a couple of

Speaker 6:

things with that, though, I was trying to think because, first of all, you know, there's still 2 states that don't follow daylight savings time period already. Arizona and Hawaii. But even before that, one I'm quite familiar with, Indiana didn't follow daylight savings times for years. But in the 2000, they started following it. So you had a situation before they changed you know, shifted the thing around, you know, nationwide where you had now a state, you know, where before, you know, never followed due date savings time.

Speaker 6:

Now it did. So I'd imagine there'd have to be bugs just from that because

Speaker 2:

And and, Jason, did you grow up in Indiana? Did you spend time in Indiana? But it because Indiana, if I recall correctly, was, like, county by county. Well spend the time versus these 2 types.

Speaker 6:

To work. So it's, like, in the time zone database, yeah, you have, like, Crawfordville and all this. That's where, like, yeah, if you want to compute time, like, back in, like, you you know, back to, like, say, 1920 or 1860, you know, then but for you know, since computers were a thing, you know, basically, most of the state this is also where it gets interesting. So I grew up in, it's Lake County, which is Northwestern most county, which is, you know, where Gary is. It's by Chicago.

Speaker 6:

So where I grew up as well as the county next to mine, we're actually on central time and did follow day daylight savings time. And so half the year, we were on the same time as the rest of the state, and half the year, we were an hour behind, which also made things interesting if you like, I have, an aunt that still lives in Indianapolis that actually the rest of my family lives there now. But, yeah. So for half the year, you know, they're hour ahead. Half the year, they weren't.

Speaker 6:

But, you know, when they switched to just following daylight savings time, you know, I just have to wonder if that didn't cause problems.

Speaker 2:

Right. The the That

Speaker 6:

was before that.

Speaker 2:

And because Indiana is now all central. Is that I think that that's like, is it all central time now? Or is it or does it still divide the state? I I do remember, like

Speaker 6:

Sorry. I Indiana is like, well, aside from and I think there's also a couple counties, like, in the southern part, but now it's all eastern. Well, all except for the ones that we're always following central. Just because, you know, like, you know, all our TV stations came out of Chicago. You know, our radio stations came out Chicago.

Speaker 6:

So if we're always an hour off, you know, a lot of people would commute to Chicago. If you Got

Speaker 2:

it. Right.

Speaker 1:

So it's it's all eastern except for in places where it's not.

Speaker 2:

Well, basically, it's like a little bit by Chicago where I think there's, you

Speaker 6:

know, a couple in Southern Indiana I've never been to. And so but, yeah, the biggest thing actually just kind of a funny footnote to that is when they did flip, as it turns out, there were a bunch of actually, I guess some kids actually started a little for a couple of years that go around to people's houses to change the clocks for them because they'd never had to do it before.

Speaker 2:

That's great. And, Adam, do you remember I mean, Jason mentioned the time zone database. And you I just remember us doing, like, out loud readings of this thing, the one letter or not.

Speaker 7:

Do you

Speaker 2:

remember this?

Speaker 1:

I remember being fascinated about there was about a month, when we were working on Fishworks, the this the storage product that's on the Brian and I worked worked on that I think it captivated us all for, for like at least a month because there was the, you know, there were, there was this period. I think I learned about it more in, reading that book about the metric system. How, the, what's the, I don't, I remember the subtitle better than the primary title.

Speaker 2:

Oh, whatever happened to the metric system.

Speaker 7:

That's right.

Speaker 1:

How America lost its feet. Wait, kept its feet.

Speaker 2:

Hold on. Yes. Hold on. Wait a minute.

Speaker 1:

What did we do again? Right. So, so I remember that book describing, you know, a bunch of standardization where, where the, where noon was, different for every railway station. And so you had these micro, time zones effectively. But I remember just pouring through the time zone database, you know, seeing, you know, this island was, you know, you know, 12 minutes ahead of the other island and and, just captivated by

Speaker 2:

that. Well, there's so much history. It's like, if you like to read the encyclopedia as a kid I I definitely did. There's so much history in the time zone database. It's really remarkable.

Speaker 3:

This But it's an it's an encoding also of sort of geopolitics as well.

Speaker 4:

Exactly. Exactly.

Speaker 3:

I Ireland decides that they no longer want to be, you know, allied with with this great nation. They want to switch to that great nation. So they need to decide which side of the international date line they are. It is fascinating.

Speaker 2:

It is really though. It's a very good point of it. And the book that Adam is referring to is what you you read this on Rick's recommendation. Right?

Speaker 1:

That's right. Yeah.

Speaker 2:

Yeah. Yeah. So our colleague, Rick Alfer, recommended this. This is like you know, you you have these books that kind of spread through a a group like wildfire that kinda everyone reads, kinda becomes canon. I felt like that way with this book, Adam.

Speaker 2:

I think a bunch of us read it. I love that book. That book was so good. I mean, relatively recent book. Did that book so it's it's talking about not just, the metric system, but the kind of standardization of all weights and measures and units of time.

Speaker 2:

Did that so in particular, that book talks a lot about the fixed calendar, the idea of a, of a 13 month calendar where each month contains 4 weeks. And then that that all that adds up to 364. So you have to inject blank days, one blank day, 3 quarters of the time, 2 blank days, roughly a quarter of the time. And then I mean, Adam, did that I didn't.

Speaker 1:

I love that.

Speaker 2:

I I there were a

Speaker 1:

couple of different proposals for for metric for for different calendars that were different levels of standard, whether it's 4 weeks or 30 days. And all of them had this like sluff to deal with, which I really enjoyed. And most of them were like, make it a holiday. Like, make these, like, days beyond time, which I which I also really enjoyed. That that was their, like, well, what are you gonna do about it?

Speaker 1:

Like, give people the day off.

Speaker 2:

Okay. So and did did you start to advocate that to others? I felt like I I was like, this idea is, like, zany, but I I I've start I get kind of, like, start I I floated it by the, I floated it by by by the kids. And, it's in I mean, they definitely you sound bonkers when you're advocating a 13th month, but it's kind of interesting. I think that the problem is that, like, we're under the it means that, like, if your birthday's on a Thursday, your birthday's on a Thursday for all eternity.

Speaker 1:

I hadn't thought about that part. Or or if you're, like, in this Interslip period in a, you know, land without dates that you're always in this, like, holiday season.

Speaker 2:

Right. You're all the the you're are and now, like, if you are someone who's like, Thanksgiving sometimes falls on my birthday, but sometimes it doesn't as it does with with my wife, Bridget. It's like, now you're either gonna always get nailed or always not get nailed. And I don't know. I I I think that that's it's intriguing.

Speaker 2:

They do you remember why they didn't end up adopting it?

Speaker 1:

No. I don't remember there being, like, a particular reason.

Speaker 2:

Well, it's like I feel like this is so telling. This is, like, such a software analog because there are there ended up being 2 different schemes that are basically this similar. One was dividing the the the calendar into 4 quartiles, and, each each quartile had so 30, 30, and 31 days. So each one had 91 days. And the other proposal added this 13th month, but they both have the same property of injecting these blank days and so on.

Speaker 2:

And these two groups of people, of course, hated one another's guts. And so everyone is spending time, like, slagging on the other person's proposal, and ultimately, like, is not good at least, and we're gonna just keep the calendar the way

Speaker 6:

it is. Didn't Kodak adopt that? I mean, I know it's more business, which isn't quite the same, but, like, is there kids? But they then they I thought they did, like, a 13 month calendar.

Speaker 2:

So yes. Very good memory, Jason. Eastman Kodak. I just feel like it was it was must have been so delightfully weird to be, like, a 19th century billionaire effectively because you get to pick all these strange causes. And Eastman, George Eastman was apparently a big, like, calendar reformist.

Speaker 2:

So he yes. He wanted to to impose a 13 month counter. And, apparently, this is like some companies still have 13 month or do 13 units per year according to that book anyway. Aaron, sorry. You had your hand up.

Speaker 5:

Yeah. So there was 2 bugs I wanted to bring up. The first is there's a lot of things that go wrong during the second 2 o'clock in the morning. I ran across a bug where we had a system that was doing builds automatically, and the whole process took about 4 hours and would start at midnight. And it turns out if there's 2, 2 o'clock in the morning, then have some parts of your build happening on Linux and some parts of your build happening on Windows, Cygwin's CMake will think that things are in the past that are actually from the future Oh, wow.

Speaker 5:

Because it's the second 2 o'clock in the morning, and it thinks that the stuff that was already built wasn't actual was actually from the future because it was done at 159 of the previous 2 o'clock in the morning.

Speaker 2:

Okay. Alright. This this

Speaker 1:

this is exactly, Brian, why I think, like, I'm not for DST or standard time, but just one time. Because anytime I was, like, programming Kron to do nightly builds of whatever fashion, I just, like, well, you know, I don't wanna do it at at midnight because my colleague might still be awake and doing stuff. And I don't want to get anywhere near 2 AM because then I'll run a foul of this stuff. So what am I I'm gonna start building it like and then if you start pushing it to 3 or 4 AM, then you start bumping into your colleagues on the east coast who might be wanting to use it anyway.

Speaker 2:

So you you don't like the fact that they give us

Speaker 5:

built and released binaries that we had to deal with for about 6 months because there were binaries that were released in an inconsistent state into the world.

Speaker 7:

Oh, man.

Speaker 2:

Okay. So so, Aaron, I how many years did you debug the same problem? You should be like, goddamn it. I feel like I have debugged this problem before. Oh, wait a minute.

Speaker 2:

A year ago, I debugged this problem. I feel like I would it would take me, like, 3 or 4 years of debugging the same problem before I retained it.

Speaker 5:

So what we did was the second time this happened to us, a year after the first, we added new rule to the build system that it would not build anything at 2 o'clock. Like, it didn't need to know what things or what days of the year. It was just like, you know what? 22 o'clock's a bad hour. Let's just not.

Speaker 2:

Okay. So this is Adam's approach. This is just like

Speaker 1:

That's right. That's would've waited us not to play.

Speaker 2:

Yeah. Hey. Hey, kid. Well Stay off at 2 AM. There's a

Speaker 3:

But this

Speaker 5:

is still my second favorite. It's alright. My favorite one was a bug that actually came from a contract I had, which was I ended up starting at a startup on leap day that had a standard language contract that said, your equity vests on the same date on the following year. Oh, no. So the day in which my equity would theoretically vest was not a day that actually existed.

Speaker 2:

And it

Speaker 5:

was just like, that's get you gotta read those start up contracts.

Speaker 2:

Okay. I I take it from your light tone that either the issue was resolved or the company went nowhere. Clearly, this did not you did not end up having to leave a life changing lot of money on the table because of this this issue. Did you get it resolved? Or how did

Speaker 5:

it It it was resolved by the company not existing anymore.

Speaker 2:

It's the very clear I had to hunch that one, but that might be the answer. That's, yeah. That's pretty funny. Read that documentation. When I would I don't know anyone born on February 29th, but I don't know if that like, I would love to know I surely, those people have an opinion of people born on February 29th.

Speaker 2:

And I could see it being kind of maybe it's a special thing, but maybe it's a shitty didn't think that 24 hours in a day was enough. So he devised, like, a 6 day work week and, 28 hours a day. Okay. So and now is he so this is an interesting is he just gonna ignore the sub and the fact that we're just gonna, like, I'm gonna take the existing a 100 and 68 hours and just divide it differently into days that I have invented. Is that what he was doing?

Speaker 2:

Yeah. That's what the that's what the story was. Someone who worked there told me. I don't know if it's true or not, but it seemed really interesting. And he had weird hours of coming and going to the lab.

Speaker 2:

Well, that would actually make sense. You know, I mean, that was, like, I when when I was actually first working for Sun with Jeff Bonwick, Barnwick worked super strange hours. And I was working on, like, a 28 hour clock, and it was very difficult to at, like, 5 PM and leaving at, like, 8 AM. And I had a reverse commute, but I was turning my reverse commute into a non reverse commute by commuting on basically the going the wrong direction at the wrong time. So I, I I I'm sympathetic with Ken Thompson's point of view, but I think you gotta, like, unfortunately, have to yield to to the, the sun on that one.

Speaker 2:

And, Sumit, I wanna get to you in a second. I on on yielding, I'd say, I wanna get to leap seconds here because I I do feel we need, to so, Simeon, you'd mentioned that that the, unfortunately, it's not constant. That the the Earth the the the speed of the Earth's rotation on its axis varies with time, and we've injected these things that the these leap seconds from time to time to to keep so or noon at so or noon. I are are you I mean, I leave seconds, I think, are are pointless. I really think leave seconds are a problem, but I would love to hear the if someone defend leave seconds.

Speaker 2:

Do you have an opinion one way or the other on leap seconds?

Speaker 3:

I guess a weak one. I mean, it kinda makes sense. You you have a calendar that aligns with astronomical time. If you if you have slippage, then, you know, it becomes progressively more wrong.

Speaker 2:

Okay. But it becomes progressively more long over, like, 100 and 100 of years. And also So it's

Speaker 3:

it's just it's like someone else's problem, like

Speaker 2:

That's exactly what it is. Give someone else's well, it's like, hey. Look. Who left the astronomers in charge? Why aren't the astronomers making it our problem?

Speaker 2:

I mean, I I like, look. It's a problem. But it's like, why does it have to be why does every software engineer have to deal with this problem? I don't I I Yeah.

Speaker 3:

I guess that was all I wanted to mention is that Adam's in good company because, and and maybe there's some Googlers on the call who can confirm this for me. But, a Googler friend of mine told me that Google has mountain view time

Speaker 2:

That is.

Speaker 3:

Yep. Which which is Pacific time without leap seconds because it was like, fuck it. This is too hard. And, to me, it seems quite consistent with the sort of view that mountain views also the center of the universe. So, you know, why not?

Speaker 2:

Yeah. I believe that is called it is called it was told to me as Google Standard Time, so I would love to hear. And I I didn't realize that it was Pacific time minus the the leap seconds. It it so does it does not respect DST, I would assume. I know that there's at least truth to this, because, Adam, I don't know if you've gotten our ex Googlers going on this subject.

Speaker 2:

But anytime anyone even, like, comes close to something having something other than UTC in a log, they're like, no. UTC only. I feel it's like it it it's like you wanna roll out, you know, the OKRs and perf plans. I mean, they they immediately, like, lunge at it. So it's yeah.

Speaker 2:

I Google it'd be, I did not realize they don't include leap seconds. I can understand that from a I would so do you remember having this discussion with we had a coworker who, also was a leap second denier and, wanted to understand the source of leap seconds. It's under it's my understanding that we don't totally know all of why the rate is changing, but a big part of it is title friction, I believe.

Speaker 3:

Yeah. And it it tends to also so the the the the need to introduce sleep seconds, is correlated also with seismic events. So, you know, the major, you know, tsunamis in, you know, in the Pacific has been there's been periods of lots of leap seconds and periods of quiet. Right. So clearly, something affects the rotation of the earth.

Speaker 3:

You know, it it is not really a rigid body. It's a fluid body. It wobbles. Right? So, yeah.

Speaker 3:

It's it's def it's definitely a natural phenomenon.

Speaker 2:

It's a natural phenomenon.

Speaker 1:

I feel deeply ignorant here. So why why are you so painful?

Speaker 2:

No. But you okay. But you obviously you don't remember this lunchtime conversation that went on for weeks. This is where Keith wanted to blow up the moon. Don't you

Speaker 1:

remember this? Oh, I do remember Keith wanting to blow up the moon.

Speaker 2:

Yes. So Keith is like, we're gonna blow up the moon. Yeah. And I actually still don't have, like, a totally straight answer on I don't you I remember you being like, well, I can't imagine there would be any side effects of that.

Speaker 4:

I mean,

Speaker 2:

that seems like a great answer.

Speaker 1:

That's right. Right. Full speed ahead. Let's ship it.

Speaker 2:

Full speed ahead. Let's ship it. Let's blow off the moon. I'm sure nothing relies on tides and, you know, anyway, let's figure it out. The but the then it was like, well, actually, if you blow up the moon, what does actually happen?

Speaker 2:

You don't remember this? We I'm sure

Speaker 1:

you you I I do remember, for example, thinking that we could corner the market on, like, 25 hour watches and then and then detonate the moon and then make a killing, but that plan never came to fruition.

Speaker 5:

But those seconds do matter because, for example, systems like Spanner care about time stamps to a significant degree. So if you have one system inside of a Google cluster that is on UTC UTC instead of UTC 1, it's not actually going to be able to commit anything into Spanner because its clock skew is enough off that the entire rest of the system will disown it.

Speaker 2:

Yeah. I mean, they are then this way, Adam, I think that's the kind of answer to your question. I mean, they are actually really important. I mean, the in this this differentiation between what UTC and UT1, it's it is it is really, problematic. And and, Tom, I mean, it's something you've you've dealt with as well?

Speaker 7:

Yeah. Not not too much, but, you know, that that was the prop the mistake early on in Unix was trying to track UTC, including the leap seconds and do that in the kernel. Whereas it should have just been a, you know, a constant timer off offset by user level code somewhere, and so that that screwed up all kinds of things.

Speaker 2:

Yeah.

Speaker 5:

And I would still argue it is the lesser of 2 evils because so most of the systems we use deal in cesium seconds, not geoseconds. So either you're going to have to keep adding seconds periodically or every single year, NIST is going to have to produce a new, here's the number of cesium atom decays in a second that we're gonna use this year, and that seems worse.

Speaker 7:

Yeah. But there should should never have been a way to get the time without specifying a time zone, which becomes a user level presentation thing. All all this kernel and middleware and stuff should should always have run with just, you know, seconds since the epoch.

Speaker 2:

Seconds since the epoch. Yeah. Yeah. Agreed.

Speaker 5:

I assume even cesium seconds, not

Speaker 2:

geosecond. Yeah.

Speaker 7:

Yeah. There's no such thing as a geosecond. Right?

Speaker 3:

Yeah. And even even the way that you arrive at what is the universal time in cesium seconds is, you know, it's kind of interesting. It's it is determined a month in arrears by averaging 400 atomic clocks adjusting for their altitude because of the, you know, gravitational well. You know, it's it this rabbit hole goes super deep. There are time nuts who love these these things.

Speaker 7:

Well, there there's a definite there's a hard definition of a second, but it's just impossible to measure it that way. Right?

Speaker 2:

I I didn't realize that the gravitational wells were an issue. I mean, of course. I mean, why, of course, why would I think it would be as as simple as measuring a cesium second?

Speaker 5:

It does get relatively complicated.

Speaker 7:

Oh, okay. And then this gets really complicated if you're maintaining GPS and trying to tell the satellites what time it is so they can tell everyone else what time it is.

Speaker 2:

Right. And that's where, like, relativistic differences come in. Right? I mean, the

Speaker 7:

the Right.

Speaker 2:

Right. Which is really get gets very mind bending. So I I I personally think that the the lesser two evils be to and we we have been on a a streak of not having leap seconds for a while, since, what, 2017? It's been it's been a couple of years since we've we've not had a leap second. So I I don't know.

Speaker 2:

I was kind of optimistic that we were just kind of seeing the error of our collective ways, but I don't think that's it. I think we're gonna see leap seconds.

Speaker 7:

I thought they were threatening to have a negative leap second at some point.

Speaker 2:

Okay. That I at some okay. That can we agree that negative seconds are out of bounds? That the Earth is either spinning up or slowing down, but it may not do both?

Speaker 7:

No. I don't know. It's due to seismic events, it might change.

Speaker 2:

I I'm saying that I may be on team Blow Up the Moon if we if if

Speaker 1:

time zones go backwards.

Speaker 2:

If time starts going backwards, like, tell me more about the the newsletter that I I will also go subscribe to the blog if it is the the blog for the newsletter. We actually did have a so the keeping time in a you know, Tom mentioned kind of historic Unix. Key and keeping time in Unix is historically really complicated. Keeping time in system is complicated because you need to have a reliable oscillator. And the you we used the, the the tick counter on UltraSpark to, to count the the the time spent spooked.

Speaker 2:

In order to do that, we would have to know the number of ticks per per second. In order to do that, we actually like I said, written the code for this. The most reliable thing in the system was actually the TOD part. And so we would, in a very tight loop on boot, watch the TOD part roll over for watch basically 2 boundaries in seconds, and we would measure it. And that would be accurate enough for NTP to be able to to that was in within the 64 parts per million that NTP needs.

Speaker 2:

Well, no, that was very funny. So we had a we we replaced they made a, a replacement of a different TOD part. And this TOD part, if you queried it too frequently, it would freak out and the seconds would roll over. So I get this call from the manufacturing facility, and godly knows what route they did. I mean, they they found the right person at the company.

Speaker 2:

They're like, hey. We understand that you could help us look at a time problem. Like, yeah. Sure. I yeah.

Speaker 2:

I'm the right person. Like, what's going on? And they're like, well, could you just log in to the system? I'm like, sure. I log in to the system and I run the date command, and it's like the year 23/75.

Speaker 2:

And I'm like, is it the 24th century? Like, what is going on on this box? Like, yeah, we're hoping you could tell us because time is passing very, very quickly. So then you run the date command again, and, like, 18 years has passed in a second. And you're like, what is going on in this alternate universe?

Speaker 2:

And, of course, what's going on is that we early in boot, we hit this thing hard a couple of times. The seconds rolled over, and we we concluded because my code had no, like, check on this. Like, wow. This is a very slow CPU. This is like a 3 hertz CPU.

Speaker 2:

And therefore, time is now running at, like, a 167000000 times faster than it should be.

Speaker 7:

I can tell you I I wanna mention, you know, IBM has has had the virtual machine system for a long time. But one time, they hacked it to do virtual time so they could run benchmarks, assuming things were running Woah. Different speeds. Very clever.

Speaker 2:

Of of course. I mean, leave it on IBM for both, like, both invent virtual machines and then also invent the way to manipulate dirty virtual machines to make a benchmark look good. That's pretty funny. So I I have other folks at I mean, are there other, time war stories? I I again, I'd be because I'm curious about, like, what if we decide to go to, hopefully, all standard time and not all daylight time, what would be the IT ramifications of that?

Speaker 2:

Is that gonna be and, also, 2038, actually, is another one. Adam, you you made reference to 2038 earlier. Maybe it's worth describing that problem a little bit.

Speaker 1:

Yeah. I mean so, I mean, time for many for I mean, for a long time has been represented as as, second since the UNIX epic, since January 1, 1970. Right? And as I recall, it was, for many systems, it was stored as assigned values. We have 31 bits of representation.

Speaker 1:

And That rolls over at some point in 2038, which feels like a much more common pathology than the Y2K problem. Now since then, the representation has gone to 64 bits or 63 bits or 64 bit side rather. But I think there are probably a lot of systems out there that are still susceptible. At least that's that's my gut.

Speaker 2:

Yeah. That's why I get

Speaker 7:

By the way, the the the y two k problem was a real problem on mainframes where the the the y the year year day to day format was very, very common.

Speaker 2:

Well, yes. And I I one of the I didn't I didn't I didn't I

Speaker 5:

didn't I didn't share it here, like, in hardware.

Speaker 2:

Aaron, you're breaking up for me on that. I'm I'm not sure if you're breaking up for other Yeah. But that I guess I thought one of the

Speaker 5:

IBM chips had dedicated hardware that was in the 2 year dates. So so the well, what you maybe were calling

Speaker 2:

and or maybe you heard this elsewhere, but the after y two k, I was meeting with a the CIO of a big bank who had told me that 45% of their y two k problems came from a single platform. So you can imagine I was on the edge of my seat, and that was from the IBM 1401. So the I b in the IBM 14 o one and 1410. There there were only 2 digits. But to me, as I that was actually it it was an incredible moment for me because this, again, in 2000, this is a machine that is at this point, I'm an adult.

Speaker 2:

I'm a young adult, but I'm an adult. And the 1401 is a machine that was too old for my father to work on when he was undergraduate. So this is a the a machine I mean, Tom, you probably never worked on 1401. Right? I mean, that is way too Right.

Speaker 2:

Right. Yeah. Yeah. Yeah. It's like

Speaker 7:

It it shipped in 1960, and

Speaker 2:

it Right.

Speaker 7:

So this is like But it was popular.

Speaker 2:

It it was very popular, but the the fact that the software had outlived the hardware was a big moment for me personally of, like, wait a minute. The software is still running in production. And this is hardware that is moments after the big bang of software. And you you begin to or software at this moment and most of the big bang of hardware. And you begin to realize that, like, wait a minute.

Speaker 2:

Our software is gonna survive in perpetuity. This stuff is just gonna not it doesn't wear out. And so that is the, and I think that stuff, I believe, just can't work with a 4 digit year. I think that they had to, like, actually rewrite it. I don't think that which is like it's, you know, 1401 software.

Speaker 2:

I think it's time to turn it off. It's time to rewrite it. Really?

Speaker 7:

So I resubmitate. Do not resuscitate

Speaker 2:

exactly what old DNR put on me on the 1401 for for sure. But and I'm I'm sure there are other systems. So but so, yeah, Adam, the 20 38 problem feels like it's real. I mean

Speaker 1:

Yeah. I mean, I'm sure that y two k felt real. I mean, it's it definitely felt real. I'm sure there were real ramifications, but, I mean, it feels real and terrifying to me. But maybe that I'm, you know, by the time 2,038 rolls around in a surprisingly short amount of time suddenly, All sounds like a crazy gray beard, who's living in a time when 31 bits was enough to to store anything.

Speaker 2:

Well, so why do you but you remember the Y2K hysteria, though, surely. Do you I mean Yeah.

Speaker 1:

No. I remember I remember the stereo for sure. Right? Like, don't fly on y two k. Like, peep you know, people, panic buying, you know, before it was popular.

Speaker 1:

Yeah.

Speaker 2:

Right. And I believe it was, China. I for China Airlines. China forced all of their because it stayed on, airline. All the execs for the airline had to be aloft at midnight on y two k.

Speaker 2:

That's awesome. We're just like, oh, you know what? That's a pretty good idea, actually.

Speaker 1:

Why? No

Speaker 2:

reason. Just No reason. Just Because it's totally safe to fly. Right? Because we're gonna have no Y2K problems.

Speaker 2:

That's why you're gonna be a lot. But I and there were I mean, the the Matt's kinda Elm problem aside, there were very few actual problems. But I think there were very few actual problems because we did focus on it so much. I think it's part of the reason we so with 2038, like, 2038 is not that far away. I mean, Adam, we are much closer to 2038 now than we were when we started our careers.

Speaker 1:

We're closer to that than Right?

Speaker 2:

Yeah. Exactly. That's right.

Speaker 1:

That's pretty scary.

Speaker 2:

That is scary. G

Speaker 3:

p GPS also has it has a week number. I think it's a 10 a 10 bit week number, that rolls over every 20 or 40 years. And that's, you know, it's particularly funny because, when we've talked about GPS, you know, in these spaces before about, you know, how GPS really needs good time. I mean, it's like, you know, one of the 4 variables that that a GPS receiver, solves for. So it's kind of weird to build GPS with, you know, a time rollover problem built into it.

Speaker 3:

But that was you know, it it occurred in 99 and 2019, and, I know that, you know, some airlines canceled flights in 2019 because of it, and, you know, this kind of thing persists. Like, it's not just the 32 bit, you know, UNIX epoch counter problem. It's really it it occurs in many different places in many bits of software. I cannot really see that it's gonna stop.

Speaker 1:

That that's so true. There there, at least in, versions of Oracle up to 11, I believe. There was a transaction counter, that had a similar property. I think their epic was, like, sometime in 1980, probably, like, Larry Ellison's favorite day or something. And if it detected that you were effectively breaking the transaction speed limit, which meant that you were, like, not gonna roll over the 60 before bit counter before 2020 or something like that, it would reboot the database.

Speaker 1:

Oh, God. So, some some bugs in software that had at Delphix would, basically cause lots of transactions to pile up and, and break the speed limit and they would have to kind of reboot and cool off the database until you had enough headroom on this, on this sort of diagonal line from 1980 to 2020 or

Speaker 2:

whatever it was. Oh, I assume that that that issue has been fixed. Would it Yeah.

Speaker 1:

It's been addressed. And at the time I remember it's like, well, why don't you just not have a, you know, 56 bit number or whatever it was? And it turned out to be baked in very, very deep in in the internals of Oracle. I'm missing some of the details here, but it it took the a significant rewrite, I think, in 11 or 12 to to absolve, resolve that problem.

Speaker 2:

Well and it is amazing when you have these these bit width limitations get very baked into binaries. So assuming this GPS weak number rollover problems, right, obviously, I've not heard of it before. And you're right. There's one in in 99 2019, and then the next one is actually in 2038. So 2038 is gonna be What a year.

Speaker 2:

What a year. If you wanna be like, oh, are you

Speaker 3:

Yes. What do you plan your retirement now. Exactly.

Speaker 2:

Well, this, this is I I I felt like my master plan was I would always joke that, like, I'm gonna make my retirement in 20:30. I didn't know how. I don't know. You're gonna rewrite my interest? What are we gonna do?

Speaker 2:

We but but it may be time to start get getting going on that. We've gotta start or maybe do we start with the hysteria now, Adam? Is that what we need to do to, like

Speaker 1:

We'll certainly start registering those domain names because, like, you know, you're you're welcome to those for the next 16 years or whatever.

Speaker 7:

2038 is just not as catchy a name as YTK.

Speaker 2:

It's no. We

Speaker 1:

gotta work on the branding. That's right, Tom. Yeah.

Speaker 2:

It's it's right. There's a right. Exactly. We gotta, take something from, like, the vulnerability folks. You gotta get it's all about the branding.

Speaker 2:

I, also feel that it happens in a super weird time of, toward the 2038 rollover happens at, like, you know, 3 in the morning on January 19th or whatever. It doesn't line up at all with the calendar. But that it'll be that one will be interesting because I do feel that it's gonna be I'll be interested to see how much kind of cultural traction we have and how much people kind of model what their software is gonna go do. And the one thing is that it is it is somewhat straightforward to actually test. But, Matt, yeah, what's up?

Speaker 4:

So I don't have a war story, but, I I do, have a recommendation about a short story about the 2038, the 2038 problem. Have any of you guys read the story, epoch by Cory Doctorow? No. Oh, no. So it's, it's, the narrator.

Speaker 4:

And this this story was written right around the sun, right around the time of the Oracle acquisition of Sun. And I I think doctor o had a more optimistic outlook on that than what actually happened because in the story, the company is called Sun Oracle. And and so the narrator is a system administrator who is working at Sun Oracle around the time of the epoch, rollover. And, and and he he is responsible for, looking after this this AI that was developed by a research team a while back. And I won't spoil it, but let's just say that it involves the AI and a patch to fix all of the 32 bit systems out there.

Speaker 2:

That's pretty great. It was according to this Hacker News comment I just found. Matt, it was commissioned by Mark Shuttleworth.

Speaker 4:

That's true.

Speaker 2:

Yeah. I'm not sure if that's his who is it? Hacker News coming, but it's yeah. Wow. Interesting.

Speaker 4:

By the way, I've got your branding for you. The epocalypse.

Speaker 2:

That's pretty good. That's pretty good. Register the domain name. Yeah. Is it already registered?

Speaker 2:

That is actually a pretty good one. I mean, no. Like, only serious. Like, that is that. It's actually that's a that's a good one.

Speaker 2:

You know, I actually really wanted to read, the, what is it? Dinghy has got some, like, I would read science fiction about software, but I I'm not really into science fiction in general. So, like, I I would like science fiction about software that does not involve, like, you know, civil wars and galaxies and, you know, starship cruisers. I you know what I mean? I I beat because I wanna read what is it?

Speaker 2:

Is it a a Fire in the Deep? No. No. There's there is a a a Walter Benge book that that that that has, like, software archeology at its root that I'd be This core doctor of short story, Matt, may be right up my alley. I think that maybe if we can get to software science fiction without intergalactic warfare, I think that that's gonna be

Speaker 1:

Brian, you know, I was thinking back, to some of the other Fishworks workarounds and pedants that we were. Do you remember, we baked in some, so we had some like Kron like facilities in there for scheduling activities on the last day of the month, the second to last day of the month and the third to last day of the month. Do you remember the terms that we used for those?

Speaker 2:

Ultimate penultimate, and what was the what would be the third?

Speaker 1:

It was anti penultimate. That

Speaker 2:

is Which

Speaker 1:

which caused us to, like, resize that that dialogue box. But, like, once we had looked it up, I'm sure we could've could've see it.

Speaker 2:

And is I can't tell if your tone is boasting because you did it or accusing because I did it. I I I No. No.

Speaker 1:

I don't know who did it, but I'm sure we were all equally pleased by it.

Speaker 2:

I'm sure we're all equally pleased by it. Yes. I'm sure I I one of us did it, the other one wishes they did it. So

Speaker 1:

The the the most thing I will do is I remember in JavaScript at the time, at least I claim, that there was no facility for knowing what time zone the the browser was in. In. And so the way we determined this was by taking that that time zone database Right. Compiling it into this function that would say, okay, if the year were, you know, 2017 and we were on this island, you know, then what time would it or if you 2017 at this time, then what time would you think it was? And we would play this back and forth to do this decision tree to infer what time zone the browser was in.

Speaker 1:

Even if it was in, you know, New York Standard Time or whatever.

Speaker 2:

Oh, that's right. That I think you're right. That's why we ended up doing the because you I think you wrote that code. That's how we ended up database.

Speaker 1:

That that's right. We got into the time zone database because it was the only way we could think of to infer what the browser was in. And of course, maybe it was like some version of IE that we were working on. A a a a problem truly older than time.

Speaker 2:

And we and we were living in such sod houses. I mean, we were the the because that was, like, very early. It's 2,006, 2,007,008. I mean, it's very early for writing things on the browser. And, I mean, obviously, Google and others had kinda paved the way here to a certain degree.

Speaker 2:

But, boy, discovering just how much was missing on that. Because I was just as you were mentioning JavaScript, I was reminded of the the fact of the the fact that January is the 0th month in in JavaScript. It's like, come on. No. But that you're what are we doing here?

Speaker 2:

This is like the week start the week day the day starts at 1, if I recall correctly, speaking of time, I know we wanna, Adam, I know speaking of time, I know we wanna, I know we'd be mindful of the fact that you're on, on childcare duty at the moment.

Speaker 1:

So That's right. Such as this. Right? And and we, we were gonna mention the top.

Speaker 2:

Oh, yeah. Right. Now

Speaker 1:

that, you know, our our new year's resolution was to publish these recordings as a podcast. And here it is, only March, end of March, and we've done it. So I'll post the link in a sec. But the the first season, the 2021 season is available on podcast, and it's everywhere except for Apple because Apple makes us sit in some penalty box for a week or something before they decide we have a real podcast. But, yeah, it's all gonna be available, and we'll start to trickle out the 2022 season as well.

Speaker 2:

Yeah. And these have been it's been actually a lot of fun to go back to, some of these past recordings. It's been great to have everyone participating. So it's been fun to have that in in a a vector that, is gonna be more readily consumable. Past recordings when, when driving.

Speaker 2:

I find it actually be, so I'm I'm gonna be really excited to not have to use YouTube for that. So thank you very much for

Speaker 1:

Good. Yeah. Bright, you're you're our number one fan, and then I'm the number 2. But that's not that surprising. I agree.

Speaker 1:

That's

Speaker 2:

right. And a huge thanks to to call by Derek who's who has been, done a ton of work on on show notes, and the Adam, you've got the show notes also uploaded there as part of that in in the podcast.

Speaker 1:

Yeah. So we'll keep on doing we'll do the live shows, get YouTube up. Once we have some show notes, then it'll make it over to the podcast. So just another step of the pipeline.

Speaker 2:

And then as long as we're teasing out, upcoming events, I think we wanna, we wanna update folks on, we we had our tails from the bring up lab in December. We wanna update folks from our, bring up of the Switch, and, some of the other things we we have been been bringing up. We got I wanna get the the, w team back to do that. That was a lot of fun. We got we got a lot of got got some new war stories that we wanna make sure we tell around the campfire before we forget them, although some of them were unforgettable.

Speaker 2:

And then also wanna have, Laura back. Folks may have seen, but, Laura Abed at Oxide found another vulnerability in the LPC 50 5, that was on Hacker News last week. Terrific blog entry from Laura, but I'd love to have her back to this.

Speaker 1:

Awesome stuff.

Speaker 2:

Yeah. Really good stuff. So that that we'll get we'll get have that upcoming over the next couple of weeks. As always, we've got other things that you want us to talk about. And your name is not Sam Alban.

Speaker 2:

Please don't troll us. The but always always happy to to pick up topics. DeJourne, this is a good one, I think. This will get to a lot of a lot of good stories here. So

Speaker 3:

Thanks. It was a lot of fun.

Speaker 2:

Yeah. Thanks, everyone. Take care. Thanks, everyone. See you next time.