Hello and welcome to Full Time Nix.
This is an episode about release notes of Nix 2.23.0 with Jacek Galowicz.
Hi, Jacek.
Hey, Shahar how are you?
Pretty good, pretty good.
Well, let's talk about it.
Yes, let's go.
There are two new features.
One is a new built-in.
and as all built-ins it can be found under the global scope `builtins` attrset and
it's called warn.
So it'll be `builtins.warn`.
What does it do?
Yeah, so when you use `builtins.warn`, it has a very similar semantics to
`builtins.trace`.
Basically, `builtins.trace` is a function that takes two parameters.
The first one is a string that will be printed on the user's shell.
It will not go into the program itself.
It will not change how your program behaves.
and the second parameter to it will be some kind of value that goes from
somewhere somewhere else and will just be forwarded while your message is printed on
the terminal only visible to the user.
Developers use this for deprecation warnings or just printf style debugging or
whatever else.
Generally, traces don't really do anything and it's
relatively difficult for the end user if you are triggering a warning somewhere
that is disguised as a trace kind of to find out where it comes from and we kind
of lately got this new debugging features in Nix and you can start a debugger on
traces but then the problem is if you're
.
you just want to catch the one specific trace that is a warning about something
that you're doing wrong because of deprecation or whatever.
Yeah, you get in trouble distinguishing that.
So they introduced `builtins.warn`, which is kind of a special trace, which is meant
to be only used for actual warnings, warnings that users ought to fix.
it is now possible to tell the debugger that it should only trigger not on traces
but only on warnings or even abort on warnings.
So you could either as a developer say, OK, you activate the option debugger on
warn and then your debugger will jump into the code and give you the context as soon
as it sees a warning being triggered.
You could also in your CI,
or whatever else, set your Nix builds with the option `--abort-on-warn` and then it
will go red in front of your face when your code triggers a warning.
So this is very nice because all the traces could still be in there.
They wouldn't produce any noise because you can already distinguish between
tracing and warning, right?
Also, the warning messages have a little bit of a different
route of character.
it's easier to grab for warnings or whatever because of their fixed format.
So this is in general useful for the debugging mix developer and also for
people who simply want to get rid of all warnings for example after an update or
similar.
That is great.
Sorry, what is output character?
Mean.
I mean, not output character as a character, but the general character of
the outputs being printed.
Before that, you had `builtins.trace` with maybe a warning prefix, maybe not.
So there was no real guarantee that all the warning output that comes from faces
being used would look similar.
There was no guarantee, right?
Now...
with `builtins.warn` being a built-in primitive, they will have exactly the same
output formatting.
So you will have a nicer time simply distinguishing that.
from.
This feature certainly makes it even easier because you don't get false
positives.
Everything that is not a warning is not being triggered any longer as long as
developers use this warn feature.
I asked for this on this course.
nice.
Anyway, thank you, who did this?
Robert H.
Hensing.
Thank you Robert.
new command nix env shell as the new nix shell so what is a nix space shell what is
that the existing command
generally want to start a new shell with specific packages installed, you could
write like, Nix space, shell space, and then nixpkgs, hashtag, some package name.
Then you would be launched into a new shell that seems like this package is
installed there.
And when you exit that shell, it's gone again.
It's one of the very nice killer features of Nix, right?
I mean, people who don't use Nix, they do Docker run something, something, and then
they simply start Docker images where one or multiple tools are installed in.
If you want to customize that, you have to write your own Docker image.
And that is not really composable.
You cannot combine two Docker images easily that have all the tools that you
like.
With the Nix shell, this is different.
You simply line up all the...
packages that you would like to have in your shell.
Launch the command.
Boom, you're in a shell now that has all these packages installed, right?
Also, not to underestimate that or not to forget, Nix.
I mean, you could do this from any Linux distro, right?
Many people are on Fedora or Ubuntu or whatever.
Maybe they don't have this package, but Nix gives you access to the biggest and
freshest open source package repository in the world, right?
This is two features at the same time.
One is these ephemeral shells.
You go into this shell, have this package, you exit the shell and it's gone again,
right?
At least it looks like this.
And access to this huge package collection, right?
So that is what nix shell generally does.
And it's not only for ad hoc invocations,
Right?
Generally, when we look at the old commands, nix-shell, not nix -space
-shell, but nix-shell, it was kind of meant to be used in two different
scenarios.
One is with the list of packages on the command line, so you get an ad hoc shell
that you just define on the command line.
The other was to source a shell.nix file or similar.
where someone already prepared all the projects packages that you want to work
on, then you wouldn't have any other command -line parameters.
With Flakes, this developed a little bit in a different direction.
So we have two different commands for doing these two things.
One is for an ad hoc shell, where you simply line up all the package names on
the command line, you use `nix shell` for that.
If you want to start a development environment from a Flake project, you
would go with the next develop command.
So, Nix Develop will look into the Dev Shells section of your Flake and without
any parameters, select the default one.
Otherwise, you could also select a specific one on the command line so that
you can have different developer environments per project if that is
useful.
And then it would simply launch that.
And that's two separate commands now, right?
Correct, I wonder why it was renamed, but I also, right, but there is a Nix, so
there's the old Nix hyphen shell command, which does not work with Flakes, and it
works with ad hoc invocations, and otherwise it looks for, by default, a
shell.nix, but it can be arbitrary file.
And with Flakes or,
Flakes and nix-command experimental features.
There is Nix space shell, which is only equivalent to the Nix hyphen shell ad hoc
invocation.
Accepted.
The equivalent to the shell.nix usage of the classic nix-shell is nix develop.
Yeah, so when you look at how `nix-shell`'s feature with ad -hoc shells
was implemented, I mean, you use the nix-shell command and then dash p and then
list of packages.
It would kind of take all your package names, put them into a bigger Nix
expression and launch that as it's kind of ad -hoc Nix shell or shell.nix.
So it would kind of create one on the fly and then simulate this behavior.
The problem was there that it selects the nixpkgs for you where the packages come
from.
You could not really easily change that without tampering with environment
variables.
And the new Nix shell, Nix space shell, it waits for Flake input and then the package
input from the Flake.
You write Nix shell.
nixpkgs, hashtag, some package name.
And then this command line defines not only the package that you want, but also
the policy decision, where should this package come from?
So you could mix up different nixpkgs commits on the command line.
You want package A in very new and package B in very old.
Or you could simply access completely
different third -party flakes.
So you have some hobby repository somewhere and tell someone, hey, you can
simply run my package from the command line without cloning my flake.
So you would write nix shell github colon shahar slash saharsproject hashtag
shahar's package name.
And boom, that's it, right?
So the old nix shell command couldn't do this.
Or not without a lot of hassle.
It was trying to do this wouldn't be a nice user experience with the old command.
And with the new command, it's really easy, right?
So it's more powerful, actually, much more powerful.
Yes.
And what is this `nix env shell`?
Is it supposed to replace nix space shell?
So it's nix space env space shell.
Exactly.
So if you look at this alone in isolation, it looks like a completely random rename,
but that's not what it is.
There's a concept behind that.
And the whole idea is that we have so many different Nix commands, right?
Nix shell this, sorry, Nix space this, Nix space that.
The developers tried to make it more intuitive by...
giving these subcommands a common scheme where you know, okay, I want to do
something with development environments.
So there are multiple Nix, space, dev, space, something commands, or I want to do
something with package installations or building or running.
Then there's a Nix space, package space, something, something subcategory.
So they are kind of subdividing the commands which kind of randomly piled up
over the years.
into new categories that make more sense because they're more coherent.
So for example, there will be `nix dev shell` and `nix dev run` and `nix dev
print-env`.
So these things all have to do something with development shells that are from a
flake.
But you can also run `nix env shell` in contrast to `nix dev shell`.
That's better
shell` will be kind of ad hoc shells.
Everything with `nix env` will be something like an ad hoc thingy.
Everything with `nix dev` will be something with prepared development
environments from Flakes, right?
So this totally makes sense.
And the other commands will be like NixPackageBuild, NixPackageRun,
NixPackageInfo, NixPackageEdit.
And that will be something that will be like per package do something with one
specific package.
And yeah, so they're kind of there's a lot of discussion going on.
So I went through all the GitHub issues where they were discussing how stuff
should be named.
Of course, naming is one of the hard problems in IT.
People put like five people in a room to discuss how something should be named and
you get 10 different opinions out of it.
So I think we will.
have more discussion about that in the GitHub issues and forums, how this should
be.
Some people will like it, some people won't, but in general, I'm looking forward
to this improved structure, right?
Yes, it sounds right.
It sounds right to me and I also trust that the intelligent beings who tend to
this project are making reasonable decisions.
Alright, so that's a small and yet part of a greater improvement change.
absolutely.
So one other note.
They will not remove the `nix shell` command because of `nix env shell`.
So both will coexist for a while, I guess.
So this is not a breaking change where everyone who updates to Nix 23.0 will have
to rewrite all their commands.
That is not the case.
They are simply added at this point.
and I hope that it will eventually be removed because this is the time to do
this all of these are supposed to be experimental it's just that we're we took
our time
Yeah, of course.
But I mean, commands like nix develop, nix shell, nix build with the spaces in
between and not the dashes.
Yeah, so many people use it already.
And of course, this is officially experimental, but it's experimental for so
long already that people ignore that.
So our experimental semantics are not really not equally useful everywhere right
now.
Mm -hmm.
It totally makes sense that they are not deprecating the nix shell, nix develop,
nix build and nix run commands too quickly.
But eventually you are right, eventually the whole interface will look chaotic and
noisy if you keep all of this, right?
Yeah, I'll leave it up to the Nix team.
Absolutely.
Yeah, they will figure it out and do it well.
So there are some more improvements.
One is to the consistency.
It's about making a nix build with the keep going flag consistent with the
nix-build with the keep going flag.
How is it not consistent?
So first, the use case for the keep going flag is usually when you're building
something on your laptop and that rebuilds many, many different derivations at the
same time.
For example, if you updated your inputs and then rebuilt, of course, everything is
going to be rebuilt usually.
And then without the keep going flag, the whole build will simply abort on the first
error.
Mm -hmm.
And maybe there are 10 errors and you want different packages and you want to see
them all.
Typically what's also the case when you are, let's say you have like two packages
building at the same time and one builds for an hour and will be completely fine.
And the other one builds for like 10 minutes until it breaks.
And without keep going the
whole build will break after 10 minutes after the first error, while the other
package, which you could simply let continue, is aborted too.
So on the next nix build, the longer package build will be completely
restarted, and that might not be necessary.
So if you run Nix build with the keep going parameter, it will simply print all
the errors that it finds on the way, so you can on your other screen start to
fix problems and bugs and at the same time all the builds that are completely fine
will continue to build, right?
So you're kind of not wasting time.
While you're waiting anyway, you can fix the bugs of the builds that don't work,
right?
So that's very useful of a feature.
Typically, I don't enable it in CIs because I want CIs to get red pretty
quickly and then abort so they have the resources for other builds that are not
going red.
while at home while debugging stuff, I'm using the keep going flag to just keep the
builds that work building while I fix the problems, right?
But now it was like this feature existed in old times already with the old
`nix-build` command and with the new `nix build` command, they still have it, but
they are slightly, slightly different.
So for example, if you have like a...
output derivations that fail to build.
Nix is basically comparing hashes and hash mismatches are displayed in terms of the
error message.
And maybe you have like three, four, five different hash mismatches that you want to
fix quickly.
But then you can't because it's already aborting after the first one.
So the...
old `nix-build` command showed you all of them, the new one didn't and now this is
fixed so this will be as useful as the old one.
Sorry, so what was the difference?
There was a difference in what it keeps going and what it fails on with regards to
this particular hash mismatch?
In case of hash mismatches, there were cases where not all of them are displayed
that were found.
It's only with regards to what is displayed, you say.
Yeah, kind of.
In general, the whole idea of keep going is simply print all the errors that you
see, but also keep going with everything else.
And this promise of really keep going during a build was, so to say, not
completely followed by the new nix build command.
So sometimes it was the case that you run nix space build keep going, then it
displays you some kind of hash mismatch.
And then, okay, then you fix it.
Then you run it again, and then it will come up with a new hash mismatch that
could have been shown on the first run already, but didn't.
Of course, that's irritating, that's stupid, but now they fixed that.
so some things were...
triggering a complete failure short -circuiting everything in spite of the
flag keep going and now it's fixed now they will not cause the whole build to
fail it will keep going
say keep going now, like really keep going.
The promise wasn't met completely right now it is.
And it's kind of, it's not forgetting to print all of the errors that I've seen.
There were simply corner cases in which this was the case and some cases in which
it wasn't.
I didn't really experience this myself, so I'm just reading from the issue
description on GitHub and the pull requests what was the case.
And it really seems like there were simply a few special cases, like with fixed
output derivations, where it didn't keep up the promise.
It's good to know.
I mean, if you don't know, then you can already forget.
Because it's now in parity now, right?
I just would like to thank the author of this fix.
And so whoever you are, thank you.
Okay, by the way with this feature flag keep going, this keep going flag, errors
are being printed, there is no summary, right?
It's not like you get a summary of errors at the end, right?
Yeah, that's true.
I mean you get a summary of all the, I mean a list of packages that failed
obviously.
And for each of them you can run the show log command and then it will give you the
whole log of this specific package built in isolation, which is very useful because
typically they are just mixed on the terminal, right?
But if you really want to understand a more complex error
then you want to see the package build log in isolation which you always could.
It's just another command.
okay great JSON format update modify `nix derivation add` and `show` `nix derivation
add`, `nix derivation show` JSON format.
What is `nix derivation add` and `nix derivation show`?
So, `nix derivation show` is a command which allows you to get a more human
readable form of a Nix Derivation, right?
When you evaluate a Nix expression, it will kind of, yeah, interpret your Nix
script and create a data structure.
This data structure is called a Derivation.
And when you build a package, this Derivation is actually
the thing that is being built, right?
Your Nix expression is full of variables and whatever, you can update the nixpkgs
inputs and whatnot, but the derivation is completely fixed, completely static,
contains no variables any longer whatsoever.
All the Nix paths are set there.
And this is the format that kind of doesn't know about the Nix language any
longer.
It's just the pure static data structure.
And this data structure can be sent to other remote builders, for example, and
they will be able to build stuff.
So you could, in theory, even create different programming, use different
programming languages to create a derivation structure, right?
You wouldn't completely have to use Nix.
So this is a useful abstraction layer.
Now the question is in what format are these derivations defined?
How to read them, how to create them and so on, right?
And below the code, when you look at the derivation file, it is implemented in so
-called ATerm format.
ATerm is a very old serialization format.
It is halfway human readable.
So it's not really binary, it's ASCII.
But nowadays people use JSON to describe all kinds of data structures because
that's a very nice language agnostic thing.
All the programming languages have JSON serialization and deserialization
libraries and so on.
So the `nix derivation show` command gives you a very nice JSON object from a
derivation.
The thing is that this
feature has been used in the past by humans to simply read format and search
derivation structures.
So it was fine if this was not completely valid JSON.
But they are going to change this.
So they want to have it like real JSON by following the spec.
And in this case, what they did is they revised the output
to better conform their own JSON guidelines.
For example, when you have content address derivations, the content address
derivations specify the output hash that comes there and the used hash algorithm
and also the kind of serialization that they did.
So that's very much content addressable.
storage details here.
The thing is that they had like their own format of describing a content addressable
derivation and they're kind of pulling this apart to be more composed.
In the past they put multiple different kinds of information into the same string
and separated them by columns and now they are trying to make this individual JSON
fields.
So it becomes easier to programmatically
do something with the derivation files.
This is a pretty much deep below the hood change.
So normal users who don't really play around with derivation files will not
notice this.
But it will enable tooling that reads derivation files to do something else with
that to have easier implementations because of this improved format, right?
Sounds like it.
So thank you.
John Ericson.
for this change.
Warnings, warn on unknown settings anywhere in the command line.
Right, so this is extremely useful.
With Flakes this is not really relevant because Flakes don't really take command
line like options on the command line.
Flakes are so hermetic that you cannot really change the output by something on
the command line.
But this is relevant for the old non -flakes commands.
So for example when you run nix-build or nix-instantiate on something
on a Nix expression that takes a parameter on the top level, you could provide this
parameter on the command line, like a flag, a string, or whatever.
And you could simply run nix-build and then your Nix expression file, and then
`--option`, and parameter name and parameter value, right?
And it would simply inject that into your Nix script.
and then evaluate it and that's a very useful feature in general that kind of got
lost with Flakes.
I hope Flakes will have this in the future too.
However, if your option has a typo in it, Nix would simply do nothing with that,
right?
So it's like with JavaScript or similar stuff when you have a typo in a variable
name, it will simply invent a new one.
and you have like undefined values and so on.
And with this change, Nix actually warns you.
So it sees, sorry, I only have parameters A and B and C.
And the one that you provided is unknown to this set.
So this is an error now.
And this way you will never ever struggle any longer with expressions that for some
magic reasons don't work while you for hours just had a typo somewhere and didn't
notice it.
nice i guess Cole had such an incident enough something to make this PR so thank
you Cole Helbling for fixing this for us.
Hehehehe
What does that mean?
Right, so for example in flakes, flakes have a very specific format, right?
You have output categories like packages, devShells, nixosConfigurations,
nixosModules, overlays, blah blah blah.
That list will get longer over time, I guess.
However, for example, if you run...
yeah, so schema will make them even user -definable.
So the list will grow longer.
However, for example, let's stay on the basic case when you run nix build
myPackageName.
And your package is on the nix level, not really a derivation.
Then, yeah, it won't work.
Nix will tell you, sorry, this is not a derivation.
And if you ask, OK, what is it else?
Because you're
Of course, you didn't do this on purpose.
It's kind of a mistake, right?
Then it wasn't really easy to find out what it was.
Maybe you are like pointing at an attribute set and forgot to add the sub
attribute or something like this.
Then nix won't really tell you.
Even if you open your flake in the REPL just to browse on the command line to your
package path, then it will tell you, sorry, this is not a derivation.
And you're like, okay.
How the hell am I supposed to debug this now?
Well, you have to put it outside of the packages into some undefined, unspecified,
and then it'll tell you.
Of course, that's an idea.
However, even without doing that, the flake command will now print you that this
is not a derivation, but it will also print you what it found otherwise.
So you will see, this is not a path, but for example, a set or a list or whatever.
And then you will most probably recognize what you did wrong when you see this
output.
So this is another improvement that will make the next programmer's life easier.
Mm -hmm.
That's very nice.
So thank you to whomever fixed this quality of life improvement.
Thank you.
fetchTree now fetches git repositories shallowly by default.
I'm not familiar with fetchTree.
What is fetchTree?
Yeah, so fetchTree basically is able to clone Git repositories.
The thing is that Git repositories can be regarded as a tree of Git repositories
again, but then also as a tree of commits.
So this deep or shallow cloning of Git repositories can have two different
meanings on two different axes.
Mm -hmm.
is, if you check out a Git repository that, for example, has sub -modules, then
you maybe want to check out the sub -modules too.
So this is a yes -no decision.
And that's useful in different scenarios.
The other thing is, when you check out a Git repository, you may want to check it
out with all its branches and all its history, which is typically the case when
you Git clone something, like without Nix even.
If you just use Git.
And do `git clone `, it will download the whole history.
For example, on nixpkgs, this takes a very long time because it's such a huge
repository.
Same with the Linux kernel, for example.
And the thing is, you can tell Nix to please check out all the sub -modules, yes
or no.
But you weren't really able to tell Nix or...
it was was wasn't told by default to check out the whole history yes or not it was
checking out the whole history so typically all the nix nixpkgs and so on
which reference some github repository they wouldn't even use the git protocol
but download via HTTPS
And GitHub has this nice feature that when you provide a certain URL to a repository
and a commit, it will give you a tarball which doesn't even contain all the Git
information any longer.
So they basically used this GitHub feature as an easy way to have a shallow clone of
just the latest commit of a Git repository.
And the whole idea of
this change in fetchTree is now that you basically get the same very quick checkout
of just the latest commit with Git by default, which will accelerate many
derivations that already reference Git repositories because all the history that
happened before the latest commit is most of the time not relevant to the build,
right?
That's correct.
for this podcast, FullTimeNix for a different format episode.
In fact, it's very similar.
It's news, but it's news for nixpkgs instead of Nix.
We have been working on a program that will generate the script, or at least a
list of packages that are new since last week in nixpkgs and their descriptions and
their maintainers.
And for that, we want indeed to, well, we want to test the, both in testing and in
implementation, we want to clone nixpkgs.
and we don't want to clone the entire history.
But also we don't want only a depth of one latest commit.
We want a depth that is possibly by date because we know the date, but that would
have to kind of be resolved.
You just can't know that.
It's not exposed.
You have to get the commits and then examine them.
But details aside, we would need an arbitrary
depth.
So it's just that this little change in the default can have a huge impact on the
checkout performance of your repositories or your Nix builds.
But it's actually not really a revolution going on.
Yes.
And my point is only adjacent and not really related, but I thought it would be
interesting.
So thank you.
David Hauer
David Hauer for optimizing this for us.
should save AWS a lot of money.
Mm -hmm.
Store Object Info JSON Format now uses null rather than omitting fields.
Yeah, so this is a general thing in JSON parsing or deserializing JSON into your
data structures.
Generally, when you have some kind of optional field that is set, yes or no, for
example, I don't know, some extra information in your JSON object, there are
two ways to provide this when there is no information.
So if you have some field that is called, for example, extraField,
and it contains a string.
You can either, when you don't have this information and you create a JSON object,
say extraField, colon, null.
Then you say, okay, this field has no content.
Or you simply don't provide the extra fields.
So that's from, if your JSON structure generally has like 10 fields, you leave
one of them empty, then you have just nine fields and you don't provide this field at
all.
This is an annoyance for the programmer who is going to deserialize your JSON.
Because the question is, when this extra field is not there, is this an error
because someone provided a different kind of JSON object?
Or would you just say, okay, because this field is optional, we ignore the missing
existence of this field?
And the next team decided that wherever they create JSON objects, every field that
has no value will exist as a field and its value will be null.
This is not a huge change, but it will make all the code that will later
deserialize JSON again simpler to deal with because you can assume that all the
fields will be there.
It's just that their value will be distinguished if it's usefully set or not.
So this is something that the users will not really find relevant most of the time,
but everyone who is programmatically involved with dealing with JSON objects
from Nix tooling, they will experience a quality of life improvement.
This PR is by John Ericson.
Thank you, John Ericson
And the last one is large path warnings.
Introduce warnings for large paths.
What are large paths?
Yeah, so generally this feature is also useful, especially for beginners.
For example, something that I experience a lot when I'm helping organizations to
adopt Nix is that many people will try to Nixify huge projects and then many
projects are, I don't know, I mean, one gigabyte of source code is something that
happens in real life.
And then, of course, in 1 gigabyte of source code, you will have multiple
packages that simply live in the same repository.
And then when you want to build these packages, what happens very often is
people use the src attribute and make derivation and just put in huge folders
that contain much, much more than the actual package source code.
Imagine you have a Nix package whose source attribute points to 1GB source
folder and you change the readme for example.
Then you have one gigabyte of changed source folder.
And if you rebuild this Nix derivation, it will copy the whole one gigabyte of source
code into your Nix store again and rebuild it all due to relevant or irrelevant
changes.
So this is generally a code smell in projects where the answer is better
project structure with subfolders and or source filtering.
But generally, what you see as a user is your builds are extremely slow.
And then the question is, why are they so slow?
Even without knowing the project structure, you want to know why they are
so slow.
And what you can do now is there's a new option.
It's called `--warn-large-path-threshold`.
And it takes one parameter, the size of the...
megabytes to warn after and if you say for example warn large path threshold 100m, it
Nix will warn you whenever it copies more than 100 megabytes of folder content into
the Nix store and then you can see I didn't intend to copy so much.
Thank you for this warning line right now and then you will see which derivation
copies so much stuff into the store.
So this is
Imagine for example you have a repository with, I don't know, for example a video
folder with 10 gigabytes of videos and somehow you are accidentally copying this
all to the store, right?
There was...
You could look into your nix store paths if they contain stuff that you wouldn't
suspect to be useful but that would be very laborious and with this new warning
it will become very easy to check if you are accidentally copying too much stuff,
right?
Yes, is there a default for this or is it off?
It's off by default.
Hmm.
I mean, if you really intend to copy so much stuff into the nix store, then it's
perfectly fine to do that, right?
Rarely.
Rarely is.
but legitimate and it's completely thinkable that this is legitimate in
specific cases, right?
Most of the time, of course, it's not and now it's I mean, it's easy to simply add
this warning flag, get a few warnings printed and then you tell people like look
We are not filtering in this package now is copying 500 megabytes of source code
into the nix store and after 10 rebuilds with
little irrelevant changes you have five gigabytes of wasted space on your disk and
that's unnecessary let's fix that right so it's easier to find out which is the
package that actually does that
I kind of am like fantasizing at the moment that Nix would...
read somehow read all the files
and try to somehow intelligently figure out that they're all source code or maybe
otherwise tiny.
To help with preventing this, I know that we're supposed to be engineers and we're
supposed to be careful about this.
But...
Sometimes you don't even know what exactly you have in your tree because projects are
so big.
Mm.
Maybe there are assets hidden where you didn't know that they are.
Maybe Nix can help you and say, you know what?
I noticed you have videos in your source.
and warn you for that.
So that would be a warning by file type instead.
That would be most probably very useful to many, but I'm not sure if this is the
right layer to do this.
I mean, if you are not filtering your source code at all, Nix will simply take
it all and use it.
And if you would add intelligence to Nix, that actually looks through the whole
folder to inspect content of each file.
This could be extremely slow then, because this would be a very file system
intensive.
workload.
Of course, you could have extra tooling that simply evaluates all your Nix
expressions and then goes through the list of derivations which lists up all the
paths that they're going to use.
And then it could kind of scan these paths and see what they contain and then warn
you in specific cases.
But I guess even this
I mean, you would need some kind of heuristic, right?
Because if you are not the build system which actually uses this, it will be hard
to distinguish what will be useful and what not.
Of course, the build system could tell you, hey, look, I used the following 100
files and the rest of the 1 ,000 files in this repository hasn't been touched by
anyone.
But that would also not be trivial.
You would have to kind of trace every processes.
file system operations to see what's actually used.
So something like this exists, but this is very invasive tooling.
I think it's a...
Okay.
track, read the whole tree after the build is done and say, well, because file
systems can keep track of what file has been read, when's the last time it's been
read.
I see.
Yeah.
Okay.
True.
However, just because something hasn't really been used already, the question is,
if you change some kind of build flag, will the file then be used or not?
So this is something that is, they're like, from the outside domain, what the
build system does with which file could look like completely random.
And then the question is, if the tooling has too many false positives, then it will
be annoying to use, right?
And then all the nice ideas and intentions don't lead to a nice user experience,
while maybe potentially making all the builds slower, right?
So that's not too easy to solve.
Yes, so this PR adds a simple tool that we can explicitly choose to use and this is
by Elco Dolstra.
Thank you, Elco.
Yeah, this is nice and easy to use, right?
You can say, OK, 6 megabytes are too much, 600 megabytes are too much, 6 gigabytes
are too much.
For example, when you work with VirtualBox, VirtualBox has a huge source
code folder.
And the build system doesn't make it easy to just build portions of that.
So you typically end up building the whole thing.
And it simply has huge source closures that you're going to live with.
With other projects, you could define completely different thresholds.
And you as the developer typically know what threshold you want to use on which
project.
And that's simply easy to do.
So they are only providing the mechanics of warning after a specific threshold.
But this policy has to be defined by you ad hoc on each project, which is
completely fine, right?
Yeah, well, Nix is very useful and there are always ways to make it more useful and
more pleasant to use.
Right.
That's all, isn't it?
Yes, that's all.
So I'd like to thank all the Nix developers and you, Jacek, for sharing
these updates and explaining them to us.
And I'm looking forward to the next Nix release.
Yes, thank you for having me.
It was a pleasure again.
Be well, Jacek.
Everyone, thank you for listening.
Have a nice day.