Atemu

Have you checked for open ports?

There’s a non-zero chance that there’s a long out of date apache or something running on it.

Unallocated means free to use for any purpose.

Show the full output and tell us how much disk usage you can explain.

You need to point it at your btrfs, so i.e. /.

Like with image handling or whatever.

I’d expect TOR browser to mitigate this. Canvas2D is disabled for instance and system fonts aren’t exposed.

Most other things could be mitigated by making every platform use the same code paths for e.g. font rendering. It should be pretty damn hard to determine which OS it is when the userspace is the same. I don’t know whether TOR browser currently does this though.

Please show sudo btrfs filesystem usage.

https://github.com/CyberShadow/btdu?tab=readme-ov-file#installation

the 1TB drive just magically lost 300+GB of capacity that shows up in use but there is nothing using it

How did you verify that “nothing” is using it? That’s not a trivial task with btrfs because any given btrfs filesystem can contain an arbitrary amount of filesystem roots and that filesystem roots can be duplicated in seconds.

If you have ever done a snapshot or enabled automatic snapshots via e.g. snapper or btrbk, data that you have since deleted may still be present in a snapshot. Use btrfs subvolume list / to list all subvolumes and snapshots.

If you ever feel lost in analysing btrfs data usage, you can use btdu to visually explore where data is located. Note that it never shows 100% accurate usage as it’s based on probabilistic sampling. It’s usually accurate enough to figure out what’s using your data after a little while though, so let it scan for a good minute.

It’s the other way around: The memory failure causes the corruption.

Btrfs is merely able to detect it while i.e. extfs is not.

It only works if the hardware doesn’t lie about write barriers. If it says it’s written some sectors, btrfs assumes that reading any of those sectors will return the written data rather than the data that was there before. What’s important here isn’t that the data will forever stay in-tact but ordering. Once a metadata generation has been written to disk, btrfs waits on the write barrier and only updates the superblock (the final metadata “root”) afterwards.

If the system loses power while the metadata generation is being written, all is well because the superblock still points at the old generation as the write barrier hasn’t passed yet. On the next boot, btrfs will simply continue with the previous generation referenced in the superblock which is fully committed.
If the hardware lied about the write barrier before the superblock update though (i.e. for performance reasons) and has only written e.g. half of the sectors containing the metadata generation but did write the superblock, that would be an inconsistent state which btrfs cannot trivially recover from.

If that promise is broken, there’s nothing btrfs (or ZFS for that matter) can do. Software cannot reliably protect against this failure mode.
You could mitigate it by waiting some amount of time which would reduce (but not eliminate) the risk of the data before the barrier not being written yet but that would also make every commit take that much longer which would kill performance.

It can reliably protect against power loss (bugs not withstanding) but only if the hardware doesn’t lie about some basic guarantees.

I am well aware of how CoW works. What I wrote does not stand in conflict with it.

Perhaps I wasn’t clear enough in what I said though:

Each metadata operation (“commit” I think it’s called) has a generation number; it first builds this generation (efficiently in a non-damaging way via CoW) and then atomically switches to it. The next generation is built with an incremented generation number and atomically switched again.
That’s my understanding of how btrfs generally operates.

When things go awry, some sector that holds some of the newest generation may be corrupt but it might be that a relatively recent generation does not contain this data and is therefore unaffected.

What I’m suggesting is that you should be able to roll back to such a generation at the cost of the changes which happened in between in order to restore a usable filesystem. For this to be feasible, btrfs would need to take greater care not to overwrite recent generation data though which is what I meant by making it “more log-structured”.

I don’t know whether any of this is realistically doable though; my knowledge of btrfs isn’t enough to ascertain this.

The complex part of Guix comes from it being a inmutable distribution based on the ideas from NixOS

That’s not the most significant factor in what makes it hard/different. There are immutable distros that come with less complexity and are arguably more immutable than NixOS or guix.

What actually sets it apart and can make these harder to use is:

1. They’re declarative rather than imperative. You describe the desired end-result rather than providing (or manually executing) the steps to construct it.
2. There is not a single global dependency dependency state upon parts of which any given package depends. Dependencies are explicit, direct and encapsulated on a level that’s as fine as you’d like instead (down to the per-file level).

could be due to some bad block or read failure from the hdd (it is supposedly brand new, but i guess it could be busted)

I’d suspect the controller or cable first.

shit like this never happens to me on extfs, even if the hdd is literally dying

You say that as if it’s a good thing. If you HDD is “literally dying”, you want the filesystem to fail safe to make you (and applications) aware and not continue as if nothing happened. extfs doesn’t fail here because it cannot even detect that something is wrong.

btrfs has its own share of bugs but, in theory, this is actually a feature.

i have an ext4 and an ufs partition in the same hdd without any issues.

Not any issue that you know of. For all extfs (and, by extension, you) knows, the disk/cable/controller/whatever could have mangled your most precious files and it would be none the wiser; happily passing mangled data to applications.

You have backups of course (right?), so that’s not an issue you might say but if the filesystem isn’t integer, that can permeate to your backups because the backup tool reading those files is none the wiser too; it relies on the filesystem to return the correct data. If you don’t manually verify each and every file on a higher level (e.g. manual inspection or hashing) and prune old backups, this has potential for actual data loss.

If your hardware isn’t handling the storage of data as it should, you want to know.

even if we suppose this is the hardware’s fault and not btrfs’s, should a file system be a little bit more resilient than that? at this rate, i feel like a cosmic ray could set off a btrfs corruption.

While the behaviour upon encountering an issue is in theory correct, btrfs is quite fragile. Hardware issues shouldn’t happen but when they happen, you’re quite doomed because btrfs doesn’t have the option to continue despite the integrity of a part of it being compromised.
btrfs-restore disables btrfs’ integrity; emulating extfs’s failure mode but it’s only for extracting files from the raw disks, not for continuing to use it as a filesystem.

I don’t know enough about btrfs to know whether this is feasible but perhaps it could be made a bit more log-structured such that old data is overwritten first which would allow you to simply roll back the filesystem state to a wide range of previous generations, of which some are hopefully not corrupted. You’d then discard the newer generations which would allow you to keep using the filesystem.
You’d risk losing data that was written since that generation of course but that’s often a much lesser evil. This isn’t applicable to all kinds of corruption because older generations can become corrupted retroactively of course but at least a good amount of them I suspect.

Linus didn’t only call out people posting flame replies, but also folks interested in a serious discussion on that topic, who also contributed to the kernel before (see PeterCxy’s blog: https://typeblog.net/55833/getting-called-paid-actor-by-linus-torvalds).

Here’s what he actually said:

I’m also not going to start discussing legal issues with random internet people who I seriously suspect are paid actors and/or have been riled up by them.

That’s ambiguous at best.

I find it more likely that it’s targetted at the greater crowd replying to that thread rather than this person specifically due to it’s general tone. If Linus thinks you are full of shit, you can be that he will call you out specifically rather than wording it like this.

AFAIK we still don’t know the exact reasons for the removal, which is just intransparent.

We do: https://www.phoronix.com/news/Linux-Compliance-Requirements.

where he called several kernel developers paid actors

He didn’t. He (rightly) called out people who have never contributed to the kernel and posted flame replies to the initial announcement on the mailing list.

This isn’t some wild conspiracy theory either, this is a thing Russia actually does: cause disarray and split “western” communities; divide and conquer. It’s an their explicit goal of the Kremlin to do this; there’s credible accounts of that.

If you want a community that’s resistant to such influences, you ought to call this sort of thing out whenever you see it.

AND have backups.

That isn’t going to help them one bit if they have a SteamVR HMD.

Ahhhhh whyyyyy, you’ve got all of these standard response codes made for you, why would you blatantly ignore them like that?!