I basically never used swap until I used some script that was supposed to de-duplicate text messages from SMS backups.
Turns out feeding it a 100,000 message archive spanning back 15 years will somehow eat up ~80GB of RAM. I doubt it should, I guess it probably just wasn’t optimized for such a large amount.
I basically never used swap until I used some script that was supposed to de-duplicate text messages from SMS backups.
Turns out feeding it a 100,000 message archive spanning back 15 years will somehow eat up ~80GB of RAM. I doubt it should, I guess it probably just wasn’t optimized for such a large amount.
So I made a 64GB swap file and it completed fine.
Not everything should work from ram. This is why we stream files for processing.
The backup itself was only ~8gb.