this post was submitted on 16 Dec 2023
7 points (88.9% liked)

Selfhosted

38707 readers
677 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
7
Proper HDD clear process? (poptalk.scrubbles.tech)
submitted 9 months ago* (last edited 9 months ago) by [email protected] to c/[email protected]
 

Usually my process is very... hammer and drill related - but I have a family member who is interested in taking my latest batch of hard drives after I upgraded.

What are the best (linux) tools for the process? I'd like to run some tests to make sure they're good first and also do a full zero out of any data. (Used to be a raid if that matters)

Edit: Thanks all, process is officially started, will probably run for quite a while. Appreciate the advice!

all 27 comments
sorted by: hot top controversial new old
[–] [email protected] 11 points 9 months ago (3 children)

Dd. It writes on disk at a block level and doesn't care if there's any kind of filesystem or raid configuration in place, it just writes zeroes (or whatever you ask it to write) to drive and that's it. Depending on how tight your tin foil hat is, you might want to write couple of runs from /dev/zero and from /dev/urandom to the disk before handing them over, but in general a single full run from /dev/zero to the device makes it pretty much impossible for any Joe Average to get anything out of it.

And if you're concerned that some three-letter agency is interested of your data you can use DBAN which does pretty much the same than dd, but automates the process and (afaik) does some extra magic to completely erase all the data, but in general if you're worried enough about that scenario then I'd suggest using an arc furnace and literally melting the drives into a exciting new alloy.

[–] [email protected] 5 points 9 months ago (1 children)

The one thing DD won't overwrite is bad sectors. If the disk has any reallocated sectors, the data in the original sectors may still be there.
If there are reallocated sectors, then the disk is reaching the end of it's life and is not worth reusing anyways.

[–] [email protected] 4 points 9 months ago (1 children)

And if you're concenred on data written on sectors since reallocated you should physically destroy the whole drive anyways. With SSDs this is even more complicated, but I like to keep it pretty simple. If the data which has been stored on the drive at any point of it's life is under any kind of NDA or other higly valuable contract it's getting physically destroyed. If the drive spent it's life storing my family photos a single run of zeroes with dd is enough.

At the end the question is that if at any point the drive held bits of anything even remotely near a cost of a new drive. If it did it's hammer time, if it didn't, most likely just wiping the partition table is enough. I've given away old drives with just 'dd if=/dev/zero of=/dev/sdx bs=100M count=1'. On any system that appears as a blank drive and while it's possible to recover the files from the drive it's good enough for the donated drives. Everything else is either drilled trough multiple times or otherwise physically destroyed.

[–] [email protected] 2 points 9 months ago

Some SSD drives can do a secure erase via block encryption where the key is stored on the drive itself. There is a command that simply generates a new key - Voilà your drive now contains random bits. I don't know if newer spinning rust drives have this feature too.

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago)

I claim my new rock band name "exciting new alloy"

[–] [email protected] -1 points 9 months ago* (last edited 9 months ago)

Just overwrite with /dev/zero and be done.What dd always has to be abused is incredible.

[–] [email protected] 5 points 9 months ago (1 children)

Wow, was your porn that much questionable?

[–] [email protected] 2 points 9 months ago

Since the disks are going to a 'family member' any porn at all, even the most tame, might get talked about.

[–] [email protected] 5 points 9 months ago

Just FYI: the often-cited NIST-800 standard no longer recommends/requires more than a single pass of a fixed pattern to clear magnetic media. See https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-88r1.pdf for the full text. In Appendix A "Guidelines for Media Sanitation" it states:

Overwrite media by using organizationally approved software and perform verification on the
overwritten data. The Clear pattern should be at least a single write pass with a fixed data value,
such as all zeros. Multiple write passes or more complex values may optionally be used.

This is the standard that pretty much birthed the "multiple passes" idea, but modern HDD technology has made that essentially unnecessary (unless you are combating nation-state-sponsored attackers, in which case you should be physically destroying anything anyway, preferably using some high-heat method).

[–] [email protected] 4 points 9 months ago* (last edited 9 months ago) (1 children)

# cat /dev/zero > /dev/your-disk

If you want progress bar, use pv instead of cat.

[–] [email protected] 3 points 9 months ago (1 children)

Or # dd if=/dev/zero of=/dev/your-disk status=progress

[–] [email protected] 4 points 9 months ago (1 children)
[–] [email protected] 1 points 9 months ago (1 children)
[–] [email protected] 0 points 9 months ago

Well, at least add blocksize.

[–] [email protected] 3 points 9 months ago (1 children)

it's not much use now, but to basically avoid the entire issue just use whole disk encryption the next time. Then it's basically pre-wiped as soon as you "lose" the encryption key. Then simply deleting the partition table will present the disk as empty and there's no chance of recovering any prior content.

[–] [email protected] 1 points 9 months ago (2 children)

Does one have to supply the password at each boot with what you are describing - this sounds like the password is somewhere in the partition table. If so what do I google to learn more?

[–] [email protected] 2 points 9 months ago* (last edited 9 months ago)

There are many ways to setups full disk encryption on Linux, but the most common all involve LUKS. Providing a password at mount (during boot, for a root partition or perhaps later for a "data" volume) is a but more secure and more frequently done, but you can also use things like smart cards (like a Yubikey) or a keyfile (basically a file as the password rather than typed in) to decrypt.

So, to actually answer your question, if you dont want to type passwords and are okay with the security implementations of storing the key with/near the system, putting a keyfile on removable storage that normally stays plugged in but can be removed to secure your disks is a common compromise. Here's an approachable article about it.

Search terms: "luks", " keyfile", "evil maid"

[–] [email protected] 1 points 9 months ago

store it in tpm

[–] [email protected] 2 points 9 months ago

Usually, I use shred:

shred -vfz -n 2 /dev/device-name
  • -v: verbose mode
  • -f: forces the write permissions if missing
  • -z: zeroes the disk in the final pass
  • -n 2: 2 passes w/ random data
[–] [email protected] 2 points 9 months ago

So I run encrypted drives except for the odd thing where it would be a pain in the anoos, but what would... drive... someone to physically destroy the drives?

Literally the only thing I can think of is having massive amounts of illegal data but what would that even be.

[–] [email protected] 0 points 9 months ago

You could also use DBAN to perform the erasure from outside the operating system.

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (2 children)

dd works fine, you'd use it something like this

dd if=/dev/zero of=/dev/[the drive] status=progress conv=fsync bs=4M

if: input file

of: output file

status=progress: shows progress

conv=fsync: basically does the equivalent of running "sync" after the command, makes sure all the kernel buffers have actually written out and are on the device. This causes the command to "hang" near the end depending on how much RAM is installed on the computer. It's not actually hanging it's just finishing writing out the data that's still cached in RAM. This can take a while depending on drive speed and quantity of system RAM.

bs=4M sets the block size to something high enough you're not CPU bottlenecked. Not particularly important exactly what the value is, 4M is a good sane default for most things including this full disk operation.

edit: one pass of zeros is enough to protect against all trivial data recovery techniques. If your threat model includes three letter agencies the hammer and drill bit technique is 👍

[–] [email protected] 0 points 9 months ago (1 children)

Thanks! I've used dd for things like recovering/cloning drives but it makes complete sense I can wipe it too. Thanks for the progress trick too, it was always just a blank cursor to me when I ran it before!

[–] [email protected] 3 points 9 months ago* (last edited 9 months ago)

I recommend using a different set of flags so you can avoid the buffering problem @[email protected] mentions.

This next example prevents all of your ram getting uselessly filled up during the wipe (which causes other programs to run slower whenever they need more mem, I notice my web browser lags as a result), allows the progress to actually be accurate (disk write speed instead of RAM write speed) and prevents the horrible hang at the end.

dd if=/dev/urandom of=/dev/somedisk status=progress oflag=sync bs=128M

"oflag" means output flag (to do with of=/dev/somedisk). "sync" means sync after every block. I've chosen 128M blocks as an arbitrary number, below a certain amount it gets slower (and potentially causes more write cycles on the individual flash cells) but 128MB should be massively more than that and perfectly safe. Bigger numbers will hog more ram to no advantage (and may return the problems we're trying to avoid).

If it's an SSD then I issue TRIM commands after this ("blkdiscard" command), this makes the drive look like zeroes without actually having to write the whole drive again with another dd command.

[–] [email protected] -1 points 9 months ago (1 children)

Why don't you just destroy them? All it takes is a screwdriver, some sandpaper, a magnet and maybe a hammer.

[–] [email protected] 1 points 9 months ago

but I have a family member who is interested in taking my latest batch of hard drives after I upgraded.