258 lines
15 KiB
Markdown
258 lines
15 KiB
Markdown
---
|
||
created_at: '2015-11-30T16:28:33.000Z'
|
||
title: dd – Destroyer of Disks (2008)
|
||
url: http://www.noah.org/wiki/Dd_-_Destroyer_of_Disks
|
||
author: opensourcedude
|
||
points: 82
|
||
story_text:
|
||
comment_text:
|
||
num_comments: 58
|
||
story_id:
|
||
story_title:
|
||
story_url:
|
||
parent_id:
|
||
created_at_i: 1448900913
|
||
_tags:
|
||
- story
|
||
- author_opensourcedude
|
||
- story_10650177
|
||
objectID: '10650177'
|
||
year: 2008
|
||
|
||
---
|
||
[Source](http://www.noah.org/wiki/Dd_-_Destroyer_of_Disks "Permalink to Dd - Destroyer of Disks - Noah.org")
|
||
|
||
# Dd - Destroyer of Disks - Noah.org
|
||
|
||
# Dd - Destroyer of Disks
|
||
|
||
From Noah.org
|
||
|
||
Jump to: [navigation][1], [search][2]
|
||
|
||
|
||
These notes cover useful things you can do with `dd`.
|
||
|
||
## Contents
|
||
|
||
* [1 Why use dd instead of cp?][3]
|
||
* [2 Burn Linux ISO images to a USB flash drive using Apple Mac OS X][4]
|
||
* [3 Securely erase a drive][5]
|
||
* [3.1 Tinfoil hat paranoia][6]
|
||
* [3.1.1 caveat on Flash memory storage][7]
|
||
* [3.2 One step disk wipe tool][8]
|
||
* [4 Erase MBR][9]
|
||
* [4.1 disk signature of boot disk][10]
|
||
* [4.2 ms-sys][11]
|
||
* [4.3 See also][12]
|
||
* [5 Erase GPT (GUID Partition Table)][13]
|
||
* [5.1 Error: **Unable to install GRUB**][14]
|
||
* [5.2 Easter Egg][15]
|
||
* [6 Fill a file with bytes][16]
|
||
* [7 Copy a drive to an image file][17]
|
||
* [7.1 restore from an image][18]
|
||
* [8 Image a CD or DVD][19]
|
||
* [8.1 Image a drive with compression][20]
|
||
* [8.2 Image a drive over a network with `dd` and `ssh` or `nc` (netcat)][21]
|
||
* [9 Show progress status statistics of `dd`][22]
|
||
|
||
## Why use dd instead of cp?
|
||
|
||
In many cases you can use **cp** where **dd** is used. What **dd** adds is filtering. It lets you set block sizes of data, you can specify how bad blocks are handled, you can limit how much data is copied. **dd** isn't much more than a fancy **cp** command.
|
||
|
||
|
||
sudo dd if=ubuntu-14.04-desktop-amd64.dmg of=/dev/sda1
|
||
sudo cp ubuntu-14.04-desktop-amd64.dmg /dev/sda1
|
||
sudo cat ubuntu-14.04-desktop-amd64.dmg >> /dev/sda1
|
||
|
||
|
||
## Burn Linux ISO images to a USB flash drive using Apple Mac OS X
|
||
|
||
This example burns an ISO image to a USB flash drive. In this example, the source image is **ubuntu-13.10-desktop-amd64.iso**. On Mac OS X the **iso** image must first be converted to a **dmg** image.
|
||
|
||
|
||
diskutil list
|
||
diskutil unmountDisk /dev/disk1
|
||
hdiutil convert -format UDRW -o ubuntu-13.10-desktop-amd64 /Users/noah/Downloads/ubuntu-13.10-desktop-amd64.iso
|
||
sudo dd if=ubuntu-13.10-desktop-amd64.dmg of=/dev/rdisk1 bs=1m
|
||
|
||
|
||
## Securely erase a drive
|
||
|
||
If you are in a hurry then just drill a hole through the top of case into the platters. A professional data recovery service might be able to get some data off the platters, but it will be very expensive to do so. I don't know of any that actually offer this service.
|
||
|
||
You can use `dd` to destroy just the data without destroying the drive.
|
||
|
||
|
||
dd if=/dev/zero of=/dev/sdXXX bs=1M
|
||
|
||
|
||
You can also use `cp` or `cat`:
|
||
|
||
|
||
cp /dev/zero /dev/sdXXX
|
||
cat /dev/zero > /dev/sdXXX
|
||
|
||
|
||
Some say you should write random data to the drive (see [Tinfoil hat paranoia][6] below), but it is nearly ten times slower to use _/dev/urandom_ than _/dev/zero_. It is practically impossible to use _/dev/random_ instead of _/dev/urandom_.
|
||
|
||
|
||
dd if=/dev/urandom of=/dev/sdXXX bs=1M
|
||
|
||
|
||
### Tinfoil hat paranoia
|
||
|
||
It takes about 15 minutes to destroy a 1GB file using GNU `shred` (default options). It takes 30 seconds to destroy the file using `dd if=/dev/zero of=somefile bs=1024 count=1M`. This is on a laptop with a 1.6 GHz dual core CPU, 2 GB RAM machine, and a Seagate Momentus ST9160823AS drive with ext3 filesystem -- in other words, nothing fancy.
|
||
|
||
Some people will tell you that simply overwriting data isn't truly secure because they heard that it's possible to read data that has been overwritten (See **data remanence**). Some believe that you must overwrite a bit multiple times to ensure that there is no way to recover the bit that had been stored there. There are official guidelines based on this belief. My belief is that this is a myth. The origin of this idea came from Dr. Peter Gutmann who speculated that overwritten data might be recovered through the use of **Scanning Transmission Electron Microscopy**. This is an interesting idea, but the key fact to point out is that this is an unsubstantiated theory -- no one has ever demonstrated recovering even a single bit of data using this technique or any other technique. No commercial forensics or data recovery firms offer any services that can recover data once it has been overwritten. Obviously the NSA is going to advertise this capability if they had it, but I believe neither they nor any advanced species of space aliens that may be visiting us have this ability. The point is that you can't hire anybody for any amount of money to recover overwritten data for you. Forget the NSA. If your data is so sensitive that you can't accept the risk that the NSA or space aliens might be able to unerase data from your drive then you don't need my advice. You might need advice from someone in a different profession... **The bottom line is that most tools that claim to "securely" erase a drive use such extreme measures that it can take hours to erase a drive.** Yet there is not a single example of anyone recovering data after it has simply been overwritten once with zeros.
|
||
|
||
If you want to erase a drive fast then use the following command (where sdXXX is the device to erase):
|
||
|
||
|
||
dd if=/dev/zero of=/dev/sdXXX bs=1048576
|
||
|
||
|
||
If you prefer to use the GNU `shred` command then you may want to put this in your ~/.bashrc or alias file to make it a little more sane:
|
||
|
||
|
||
alias shred='shred --iterations=1'
|
||
|
||
|
||
My argument also applies to Flash memory media, which, in consumer devices, is slowly replacing magnetic media. In fact, it's probably easier to decap a flash chip and read the electron potentials trapped in the floating gates. Assuming this is possible, this would still require a laboratory and lots of money. The problem becomes even harder with MLC flash memory, which is the most common.
|
||
|
||
#### caveat on Flash memory storage
|
||
|
||
Note that with Flash memory storage there is the Possibility a few random chunks of your files may be frozen in the flash device in such a way that you can't access it or delete it or even be sure if this has happened. Flash media is inherently unreliable so they all contain spare memory. As the device runs it is able to detect when parts of the memory become too unreliable to be trusted. When this happens the device will substitute some of the spare memory for the sections that are marked as bad. The data in the bad sections is copied to the spare memory. Most manufacturers doesn't mention this feature or provide a way to tell when and where this happens. The substitution is totally transparent. The bad memory doesn't go anywhere, but it's never used again. The problem is that the forgotten sections of bad memory remain on they device and they may be readable. It may be as simple as removing the flash storage chips and attaching them to a different device that can read the spare sections of memory. If the controller is built into the flash chip then it may be possible to cut off the top of the chip and read the values of the individual memory cells. ... I don't personally worry about this because while it may not be outside the realm of science fiction it is still difficult. Plus the recoverable sections of bad memory would be tiny random chunks out of the entire filesystem, though perhaps enough to cause concern for some people. ... **I'm curious if any forensic professionals have ever made use of this idea to recover any useful information.**
|
||
|
||
### One step disk wipe tool
|
||
|
||
This is useful if you want to recycle a lot of drives: [Darik's Boot and Nuke][23]
|
||
|
||
## Erase MBR
|
||
|
||
I had Linux with GRUB installed on a machine. I needed to get rid of it and put Windows on the machine. I used a Ghost recovery disk to restore Windows on it, but Ghost didn't restore the MBR. GRUB was still lurking in the Master Boot Record. On boot GRUB would try to start but would error out. Wiping out the MBR fixed the problem. This will wipe out the MBR of a disk (sdXXX in this example) but keep the partition table and disk signature:
|
||
|
||
|
||
dd if=/dev/zero of=/dev/sdXXX bs=440 count=1
|
||
|
||
|
||
If you also want to totally erase the entire MBR include disk signature and partition table then use the following command:
|
||
|
||
|
||
dd if=/dev/zero of=/dev/sdXXX bs=512 count=1
|
||
|
||
|
||
If you want to not worry about remembering the exact amount of the disk to erase and you do not care about erasing other data on the drive, then simply blast away the first megabyte or so. It doesn't matter.
|
||
|
||
|
||
dd if=/dev/zero of=/dev/sdXXX bs=1M count=10
|
||
|
||
|
||
### disk signature of boot disk
|
||
|
||
The disk signature is an obscure topic. These are the 4 bytes in the MBR starting after the first **440** bytes (offset **0x01B8** through **0x01BB**). Often you can mess with it without problems, but in certain circumstances Linux may need to see a specific disk signature on the boot disk. The most critical fact is that the disk signature of the primary BOOT disk must be unique. In days past, I did not know the significance of the disk signature so I would often zero it out along with the MBR boot code using `dd if=/dev/zero of=/dev/sdXXX bs=446 count=1`. That is not guaranteed to be harmless. It may cause problems; although, usually it is harmless. It is also bad to COPY a disk image including the MBR and then mount both copies on the same system. **The system may not boot or nothing will go wrong at all!**
|
||
|
||
**Do not confuse the disk signature with the MBR signature.** The MBR signature is always **0xAA55** starting at offset **0x01FE**. It is stored little endian, so **0x01FE:0x55** and **0x01FF:0xAA**.
|
||
|
||
### ms-sys
|
||
|
||
The `ms-sys` command may be helpful in working with the MBR and disk signatures.
|
||
|
||
### See also
|
||
|
||
EDD
|
||
: Bios Enhanced Disk Drive Services (EDD) 3.0. This protocol determines which disk BIOS tries boot from. This uses the Disk Signature bytes. These are the 4 bytes in the MBR starting after the first 440 bytes (offset 0x01B8 through 0x01BB).
|
||
|
||
## Erase GPT (GUID Partition Table)
|
||
|
||
If you see this error while using **fdisk** then you may want to remove all trace of **GPT**.
|
||
|
||
|
||
WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.
|
||
|
||
|
||
To erase the **GPT** you need to erase the table at both the beginning and end of the disk. You need to use **blockdev** to calculate the block number at the end of the drive.
|
||
|
||
|
||
dd if=/dev/zero of=/dev/sdXXX bs=4096 count=35
|
||
dd if=/dev/zero of=/dev/sdXXX bs=4096 count=35 seek=$(($(blockdev --getsz /dev/sdXXX)*512/4096 - 35))
|
||
|
||
|
||
Older:
|
||
|
||
|
||
dd if=/dev/zero of=/dev/sdXXX bs=512 count=2
|
||
dd if=/dev/zero of=/dev/sdXXX bs=512 count=2 seek=$(($(blockdev --getsz /dev/sdXXX) - 2))
|
||
|
||
|
||
### Error: **Unable to install GRUB**
|
||
|
||
While installing Ubuntu on a disk that may have been previously used you may get this error when you get to the very final end of the installation process.
|
||
|
||
|
||
[!!] Install the GRUB boot loader on a hard disk
|
||
|
||
Unable to install GRUB in /dev/sda
|
||
Executing 'grub-install /dev/sda' failed.
|
||
|
||
This is a fatal error.
|
||
|
||
<Go Back> <Continue>
|
||
|
||
|
||
The cause of this is **GPT**. You must remove the partition table before you install Linux and GRUB. Do this with an Ubuntu Desktop LiveCD (Ubuntu Server CD does not have a live option for debugging... go figure.). Remeber, the **GPT** is both at the beginning and end of the disk. You must remove both of them.
|
||
|
||
|
||
dd if=/dev/zero of=/dev/sdXXX bs=4096 count=35
|
||
dd if=/dev/zero of=/dev/sdXXX bs=4096 count=35 seek=$(($(blockdev --getsz /dev/sdXXX)*512/4096 - 35))
|
||
|
||
|
||
**Note 1**, you may have to zero out a larger range of blocks for the secondary GPT. This is because I am not certain of my math. Most modern disks use 4096 byte sectors internally, but may report 512 bytes to the OS. I'm not sure what size the drives use for the **LBA** arithmetic. I think LBA blocks are 512 bytes, but in these examples I pretend it may be 4096 bytes just to be sure. **blockdev** uses 512 bytes for the **\--getsz** option, but the **seek** option in **dd** uses 4096 byte blocks, so the results of blockdev have to be converted to a seek point using 4096 byte blocks... Note because I use 4096 bytes for LBA block sizes this may over-estimate the size of the GPT tables and so could remove more than just the GPT tables. For my purposes this is OK because I just want to ignore whatever was on the disk before and get GRUB to install properly. This is bad if you are trying to surgically remove the GPT tables while preserving all other data.
|
||
|
||
**Note 2**, If you are exploring GPT by using **dd** to dump disk information, remember to use **skip** instead of **seek**.
|
||
|
||
**Note 3**, While using a live CD to get a shell to do any of this, you may also need to remove device-mapper targets (a wild guess), **dmsetup remove vg--vmh--root**, or something like that.
|
||
|
||
### Easter Egg
|
||
|
||
While searching for strings in the **dd** dumps of the GPT table of a drive I noticed the following string, **Hah!IdontNeedEFI**. A little research shows that this is the actual official the GUID of GPT.
|
||
|
||
## Fill a file with bytes
|
||
|
||
This creates a 10MB file filled with zeros (0):
|
||
|
||
|
||
dd if=/dev/zero bs=1M count=10 of=test_data.bin
|
||
|
||
|
||
You can use /dev/zero and `tr` to generate and fill a file with any given byte constant. This creates a 10MB file filled with ones as a bit pattern (0b11111111, 0377, 255, or 0xff).
|
||
|
||
|
||
dd if=/dev/zero bs=1M count=10 | tr '
|
||
|
||
|
||
[1]: http://www.noah.org#column-one
|
||
[2]: http://www.noah.org#searchInput
|
||
[3]: http://www.noah.org#Why_use_dd_instead_of_cp.3F
|
||
[4]: http://www.noah.org#Burn_Linux_ISO_images_to_a_USB_flash_drive_using_Apple_Mac_OS_X
|
||
[5]: http://www.noah.org#Securely_erase_a_drive
|
||
[6]: http://www.noah.org#Tinfoil_hat_paranoia
|
||
[7]: http://www.noah.org#caveat_on_Flash_memory_storage
|
||
[8]: http://www.noah.org#One_step_disk_wipe_tool
|
||
[9]: http://www.noah.org#Erase_MBR
|
||
[10]: http://www.noah.org#disk_signature_of_boot_disk
|
||
[11]: http://www.noah.org#ms-sys
|
||
[12]: http://www.noah.org#See_also
|
||
[13]: http://www.noah.org#Erase_GPT_.28GUID_Partition_Table.29
|
||
[14]: http://www.noah.org#Error%3A_Unable_to_install_GRUB
|
||
[15]: http://www.noah.org#Easter_Egg
|
||
[16]: http://www.noah.org#Fill_a_file_with_bytes
|
||
[17]: http://www.noah.org#Copy_a_drive_to_an_image_file
|
||
[18]: http://www.noah.org#restore_from_an_image
|
||
[19]: http://www.noah.org#Image_a_CD_or_DVD
|
||
[20]: http://www.noah.org#Image_a_drive_with_compression
|
||
[21]: http://www.noah.org#Image_a_drive_over_a_network_with_.60dd.60_and_.60ssh.60_or_.60nc.60_.28netcat.29
|
||
[22]: http://www.noah.org#Show_progress_status_statistics_of_.60dd.60
|
||
[23]: http://dban.sourceforge.net/
|
||
|