Monday, March 12, 2012

Why format your camera's memory card?

One topic that I've seen come up periodically is how to clear out all the photos from your camera's memory card.  Some say that it's better to simply erase the photos, while others say it's better to completely reformat the card.  I'm a firm believer that unnecessarily reformatting a flash card is a bad idea.  Here's why...

The most common argument for regularly formatting your flash card is that doing so will identify any bad blocks on the device and remove them from the filesystem's pool.  Unfortunately, this is a naive idea.  This methodology may work well for spinning hard drives, but it actually does more harm than good on flash drives.  The explanation why applies to both memory cards for your camera or phone and SSD drives in your computer.

Modern CompactFlash (CF) and Secure Digital (SD, SDHC, & SDXC) cards are all formatted with the ancient DOS FAT32 filesystem, the origins of which has been in existence far longer than digital cameras have roamed the earth.  File and directory layout within this filesystem is governed by the industry standard Design Rule for Camera Filesystem (DCF), which specifies a standard naming convention for files & folders.  Sharing cards between cameras can screw up the file numbering in your camera if the card isn't empty when you start shooting, but that's the extent of the potential damage that you may incur by such promiscuity.

I have three Canon DSLR's that all share the same pool of CF cards, and the only issue I've encountered came when I put a non-empty card from one body into another, then took pictures on that card.  The second camera increased its image counter to match the last photo on the card.  As long as you erase all images on the card as soon as you insert it into the camera & before you take a shot, this renumbering isn't an issue (on Canon, anyway).  YMMV with other brands.

New flash cards all come pre-formatted with a FAT32 filesystem.  If you've just gotten a used memory card, it's probably wise to reformat it in your camera to ensure that it contains the proper filesystem.  Heaven only knows what the previous owner may have done with it.  In everyday use, if you've got a really large card, it may be faster to reformat it than to erase 32GB worth of photos.  Those are the only legitimate reasons I can think of to reformat a functioning card.  However, doing so still reduces the life expectancy of your card (keep reading).

I never reformat my cards in-camera for a few reasons.  First, formatting them on the computer allows me to set the volume name for each card so I can keep straight which removable device is which (use the "label" command on Windows or the "mlabel" command on Linux).  Second, I keep a TXT file at the top level of each card that tells my name & contact info, just in case I ever lose one.   Erasing all images won't touch this TXT file, but reformatting the card would delete it.  Third, formatting the card marginally decreases the life expectancy of the card.  Let me elaborate...

When you use a traditional hard drive with spinning platters, little bits of the drive occasionally go bad and fail to correctly store your information.  If you select the correct option when you reformat a drive (this generally doesn't happen by default), each block of space will be tested for validity.  If a block fails the test, it gets marked as "bad" and will not be used as part of the new filesystem.  It's a good system, and will prolong the useful lifespan of a drive by allowing you to continue using the good blocks.  The problem comes when people try to apply this same "mark the bad blocks" method to flash memory like SSD drives and CF/SD cards.

Controllers remap filesystem blocks into memory
Each bit of flash memory can endure a limited number of write cycles before it wears out and is no longer reliable.  Virtually nobody uses every bit of their memory cards uniformly across the entire card.  In order to maximize the lifespan of a card, manufacturers use a technique called "wear leveling" to spread the write cycles evenly across the card.  The card's memory controller presents a certain number of blocks to the filesystem, just like a hard drive does, but under the covers, the controller remaps each block to an arbitrary location in memory.  When one bit of memory has been used too heavily, the blocks will get rearranged (without the filesystem's knowledge) so that the more heavily used blocks now reside in less heavily used bits of memory.  (When I say "bit" here, I mean "a small amount," not a single binary digit.)

Flash cards also keep a cache of spare bits of memory which are hidden from the filesystem.  If the card's memory controller detects that a bit of memory is starting to throw errors, it will mark that bit as "bad" and remap the filesystem block from that bad bit to a new, good bit out of the pool of spares.  The filesystem never knows this has happened.

This has a couple implications:  First, the flash controller's low-level remapping will hide all block errors from the filesystem until there are so many errors that its pool of spare bits is exhausted.  When your memory card gets to that stage, it's on its last leg, and desperately needs to be replaced.

Second, when you do get to the point where those bad blocks are finally exposed to the filesystem, the wear leveling will still continually move data around such that a single bad bit will show up in a variety of different filesystem blocks from time to time.  Marking a block as bad at the filesystem level won't prevent that bit of memory from showing up in a your filesystem at a different spot tomorrow.  It's like a tiny game of "Whac-A-Mole."

In short, bad blocks are a non-issue for flash memory.  If you're seeing them, your card has deteriorated to the point where it's time to pitch the card and buy a new one.  Formatting the card won't help, because the filesystem has no clue where the bad bit of memory really lives.

SanDisk Tech Support confirms that unnecessary formatting is a bad idea:

"I understand that you would like to know whether formatting can harm your card; please allow us to inform you that while format helps restoring the card to the original state, performing it too often will indeed wear out your card faster. Each card has a specific number of read/write cycles before it becomes unreliable; format is considered an erase cycle and counts."

So that's my opinion on formatting memory cards.  Don't do it just for the fun of it.  Don't do it to try to string along a dying card for a few more shoots.  If you've got any questions, or if you've got a different opinion that you can back up with facts, please speak up in the comments below.

If you're curious about the taco'd SD card pictured in the title image, you can read the whole store in this post.

No comments:

Post a Comment

Please leave your comment below. Comments are moderated, so don't be alarmed if your note doesn't appear immediately. Also, please don't use my blog to advertise your own web site unless it's related to the discussion at hand.