File Recovery Postscript – Do You Know Where Your Files Are?

Update: Results of our trip to Geek Squad for recovery help are in.

I thought I wrapped up the saga of Mary’s hard drive failure and the recovery of critical data in a recent post. Wrong! The Quicken files are missing. Read on to learn how to avoid my stupid mistake.

We only backed up data, not programs. Theory was that we can restore programs from the CDs they came on or re-download them. I’ll stand by that but we did lose the Quicken install CD. No big deal, I said, it was the 2005 version and we’ll just get a new one. Maybe we even registered and we can just upgrade. But then I began to think about the data…

A quick search of Mary’s new hard drive for files “*.q*” yielded nothing. (Note for non-nerds: Quicken file extensions are three letters starting with “q” so I was looking for all such files on the hard drive.) Nada! Went on Mozy (the online backup service we use) to see if I had just somehow neglected to download it; nope, no “.q” files there, either. Youch! And it’s all my fault.

Here’s what happened: Mozy makes it very easy to backup the files that are in your “My Documents” folder and the folders that are in that folder. Mozy will LET you backup files from anywhere on your hard drive; you just have to know where they’re located when you do Mozy setup. Most modern applications store their changeable data in folders that are in the “My Documents” folder. For example, here’s what the online FAQs for Quicken 2007 say:

“Quicken puts your data files into the Quicken folder where they're easy to find. …

“Quicken stores your Quicken folder in C:\Documents and Settings\User Name\My Documents\Quicken by default..”

But it wasn’t always that way. The FAQs for Quicken 2005 say:

“The Quicken program automatically puts your data files into Quicken’s program directory. If you prefer to keep your data in a separate directory, create a directory for your data and move your data files to that directory, using Microsoft® Windows® Explorer or File Manager.”

Program directories are NOT in the My Documents folder. Sometime they’re in folders on paths like “C:\Documents and Settings\Tom Evslin\Local Settings\Application Data\Microsoft\Outlook”.  This is the Outlook program directory where my Outlook data is. I know it’s there so I DO back it up but that didn’t make me think that other programs might have also stored their data in folders like this one. BTW, these folders are very hard to find because you have to turn on options for seeing hidden and system folders and finding things in them or Windows pretends they don’t exist.

So, if you have an old version of a program, you have to be sure you know where it stored its data so you can back that data up properly.

However, even if you have a new version of the program, you may still have a hidden data problem if you upgraded from an old version. Reading on in the Quicken 2007 FAQs:

“Quicken stores your Quicken folder in C:\Documents and Settings\User Name\My Documents\Quicken by default. [I know, I quoted this above; read on] If you installed this or your previous version of Quicken to another location, Quicken leaves your Quicken folder in the location you specified until you reinstall Quicken and specify another location.” Translation: since Quicken 2005 and earlier put data in the program folder, it stays in the program folder even if you upgrade unless you move it somewhere else.

Outlook apparently works the same way. I’m sure many other programs do as well. The moral is you have to know where your data is to back it up properly.

How do you know where your programs are storing their data? You could read the documentation but that’s not much fun. If you explicitly save data you’re working on as you do with Word or Excel, you can do a “Save As…” and see what folder it puts stuff in by default or you can always control where it puts the data and remember what you told it to do.

But programs like Outlook and Quicken which work off the same files day after day are a little trickier because you don’t do explicit saves. To be safe, you should search the documentation for “file extensions” and find out what sorts of files your application creates. For example, most Outlook files have the extension “.pst”; Quicken’s main file has the extension “.qdf”.

Once you know the file extension, you can search your hard drive for files of that type “*.pst”, for example. Then you’ll know where they are. Remember, though, if the search comes up blank, choose the advanced search option to search hidden and system folders.

This whole experience with the missing Quicken files show’s how right reader Craig Plunkett was when he said recovery is something you should practice.

Meanwhile, we went to Geek Squad at Best Buy to see if they could recover the files from the now dead disk in Mary’s old computer rather than try to recover from bank records. Standby for results.

Comments

[Danette West]

Any luck getting back any of the Quicken files??
I thought I was backing mine up as well....alas...I've learned that lesson.

I did a system recovery before I realized.

[unirecovery]

The tips are very interesting but I will emphasis on one thing. If you face a hard disk crash and you need your data back, don't try to spoil the hard disk with trying to recover data back and then after several trial and errors, you go to a specialist.
Simply let a specialist undertake the work and you will get the maximum data recovered.

http://www.unirecovery.co.uk

[Neil Simon]

Tom,

Have you considered something like Acronis True Image (v9.1)? It's easy to setup, and lets you create a scheduled, automated FULL hard drive backup (usually called an "image"), that can run nightly. It's easiest when used with a direct-attached USB external hard drive. It basically enables you to replace a dead internal drive with a new one, and quickly restore it back from the latest image that's on the external drive. Drawback -- it's onsite. But, augment that with Mozy (or whatever online service) for your critical data, and it's a nice comprehensive solution, that let's you get back up and running quickly. No reinstall of Windows, settings, passwords, etc.

Neil

[Matt Ryall]

Thanks for the useful pointers, Tom. A good way of discovering which files an application has open is to use a tool like Process Explorer, which lists all the "handles" (files, directories and a few other things) that a program has in use.

http://www.microsoft.com/technet/sysinternals/utilities/processexplorer.mspx

This can help when you do not even know what file extension a particular program uses.

[Michal Altair Valasek]

I think that it may be a good idea little to clarify things.

In "My Documents" or whatever they're called in given language version programs are *expected* to store files that user would remember that he put there (like Word or Excel documents).

In "Application Data" folder programs are expected to store their files, that user would NOT remember to manually create - such as settings, data files like Outlook's etc.

So, the wise move is to backup your entire profile (usually C:\Documents And Settings\username\), except Temp and Temporary Internet Files (to save space). This way you can backup even lots of program settings, favorites etc. That's why (among other things) the profile exists and why the above recommendation is live from Windows NT 4.0 (at least).

Backing up your profile will not save you, if you have some program, which defiantly ignores the above recommendation and writes your settings to its program folder etc. Such programs usually require you to have administrator privileges on your computer.

Unless you have Windows Vista. In Windows Vista the above mentioned recommendation is not merely guideline, but rule. With User Account Control (UAC) turned on (as is by default) you basically do not have write rights anywhere else than your profile, even if you are administrator. So the program can't write anywhere.

There is very cute workaround built-in: because such poorly-written programs are widespread, there is thing called "Virtual Store" in Windows Vista. If any non-elevated program tries to write to some well-known folders (such as Program Files or Windows folder), the data are actually written to this virtual store and slipped under to the badly behaved program. What's good is that this virtual store lies in your profile as well (usually in somewhere like C:\Users\username\AppData\Local\VirtualStore). So in Windows Vista, by backing up user profile, you are actually backing up such obsolete program data as well.

So in Vista there are not only bells and whistles and transparent windows, but also such features which can save your ass^H^H^Himportant data one time.