Mike's Musings #7: FSX Disaster Recovery - Part 2

By Mike McCarthy (27 February 2007)

Summary

Nevertheless, System Restore is like having a pet tiger on a leash. The tiger can be a wonderful companion, but you must never forget that it is a wild animal capable of turning on you at any time if you do not treat it exactly the way it wants to be treated.

In Musings #6 I discussed System Restore's principles of operation in layman's terms. Let's have a quick review ...

Given a full set of architectural as-built drawings, and given a logbook containing detailed descriptions, in chronological order, of all subsequent adds, deletes and changes, the insurance company will be able to recreate your house from a "greenfield", up to and including the effect of the most recent logbook entry.

Operating system professionals will recognize that the article in question was an oversimplification in terms of how System Restore is actually mechanized, but the effects achieved are exactly as were described in Part 1, and you need to keep those principles in mind as you read this article, Part 2 in the series. If you don't recall the principles, or if the review above doesn't make sense, please go back and reread that article now.

Here's the hard thing to grasp: In Windows XP, the file system state description applies to everything that is outside of My Documents, but not to anything that is inside My Documents. This is deliberate. In fact, the whole point of My Documents is to hide things from System Restore. It's as if our hypothetical house had an outside garage (My Documents) which is not depicted in the as-builts and which is off-limits to the construction gangs that the insurance company would bring in to rebuild the house. On the surface this is a very simple statement, but it has important yet non-obvious implications which we will discuss below.

(Okay professionals, I know that the "everything outside of My Documents" statement also is an oversimplification. A complete list of System Restore folder/file includes and excludes will be found in the file c:\windows\system32\restore\filelist.xml, but for the vast majority of users my statement can be treated as if it were literally true.)

All of the state will be recreated, subject to the obscure exclusions cited in filelist.xml. This includes the state of additional hard drives beyond C as well as additional hard drive partitions beyond C. It also includes the deletion of folders and files which did not exist at the time the restore point in question was created.

I'll say it again: If a given folder outside My Documents didn't exist when you made the restore point, and if subsequently you directly or indirectly created the folder and filled it with important files, then rolling back to the restore point will DELETE THE FOLDER AND ITS FILES, because System Restore recreates file system state, not just files.

Some examples: If you had a virus-infected file which your anti-virus software deleted, System Restore may recreate the file complete with the virus, again depending on when the restore point was created. If a driver installation or update created or modified folders and files you weren't aware of, outside My Documents, and if you later uninstalled the driver, a rollback may effectively reinstall the driver and/or repeal the driver update.

The implication of the previous paragraph is that, unless you take special measures, typically you will not know what file system state information is contained in a given restore point. Therefore unless you take those special measures you may not be able to predict what the effect of a restore will be.

Now ... System Restore isn't just about files. It's also about the Windows "registry", and similar remarks apply here. The registry will be restored to precisely the state it had when the restore point was made. This may help, or it may hurt, depending on what you know about the content of the restore point.

Here's what I do to head off uncertainty ...

First, before making a restore point I always empty the recycle bin and then cut certain things from the file system and paste them into to a folder in My Documents, one that I have named HIDER. The objects that I move in this way are a) the Flight Simulator 9 and Flight Simulator X folders, b) the Microsoft, ThunderBird, Mozilla and TeamSpeak folders from my Application Data folder, and c) my AirBoss ™ software development project. (Mozilla and ThunderBird are my browser and email client respectively. TeamSpeak is a voice client.)

The Application Data folders are located at c:documents and settings\[yourame]. The one named microsoft contains essential state information for FS9 and FSX along with other Microsoft programs such as Outlook Express and Internet Explorer. You don't want System Restore to monkey with the microsoft folder so you should exclude it from your restore points by moving the whole folder to HIDER. (On Windows Vista you should move not the whole folder but instead its contents, excluding the folders named Internet Explorer and Windows.)

By the way, can you achieve all of the the cut-to-HIDER effects by editing System Restore's filelist.xml file so as to do the desired exclusions? Perhaps so, but I consider that to be dangerous because filelist.xml is not itself mentioned in filelist.xml, nor are its parent folders, so I don't know whether filelist.xml itself is included or excluded. In other words, I don't know how System Restore treats that file on a rollback, and I don't want to find out the hard way.

Anyway, if you have cut to HIDER the application data / microsoft folder as discussed above, when you do a restore you will find that another microsoft folder has been created behind your back, so to speak. Just overwrite all of it when you do the cut from HIDER.

Second, I plan my restore points very carefully. Every time I make a significant change to the system I go through the cut-to-HIDER procedure, after which I make my own restore point. I don't rely on XP-generated restore points because, and pay attention here, if you let nature take its course, sooner or later a vital restore point will fall off a time cliff owing to it having been displaced by subsequent system-generated restore points.

That's right, folks. IF YOU IGNORE WHAT SYSTEM RESTORE IS DOING BEHIND YOUR BACK, IT CAN ACTIVELY HURT YOU. The only way around this problem of crowded-out restore points is to do what I call "leapfrog restores", frequently rolling back to a restore point that I usually call "base restore point", and then making a new base restore point from the leapfrog. This forces the leapfrog data to remain near the front of the System Restore history queue, ensuring that what falls off the back of the queue will be nothing but uninteresting system-generated snapshots, or at least snapshots made by me that I no longer care about.

I do the leapfrog procedure weekly, more often if there have been a lot of recent file adds/deletes/changes. Is this a lot of work? On a scale of one to ten it's a one after you learn how to do it efficiently. Is it worth it? Well, it's like insurance: How much are you willing to lose, versus how much are you willing to pay to negate the possibility of loss? Remember, as with insurance, when disaster strikes you either will or will not have a policy in force. There is no middle ground.

Now for another subtle point: The smaller your hard drive the more frequent the leapfrogs must be. This is because the smaller the drive, the smaller the capacity of System Volume Information, and the more quickly it will overflow if you don't force your leapfrogs to the front of the history queue.

Finally, I do my restores equally carefully, especially when doing a disaster recovery such as I had to do yesterday for FSX. What I did needed to be surgically precise, and here's why ...

Realizing that I did in fact need to uninstall for the second time, and being cognizant of the System Restore database overflow risk, instead of doing a formal uninstall I simply cut the newly reinstalled FSX and Microsoft Flight Simulator X folders to HIDER. Then I rolled back to my leapfrog restore point named "Just after tuning up FSX." After that I deleted the few FSX-related folders that the rollback had incorrectly created. Then, to avoid a potential future overflow of the System Restore history queue, I made a new base restore point, and after that I moved the contents of HIDER back to the usual places.

(If necessary, read the preceding paragraph several times until you fully understand it. You can perform several kinds of tricks if you understand how System Restore works, and if you keep its workings in mind. My goal here had been to restore the registry to a clean state as well as ensuring a clean set of FSX files. To make both things happen I had to address the issues separately as described in the preceding paragraph.)

Now ... Are you still awake? I hope so because this stuff is really important ...

Mission accomplished, and you can do it too if you follow my System Restore guidelines. If you would like to take advantage of System Restore, it's imperative that you come to understand everything I have said above. Read and reread the material till you do understand it because sooner or later Mother Nature will be giving a quiz about your overall system, not necessarily just about FSX. If you give the wrong answers you may be sentenced to rebuilding your system from a zeroed hard drive. If you don't know how to do that, you may be sentenced to buying a whole new computer.

When installing a new application, or a new device driver, do the cut-to-HIDER procedure and then make a restore point. After that, do the application or driver installation, and then make yet another restore point. (Yes, in this situation XP will make a suitably-named after-installation restore point for you, but you should develop the habit of doing yourself everything that is System Restore related. Making this second restore point will take up very little space in System Volume Information because nothing will have changed, so there will be no changes that need to be recorded by XP.) If you then decide to back out the application or driver, for safety's sake first do an uninstall in the usual way, then do your cuts to HIDER, and then do a rollback to the restore point that you created just prior to the install. The rollback will compensate for any deficiencies in the application or driver uninstall registry clean-up logic. It will also delete any residual application/driver folders, icons, and so on.

Can you make registry backups without using System Restore? Yes, though I don't recommend it. Do Start, Run, Regedit, File, Export. However, if you later do a corresponding registry import but you forget that an application or driver had been installed or uninstalled since the export, you may seriously damage the very system you are trying to protect. It is much better to keep things synchronized by doing full System Restore snapshots, giving the snapshots meaningful names such as "After reinstalling FS9."

Should you rely on System Restore to uninstall a large application like FSX? No. When it comes to large file system state changes, System Restore's rollback logic has unpredictable limitations. It is much better to do your own uninstalls followed by rolling back to your own "Before installing [whatever]" restore point.

Can you uninstall drivers by rolling back to "Before installing [whatever driver]" snapshots if you haven't done the cut-to-HIDER procedure? Generally yes, but not always; do this only if there is no viable alternative. Again it is better to make and use "Before installing [whatever]" snapshots. If you forget to make my recommended restore point you can try rolling back to the most recent system-generated restore point, but now things get really iffy because you may not know what other changes were made subsequent to the system-generated restore point. (The rollback logic will attempt to undo those changes.)

Let's say that you make a snapshot while a particular file is in existence outside of My Documents. If you then do a restore with a modified version of the file in existence in the same folder, System Restore will create a second instance of the file in the same folder. The earlier version will be given the instance number 2, and the newer version will be given the instance number 1. There's a lot of room for confusion in this kind of situation, especially if you do multiple snapshots and restores without following my prescribed procedure, because you will get instance numbers 3, 4, 5, and so on, and you will be clueless about what the numbers now mean. Avoidance of this problem is why I cut things to HIDER before making a restore point, and it's why I cut things to HIDER before doing a restore. My approach makes it impossible for System Restore to be aware that the HIDER files ever existed, much less that they should be "restored". (Of course, as earlier mentioned, you must make your own backups of your HIDER files.)

What if you make a minor tweak to, say, your System Tray, or to your Start Button menu. Do you have to go through the entire leapfrog procedure just to lock in that one tiny change? Well, it's up to you. If I'm very (repeat very) confident that my system is working well, I don't bother with a leapfrog restore followed by a re-application of the change I had just made. Instead I simply cut things to HIDER and then make a new base restore point. This will lock in whatever changes I made, including changes I may have forgotten about. But be careful ... Doing this will also lock in any system corruptions that you may not have been aware of.

Conclusions

For purposes of review, remember this guiding principle: Things that you don't want to be changed or deleted by a rollback must have been cut to HIDER (or to somewhere else in My Documents) at the time the restore point in question was created. They must be cut to HIDER again before you do the restore, if you are to stop System Restore from creating confusing multiple folder/file versions. Only after the restore may you move things from HIDER back to where they really belong. (In the immortal words of Paul McCartney, "Get back, JoJo ... Get back to where you once belonged.")

To avoid mistakes you had better plan on making a HIDER files checklist, and you had better maintain this checklist somewhere within My Documents. If you use NotePad or WordPad you'll be able to keep the checklist open as you use Windows Explorer to move files into or out of HIDER.)

Which folders and files to cut to HIDER is a question that only you can answer -- because the answer depends on what you have done to your system. Also bear in mind that restores will roll back the system registry, so you must keep your restore points in sync with application and driver installs and uninstalls, as discussed earlier.

How important is it for you to do these things? That's up to you. My wife knows non-technical people who have bought whole new computers when their old ones became inoperable in a software sense. One of her friends lost all her data files that way. The files weren't gone but she couldn't get at them, and she didn't know that commercial data recovery services exist.

Mike McCarthy
mike@pcgamecontrols.com

Addenda ...

The exceptions were a) a simmer who has nevertheless decided to fall back to FS9, b) a simmer who said he didn't want to be bothered with my approach because he's very happy with the performance he gets from his ultra-powerful Great Computer In The Sky, c) a simmer whose problems I was unable to troubleshoot, and d) another simmer in the same boat as c).

As I mentioned to person b), Musings #5 was aimed at people (like me) who have average computers of today and yesterday rather than at people who had all-out racing machines like his.

Now: Why do I write? Because I must.

Not surprisingly I know several professional writers, and many years ago I too was a professional, a software reviewer for Computers & Electronics Magazine. Today I have several unpaid writers in my fan club. (You folks know who you are.) I also have non-writer fans. (You too know who you are.)

One thing we writers all seem to have in common is a need to express ourselves in black and white whether or not we get paid. This is accompanied by a) our appreciation of each other's work, and b) a collective need to expose our work to a public audience -- to take the risk that people will actually read our stuff and form opinions about it, or even comment on it.

What appears here on FlightSim.Com is but a part of my output. Most of what I write is sent to individual friends and has nothing to do with flight simulation, or even with aviation. The poet T. S. Eliot did much the same thing. The only work of his that I'm capable of appreciating, "Old Possum's Practical Book of Cats", is a simply a collection of poems about cats that he sent to friends anonymously. (Not that anyone could have failed to guess who the author was.)

My favorite in the series is "Growltiger's Last Stand", but it's too long to repeat here. Instead I will refresh your recollection of the better known "The Naming of Cats", and then I will retire for the night. Here is Eliot's masterpiece, long since in the public domain ...

The naming of cats is a difficult matter,
It isn't just one of your holiday games;
You may think at first I'm mad as a hatter
When I tell you a cat must have three different names.

First of all, there's the name that the family use daily,
Such as Victor, or Jonathan,
George or Bill Bailey--
All of them sensible everyday names.
There are fancier names if you think they sound sweeter,
Some for the gentlemen, some for the dames;
Such as Plato, Admetus, Electra, Demeter--
But all of them sensible everyday names.

But I tell you, a cat needs a name that's particular,
A name that is peculiar, and more dignified,
Else how can he keep up his tail perpendicular,
Or spread out his whiskers, or cherish his pride?

Of names of this kind, I can give you a quorum,
Such as Munkustrap, Quazo or Coripat,
Such as Bombalurina, or else Jellyrum--
Names that never belong to more than one cat.

But above and beyond there's still one name left over,
And that is the name that you will never guess;
The name that no human research can discover--
But The Cat Himself Knows, and will never confess.

When you notice a cat in profound meditation,
The reason, I tell you, is always the same:
His mind is engaged in rapt contemplation
Of the thought, of the thought, of the thought of his name:
His ineffable effable effanineffable
Deep and inscrutable singular Name.