There are some quirk using Custom Action in Visual Studio 2005 and if your development machine is Vista you have even more trouble to sort. Here are what I faced recently :
- Unhelpful error message. See : "Windows Installer fails on Vista with 2869 error code.". In my case, I just run the resulting installer in an XP machine to get more informative message.
- When you add Custom Action for Commit phase you need to include your Custom Action Library to Install phase although you don't do anything on it e.g: override it. If you don't do this it you will have error can not save state or something. More about it here.