Testing and Volatility of Code

When does automating test worth doing and when it does not? one of the consideration I think the is volatility of the code.

The more volatile the code the more it is probably not worth automating, at least not directly. If you just code something to be changed not long after or will be changed quite frequently keeping it automatically tested will be costly. It is not a burden if it is only several dynamic part that we tested, but if there is a lot of them the accumulation of little frequently-changing tests that need to be maintained will be burdensome.

I know, if you use TestDrivenDevelopment, you make test first before any code. However, I think of it in conceptual level. You have to have test before any code, not to have The Test e.g: xUnit-based test. Your test could be in your mind and the value could be the one that your command line program print in console and you do Assert with the value in your mind. It's TestDriven enough for me and for not yet stable problem sometime I feel it's more flexible that way.

My concern here is on the word "automating" not on the TestDriven-ness. Automation is investment and it should be managed so that it gives us benefit more than it cost. Volatile code makes the tests volatile too, so lots of volatile code to test, lost of cost in changing/maintaining those test accumulatively.

There are a lot of solution for this e.g: test it indirectly through more stable code, but the main point is that we could test in a more efficient manner by paying attention to volatility of the code.