Should we use customer data or static test data when testing an application?

My general approach when testing a legacy system is to use the customer data in the system for testing. I'll often find the application will let me do something that creates bad data in a persistent storage (e.g. database). Before we ship version 8.0, development will fix the application so the user interface will not allow the bad data to get stored.

The idea is to control the scenarios by restricting the user input. Unfortunately, the application has been in use for years and that ability to store bad data has been there for years. There is a good chance that someone stored that bad data in the production database. It wasn't a problem in version 7.7 but that nice new feature in 8.0 breaks on the bad data.

So although version 8.0 will stop any future customers storing the bad data, there is still the old data in the system. What we need to do is get rid of the bad data, ideally. If the user interface had of stopped the customer from entering the bad data in the past, what would the customer have done? If the customer would have skipped entering the data altogether then we can delete the data now. If the customer would have changed the input to be valid data (e.g. don't leave fields blank) or if we now store bad inputs as default good inputs (e.g. blank fields are no longer stored as null but as an empty string or -1 for a numeric value). But what if different customers would have done different things. The business rules could become quite complex. Especially if you have hundreds of customers using the same application (e.g. web based service).

So the solution is to leave the bad data and change version 8.0 so it handles bad data.

How do we test this? There are two camps of thought on this. We can use the existing customer data or we can generate static data.

How do I test this? I like to use both ways. The short term solution is to use customer data. Often during my testing it is hard to imagine what bad data might be hiding in the system. For the long term I will create static data which tests all the scenarios I think are valid. Trying to test all the possible inputs would be impossible. So I'm going to test what the application will let me use. If I didn't use customer data and simply used what version 8.0 was willing to generate, I'd miss the hidden bad data.

On pretty much every project I have worked on, using customer data has resulted in old data causing problems that could never be created with the application as it exists when I joined the project.

So create test cases with known, static data but use legacy data as well.