No matter what you do keep in mind that any time you alter the content in the library you will need to re-scan the whole library. (hold down SHIFT and it goes a bit faster)