Problem might be your TB-VGA/TB-DVI adapters.
I used the active TB-dual link DVI and had been able to run up to 6 displays.

At the moment I´m using 3 Outputs on a tour without any problems with the original apple-adapters (1920x1200 50Hz, 1920x1080 50Hz, 1024x768 50Hz)