just check if you're exactly at 60.000 or 50.000Hz
when locking the refresh rates you have to verify with switchres if it's correct
catalyst only gives an indication

turn off the hud during playback

preferably use 60Hz 30fps or 50Hz 25fps

what codec are you using?
the movie width needs to be dividable by 16
which playmode?

try without phoenix installed
try m466 or lower
if possible try a 5.1 macpro with quadro4000, snow leopard or GTX680 with mountain lion

we had 3 phoenix cards running with a 4.1 server, flashed GTX285 snow leopard without problem, there was a bit of tearing only with camera fast movements
i think it was on m329 or m364

hopes this helps
P