Hi! I have the same problem on 2 MacPros 4.1 (both with gtx 470) when using OSX 10.7 or newer... version 10.6.8 with CAT 466B seems to be OK... but your graphic card is not supported in 10.6.8...

p.s.: few weeks ago my colleague have this problem on 2 catalysts at the same time: trashcan on 10.9.5 and Mac Pro 4.1 with 10.6.8. same project (CAT ver. 466C) on both machines (1 - working and 1 spare) - same horizontal tearing. files was OK in Quicktime player on same Mac. output 2x1920x1080@50 (first - for HUD, second - LED wall)