I believe I understand what you are asking. The only way to have a specifically masked video input or a specific mask of content over a background is to create an object in a 3D program which is a 2D surface which has all the areas you want invisible deleted from the object. Render this object and put it in the object folder of the Catalyst and call it up in the VFX. Once you do this, you can run it with your content assigned to it so that all the invisible parts are simple gone because they do not exist in the object.

Does this make sense?