Can I use a photoshop image with a transparent background, or with an alpha channel in catalyst?

Yes. It should work.
Alpha channels are supported in photoshop images, and will be transparent. ( Thats how its supposed to be...unless i made a mistake )