I just ran tests with file sharing and management on. I see a very small difference. The difference is only when there is activity.

Once I disconnect RDP and all file shares, the performance is fine. I even tried to see if I could bog the system down doing index queries from an SSH command line And no hiccups.

I'll still promote turning it all off but unless someone can show results of the performance losses they have seen, I would say it is negligible.