Don’t start a sentence with plus. Using plus adverbially as a sentence connector seems to be quite common on the Internet: These three reasons should be enough to convince them of their mistake. Plus, they can hardly deny their responsibility in this matter. Using moreover or finally instead of plus would be correct and far better.

You can start a sentence with “plus” when you want to use it as an introductory clause. If the sentence beforehand had been too long with “plus” as a part of it, it makes more grammatical sense to include “plus” as a new word. “Plus” at the beginning of a sentence is a great way for you to break up the flow of your writing.