phpBB has to be kept fully patched. If phpBB (and any installed mods) isn't kept updated/patched, then it can be exploited. But an up-to-date installation of phpBB
is completely secure.
vBulletin really isn't any more secure than phpBB. Same with any other forum, same with any other PHP-based script. vBulletin will have to be kept patched, and good passwords used, in order to protect it from being cracked.

Bailey