I am starting to think it is the envolve chat that is causing a lot of the delays. I looked around and that seems to be a common complaint with the system.
I am not sure how many people use it, or if there are any other chat room type add on with vbullitein.
It all comes back to the host that is being used. I use Knownhost for my forums due to the fact that they don't oversell their hosting. Even a decent sized VPS on a non oversold host makes all the difference.