It depends on the time of year. I went to Bermuda July and thought it was really nice, conveniently close by, and had a lot of mileage availability. However, during that time you are risking hurricane season. Additionally, as you get into winter Bermuda isn't as warm as the Caribbean- so the Caribbean is definitely preferable during that time of year.
It was definitely a really nice island with nice hotels, but since the weather doesn't work into your favor most of the time, the Caribbean might be easier.
To note: The hotels are on the expensive side. There are 2 Fairmonts and mostly independent hotels,