@davidd75 here are my two cents. I think you need to separate the investment analysis from the personal feelings about owning a second home.
For the investment analysis, you need to build a pro forma which takes into account all income and all expenses. You're going to have to make estimates for rental rates, occupancy rates, and all costs, but try to find real data to inform your estimates. Try to include everything, both at the time of acquisition (like any renovations, the cost of furnishing the place, etc.), and ongoing costs like management, an allowance for damages, cancellations, etc. Speak to people who have done this to try to figure out realistic expectations for each item.
If you would otherwise be renting a place for a number of days or weeks annually, then maybe add that to the rental income line. Make sure to adjust the actual rental income to reflect whichever days you're renting it out for.
All that will help you analyze the cashflow (positive or negative).
Then build in whatever principal paydown you're making, and any market increase in the value of the home, if you're assuming one.
Now before getting into the emotions of why you want to own a vacation home, at least you can analyze the investment itself. Is it looking a compelling enough investment to justify the headache, or does it look a lot less exciting in excel than it does in a dream? How does this investment compare to others that you could make? How diversified are you if something in this investment goes wrong?
Now separately you can think about the emotional or intangible value to you in owning a second home , but at least you're not muddying the investment analysis with that excitement. That certainly won't do you any good.
Obviously this is all irrelevant if you can comfortably afford a second home and think it's a good use of your money, but if you're actually looking at this as an investment, then I think you gotta analyze it as one.
/.02