I'd venture that your guests weren't eating it for breakfast, but for lunch or even later.
Wouldn't surprise me in the slightest. Though I didn't have any guests because the minyan was full to capacity. Except my brother-in-law (and a cousin who came afterwards), no one came to the bris except the mispallelim.