The way I understand it.
Place the square and round donuts on top of each other.
The corners of the square donut stick out further than the round one.
But the hole of the square donut also 'sticks out' further than the hole of the round one, and has more donut missing.
But the outer edge of the donuts are larger than the inner edge by the hole.
So there's more extra donut there (at the outer edge) than is missing (at the inner edge/ hole).
The juicy part here is the second half (paragraph that begins “The interesting thing...”). You didn’t follow through:
Using your approach, you can cut off a “corner” of the square doughnut, you’ll be cutting along the round edge so you’ll end up with a triangle that has one side rounded inwards. You then do the same for the inner corner (you ostensibly need to flip the doughnuts for this cut).
Then you align both triangles and cut larger (outer) one to subtract the size of the smaller one. You measure the surface of what remains of the outer/larger triangle and multiply by four (to account for all four corners), that number is the absolute measure of what’s extra in the square.
Now measure the surface of the round doughnut and divide the previous measurement of surface of the extra in the square doughnut by the entire surface of the round doughnut, the result is how much larger the square doughnut is.