After thinking QuickTime had a really annoying bug that would show some videos at an incorrect size (1248x702 instead of 1280x720 for example) I finally spent enough time trying to find a fix. I came across this very useful post at the Apple Discussions site.

It turns out the incorrect size is caused by a QuickTime feature (documented here). Unfortunately it looks like it can actually only be changed in QuickTime 7 Pro and not QuickTime X. Even so this is a significant hint as I'd always been frustrated knowing I'm not looking at a clip at 100%, pixel to pixel.