I watched most of your linked clip, and it actually looks pretty darned good considering you're using a 7600.
Recording uncompressed analog video to a 1996 Mac is a daunting task, even at 320x480.
You've got a 333MHz G3 CPU, but that means your System bus is probably running at 47.5MHz, which is pretty darned slow. I don't know what vintage your hard drive is, but it may have a relatively slow rotational speed and read/write speed, and a small cache buffer. Your drive may also be fragmented, forcing the Mac to store bits and pieces of your video in different parts of the drive, which would produce hiccups while capturing as well. And finally, your capture software is working hard to do all that digitizing - that could be the bottleneck.
If you have a Firewire-equipped Mac and a digital video (DV) camera, you can use the camera as a bridge between a VCR and the computer. It'll work fine and won't cost a dime, if you have (or can borrow) the equipment already.
Or you can buy an analog-digital video bridge device. About a year or so ago, I needed to dub about 60 short clips from 25 VHS tapes, accumulated over about 15 years. I priced analog-digital bridges, and most of them were $80 and up (WAY on up in some cases).
That's when I decided to just buy a standalone VCR/DVD Recorder. I found a nice open-box Sony unit on eBay for $120 and it took me maybe two hours to dub all 60 clips. Every single dub was absolutely perfect, not a single dropped frame. It was dead easy to rip the clips off the DVDs onto my Mac, where I edited them down to final length using nothing more than Quicktime Player. And now I have an updated VCR and DVD player in the bargain, and I have saved my friends lots of money by dubbing their old family VHS's onto DVD without them having to pay a video store to do it.
Today, you can find refurb DVD-R/VCR combos for less than $150. And the resolution will be a full 640x480. VHS is crap, but because it has 400 vertical lines of resolution, a 480i DVD capture is still going to look a lot better than a 320i BTV/Apple Video Player capture.
So that's my $.02.
Good luck!
M