At the very top of the forum you will find a stickie titled:
"Sticky: Why is Topo horizontally/vertically inaccurate?"
It takes you to a detailed explanation, located here:
Simple version: The USGS provided Digital Elevation Map underlying Topo is more accurate than your GPS. Your GPS has a choice in accuracy: X-Y (lat,lon), or Z plane (altitude). There's a sacrifice in Z to make X-Y more accurate, as that's what most people worry about when traveling.
This is similar in concept to your computer speeds. You have an overall max transfer speed. Download is (for home use) faster than upload. Why? Do you click and download more information, or do you sit at home and push out / upload more information?
Read the link for more detailed information. Topo 5 also came out in 2003 IIRC, the information (including the DEM) was updated in Topo 6.
In general, the roads are best at major roads and highways that are "pinning points." Accuracy is not as perfect between them, but not so bad I ever got lost. I see about 300 feet typical worst-case variation between these pinning-points.
Note that GPS variations affect accuracy, and you will find away from main cities/highways, NavTeq (MS S&T, google, etc) accurcy goes down as well. They cherry pick.
• Horizontal Road/POI Inaccuracies: Maximize your screen before clicking the link
(it's a png and sizes to your screen size). 121 meters off over the same day.
It will explain some of the inaccuracies. You might also be in a region that needs to be submitted for a correction as the roads are not located correctly and need to be shifted over.