You're both right, really.
Frankly those low freshman retention rates were a sign of poor instruction as much as anything. A few of the Old Guard were still around when I started in 2004 (4 Horsemen of Calc, Physics 1&2, etc), and those guys were just abysmal teachers.