I assume you are referring to the rolling start video's posted on the M5board? Granted, once both cars are rolling, in a straight line the Gallardo pulled barely a car length on the M5. This may have to do with 4-wheel drive bogging the car down a bit, but nevermind that. Off the top of my head, an EVO magazine comparison pitted the 'unstoppable' E60 M5 against a Gallardo (non-SE) sometime last year, and the speed and dynamic superiority of the Lambo became starkly clear. The M5 was fighting its weight, but the in-depth feature revealed the Gallardo to be miles faster, and easier to drive at speed along winding roads. Only in the most ideal conditions (M5board being IT) will that sedan come remotely close. In a recent Top Gear shoot-out between the F430 and Gallardo spiders, the lambo won, and was reportedly the quicker of the two across varying conditions. In some motoring video clips on my computer, I've seen the F430 outpaced aswell. I like Ferraris' too, but their power-to-weight ratios have never translated into expected pace (look at an F355 for example). I don't know why.
And by the way, the RS4 is faster than an M5 at Hockenheim, Nurburgring, Top Gear track and EVO handling circuit. Don't believe the hype mate.
EDIT:
http://www.youtube.com/watch?v=wzisa6wcMNU The B6 S4 beat the E46 M3 on a bone dry track. Is the M3 more fun, yes, but even against logic (the weaknesses the S4 has compared to M3 you pointed out) and odds, the S4 makes the best of Quattro- the best Ad there is- beating a BMW M3 in a comparison that was probably not a primary concern during S4 development. And this S4 doesn't even have the latest 40/ 60 torque split!