After reading an interesting article in the last Mojo, I took the box-set live "At The BBC" (1 dvd + 3 cds). It's very rich. It kicks off with McKay'era (Morrissey's favourite period) with their first Whistle Test performance. Then, one sees a session and a concert from McGeoch'era (hailed by Marr and Radiohead). The quality of remastering is remarkable.
I also re-discovered "A Kiss In The Dreamhouse", it's probably the best album with McGeoch. It's experimental but very accessible in the end. This record particularly influenced acts like The Beta Band and LCD Soundsystem.