There is a musical instrument neighborhood, with a lot of shops selling new and used instruments, near Callao y Sarmiento in the Microcento.
Old Vinyl is for sale at the better music stores- Miles in Palermo, which has a great CD selection, also sells used Vinyl.
There are used Vinyl stores in the two hipster gallerias on Santa Fe- kind of indoor minimalls of vintage clothes, punk and skater stuff, tattoo parlors, etc- One is called Bond Street, the other is called 5th Avenida-
http://www.galeriabondstreet.com/
http://www.buenosaires-argentina.com/attractions/galeria-5ta-avenida.html
There are a bunch of weekend flea markets with individual vendors selling vinyl, sometimes instruments.
The fancy and expensive one is Sunday Mornings at Plaza Dorrego in San Telmo, but cheaper and potentially more interesting ones include the one on weekends at Parque Centenario and Parque de los Andes. These are both Saturdays and Sundays, much funkier and more hit or miss than San Telmo.
There are also several permanent street markets with kiosks of booksellers, some of whom sell vinyl. One on Santa Fe near Plaza Italia, another in parque rivadavia.
And lots and lots of small antique stores and book dealers that might have something.
Its a big city, and due to geographical location and expense and customs laws, a lot of stuff from the past is still here.