Surprisingly, I have never had a hard time finding hot peppers... barrio chino and san telmo markets will aways have a stock, but first check your local verduleria - you probably won't see it advertised, just ask.
I first asked my peruvian verduleria on Santa Fe and Carranza for the most...