I heard two good news for foreign travelers to Matsumoto.

One is a new ATM machine has been installed in the tourist information center at one block south of Matsumoto Castle. You can withdraw Japanese yen by your international cash or credit card as well as ATMs in Seven-Eleven convenience stores and post offices.
See this PDF leaflet for the details.

Another is new bus service to Utsukushigahara Heights. Until last year, a bus to the Utsukushigahara Heights Art Museum operated only early August to mid-August. This year, another direct bus to Utsukushigahara Heights operates weekend of June 18 to Sep 19 and everyday of July 25 to August 31. It departs from the Alps (west) Exit of the station at 8:00 and 13:30 (twice per day).