After 6 or so years here, my Japanese still leaves much to be desired. I have several Japanese friends, but only one of whom I speak to in Japanese. Hiro is at work most of the time and our language of communication has never switched to Japanese - though there a fair bit of Japanese mixed in with our English. (including Japanese words I mangle to fit with English grammar).
There are several obstacles to improvement, aside from age.
One is that I don't actually need to use it particularly. I have plenty enough Japanese to buy the groceries, go to the dry cleaner, exchange pleasantries, ask for cooking advice at the vegetable shop. It's a big jump between this and being able to express informed opinions on current affairs.
A further problem is that no-one corrects me.... sigh.... sigh... and triple sigh... I feel like such a dufus when I realise something that I thought was OK, and have been using for years, is actually a mistake.
Additionally going to classes doesn't make so much sense as there will be people who are much better and much worse - intermediate is a very big grey zone.
And something that use not to bug, but is starting to, is Japanese people with very limited English speaking to me in very broken English. For some reason there is a preconception, particularly among railway, restaurant, and shopkeepers that people with white skin can't speak Japanese. eg At Hamamatsucho station the other day I asked the JR staff which way to proceed through the station to get to the Pokemon Centre. The answer... : downu sutepo, ri to, ... I will have my assertive boots on next time and repeat back in Japanese 'what you are saying is.....go down these steps and turn right'. It seems very petty, I know... I find at the local govt. classes I need to be insistent about speaking English as everyone there also would like to speak better English as well.... grrrr
Going to the local govt. lessons is good though. Last Fri I went and there was a very sweet Japanese woman in her 30s or so who was paired with me. After almost 2 hours I persuaded her that correcting me was actually very helpful not very rude. In some ways that is the best way to learn - having a patient person correct, while at the same time working at home on improving my vocab.
Having said that, I'd better get back to it...:)