Yep, really common.
"I've heard" that Toyota initally tried all sorts to remedy the issue.
Replace bearings, replace gear sets, replace gearboxes (while cars were still under warranty, had this issue)
But the issue remained.
Some people have had better luck with aftermarket single piece flywheels though, but it seems its just an annoying noise rather than anything that'll damage your gearbox.
Thread about it
here