Deffo the flywheel breaking up, all the metal filings are contaminating the starter and clogging it up.
If you replace just the starter motor there will probably be a note in the box saying warranty will be invalid if the new starter fails due to flywheel contamination so just be aware of this if you dont replace the flywheel !!