The historical reason this sort of thing doesn't happen is it is a massive burden on authors and players, for very little gain.
Spelling out every planet, or even just a significant amount of them, would take an immense amount of time, effort and resources. It would also decrease the flexibility of campaign generation and story writers.
While it is true that very little of the canon actually makes a great deal of sense, particularly from the initial stage of development, it's been outright stated that most planets have their own militia with it's own infantry and vehicles, so it's not hard to go from there.
Not even close! Vehicles were about the only thing the successor states could build from scratch, as a result they proliferated throughout the Succession Wars in all realms. If anything the Star League likely used light mech recon in force, with the Successor States using vehicles and light mechs being front line combatants. Of course, those new build vehicles all had ICE power plants and low tech weapons and sensors, but the Successor States could build them in relatively large numbers and that made up for it.
I don't know that I explained my head-canon of it very well.
If you've got the kinds of resources that relatively important (i.e., invasion target) pre-SW planets would have, you'd have huge standing armies with lots of specialized units.
You've got satellite surveillance over your own planet.
You've got aerospace fighters to defend your satellites and to spy on invaders.
You've got conventional fighters to hit conventional forces and to spy on invaders.
You've got pre-placed autonomous sensor devices along major highways and near important installations.
You've got infantry squads and platoons that take up defensive positions and radio any contact with the enemy.
You've got small patrol and scout vehicles (probably with active probes) that roam the countryside looking for anything you might miss.
You've got VTOLs that patrol the countryside.
You've got light mechs that would act as a quick response to any attackers.
Pre-SW, you would have amazing situational awareness of your home planet. Any invaders who came to your world would bring their own scouting forces, a more transportable version of what you see above. And they'd also bring countermeasures versus a lot of the stuff they expect you to use.
For instance, one of the things people always bring up is how effective infantry can be in the game. Presumably, this fact is not lost on the military commanders of the day. But one thing that should be pretty common is highly mobile artillery. They should really have something like an AC-130 gunship. Mount a Thumper artillery piece on a conventional aircraft (I know that the rules say you can't fire artillery while airborne except for Arrow IV, but you really should be able to do it) or a VTOL. Now you've got supremely mobile firepower that can wipe out defending infantry forces without risk of return fire. Alternatively, use a group of Meteor conventional fighters (AC-10s) with flechette ammo.
None of this should be that controversial, this is how war has been waged for all of modern history in real life. You spam unit X, they bring in unit Y to counter, you create unit Z to counter their counter.
While Battlemechs are "kings of the battlefield", they were created in an environment where a lot of these conventional means of war already existed, and were in use. A Stinger mech is really useful, because it can survive a few shots from your enemy's AC-130 stand-in. A surprise Thumper round or two probably won't kill it, and so your scout can survive to report back. The Stinger has two machine guns, but it's not like it can kill its BV in rifle infantry. But it doesn't have to -- you've got your own AC-130 gunship that will make short work of massed infantry formations. The Stinger just needs to be able to massacre the occasional stray squad.
Battlemechs would have been powerful enough and durable enough that they disrupt some of the move/countermove of conventional war. A Rifleman can not only shoot down aircraft, it's also not nearly as vulnerable to an air-dropped inferno bomb as an AA tank would be. Mechs would have outshined their competition in the warfare that existed at the time. A lot of early mechs were Age of War designs, and they were created to fight in the environments of that day. The Rifleman doesn't have to be loaded down with max armor -- it just has to be durable enough to survive against its likely opponents (probably conventional aircraft, mostly).
After the 1st Succession War killed billions and billions of people, these massive conventional armies that had existed for a long time would have been completely smashed. Now you begin the process of rebuilding. And that's when people decide it's not worth it to replace some of this stuff. You don't flood the field with infantry anymore, because you know your enemy still has some AC-130s they can use (or can put back into production if they have to). You don't fill the air with conventional fighters, because all it takes is a handful of ASFs and some patience, and your men will be sitting ducks.
When money is no object and you have your local factories running for decades, producing local variants of popular vehicles, then sure it pays off to have a huge conventional force. But when your army is shattered and you have to dedicate precious resources to giving yourself as effective a force as you can get, you put it into Battlemechs. And thus warfare in the Inner Sphere becomes somewhat ritualized. Yes, they can give every Joe Citizen a rifle and take down that mech company you sent to attack the world. But if they do that, you'll feel compelled to bring out the infantry-murdering artillery. And then they'll have to produce a counter for it. And you'll have to bring in a counter for that. And every step of the way, you're paying more and more money and sacrificing more and more lives. And at the end of that long chain of events, you're still left with the fact that Battlemechs are the ultimate trump card. So why not just skip those intermediate steps and go right to the end?
In the 2nd SW, people still hadn't quite learned their lesson. They tried to rebuild their original armies and they suffered the same fate. By the 3rd SW, people just threw up their hands and said "for god's sake, let's just use the mechs and forget about everything else".
So mechwarriors and light mechs make excellent scouts, once you've reached this point with society. Yeah, vehicles are cheaper, and can be faster, but that method of waging war is actually significantly more costly in the long run.
