I think you are misunderstanding something crucial - The reason why conventional vehicles & infantry don't have highly unlikely chances of dealing very limited but potentially crippling damage to themselves isn't because the game creators are looking to make them as good or better than 'Mechs - it's because the game designers understand that they aren't the focus of the game, so their rules are made intrinsically simpler. There aren't rules for the possibility of an infantry platoon's HMG jamming not because it's impossible in the Battletech universe or because the authors want to buff infantry, but because infantry are plenty weak already and adding a further complicated rule for infantry machine gun jams that will at most reduce damage output by 2 just isn't worth the page space to print it and would be no fun for the majority of players, because it would take time away from playing with the 'Mechs.
So, since they're cannon fodder, who cares?
I completely understand the sentiment. No matter how cool-looking or well statted a tank may be, I too would prefer to play a Mech any day. I have a couple players in my groups who refuse to play anything but Mechs, but they have no problem facing off against them. I only run vehicles and battle armor and maybe some infantry as opposition for somebody's hero force, or because it's strongly implied in the force I'm designing.
But, I don't dislike taking vees or infantry because they're week. It's because they're too simple, and quite boring. As a game, I'd rather be entertained by
all that it has to offer, and not just the primary focus element.
I believe you're a widely versed gamer with a wide range of systems under your belt. I strongly suspect that you know, like I know, that there are many better ways to add significant detail and give infantry and vehicles flavor, character, and options - making them entertaining - while still streamlining them and keeping them in their cannon fodder position. As it stands, the current style, and even some of the older styles, are highly unsatisfactory. It's not that they simply don't do certain things; they aren't even given the choice, regardless of how bad the results may be afterward.
If you go back to older, simpler rulesets, you'll find Mechs feel more powerful. Why? Because the rules are simpler. You attacked a wood hex and it was automatically cleared, for example. Sure, certain weapons were left out when choosing to implement that option, but it was there. The same applies with vehicles and infantry. With simpler rules, they don't have as many risks or drawbacks to weaken them.
I, myself, would welcome detail, and I'm not the only one. I see posts every now and then, and even short threads, asking about a revamped BattleTroops. Some people go so far as to look to the RPGs for a method of moderately detailed personnel combat that can still integrate with the standard BattleTech game. So, it's not like people don't care about the more conventional units.
As I said, I can think of all kinds of ways to make a better system, but to post it would be a fan-design. So I won't. I'll just leave it at I know the new game designers could have done better, and still can.