In a nutshell, it comes down to the self-ghettoizing of Western mecha franchises by their fans & creators.
Head over to the "other Mecha Games" thread. I don't think anyone's mentioned a single game that's doing well in the current environment. It's pretty much the same story for Western video games, where Battletech games are having fits & starts via kickstarter, Titanfall 2 did decently as a hybrid Mecha/infantry shooter, but otherwise it's a dead genre. Meanwhile anime, including mecha anime, is more popular with Western audiences than ever. Why?
While I don't think this applies to anyone in this thread for obvious reasons, there are large swaths of both the fanbase & the creative community that knee-jerk reject anime & anything that doesn't fall into their established sense of aesthetics. This is especially ironic considering both modern anime & Western mecha franchises are both descended from the same 70's & 80's anime franchises, but that seems lost on the purveyors of this outlook.
While it's one thing to say "Yeah, I'm not a fan of Code Geass's mecha, they're too flashy." It's quite another when a community develops a cultural contempt for another part of the genre & its fans. As traditional marketing avenues for the world of table-top games & paperback novels continue to fade from relevance, anime is thriving in the digital age! This contempt from older fans of Western mecha for anime & most importantly, it's fans, combined with a unwillingness on the part of many creators to engage them in the spaces they frequent is pretty much strangling traditional Western mecha properties.
Twenty years ago when I got into Battletech, I walked into a comic book store, saw the shiny 3rd Edition boxed set for Battletech sitting in the shelf, & I bought it, because I'd been infatuated with giant robots since I was much younger watching Transformers & Robotech. If I were that kid today, the local comic book store doesn't stock table-top games not named Warhammer anymore, & there's hardly a FLGS to be found & barring an older relative, there's hardly anyone who could point you to one. Every young person with an interest in giant fighting robots right now is meeting that fix with anime they found online.
The only rational course of action for someone attempting to market a similar product should be to try to engage those customers and win them over to your product - instead, companies producing Western mecha almost completely ignore anime conventions & web communities, and fans often direct derision at anime fans who approach their games at stores or events. Western mecha games & fiction are killing themselves, & there's no reason for it other than the community's own bigotry towards people and properties that have substantial thematic, aesthetic & narrative similarities to those they claim to love.
I love both aesthetics, so I'd hate to see Western mecha disappear. I don't want it to turn into anime either - Western mecha's greater focus on a gritty military aesthetic is fantastic & I'd never want to lose it. But if we refuse to engage with a new generation of mecha fans who naturally gravitated towards anime because it was the most readily available outlet for their interests, then we're just waiting for the inevitable end. I know asking the fan community to change their outlook is shouting at the wind, but I at least would like the industry figures to wake up before it's too late.
