Speaking for my personal games over the last 38 years: my friends and I always used pick-up games to test out custom designs. We usually hash out what's acceptable for the session - Clan tech only, UrbanMechs only, 10,000 BV with any technology, 3025 Inner Sphere only, super heavy designs only, etc., and build per appropriate rule books.
In campaigns we generally stick to whatever designs are available and, later, salvaged. As the force gets sufficient technical knowledge and cash, they'll customize within the rules and whatever restrictions the GM thinks are appropriate.