It looks like the Sworders are seen more as a political unit, which is disingenuous. To apply for the SoL, you need five years of spotless service in another regiment. They test them for orthodoxy, but also military skill, and they only take the top 5% of all warriors tested, and need to maintain that edge. They are on the top of the heap for training and procurement, even in the old 3025-days of low industry. In 3025 (the only books I have available at work), two of the Sworder regiments were elite, two were veteran and one was regular quality, and the 2nd Sword of Light was often held as the best single unit in the Sphere. In the same era, all of the Warrior House battalions were Veteran quality.
As a Davonista I hate them, but the Sword of Light units have a well earned, long standing record as the fanatical, elite warriors of baddassery in the most militant of the Successor States. They have the history. They have the numbers. They have the skill. They have the support of a much more powerful military support system.