Ok I figured out what I was reading wrong. The last sentence was worded slightly confusingly and my reading comprehension failed. The unit with AMS takes the 8 line of the cluster table of the number of missiles that are hitting from the roll on the cluster table, not from the originally launched number.
So if the cluster roll on a LRM 20 is a 2, then you take the 8 from the 6 column not the 20 column. Meaning 14 move on, 6 target the unit, and AMS shoots down 2 of the 6 missiles, and the unit takes 4 damage.
14+2+4 = 20 no missiles magically appearing out of nowhere.
Again my reading comprehension failed, sorry about that; and you can close this question. Thank you.