Just as badly against T-34. Against then-known heavy tanks T-28 and T-35 the 37mm could penetrate only at under 100m distance. The assumption at the beginning of Barbarossa was that the Soviets only had BTs, T-26 and such plus a handful T-28/T-35, with the already introduced T-34 and KV-1/2 completely unknown and unexpected (but encountered on Day 2, with devastating results in favour of the Soviets).
It's somewhat more interesting with the 50mm KwK39 on the Pz III, since this is always claimed to have been equally ineffective against current Soviet tanks - however, the 50mm KwK39 accounted for 54% of all T-34 losses within the first year, giving it some credit in that regard.