Always so much detail missing from the anime or manga.
I prefer the manga. The light novel - as usual for the format - is rather all over the place and occasionally jumps slightly where you feel like "wait, shouldn't there have been..." while in other places losing itself in irrelevant sidetracking.
In comparison you really notice in the manga that they attempted to round out the story, that there was proper editing involved and that they tell it "better".
Mentally both are adults but physically 11 and 15. Add on the fact they are in a fantasy setting where there is no legal or moral limitation to what they can do, yet she has no issue with it while he think of ending up a lolicon.
The problem for him is that he does have a defined strike zone based on his mental age - and no, he's not a lolicon, rather the opposite. They're editing that out in the anime, but the manga has it; and the manga actually defines it better than the light novel. Key scene is where he's invited by Martha to the inn in Seiryuu. It's one place where the anime actually rewrites characters compared to manga and LN.
Actually that makes me wonder if there is any specifically isekai media out there with the opposite way of jumping. Someone becoming older. I know there's some older stuff that goes that way - Nanaka 6/17 comes to mind.