

Chiming in with more context, my PhD was in neuroscience and I worked in a language lab. As others have stated, there is a critical window for learning a language. The biology behind it is fascinating.
As early as about 9 months of age, your brain begins to decide what speech sounds are important to you. For example, in Japanese the difference between /r/ and /l/ sounds doesn’t matter, but in English it does. Before 9 months, most babies can tell the difference between the two sounds, but babies living in Japanese-speaking environments (without any English) LOSE this ability after 9ish months!
Language is more than just speech sounds, though. Imagine all these nuances of language - there are critical moments where your brain just decides to accept or reject them, and it’s coded somewhere in your DNA.
Have you tried learning Japanese / English after learning the other? I studied Japanese and learned how to pronounce the /r/ in Japanese correctly.
For some people, the difficulty is less in production, and more in interpretation for someone who is native Japanese speaking and later learned English.