Turning it on only when you’re using it doesn’t stop it harvesting all your Bluetooth interaction data (like covid contact tracing). That’s what Android Auto does with Ford cars - the car isn’t just tracking you when you drive it and reporting back to Ford, it’s tracking where you go with your phone based on all the Bluetooth devices you ping.

The best solution is to just not have it. Or, if you have to have it, have it on a spare device.