Speech-to-text or speech recognition can be a generative (transformer) model architecture, but there are a lot of models that are not. The transformer ones seem to be better in quality.

Non-transformer examples:

Coqui STT https://stt.readthedocs.io/en/latest/

Kaldi https://kaldi-asr.org/models.html

Creating subtitles / captions requires some more programming to create the text file with the correct timings.