Speech-to-Text (STT)

Speech-to-Text (STT) in Mastra provides a standardized interface for converting audio input into text across multiple service providers. STT lets voice-enabled applications respond to human speech. It supports hands-free interaction and improves accessibility for users with disabilities. It also provides a more natural interface.

Configuration

To use STT in Mastra, you need to provide a listeningModel when initializing the voice provider. This includes parameters such as:

• name: The specific STT model to use.
• apiKey: Your API key for authentication.
• Provider-specific options: Additional options that may be required or supported by the specific voice provider.

Behavior: All of these parameters are optional. You can use the default settings provided by the voice provider, which will depend on the specific provider you are using.

const voice = new OpenAIVoice({
  listeningModel: {
    name: 'whisper-1',
    apiKey: process.env.OPENAI_API_KEY,
  },
})

// If using default settings the configuration can be simplified to:
const voice = new OpenAIVoice()

Available providers

Mastra supports several Speech-to-Text providers, each with their own capabilities and strengths:

• OpenAI: High-accuracy transcription with Whisper models
• Azure: Microsoft's speech recognition with enterprise-grade reliability
• ElevenLabs: Advanced speech recognition with support for multiple languages
• Google: Google's speech recognition with extensive language support
• Cloudflare: Edge-optimized speech recognition for low-latency applications
• Deepgram: AI-powered speech recognition with high accuracy for multiple accents
• Sarvam: Specialized in Indic languages and accents

Each provider is implemented as a separate package that you can install as needed:

pnpm add @mastra/voice-openai@latest  # Example for OpenAI

Using the listen method

The primary method for STT is the listen() method, which converts spoken audio into text. Here's how to use it:

import { Agent } from '@mastra/core/agent'
import { OpenAIVoice } from '@mastra/voice-openai'
import { getMicrophoneStream } from '@mastra/node-audio'

const voice = new OpenAIVoice()

const agent = new Agent({
  id: 'voice-agent',
  name: 'Voice Agent',
  instructions: 'You are a voice assistant that provides recommendations based on user input.',
  model: 'openai/gpt-5.6-sol',
  voice,
})

const audioStream = getMicrophoneStream() // Assume this function gets audio input

const transcript = await agent.voice.listen(audioStream, {
  filetype: 'm4a', // Optional: specify the audio file type
})

console.log(`User said: ${transcript}`)

const { text } = await agent.generate(
  `Based on what the user said, provide them a recommendation: ${transcript}`,
)

console.log(`Recommendation: ${text}`)

Transcribe an audio file

The listen() method accepts a stream of audio data from a microphone or file. Use createReadStream() when you want to transcribe an audio file:

import { createReadStream } from 'fs'
import path from 'path'

const audioFilePath = path.join(process.cwd(), 'agent.m4a')
const audioStream = createReadStream(audioFilePath)

const transcription = await agent.voice.listen(audioStream, {
  filetype: 'm4a',
})

console.log(`Transcription: ${transcription}`)

For a broader overview of voice providers on agents, see Voice in Mastra.