Type4Me macOS Voice Input Skill by ara.so — Daily 2026 Skills collection. Type4Me is a macOS voice input tool that captures audio via global hotkey, transcribes it using local (SherpaOnnx/Paraformer/Zipformer) or cloud (Volcengine/Deepgram) ASR engines, optionally post-processes text via LLM, and injects the result into any app. All credentials and history are stored locally — no telemetry, no cloud sync. Architecture Overview Type4Me/ ├── ASR/ # ASR engine abstraction │ ├── ASRProvider.swift # Provider enum + protocols │ ├── ASRProviderRegistry.swift # Plugin registry │ ├── Providers/ # Per-vendor config files │ ├── SherpaASRClient.swift # Local streaming ASR │ ├── SherpaOfflineASRClient.swift │ ├── VolcASRClient.swift # Volcengine streaming ASR │ └── DeepgramASRClient.swift # Deepgram streaming ASR ├── Bridge/ # SherpaOnnx C API Swift bridge ├── Audio/ # Audio capture ├── Session/ # Core state machine: record→ASR→inject ├── Input/ # Global hotkey management ├── Services/ # Credentials, hotwords, model manager ├── Protocol/ # Volcengine WebSocket codec └── UI/ # SwiftUI (FloatingBar + Settings) Installation Prerequisites

Xcode Command Line Tools

xcode-select --install

CMake (for local ASR engine)

brew install cmake Build & Deploy from Source git clone https://github.com/joewongjc/type4me.git cd type4me

Step 1: Compile SherpaOnnx local engine (~5 min, one-time)

bash scripts/build-sherpa.sh

Step 2: Build, bundle, sign, install to /Applications, and launch

bash scripts/deploy.sh Download Pre-built App Download Type4Me-v1.2.3.dmg from releases (cloud ASR only, no local engine): https://github.com/joewongjc/type4me/releases/tag/v1.2.3 If macOS blocks the app: xattr -d com.apple.quarantine /Applications/Type4Me.app Download Local ASR Models mkdir -p ~/Library/Application \ Support/Type4Me/Models

Option A: Lightweight ~20MB

tar xjf ~/Downloads/sherpa-onnx-streaming-zipformer-small-ctc-zh-int8-2025-04-01.tar.bz2 \ -C ~/Library/Application \ Support/Type4Me/Models/

Option B: Balanced ~236MB (recommended)

tar xjf ~/Downloads/sherpa-onnx-streaming-zipformer-ctc-multi-zh-hans-2023-12-13.tar.bz2 \ -C ~/Library/Application \ Support/Type4Me/Models/

Option C: Bilingual Chinese+English ~1GB

tar xjf ~/Downloads/sherpa-onnx-streaming-paraformer-bilingual-zh-en.tar.bz2 \ -C ~/Library/Application \ Support/Type4Me/Models/ Expected structure for Paraformer model: ~/Library/Application Support/Type4Me/Models/ └── sherpa-onnx-streaming-paraformer-bilingual-zh-en/ ├── encoder.int8.onnx ├── decoder.int8.onnx └── tokens.txt Key Protocols SpeechRecognizer Protocol Every ASR client must implement this protocol: protocol SpeechRecognizer : AnyObject { /// Start a new recognition session func startRecognition ( ) async throws /// Feed raw PCM audio data func appendAudio ( _ buffer : AVAudioPCMBuffer ) async /// Stop and get final result func stopRecognition ( ) async throws -> String /// Cancel without result func cancelRecognition ( ) async /// Streaming partial results (optional) var partialResultHandler : ( ( String ) -> Void ) ? { get set } } ASRProviderConfig Protocol Each vendor's credential definition: protocol ASRProviderConfig { /// Unique identifier string static var providerID : String { get } /// Display name in Settings UI static var displayName : String { get } /// Credential fields shown in Settings static var credentialFields : [ CredentialField ] { get } /// Validate credentials before use static func validate ( _ credentials : [ String : String ] ) -> Bool /// Create the recognizer instance static func createClient ( credentials : [ String : String ] , config : RecognitionConfig ) throws -> SpeechRecognizer } Adding a New ASR Provider Step 1: Create Provider Config Create Type4Me/ASR/Providers/OpenAIWhisperProvider.swift : import Foundation struct OpenAIWhisperProvider : ASRProviderConfig { static let providerID = "openai_whisper" static let displayName = "OpenAI Whisper" static let credentialFields : [ CredentialField ] = [ CredentialField ( key : "api_key" , label : "API Key" , placeholder : "sk-..." , isSecret : true ) , CredentialField ( key : "model" , label : "Model" , placeholder : "whisper-1" , isSecret : false ) ] static func validate ( _ credentials : [ String : String ] ) -> Bool { guard let apiKey = credentials [ "api_key" ] , ! apiKey . isEmpty else { return false } return apiKey . hasPrefix ( "sk-" ) } static func createClient ( credentials : [ String : String ] , config : RecognitionConfig ) throws -> SpeechRecognizer { guard let apiKey = credentials [ "api_key" ] else { throw ASRError . missingCredential ( "api_key" ) } let model = credentials [ "model" ] ?? "whisper-1" return OpenAIWhisperASRClient ( apiKey : apiKey , model : model , config : config ) } } Step 2: Implement the ASR Client Create Type4Me/ASR/OpenAIWhisperASRClient.swift : import Foundation import AVFoundation final class OpenAIWhisperASRClient : SpeechRecognizer { var partialResultHandler : ( ( String ) -> Void ) ? private let apiKey : String private let model : String private let config : RecognitionConfig private var audioData : Data = Data ( ) init ( apiKey : String , model : String , config : RecognitionConfig ) { self . apiKey = apiKey self . model = model self . config = config } func startRecognition ( ) async throws { audioData = Data ( ) } func appendAudio ( _ buffer : AVAudioPCMBuffer ) async { // Convert PCM buffer to raw bytes and accumulate guard let channelData = buffer . floatChannelData ? [ 0 ] else { return } let frameCount = Int ( buffer . frameLength ) let bytes = UnsafeBufferPointer ( start : channelData , count : frameCount ) // Convert Float32 PCM to Int16 for Whisper API let int16Samples = bytes . map { sample -> Int16 in return Int16 ( max ( - 32768 , min ( 32767 , Int ( sample * 32767 ) ) ) ) } int16Samples . withUnsafeBytes { ptr in audioData . append ( contentsOf : ptr ) } } func stopRecognition ( ) async throws -> String { // Build multipart form request to Whisper API var request = URLRequest ( url : URL ( string : "https://api.openai.com/v1/audio/transcriptions" ) ! ) request . httpMethod = "POST" request . setValue ( "Bearer ( apiKey ) " , forHTTPHeaderField : "Authorization" ) let boundary = UUID ( ) . uuidString request . setValue ( "multipart/form-data; boundary= ( boundary ) " , forHTTPHeaderField : "Content-Type" ) var body = Data ( ) // Append audio file part body . append ( "-- ( boundary ) \r\n" . data ( using : . utf8 ) ! ) body . append ( "Content-Disposition: form-data; name=\"file\"; filename=\"audio.raw\"\r\n" . data ( using : . utf8 ) ! ) body . append ( "Content-Type: audio/raw\r\n\r\n" . data ( using : . utf8 ) ! ) body . append ( audioData ) body . append ( "\r\n" . data ( using : . utf8 ) ! ) // Append model part body . append ( "-- ( boundary ) \r\n" . data ( using : . utf8 ) ! ) body . append ( "Content-Disposition: form-data; name=\"model\"\r\n\r\n" . data ( using : . utf8 ) ! ) body . append ( " ( model ) \r\n" . data ( using : . utf8 ) ! ) body . append ( "-- ( boundary ) --\r\n" . data ( using : . utf8 ) ! ) request . httpBody = body let ( data , response ) = try await URLSession . shared . data ( for : request ) guard let httpResponse = response as ? HTTPURLResponse , httpResponse . statusCode == 200 else { throw ASRError . networkError ( "Whisper API returned error" ) } let result = try JSONDecoder ( ) . decode ( WhisperResponse . self , from : data ) return result . text } func cancelRecognition ( ) async { audioData = Data ( ) } } private struct WhisperResponse : Codable { let text : String } Step 3: Register the Provider In Type4Me/ASR/ASRProviderRegistry.swift , add to the all array: struct ASRProviderRegistry { static let all : [ any ASRProviderConfig . Type ] = [ SherpaParaformerProvider . self , VolcengineProvider . self , DeepgramProvider . self , OpenAIWhisperProvider . self , // ← Add your provider here ] } Credentials Storage Credentials are stored at ~/Library/Application Support/Type4Me/credentials.json with permissions 0600 . Never hardcode secrets — always load via CredentialStore : // Reading credentials let store = CredentialStore . shared let apiKey = store . get ( providerID : "openai_whisper" , key : "api_key" ) // Writing credentials store . set ( providerID : "openai_whisper" , key : "api_key" , value : userInputKey ) // Checking if configured let isConfigured = store . isConfigured ( providerID : "openai_whisper" , fields : OpenAIWhisperProvider . credentialFields ) Custom Processing Modes with Prompt Variables Processing modes use LLM post-processing with three context variables: Variable Value {text} Recognized speech text {selected} Text selected in active app at record start {clipboard} Clipboard content at record start Example custom mode prompts: // Translate selection using voice command let translatePrompt = """ The user selected this text: {selected} Voice command: {text} Execute the command on the selected text. Output only the result. """ // Code review via voice let codeReviewPrompt = """ Code to review: {clipboard} Review instruction: {text} Provide focused feedback addressing the instruction. """ // Email reply drafting let emailPrompt = """ Original email: {selected} My reply intent (spoken): {text} Write a professional email reply. Output only the email body. """ Built-in Processing Modes enum ProcessingMode { case fast // Direct ASR output, zero latency case performance // Dual-channel: streaming + offline refinement case englishTranslation // Chinese speech → English text case promptOptimize // Raw prompt → optimized prompt via LLM case command // Voice command + selected/clipboard context → LLM action case custom ( prompt : String ) // User-defined prompt template } Session State Machine The core recording flow in Session/ : [Idle] → hotkey pressed → [Recording] → audio streams to ASR client → hotkey released/pressed again → [Processing] → ASR returns text → [LLM Post-processing] (if mode requires) → [Injecting] → text injected into active app → [Idle] Updating After Source Changes cd type4me git pull bash scripts/deploy.sh

SherpaOnnx does NOT need recompiling unless engine version changed

Troubleshooting App won't open (security warning) xattr -d com.apple.quarantine /Applications/Type4Me.app Local model not recognized in Settings Verify the directory structure exactly matches: ls ~/Library/Application \ Support/Type4Me/Models/sherpa-onnx-streaming-paraformer-bilingual-zh-en/

Must show: encoder.int8.onnx decoder.int8.onnx tokens.txt

SherpaOnnx build fails

Ensure cmake is installed

brew install cmake

Clean and retry

rm -rf Frameworks/ bash scripts/build-sherpa.sh New ASR provider not appearing in Settings Confirm the provider type is added to ASRProviderRegistry.all Ensure providerID is unique across all providers Clean build: swift package clean && bash scripts/deploy.sh Audio not captured / no floating bar Grant microphone permission: System Settings → Privacy & Security → Microphone → Type4Me ✓ Grant Accessibility permission for text injection: System Settings → Privacy & Security → Accessibility → Type4Me ✓ Credentials not saving

Check file exists and has correct permissions

ls -la ~/Library/Application \ Support/Type4Me/credentials.json

Should show: -rw------- (0600)

Fix permissions if needed:

chmod 0600 ~/Library/Application \ Support/Type4Me/credentials.json Export history to CSV Open Settings → History → select date range → Export CSV. The SQLite database is at: ~/Library/Application \ Support/Type4Me/history.db

Direct query:

sqlite3 ~/Library/Application \ Support/Type4Me/history.db \ "SELECT datetime(timestamp,'unixepoch'), text FROM records ORDER BY timestamp DESC LIMIT 20;" System Requirements macOS 14.0 (Sonoma) or later Apple Silicon (M1/M2/M3/M4) recommended for local ASR inference Xcode Command Line Tools + CMake for source builds Internet connection only needed for cloud ASR providers