Audio

Analysis

Automation

Build 04

©2026 Nikhil Khedlekar. All rights reserved.

Core Expertise

Generative AI Workflows • Prompt Engineering • Content Automation • Content Strategy • Programmatic SEO • CMS Architecture • Search Intent Strategy • WordPress • Figma • Canva • Analytics

Back to Tools
Build 04

Audio Analysis & Comparison Automation

A Python pipeline that compares speech recordings using speaker embeddings, voice-activity detection and pitch features, and returns structured evidence so a reviewer can focus on judgement instead of replaying clips.

Self-initiated build · Python, PyTorch, SpeechBrain · Nov 2026
ECAPA-TDNN Speaker Embeddings
Voice Activity Detection
Cosine Similarity Scoring
Playwright Browser Automation
Stack
Python · PyTorch · SpeechBrain
Role
Pipeline design & ML integration (AI-assisted)
Status
Built and iterating
Timeline
Nov 2026
In short

A Python pipeline that compares speech recordings using ECAPA-TDNN speaker embeddings, voice-activity detection and pitch features, and reports structured similarity evidence for a human reviewer.

Comparing Voices By Ear

The work rewards patience, not insight. That is a sign it should be automated.

Judging whether two recordings come from the same speaker by listening is slow and repetitive: play, rewind, play again, switch clips, repeat.

It is also inconsistent. Judgement drifts as attention fades, and two reviewers can disagree on the same pair.

  • Hours of replaying clips to reach a single judgement
  • Subjective calls that vary with fatigue
  • No consistent record of why a decision was made
  • Manual browser steps around every clip

The pipeline turns audio into measurable evidence. It does not replace the reviewer; it gives them scores, flags and context so their time goes where it matters.

Evidence, Not Verdicts

A similarity score is only as trustworthy as the audio and the checks around it, so the pipeline follows four rules:

  • Scores support a decision; they never make it
  • Only speech is compared: silence and noise are filtered out first
  • More than one signal: embeddings first, pitch as a cross-check
  • Ambiguous cases are surfaced, not hidden

The goal is a reviewer who trusts the output because it shows its working.

Applied ML Stack

Pre-trained speaker models do the heavy lifting. The engineering is in the preparation and reporting around them.

Language
Python
Deep Learning
PyTorch
Audio Processing
torchaudio
Speaker Model
SpeechBrain ECAPA-TDNN
Scoring
Cosine similarity
Browser Steps
Playwright

How It Works

A preparation-then-comparison pipeline, with a branch for results that are not clear-cut.

Collect
Playwright handles the repetitive browser steps: detecting new items and gathering the audio.
Normalise
Clips are converted to a consistent format so they can be compared fairly.
Detect speech
Voice activity detection keeps the segments where someone is speaking and drops silence.
Extract pitch
Pitch features are computed as a secondary acoustic signal.
Create embeddings
An ECAPA-TDNN model from SpeechBrain turns each clip into a fixed-length speaker embedding.
Compare
Cosine similarity between embeddings gives a similarity score for each pair.
If the evidence is clear
The result is recorded with its supporting features.
If it is ambiguous
It is flagged for a careful human listen.
Report
Scores, features and flags are written out as a structured assessment for the reviewer.

Where It Can Go Wrong

Speaker models are powerful but not magic. These are the limits I designed around.

Noisy or very short clips reduce reliability
Silence and music can skew embeddings, so speech detection runs first
Thresholds need calibrating against known examples
A person makes the final call
How it was built: I described the pipeline and its checks, then used AI-assisted development to write the code, reading, modifying and debugging it myself. The judgement calls (what to filter, what to flag) were mine.

What I'd Improve Next

Calibrate thresholds on a labelled sample set
Add a per-clip audio quality score
A visual report with waveforms and score history
An evaluation harness to measure precision and recall

What This Taught Me

  • Pre-trained models give a strong baseline quickly. The engineering is in the preparation around them.
  • Garbage in, garbage out: filtering non-speech matters as much as the choice of model.
  • A score is evidence, not a verdict, so the output has to show its working.
  • Automation is most valuable around the edges of an ML model: fetching, formatting and flagging.

What This Demonstrates

Applied machine learning with PyTorch and SpeechBrain
Audio signal processing: voice activity, pitch and embeddings
Browser automation for repetitive data collection
Designing human-in-the-loop evaluation workflows
Honest handling of model limits

Common Questions

What does the Audio Analysis and Comparison Automation do?

It is a Python pipeline that turns each recording into a speaker embedding with an ECAPA-TDNN model (SpeechBrain on PyTorch), filters out non-speech with voice-activity detection, and scores pairs of clips with cosine similarity, with pitch features as a secondary signal.

Does it decide whether two voices match?

No. It produces evidence: scores, supporting features and flags. Unclear results are flagged for a careful human listen, and a person makes the final call.

Why use voice-activity detection and pitch features?

Silence and noise can skew speaker embeddings, so only speech is compared. Pitch adds a second, independent acoustic signal as a cross-check.

What are its limitations?

Noisy or very short clips reduce reliability, and the thresholds need calibrating against known examples before the scores can be trusted.

What role does browser automation play?

Playwright handles the repetitive browser steps around the analysis, such as detecting new items and gathering the audio. The analysis itself does not depend on it.