Voice-based control represents an efficient way to interact with IoT devices and autonomous systems, but its use is often limited by privacy, security, and hardware constraints. The present study explores the open-source Python library Resemblyzer for on-device speaker authentication using short voice commands, focusing on offline operation in resource-constrained environments. Experiments with datasets of different audio quality, duration, and size, covering both same-speaker and cross-speaker cases, used cosine similarity to detect false positives, false negatives, and threshold results. We found that reliable authentication is possible with recordings as short as 2.63 seconds and at least 495 KB in size, while shorter clips (1-1.5 seconds) are less dependable, especially against high-quality references. These results show that Resemblyzer can run effectively without cloud access, offering practical guidance on minimal audio requirements for secure, real-time voice verification in IoT and robotic systems.
In the present work we evaluated the potential of publicly available Python library Resemblyzer
for user authentication based on short voice commands [
We focused on Resemblyzer library as the core speaker embedding tool due to its strong
performance in voiceprint-related tasks, public availability and active maintaining by community,
compatibility with Python’s scientific ecosystem (e.g., NumPy, Pandas, scikit-learn) and ability to
operate in both batch and near real-time contexts [
Comparison of popular solutions and their pretrained models by size can be found in Table 1.
As can be seen from this table, Resemblyzer model is the smallest one. This advantage plays an important role taking into consideration that it should be used in resource-constrained devices. Moreover, authentication based on voiceprint analyses in Resemblyzer library works purely ondevice with no need to send audio to cloud services. In other words, no network communication is required.
The implementation of the work involved a set of tests with short audio records designed to emulate voice commands for a combat drone. All audio records were saved in .wav format and produced with a standard computer microphone by ‘Sound Recorder’ application which is available at any Windows OS. We deliberately avoided high quality devices with a purpose to simulate the real-life conditions.
Two types of voices, male and female, were included into the experiments. Information about audio records used and identification of the corresponding input sets are summarized below (Tables 2-5).
Authentication.wav
Authentication Analyze.wav
Attack.wav Autopilot.wav
Check.wav Defence.wav Destroy.wav
Find.wav Follow.wav Freeze.wav
Go.wav Home.wav Identity.wav Listen.wav Stop.wav Wait.wav Watch.wav
Analyze
Attack Autopilot
Check Defence Destroy
Find Follow Freeze
Go Home Identity Listen Stop Wait Watch 1,16 1,34 1,50 1,42 1,29 1,48 1,53 1,46 1,42 1,40 1,42 1,34 1,59 1,20 1,02 1,18 1,23 The Adventures of Sherlock Holmes" by Arthur Conan Doyle is a collection of detective stories written during the late 19th century. The book
introduces the legendary detective Sherlock Holmes and his loyal companion, Dr. John Watson Authentication.wav Autopilot.wav
Check.wav Defence.wav Destroy.wav
Find.wav Follow.wav Freeze.wav
Go.wav Home.wav Identity.wav Listen.wav Stop.wav Wait.wav
Watch.wav
File name A1-f.waw
Text in audio The Adventures of Sherlock Holmes" by Arthur Conan Doyle is a collection of detective stories written during the late 19th century. The book
introduces the legendary detective Sherlock Holmes and his loyal companion, Dr. John Watson Length of audio (seconds) 12,99
For each audio recording, a voiceprint was generated using the Resemblyzer library, producing a 256-element array of floating values. Each array was saved in a separate .csv file, named identically to its corresponding original audio file.
The Python code used to perform this operation is provided below: import pandas as pd from resemblyzer import VoiceEncoder, preprocess_wav import numpy as np from pathlib import Path import os def calculate_n_save_voiceprint(audio_record: str, output_folder: str, csv_separator: str = ";", csv_decimal_symbol: str = ","): """Calculation of voiceprint for the passed audio file and saving it to a CSV file.
ValueError: If the audio record path is invalid or not a .wav file. """Calculate and save voiceprints as separate csv file for all .wav files in the input folder.
Args: input_folder (str): Path to the folder containing .wav files. output_folder (str): Path to the folder where voiceprints will be saved. csv_separator (str, optional): Separator for the output CSV files. Defaults to ";".
csv_decimal_symbol (str, optional): Symbol for decimal point in the output CSV files. Defaults to ",". """ for file_name in os.listdir(input_folder): if file_name.lower().endswith(".wav"): print(f"Processing: {file_name}") file_path = os.path.join(input_folder, file_name) try:
calculate_n_save_voiceprint(file_path, output_folder, csv_separator, csv_decimal_symbol) except ValueError as e:
print(e) except Exception as e:
print(f"An error occurred while processing {file_name}: {e}") An example of abridged output voiceprint data in single .csv file may be illustrated as: 0,13736048;0,00000000;0,16476172;0,00000000;0,00000000;0,00000000;0,00000000;0,00000000;0,06845610;0,21822383;0,04 552145;0,00026178;0,08779779;0,09342065;0,00000000;… 0,02555513;0,00000000;0,00000000;0,00000000;0,00000000;0,00000000;0,00000000;0,00602875;0,11488174;0,07095279;0,07308 850;0,00000000;0,12232912;0,14700639;0,00000000;0,07068025;0,12317624;0,00709428;0,00000000;0,00000000;0,00000000;0,03 307733;0,08165727;0,00000000;0,06436168;0,00000000
The authentication process and evaluation of similarity between two voiceprints can be quantified using cosine similarity, a widely used metric in speaker verification systems: Similarity =
A∗B ‖A‖∗‖B‖
Cosine similarity yileds a value between –1.0 and +1.0, indicating how closely two voiceprint vectors align in the embedding space. Specifically, +1.0 value means almost identical voices, 0.0 indicates no similarity, and –1.0 value represents complete opposition, an outcome unlikely in the context of our work. Heuristic or empirical result equal or greater than 0.75 means successful authentication. In speaker verification systems using cosine similarity, typical same-speaker scores fall around 0.8-0.95, different-speaker scores often fall around 0.3-0.6, so 0.75 is commonly used as a default threshold in prototyping or academic examples.
Python code which compares two voiceprints and calculates the similarity can be found in the listing below:
def compare_2_voiceprints(voiceprint_csv_file1: str, voiceprint_csv_file2: str, csv_separator: str = ";", csv_decimal_symbol: str = ",") -> float: """
Args: voiceprint_csv_file1 (str): Path to the first voiceprint CSV file.
voiceprint_csv_file2 (str): Path to the second voiceprint CSV file.
"""
df1 = pd.read_csv(voiceprint_csv_file1, sep=csv_separator, decimal=csv_decimal_symbol, header=None)
df2 = pd.read_csv(voiceprint_csv_file2, sep=csv_separator, decimal=csv_decimal_symbol, header=None)
if df1.shape[
raise ValueError("Voiceprints must have the same number of dimensions.") 3.1. Experiment 1: calculating the similarity for the short audio records produced by the same person In our first experiment we compare voiceprints received from dataset #1 between each other. The algorithm was the following: 1. Selection of the first .csv with voiceprint from the target folder (e.g. analyze.csv). 2. Comparison of this voiceprint with those voiceprints stored in all other .csv files in the same target folder 3. Record the calculated similarity scores in the results.csv file as a new row. The header of this file includes names of files taking part in comparison operation (all csv files found in the target folder) referred to here as destination files. The first column contains the names of source file, and each subsequent column stores the value of calculated similarity between the source and destination files. 4. Repetition of the process for the next .csv file with voiceprint.
The format and structure of result.csv file allows us to import easily these data into Excel and utilize more sophisticated tools for the analysis.
An example of output result.csv file received during this experiment can be found below: # Calculate cosine similarity embedding1 = df1.iloc[0].values embedding2 = df2.iloc[0].values return similarity 3. Results and Discussion
similarity = np.dot(embedding1, embedding2) / (np.linalg.norm(embedding1) * np.linalg.norm(embedding2)) ;analyze;attack;authentication;autopilot;check;defence;destroy;find;follow;freeze;go;home;identity;listen;stop;wait;wat ch analyze;1,0;0,74;0,72;0,65;0,67;0,79;0,69;0,8;0,81;0,78;0,79;0,85;0,79;0,86;0,72;0,83;0,8 attack;0,74;1,0;0,67;0,75;0,71;0,69;0,63;0,75;0,73;0,71;0,71;0,64;0,73;0,75;0,84;0,81;0,8 authentication;0,72;0,67;1,0;0,66;0,58;0,8;0,7;0,77;0,64;0,65;0,61;0,62;0,81;0,77;0,58;0,64;0,62 autopilot;0,65;0,75;0,66;1,0;0,62;0,63;0,67;0,71;0,75;0,67;0,69;0,64;0,71;0,6;0,7;0,7;0,7 check;0,67;0,71;0,58;0,62;1,0;0,57;0,65;0,72;0,78;0,76;0,81;0,67;0,62;0,64;0,74;0,81;0,75 defence;0,79;0,69;0,8;0,63;0,57;1,0;0,72;0,82;0,68;0,68;0,66;0,69;0,79;0,82;0,64;0,67;0,63 destroy;0,69;0,63;0,7;0,67;0,65;0,72;1,0;0,75;0,74;0,75;0,77;0,76;0,8;0,75;0,65;0,73;0,7 find;0,8;0,75;0,77;0,71;0,72;0,82;0,75;1,0;0,76;0,79;0,79;0,74;0,84;0,83;0,7;0,8;0,72 follow;0,81;0,73;0,64;0,75;0,78;0,68;0,74;0,76;1,0;0,82;0,89;0,82;0,71;0,73;0,77;0,87;0,83 freeze;0,78;0,71;0,65;0,67;0,76;0,68;0,75;0,79;0,82;1,0;0,86;0,84;0,81;0,81;0,78;0,86;0,84 go;0,79;0,71;0,61;0,69;0,81;0,66;0,77;0,79;0,89;0,86;1,0;0,87;0,75;0,72;0,82;0,88;0,83 home;0,85;0,64;0,62;0,64;0,67;0,69;0,76;0,74;0,82;0,84;0,87;1,0;0,73;0,8;0,76;0,85;0,83 identity;0,79;0,73;0,81;0,71;0,62;0,79;0,8;0,84;0,71;0,81;0,75;0,73;1,0;0,84;0,7;0,76;0,73 listen;0,86;0,75;0,77;0,6;0,64;0,82;0,75;0,83;0,73;0,81;0,72;0,8;0,84;1,0;0,68;0,79;0,79 stop;0,72;0,84;0,58;0,7;0,74;0,64;0,65;0,7;0,77;0,78;0,82;0,76;0,7;0,68;1,0;0,84;0,85 wait;0,83;0,81;0,64;0,7;0,81;0,67;0,73;0,8;0,87;0,86;0,88;0,85;0,76;0,79;0,84;1,0;0,93 watch;0,8;0,8;0,62;0,7;0,75;0,63;0,7;0,72;0,83;0,84;0,83;0,83;0,73;0,79;0,85;0,93;1,0 Python code which does comparison and produces result.csv can be found below: def calculate_voiceprints_similarity(input_folder: str, output_csv: str, csv_separator: str = ";", csv_decimal_symbol: str = ","): """Calculate the similarity matrix for voiceprints.
input_folder (str): Path to the folder containing voiceprint CSV files. output_csv (str): Path to the output CSV file for the similarity matrix. csv_separator (str, optional): Separator for the output CSV file. Defaults to ";". csv_decimal_symbol (str, optional): Decimal symbol for the output CSV file. Defaults to
ValueError: If the input folder is invalid or contains no CSV files. """ folder = Path(input_folder) if not folder.is_dir():
raise ValueError(f"{input_folder} is not a valid folder.") csv_files = sorted(folder.glob("*.csv")) if not csv_files: print("No CSV files found in the folder.") return file_names = [f.stem for f in csv_files] file_paths = {f.stem: str(f) for f in csv_files} # Initialize with NaN to ensure float dtype df_result = pd.DataFrame(np.nan, index=file_names, columns=file_names) for a in file_names: for b in file_names: if a == b:
df_result.loc[a, b] = 1.0 else: similarity = compare_2_voiceprints( file_paths[a], file_paths[b], csv_separator=csv_separator, csv_decimal_symbol=csv_decimal_symbol, ) df_result.loc[a, b] = round(similarity, 2) df_result.index.name = "" df_result.to_csv(output_csv, sep=csv_separator, decimal=csv_decimal_symbol)
The results of comparison for dataset # 1 are summarized in Table 7.
Given that all audio recordings were produced by the same speaker, we expect that all similarity scores would exceed 0,8. However, the results revealed numerous cases with values below this threshold, and in some instances, even below 0,6. In the context of our, scores below 0.6 can be considered false negatives, the range between 0,6 and 0,8 represents an uncertain or threshold zone, and values above 0,8 indicate a successful match. To better understand the distribution of results, we calculated the proportion of scores falling within each range. The following Python function performs this calculation:
def analyze_similarity_matrix(similarities_csv_file: str, failedRange: float, succeedRange: float, csv_separator: str = ";", csv_decimal_symbol: str = ","): """
Returns:
dict: Percentages of failed, threshold, and succeed values. """ df = pd.read_csv(similarities_csv_file, sep=csv_separator, decimal=csv_decimal_symbol, index_col=0) # Extract all similarity values except diagonal (self-comparisons) values = [ float(value) for i, row in df.iterrows() for j, value in row.items() if i != j # Classification counters
Let’s do the same experiment for dataset #3 – analogous short audio records recorded by female. The results of comparison for dataset # 3 are given in Table 9. p o t s 0,76 0,84 0,81 0,82 0,88 0,82 0,82 0,8 0,84 0,81 0,8 0,88 0,79 0,84 1 0,84 0,9 t i a w 0,73 0,81 0,8 0,81 0,89 0,83 0,8 0,81 0,85 0,89 0,79 0,85 0,82 0,9 0,84 1 0,87 h c t a w 0,75 0,8 0,77 0,79 0,87 0,81 0,8 0,82 0,84 0,87 0,83 0,86 0,79 0,83 0,9 0,87 1
The resulting score for this case is presented in Table 10: Resulting scores
The results for dataset #3 were found to be notably better than those for dataset #1 – zero failed cases and a threshold range less than half the size of the successful matches. To understand the factors contributing to this improvement, at the next recordings from the two datasets and tried to identify step the of the study we
compared characteristics that lead to audio higher similarity scores. The first step involved the detection of the cases with the largest differences between the two resulting similarity matrices. We take the similarity matrix from dataset #3, since it demonstrates superior performance, and subtract the corresponding matrix from dataset #1. In this context, the resulting delta serves as a quality indicator for the comparison (Table 11). Difference of similarity scores between dataset #3 and dataset #1 0,09 0,15 0,25 0,18 0 0,29 0,22 0,08 0,09 0,1 -0,01 0,19 0,22 0,22 0,14 0,08 0,12 e c n e f e d -0,05 0 0,15 0,09 0,12 0,18 0,1 0,1 0,01 0,03 0 0,03 0,06 0,04 0,05 -0,06 0 By summing the values in each column, we obtain a cumulative weight for the corresponding file, allowing us to identify the cases with the largest overall differences.
These results are presented in Fig. 1.
analyse attack auth-tion autopilot check defence destroy find follow freeze
go home identity listen stop wait watch
The physical characteristics of input files are given in Table 12.
Based on the above data, no unconditional correlation can be established between file size or audio recording length and the success of authentication. For example, in the case of the “analyse” recording, dataset #3 contains a longer audio clip with a larger file size, yet its similarity score is lower than that of dataset #1. To better understand these correlations, we proceed by calculating the average delta values for file size and audio length across the entire dataset, allowing us to assess the overall differences. It appeared that dataset #3 has bigger average file size (~10 Kb greater) and almost the same length of audio record, but do not have false positive cases. These findings suggest that for short audio records with the length of 1-1,5 seconds, files with acontent size exceeding 270Kb are likely to produce voiceprint suitable for reliable person authentication with a probability of 80%. 3.2. Experiment 2: calculating the similarity for the short audio records produced by different person In a previous scenario we used voiceprint received from short audio records produced by the same person. Now we try to compare the audio records from different speakers in order to determine the rate of false positive cases under these conditions.
We used the same datasets #1 and #3, but now we compare each file from one dataset with each file from another dataset. The algorithm included the following steps: 1. Selection of the first .csv file containing a voiceprint from the dataset #1 folder (e.g., dataset1/analyse.csv). 2. Comparison of the voiceprint from dataset1/analyse.csv with all voiceprints stored in the dataset #3 folder. 3. Recording the calculated similarity scores in results.csv as a new row. The header of results.csv lists the filenames from dataset #3, the first column contains the name of the source file from dataset #1, and each subsequent column stores the similarity score between the source and each corresponding destination file. 4. Repetition of the process for the next .csv file with voiceprint.
The structure of results.csv is identical to that of the previous experiment. The Python code used to perform these comparisons and generate results.csv is given below: def calculate_voiceprints_similarity_matrix_by_folders(input_folder1: str, input_folder2: str, output_csv: str, csv_separator: str = ";", csv_decimal_symbol: str = ",", suffix1: str = "_1", suffix2: str = "_3"): folder1 = Path(input_folder1) folder2 = Path(input_folder2) if not folder1.is_dir() or not folder2.is_dir():
raise ValueError("One or both input folders are invalid.") # Get common CSV files (intersection of names) files1 = {f.name: f for f in folder1.glob("*.csv")} files2 = {f.name: f for f in folder2.glob("*.csv")} common_files = sorted(set(files1.keys()) & set(files2.keys())) if not common_files: print("No common CSV files found in both folders.") return # Prepare labeled names row_labels = [f"{name}{suffix1}" for name in common_files] col_labels = [f"{name}{suffix2}" for name in common_files] similarity_matrix = [] for file1_name in common_files: row_label = f"{file1_name}{suffix1}" row = [row_label] # First cell in the row is the labeled name
emb1 = pd.read_csv(files1[file1_name], sep=csv_separator, decimal=csv_decimal_symbol, header=None).iloc[0].to_numpy() for file2_name in common_files:
emb2 = pd.read_csv(files2[file2_name], sep=csv_separator, decimal=csv_decimal_symbol, header=None).iloc[0].to_numpy() print(f"Similarity matrix saved to {output_csv}")
The results of comparison are given in Table 13. 0,6 0,6 0,55 0,65 0,63 0,46 0,63 0,61 0,72 0,64 0,7 0,66 0,61 0,54 0,61 0,71 0,68 e c n e f e d 0,53 0,49 0,52 0,53 0,54 0,45 0,56 0,53 0,65 0,51 0,58 0,53 0,53 0,49 0,47 0,57 0,53 The resulting score for this scenario is presented in Table 14:
Total number: 272
The results obtained in this test are consistent with these expectations. No successful matches or false positive results were expected when comparing audio recordings from different speakers. 3.3. Experiment 3: calculating the similarity for the short audio records with help of voiceprint built on audio record of good quality In two previous scenarios we used voiceprints received from short audio records and potentially it could affect the quality of voiceprints due to the lack of data required for the Resemblyzer voiceprint generator to analyze. To address this, we now examine a scenario in which one of the voiceprints in each comparison pair is derived from a longer recording. Specifically, we use dataset #2 (male voice, 15,78 seconds) and dataset #4 (female voice, 12,88 seconds). This experiment involves calculating similarity scores between voiceprints from dataset #1 and the voiceprint from dataset #2, and likewise between voiceprints from dataset #3 and the voiceprint from dataset #4.
To perform the above estimates, the following algorithm was implemented: 1. Selection of the first .csv file containing a voiceprint from the target folder (e.g., analyze.csv). 2. Comparison of the voiceprint from analyze.csv with a good quality voiceprint (e.g., from dataset #2). 3. Storing the similarity value in memory. 4. Proceeding to the next .csv file with voiceprint and repetition of the comparison. 5. Recording the calculated similarities in the results.csv file. The header of this file lists the names of the files involved in the comparison, and the single row (since there is only one in this case) contains the corresponding similarity values.
The Python code used for these comparisons and for generating results.csv is: def calculate_voiceprints_similarity_single_to_all(voiceprint_csv_file: str, input_folder: str, output_csv: str, csv_separator: str = ";", csv_decimal_symbol: str = ","): """Compare one voiceprint with all voiceprints in a folder and save similarity results.
Args: voiceprint_csv_file (str): Path to the input voiceprint CSV file (one line, no header). input_folder (str): Folder containing CSV files to compare against. output_csv (str): Path to output CSV file (single-row result). csv_separator (str, optional): CSV column separator. Defaults to ";".
csv_decimal_symbol (str, optional): Decimal symbol. Defaults to ",". # Load source voiceprint df_source = pd.read_csv(voiceprint_csv_file, sep=csv_separator, decimal=csv_decimal_symbol, header=None) source_embedding = df_source.iloc[0].to_numpy() # Gather and prepare comparison files csv_files = sorted(folder.glob("*.csv")) if not csv_files: print("No CSV files found in the folder.") return results = {} for file in csv_files: df_target = pd.read_csv(file, sep=csv_separator, decimal=csv_decimal_symbol, header=None) target_embedding = df_target.iloc[0].to_numpy() if source_embedding.shape[0] != target_embedding.shape[0]:
raise ValueError(f"Dimension mismatch: {file.name}") similarity = np.dot(source_embedding, target_embedding) / (
np.linalg.norm(source_embedding) * np.linalg.norm(target_embedding) )
Results of comparison of dataset #1 with dataset #2, and dataset #3 with dataset #4 are illustrated in Fig. 2.
Dataset #1 and #2 Dataset #3 and #4 Failed
Threshold
Succeed
The resulting scores are presented in Tables 15 and 16. Dataset #1 and #2 Dataset #1 and #2
Total number: 17
Range
Percentage (%) 17,65 82,35
0,0 Percentage (%) 35,29 64,71 0,0
Description
Failed scenario Threshold scenario Succeed scenario
Description
Failed scenario Threshold scenario Succeed scenario
In both cases, no successful matches were observed, with a high proportion of results falling into the threshold and false positive categories. These findings indicate that voiceprints generated from short audio recordings of 1-1.5 seconds are not suitable for reliable authentication when compared against a high-quality voiceprint. 3.4. Experiment 4: calculating the similarity for the medium audio records using the voiceprints built on audio record of high quality In the following, we tried to increase the quality of input audio records by increasing the length of audio records and file size to approximately twice in comparison with the original datasets #1 and #3. Details of dataset #5 are provided in Table 17.
As a first step, we repeat the initial experiment, calculating similarity scores by comparing the voiceprints from dataset #5 with one another. The results of these comparisons are presented in Table 18.
The resulting score is presented in Table 19.
The results were found to be very promising. Accordingly, no false positive cases were observed, and the proportion of threshold values is minimal. Next, we repeat the experiment by comparing dataset #5 with the high-quality voiceprint obtained from dataset #2. The results of this comparison are presented in Fig. 3.
The resulting score is presented in Table 20.
The obtained findings indicate strong performance, suggesting that audio recordings with a duration of at least 2.63 seconds and a file size of 495 KB or greater can produce reliable voiceprints suitable for secure user authentication. 1 0,89 0,91 0,86 0,8 0,82 0,84 0,8 0,83 0,86 0,86 0,84 0,83 0,83 0,85 0,83 0,8
The present study has demonstrated in practice that the Resemblyzer library can be effectively employed on portable devices with limited hardware resources and no access to cloud services. The experiments confirm that the library delivers highly reliable results in voice authentication using relatively short audio recordings (no longer than 3 seconds in duration and under 500 KB in size), which aligns well with the typical timing of short voice commands. These findings highlight the potential of Resemblyzer for integration into autonomous, resource-constrained systems and provide a foundation for further research into its applicability in more complex operational scenarios for independent portable devices.
The author(s) have not employed any Generative AI tools.