The study aims to compare the performance of a quantized, finetuned Llama 3 model to its advanced version of the baseline model as part of our participation in a preference prediction task at Eloquent 2025. The performance is primarily evaluated based on five criteria such as (a) relevance, (b) naturalness, (c) truthfulness, (d) safety, and (e) overall quality of how well the model judges the two responses. Among our major findings is that optimization techniques (quantization) produce useful results. Specifically, the finetuned Llama 3 is better at addressing individual qualities such as safety, truthfulness, and relevance than the baseline model.
Prior works submitted to the Eloquent Lab 2024 primarily focused on evaluating the quality of
system responses and task reports [
Therefore, this study contributes to the ongoing exploratory eforts by participating in Preference
Prediction task monitored by Eloquent Lab 2025 and reporting the optimized methodology and findings
of Unsloth finetuned Llama 3 [
2.1. Data
The following three sections describe our methods: (1) Data Collection, (2) Data Preprocessing, and (3)
Finetuning. Lastly, the findings section includes the training results and shared results evaluated by the
Preference Prediction task (2025) committee [
The dataset (2025 validation data) used for finetuning the Llama 3 model is provided by the preference
prediction committee [
Llava 3 using a Low Rank Adaptation (LoRa) approach was utilized for eficient memory usage. Finetuning involves customizing the model to generate nuanced responses based on the finetune dataset. Before the finetuning, the dataset is prepared using the Llama 3-specific prompt template. Each entry in the finetuning dataset contains examples with an instruction, input, and output for the Llama 3 to follow structured guidance and generate relevant responses.
The Llama 3 model, loaded in 4-bit quantization for eficiency, is set up using a specific training configuration. Several experiments have been conducted to track the training loss to keep it minimal. To supervise the finetuning, the SFT trainer is enabled. In this study, the SFT trainer configuration, along with LoRa setup, where the training loss is observed to be minimal, is referred to as the optimal training configuration, as shown in Table 2.
In our section, we include two kinds of evaluation where each compares baseline scores and fine-tuned Llama 3 scores as shown in Tables 2 and 3. Scores in Table 2 follow a systematic approach to evaluate whether the baseline model’s judgement aligns with human judgement for each of the criteria. The baseline/finetune model is presented with two AI-generated responses (A and B) along with the given instruction (test data). The test data has a total of 1,248 items. Then, the model judges which assistant’s answer is better for each criterion and saves it in the resultant TSV format. The results show the accuracy comparison of the baseline model evaluated against the human-annotated answers (validation dataset) for each criterion. The scores in the middle columns for baseline Llama 3.1, when judged individually for each criterion, indicate low alignment with human annotations (validation). However, the scores are notably higher for overall quality (42.42) than individual judgments, indicating that the baseline Llama 3.1 performs better at holistic judgment but lacks individual qualities such as safety (13.13) and truthfulness (11.11). In case of the finetuned model trained on the validation data (human annotations), the scores from the preference prediction committee reveal that individual judgements of the finetuned Llama 3 model are higher than the advanced versions of baseline Llama 3.1. This indicates that the model quantized finetuning has improved qualities such as safety (48.96), relevance (39.98), and truthfulness (38.62), which are major concerns. However, the overall quality (33.01) of the finetuned Llama 3 has underperformed compared to the baseline Llama 3.1 model. This score indicates that the finetuned model lacks a balanced factor efectively when making an overall judgment. However, reviewing the limitations, such as finetuning on a small dataset (99 examples), limited baseline comparisons (only with Llama 3.1), the study can enhance the performance for future preference prediction tasks.
The below sample is one of the responses generated by a nfietuned Llama that reflects an entire entry from the original TSV output file. The sample output is formatted in code syntax for better readability, demonstrating finetuned Llama predictions and their associated explanations between two generative LLM partial responses (LLM A and B), evaluated across multiple criteria: relevance, naturalness, truthfulness, safety, and overall quality. Each criterion assesses diferent aspects of the responses. For instance, relevance criteria evaluate if the answer adequately addresses the user’s original query. Similarly, naturalness criteria judge how human-like the response feels, truthfulness assesses factual accuracy, and safety examines the avoidance of harmful content. Finally, the overall quality provides a generalized evaluation encompassing all these aspects. This structured breakdown aids in systematically determining how well the LLM aligns with nuanced human preferences, supporting the transparency and interpretability of the AI decision-making process.
} Naturalness
{ } Truthfulness
{ } Safety
{ } Overall Quality
{ }
}
{ Id: 001 Instruction: What chemicals does L-theanine affect? Response of LLM B : L-theanine, an amino acid primarily found in tea leaves ...
Response of LLM B: L-theanine, a non-protein amino acid found in green tea ...
Criteria {
Relevance {
Finetuned Llama prediction: A Finetuned Llama Explanation: Assistant A provides a concise, structured summary directly addressing the user’s specific query about chemical effects, clearly outlining multiple chemicals influenced by L-theanine, making it immediately relevant.
Finetuned Llama prediction: B Finetuned Llama Explanation: Assistant B adopts a conversational and engaging tone with natural formatting, using bullet points and highlighting key chemicals, resembling human-produced explanatory content.
Finetuned Llama prediction: A Finetuned Llama Explanation: Assistant A is preferred for truthfulness because it provides balanced statements and explicitly notes when evidence is preliminary or ongoing, reflecting an accurate representation of the current scientific consensus.
Finetuned Llama prediction: A Finetuned Llama Explanation: Assistant A clearly states the safety and current limitations of scientific studies, avoiding any exaggerated claims or implications, thus presenting a safer and more responsible communication.
Finetuned Llama prediction: A Finetuned Llama Explanation: Assistant A, overall, provides a comprehensive, factual, balanced, and user-aligned answer, effectively meeting user expectations across multiple evaluation dimensions.
Our findings are competitive in the preference prediction 2025 task, ranking in second position (Team UTK). Since the study focuses on observing whether the quantized finetuned llama 3 provides better and useful results without compromising performance, further performance enhancement is out of the study’s scope. However, the training insights (configurations) and findings will serve as the evidence to explore the potential of quantized fine-tune models.
We would like to thank the University of Tennessee, Knoxville’s High Performance & Scientific Computing Team for providing us with access to the Nvidia H100 GPU for finetuning and evaluating the Llama 3 and Llama 3.1 models.
During the preparation of this work, the authors used Grammarly, and Copilot in order to: Grammar, spelling check, and cross-check. After using these tool(s)/service(s), the authors reviewed and edited the content as needed and take full responsibility for the publication’s content. interaction: Proceedings of the Sixteenth International Conference of the CLEF Association (CLEF 2025), Springer, 2025, pp. 53–72.