Silent vulnerability fixes pose significant risks to downstream open-source software (OSS) users, as the lack of vulnerability details in fix patches leaves users unaware of potential threats. Previous work predicts the key aspects of vulnerability fixes using an encoder-decoder model to aid users in understanding these fixes. However, their approach overlooks the limited expressiveness of commit messages and the varied intents underlying code changes. In this poster, we propose a semantic-augmented method for key aspect prediction in silent vulnerability fixes. Firstly, we enrich commit semantics by incorporating information from multiple external sources. Then, we design a Chain-of-Thought (CoT) prompt to analyze code semantics at the hunk level and identify security-relevant changes. Finally, we design a task-specific embedding method to represent code difs and retrieve semantically similar commits, guiding large language models (LLMs) to predict the vulnerability type, root cause, impact, and attack vector. Experiments on our constructed dataset demonstrate that our method outperforms baselines in key aspect prediction across ROUGE-L and METEOR.
CEUR
ceur-ws.org
Commit Commit Message
Code Diff Commit
Wikipedia
Software background Github RESTAPI
Pul request Regular expression
Issue Hunk1 incorporating information from multiple external sources. Then, we design a CoT prompt to analyze code semantics at the hunk level, enabling the identification of security-relevant changes within mixedpurpose commits. Finally, we design a task-specific embedding method to represent complex code difs and retrieve semantically similar commits. These retrieved examples are combined with enriched commit messages and code dif analysis to guide LLMs in predicting the vulnerability type, root cause, impact, and attack vector.
We construct a large-scale dataset comprising 10,912 vulnerability-fixing commits and their corresponding key aspects, covering 3,575 open-source projects and 9,586 CVE records. This dataset is available at https://doi.org/10.6084/m9.figshare.29693216.v1.
Experimental results on our constructed dataset show that the proposed method outperforms baselines in key aspect prediction, achieving higher scores on ROUGE-L and METEOR.
Our task is to predict four key aspects of a vulnerability—its type, root cause, impact, and attack vector—given a vulnerability-fixing commit that includes a commit message and code dif. In this section, we present our semantic-augmented prediction framework, which consists of three components: (1) enriching the semantics of commit messages using external sources, (2) analyzing code difs at the hunk level via CoT prompting, and (3) retrieving semantically similar commits with task-specific embedding method. The results from all three components are integrated to guide LLMs in predicting the vulnerability aspects.The overall framework is illustrated in Figure 1.
Commit messages in silent vulnerability fixes are often terse and uninformative, making it dificult to
infer the underlying security context. However, we observe that many such messages include references
to external resources, such as issue identifiers (e.g., “fix #248”). These references can be leveraged
to retrieve richer contextual information. We use regular expressions to extract issue numbers from
commit messages and query the corresponding GitHub issues using the GitHub REST API. In addition,
we obtain the content of the pull request (PR) associated with the commit, which often contains more
detailed descriptions of the code changes. To further enrich the semantic context, we retrieve the
software project’s Wikipedia page as a source of background knowledge. This is because key aspects
often contain software-specific features [
Commits often include multiple code changes that serve diferent purposes, such as feature updates, refactoring, or formatting, which may not be related to vulnerability fixes. To isolate the securityrelevant portions of a commit, we analyze the code dif at the hunk level, treating each hunk as a standalone semantic unit. We design a CoT prompting strategy to evaluate each hunk. For each hunk, the CoT prompt guides the LLM through six sequential steps: identifying the commit’s overall intent, summarizing the hunk’s behavior, evaluating its alignment with the commit’s purpose, assessing its security implications, analyzing potential dependency risks, and determining whether the hunk is related to a vulnerability fix. We apply this process to all hunks within a code dif. For those identified as vulnerability-related, we collect their corresponding CoT analysis results and use an LLM to generate an overall analysis of the security-relevant code changes. This enriched, hunk-filtered code dif analysis is then provided to the final prediction stage alongside the enriched commit message.
To better leverage the in-context learning capabilities of LLMs, we adopt a retrieval-augmented few-shot
prompting approach. For each commit, we retrieve top- similar vulnerability-fixing commits from the
training set as reference examples. We construct a vector database using the commits in the training set,
where each commit is represented by an embedding that captures its security-relevant code semantics.
Specifically, we use only the code hunks identified as vulnerability-related by the method described
in Section 2.2. Within each hunk, we exclude unchanged lines and retain only the added and deleted
lines, as these are the primary carriers of vulnerability semantics. To encode the added and deleted
lines, we employ the CodeT5+ 110m embedding model [
Finally, we combine the original commit with the enriched commit message, the overall code analysis, and the retrieved similar commits. These components are concatenated to form the input prompt, which is then fed into a LLM to generate the four key aspects.
We construct a large-scale dataset of vulnerability-fixing commits to evaluate the efectiveness of our
method. We first collect all CVE entries from the NVD up to April 9, 2025. Each entry includes a CVE ID,
a textual vulnerability description (TVD), and a set of external references. For OSS projects hosted on
GitHub, the URL of a vulnerability-fixing commit typically follows the format:
https://github.com/owner/repo/commit/commit_hash. Since only a small subset of CVE entries includes such URLs, we use
regular expressions to extract entries containing GitHub commit links and and then obtain the
corresponding patch files, from which we extract commit messages and code difs. We filter out commits
with code difs exceeding 2,000 tokens to keep the dataset balanced and representative of common
vulnerability fixes. Each CVE’s TVD contains information related to the key aspects of the vulnerability.
We use DeepSeek V3 [
We compare our method against two baselines: the CodeBERT-based encoder-decoder model proposed
by Sun et al. [
Our method requires several steps of interaction with LLMs, which increases computational cost. Future work will focus on exploring optimization strategies to reduce resource consumption and improve scalability. In addition, the evaluation is currently limited to ROUGE-L and METEOR, which measure textual similarity rather than semantic correctness. Future studies will incorporate expert annotation and domain-specific metrics to provide a more reliable assessment of practical utility.
In this poster, we propose a semantic-augmented framework for predicting key aspects of silent vulnerability fixes, including the vulnerability type, root cause, impact, and attack vector. Our approach enhances commit understanding through external knowledge, hunk-level code analysis, and retrieval-based few-shot prompting. Experiments on a large-scale dataset demonstrate that our method outperforms baselines across all key aspects.
During the preparation of this work, we used ChatGPT in order to: Grammar and spelling check. After using this tool, we reviewed and edited the content as needed and take full responsibility for the publication’s content.