Large language models provide an opportunity for reliable and eficient information retrieval evaluation methods. However, current evaluation metrics fall short in accurately assessing the information content of systems' responses-without resorting to expensive human judgments. In contrast, the EXAM++ Answerability Metric leverages a bank of query-related exam questions to quantify relevant information content that is covered in the systems' responses. The process involves (1) decomposing the query into detailed questions, (2) checking each for answerability with passages in the system response, and (3) devising evaluation metrics based on this information. Using the TREC Complex Answer Retrieval benchmark, we demonstrate that our LLM-based EXAM++ approach works successfully, outperforming several established baselines. In particular, we take a deep dive into diferent approaches to determine the answerability of questions in a given passage, including the use of question answering systems with answer verification and self-rated answerability determination. 1 Large Language Models (LLMs) can generate and/or retrieve responses for search queries, resulting in many systems that combine traditional retrieval with neural ranking and natural language generation. Ideally, the systems' responses cover relevant information content while being concise and complete. However, there is a need for convincing evaluation metrics to assess the accuracy and completeness of the information content in responses. This should be accomplished in a repeatable and reusable manner and without resorting to expensive human judgments. To address this scenario, Sander and Dietz [1] proposed the EXAM Answerability Metric, which evaluates retrieval/generation systems based on whether they retrieve passages that answer a set of query-specific exam questions. Given a test bank of exam questions, they automate the work-intensive part of scanning each passage for answers using an automated question answering system.
• supporting the development of exam question banks with prompt-based generation,
• modernizing the question answering system with the recently released FLAN-T5 family [
A strength of EXAM++ is that, in contrast to other work on LLM-based relevance grading, we can readily integrate humans into the evaluation by having them manage the design of the test bank of exam questions. The test questions should be designed to cover all relevant facets of a query, so that the more questions are addressed, the more relevant a passage is. Based on the long history of classroom education and exam design, we argue it is more natural for human judges to control the design of exam questions than to directly provide relevance judgments.
By virtue of automating the grading of system responses, human judges are never required to perform passage-level relevance assessments. At the same time, humans are fully in control of defining which information content is relevant via the exam question bank. 1
The evaluation approach yields reusable test collections that can be expanded by modifying the question bank at any point in the evaluation process, as the remaining pipeline is fully automated. The impact of a question bank modification can be directly observed by listing passages whose relevance grade would change.
Contributions. In this paper, we provide an in-depth study analyzing diferent choices of
the EXAM++ approach: Automatic vs. manual test banks, predicted relevance labels with
traditional evaluation metrics vs. coverage-based measures, impacts of fine-tuning vs. prompt
engineering, grading via self-ratings vs. via question answering systems with diferent answer
verification approaches. While EXAM++ is identical to the question-based RUBRIC evaluation
method [
We focus on an approach that does not require passage-level relevance judgments or source texts. Our work is unique in this regard, but aspects relate to many active branches of research, which we detail below.
In contrast to our approach, several LLM-based evaluation approaches attempt to directly imitate the relevance judgment process.
Sun et al. [
In 1SLs, MacAvaney and Soldaini [
However, several critiques have been raised about using LLMs for producing relevance
labels in general. Faggioli et al. [
The idea of anchoring an evaluation on a bank of test questions has been widely discussed in
literature on summarization [
Many approaches use a Cloze-style approach to generate questions from a given gold
summary or source text. Questions can be in the form of multiple-choice questions [
As it pertains to information retrieval evaluation, the problem with generating questions from a given source text or gold summary is that (1) such a gold standard is usually not available and (2) it is unclear which of these questions relate to relevant information in the gold summary (or source text).
The original EXAM method avoids this problem altogether by asking a human to design
questions that address the search query. In contrast, we propose to automatically generate
questions directly from the query, building on the world-knowledge of GPT [
The original EXAM method [
which topics for the TREC CAR Y3 evaluation were derived. Furthermore, they use a custom question answering system that is optimized to answer multiple-choice questions in the style of TQA questions.
Their approach considers each passage retrieved by a system submitted to TREC CAR Y3 and uses the automated question answering system to extract answers for all test questions. Each of these answers is verified against the answer key for each exam question—tracking correctly answered questions. The system’s evaluation score is based on the set of questions that is correctly addressed with any of the top 20 passages—averaged across all queries. The more questions can be answered, the higher the EXAM evaluation score for that system.
The original EXAM method relies solely on humans to design the exam, with the intent that only a human could identify the core questions that would need to be addressed in a relevant answer. This is in contrast to approaches that generate questions from a gold summary (detailed in Section 2.2), which might lead to questions derived from non-relevant aspects mentioned in relevant text.
In this work we explore a modernized version of Sander’s EXAM Answerability metric, which we call EXAM++.3 Akin to Sander’s method, we use a bank of exam questions to grade systems based on the set of questions that can be correctly answered with information in the system’s response. The more questions can be answered with the system’s response, and the more passages answer questions well, the higher the EXAM evaluation score of the system. By automating the component that determines the answerability of passages, the evaluation paradigm becomes repeatable and reusable at a reasonable cost. As a result, it can be applied to systems that retrieve passages from a corpus as well as systems that generate content with LLMs.
Our EXAM++ evaluation system assumes the following inputs: 1. A set of queries, optionally with query subtopics. 2. A set of system responses, which can come in the form of a passage ranking or a set of generated passages.
Note that the exam questions are intended to be kept secret from the retrieval/generation system, only to be used for evaluation.
The EXAM++ evaluation approach is structured into the following three phases that we
detail in the remainder of the section and depict in Figure 1.
1. Obtaining an exam question bank: A process of creating a test bank of query-specific
exam questions.
2. Grading system responses: All passages in system responses are graded using an
automated LLM-based system to determine which questions are answerable with the passage
content. For each passage, the set of answerable questions is tracked along with grades that
represent how relevant, complete, and accurate the provided answer is.
3An implementation of EXAM++ is available in the Autograding Workbench [
Our contribution difers from the original EXAM method in several important ways:
• The original EXAM method is based on manually created multiple-choice exam questions.
TREC CAR Y3 Question Bank Prompt Explore the connection between ‘{query_title}’ with a specific focus on the subtopic ’{query_subtopic}’. Generate insightful questions that delve into advanced aspects of ‘{query_subtopic}’, showcasing a deep understanding of the subject matter. Avoid basic or introductory-level inquiries. Give the question set in the following JSON format: ```json {"questions":[question_text_1, question_text_2,...]} ``` • We propose to semi-automatically generate free-text questions for each query, as described in Section 4.1.1.
B. Grading system responses: To grade each passage via the answerability of exam questions, • The original EXAM method uses a pre-neural multiple-choice question answering system with answer verification. • First, we modernize the question answering system with an LLM-based approach (Section 4.2.1). • Second, we explore the ability of LLMs to self-rate the answerability of a question with given context, without directly verifying the correctness of answer (Section 4.2.2). C. EXAM-Cover evaluation: To evaluate each IR system, • With EXAM-Cover, we follow the original EXAM method by evaluating systems according to the number of answerable exam questions (Section 4.3.1). • To improve adoption, we add a variant “EXAM-Qrels” that implements a related idea so that it is inter-operable with the popular evaluation tool trec_eval (Section 4.3.2).
4.1.1. Generating Question Banks We use a generative LLM, specifically ChatGPT, to automate the creation of free-text questions 4 that are directly tailored to the needs of information retrieval (IR) tasks and specific domain requirements. This approach allows a larger information to be broken down need into insightful and relevant questions that probe deeply into the nuances of each query, enhancing the depth and quality of the question banks. With application to TREC CAR Y3, a set of open-ended questions are generated for each subtopic, via a zero-shot prompt as detailed in Table 1.
The goal during topic development is to have a human judge ensure that essential
information about the query is covered by the question bank, and (if necessary) modify the questions
accordingly.
4This step is identical to generating question-based RUBRICs in Farzi and Dietz [
Self-rating Answerability Prompt Can the question be answered based on the available context? choose one: - 5: The answer is highly relevant, complete, and accurate. - 4: The answer is mostly relevant and complete but may have minor gaps or inaccuracies. - 3: The answer is partially relevant and complete, with noticeable gaps or inaccuracies. - 2: The answer has limited relevance and completeness, with significant gaps or inaccuracies. - 1: The answer is minimally relevant or complete, with substantial shortcomings. - 0: The answer is not relevant or complete at all.
Question: {question} Context: {context} 4.1.2. Manual Question Banks Alternatively, query-specific question banks can be manually constructed from scratch. Optionally this can include a gold answer key for verification, as described in the original EXAM method, which uses such a test bank from the TQA dataset.
The grading process leverages a state-of-the-art LLM, such as the FLAN-T5-large [
Pre-processing system responses. Before grading, a judgment pool of all retrieved passages is created for eficient processing. Longer system responses are segmented into paragraph-sized passages . Each passage is given a unique identifier ( passage_id) to ensure that every part of the response can be individually traced throughout the grading process. 4.2.1. LLM-based Question Answering with Answer Checking For every passage-question pair (, ) , we ask the LLM to extract a best efort answer from the passage. We use the prompt in Table 2 (top) with a text-to-text generation pipeline.
Once answers are extracted, they are verified for correctness against the answer key. The verification process will normalize the correct and predicted answers through lower-casing, stopword removal, and stemming. We then apply a heuristic matching function where a match is considered valid if the edit distance between normalized answers is less than 20% of the length of the longer string.
Occasionally the LLM will respond with an expression indicating that the question is unanswerable with the provided context. We count an answer as incorrectly answered (grade 0) when we encounter an ill-formed answer (such as “a.” or “(iii)”) or one of the following expressions: “unanswerable”, “no”, “no answer”, “not enough information”, “unknown”, “it is not possible to tell”, “it does not say”, or “no relevant information”.
Variation: SQuAD2 fine-tuning. We study the impacts of fine-tuning the question
answering system using the SQuAD2 dataset [
tion remains a technical challenge. Noticing that many correct answers are missed because they are phrased diferently, we additionally explore asking the LLM to verify the answer match. We verify with the prompt in Table 2 (middle) by providing the extracted answer, the gold answer, and the question.
We manually analyzed the accuracy of this verification step, based on extracted and correct
answers. To give an example from TQA exam question L_0016/NDQ_000615 “During very wet
times, the water table will...” for which the correct answer is “rise”, this LLM-based process
identifies additional answers including “increase”, “rising”, “be higher”, “increase substantially”
as well as answers that restate the question such as “During very wet times, the water table
will rise.”
4.2.2. Grading by Self-rating Answerability
Given the technical challenges of answer verification we explore an easier alternative. We use
an answerability system introduced as RUBRIC in Farzi and Dietz [
Given each passage-question pair, the LLM rates the answerability on a scale from 0 (worst) to 5 (best) using the prompt provided in Table 2, bottom. In cases where the LLM does not provide a numerical rating, we default to a rating of 1 for answered questions—with the exception of answers that denote unanswerability (as in Section 4.2.1) for which we assign a grade of 0.
This method enables an autonomous assessment of answerability and relevance, avoiding technical issues of answer verification when there are diferent ways to phrase a correct answer or if there are diferent answers that are equally correct. Moreover, this supports the use of open-ended questions for evaluation.
The output of the grading phase is, for each passage-question pair (, ) , a grade that represents the relevance, completeness, and accuracy with which the question is addressed. The grade is 0 if the passage does not address the answer. For question answering with answer verification, the grade is either 1 (if correct) or 0 otherwise. In addition, we track the extracted answer to support manual verification via human judges.
4.3.1. EXAM-Cover Evaluation We incorporate a coverage-style evaluation metric as suggested by Sander et al. It quantifies the set of exam questions ∈ for the query that are covered in retrieved passages ∈ with a minimum grade level , as defined by:
EXAM-Cover ( ) =
1 | ⋃ { | grade(, ) ≥ , ∀ ∈ | | ∈ }| To avoid gaming the evaluation metric with a very long system response, the size of the passage set is limited to a fixed budget, e.g. = 20 passages. 4.3.2. EXAM-Qrels Evaluation Alternatively, we provide relevance labels for each passage facilitating compatibility with traditional IR evaluation metrics, such as implemented in the trec_eval tool. Passage-level relevance labels are obtained by mapping grades to a binary or multi-graded relevance label, EXAM-Label() = max grade(, ) ∈
The EXAM-Label allows to use established IR evaluation metrics that incorporate multigraded relevance labels (such as NDCG), or by choosing a minimum grade indicating relevance, , to control the leniency of the evaluation.
Like all relevance-label based approaches, the pool of graded passages may impact the evaluation results—therefore, as systems reveal unjudged passages, these should be graded to update the qrel files.
The downside of this EXAM-Qrels approach is that once a relevance label is determined,
the evaluation metric is unaware of which exam questions were covered. To preserve this
information, future work should explore integrating EXAM++ with intent-aware evaluation
measures such as -NDCG [
Whether EXAM-Cover or EXAM-Qrels is a more appropriate evaluation measure depends on the goals of the information retrieval application. When users are expected to stop after the first relevant passage, then we suggest evaluating with EXAM-Qrels with mean reciprocal rank. When recall is a priority, we suggest using EXAM-Qrels with R-precision or (mean-) average precision (MAP). When the emphasis is on covering diverse facets of relevance, we suggest to use EXAM-Cover. (1) (2)
We experimentally compare variations of our EXAM++ system to the original EXAM method
[
In experiments with generated question banks, we follow ”Phase 1” to obtain ten questions for each of the 721 query-subtopics across 131 queries in CAR Y3. In experiments that use the manual TQA question bank, we use all non-diagram questions with gold answer keys. In preparation for grading (Phase 2), we build a judgment pool of all passages in oficial judgments and the top 20 of all run submissions—a total of 85,329 passages.
For question generation we use gpt-3.5-turbo-instruct; for question verification and self-rating we use the FLAN-T5-large model with the text2text-generation pipeline from HuggingFace.6 We also explore fine-tuning the FLAN-T5-large on the SQuAD2 dataset. The ifne-tuned model is available on HuggingFace as sjrhuschlee/flan-t5large-squad2 to be used with the extractive question-answering pipeline.
We compare the following variations of our approach.
EXAM++: Using our generated question banks and grading with self-ratings (Sections 4.1.1 and 4.2.2).
Manual EXAM++: As previous but using manual question banks from the TQA dataset [
Manual-EXAM-QA: As previous but grading via question answering using prompts from
Table 2 (top), with word-based answer checking (Sections 4.1.2 and 4.2.1).
Manual-EXAM-Squad2: As previous but fine-tuning the grading LLM on SQuAD2 and using the question-answering pipeline of a prompt.
the extracted answers are verified with the FLAN-T5-large LLM using the answer verification prompt from Table 2 (middle).
For all these methods we compare both the EXAM-Qrels and EXAM-Cover evaluation approach. For EXAM-Qrels, we export passage-level EXAM++ relevance labels to be used with trec_eval on traditional evaluation measures. In this experiment we use measures used in the oficial TREC CAR Y3 evaluation, such as average precision (MAP), normalized cumulative discounted gain (NDCG@20), and R-precision (Rprec).
Original EXAM: Using the results provided by Sander et al [
FaggioliB_few, Sun_few: As previous but using few-shot prompts suggested for the TREC
Deep Learning track [
Leaderboard rank-correlation: The leaderboard of systems under the EXAM-Cover and EXAM-Qrels metric should be similar to the oficial TREC CAR Y3 leaderboard. This similarity is evaluated with two rank correlation measures: Spearman’s rank correlation coeficient, which measures diferences of a system’s rank on the leaderboard, and Kendall’s rank correlation which penalizes swaps of two systems on the leaderboard.
Inter-annotator agreement: High passage-level agreement between oficial judgments and our predicted relevance labels. We provide count statistics and Cohen’s inter-annotator agreement statistics.
Since Sander’s work demonstrated that ROUGE metrics are uncorrelated with leaderboard rankings, we omit the comparison here.
Significance testing. We perform a standard-error bar overlap test for Figure 2 and only describe significant diferences in the text. For leaderboard correlation results in Table 3, we consider results within ±0.05 as equally good.
EXAM++. Each evaluation method gives rise to a leaderboard of systems. Table 3 compares how well each leaderboard correlates with the oficial TREC leaderboard. Our proposed EXAM++ with minimum grade = 5 obtains overall best results for EXAM-Qrels. In many cases this approach obtains near-perfect rank correlations above 0.9. For reference, rank correlation statistics are on a range from -1 to +1, with 0 indicating no correlation.
Table 4 presents the inter-annotator agreement between manual TREC judgments and predicted relevance labels. We see that especially high self-rating grades obtain a good correlation (Cohen’s kappa of 0.38).
For leaderboards based on the EXAM-Cover metric, we also obtain strong results with EXAM++, but observe even better results using the manually created question bank (Manual EXAM++). We believe that the manual control in question bank design would only select vetted questions that represent relevance. We find that some of the generated questions are too broad, promoting systems that provide information that is not suficiently specific. Future work should focus on adjusting the question bank generation prompt (Table 1) to obtain more focused questions.
QA + answer verification. Next, we turn to EXAM++ approaches that determine relevance by verifying extracted answers from passages against gold answer keys. We find that verifying extracted answers (Manual-EXAM-QA) obtains comparable results to the self-rated answerability approach (Manual EXAM++) when used to obtain relevance labels. However, it is slightly worse when used with coverage-based metrics.
In either case, all our proposed approaches outperform the original EXAM method [
Furthermore, the few shot examples designed for the TREC DL domain (used in FaggioliB_few, Sun_few) do not generalize to the broad information needs of the TREC CAR Y3 domain. We hope that future research analyzes which of the findings on the DL collection generalize to other information retrieval use cases.
Figure 2 presents the impact of diferent evaluation methods on how systems are ranked on the leaderboard. We choose three of the best performing evaluation methods, spanning across our diferent options (marked in blue in Table 3), namely: EXAM++ MAP (grade>=5): Generated question bank, self-rated answerability EXAM++ with EXAM-Qrels, trec_eval using (mean) average precision, relevant grade ≥ 5. Manual EXAM++ Cover (grade>=1): Manual question bank, self-rated answerability
EXAM-Cover, relevant grade ≥ 1.
Manual-EXAM-Squad2: Manual question bank, using the question answering approach with answer verification on a fine-tuned LLM model, EXAM-Qrels, trec_eval using Rprecision.
Orig EXAM [
EXAM-Cover.
Oficial leaderboard (MAP): Manual TREC judgments, (mean) average precision, relevant
grade ≥ 1, as reported in the TREC CAR Y3 overview paper [
To make the system ranking behavior more visible, all systems’ evaluation scores are renormalized so that the highest scores maps to 1.0, and the lowest to 0.0. Several systems use a similar approach, leading to near identical scores on all leaderboards (including the oficial CAR leaderboard).
We find that all evaluation methods track the oficial leaderboard. Self-rating-based EXAM++ follows the shape the best. However, the higher grade cutof of = 5 leads to much large error bars in contrast to = 4 . We find that the question answering-based method Manual-EXAMSquad2 is too unspecific, assigning the same high score to two-thirds of all systems.
We find that coverage-based evaluation with Manual EXAM++ and original EXAM promote some of the low-ranking systems. With our experiment it is impossible to say whether this is due to a bias in the oficial leaderboard (which does not acknowledge coverage) or an issue with the coverage-based evaluation metric. However, the fact that two independent coverage-based implementations agree on assigning ICT-DRMMTKS a higher score, suggests that this system might indeed provide good coverage (albeit at lower precision). 1.0 e r o cS0.8 n o i t a u l va0.6 E m e t s yS0.4 d e z il a m ro0.2 n e R
While for generated banks of open-ended questions, a higher grade cutof of = 5 obtains stronger results, we observe the opposite for manual question banks taken from the TQA dataset, where a grade cutof = 1 produces best results.
In general, we remark that the appropriate self-rating levels depend on the dificulty of the question bank. Sander et al remark that questions of the TQA collection are often phrased in an obtuse way, as they are designed to encourage (human) students to closely read the text. As a result, too few passages obtain a high grade for most questions, which then results in evaluation scores that don’t distinguish between systems.
For the open-ended questions from our generated test bank, it is generally easier to obtain a high self-rating grade—especially since multiple answers can be considered reasonably relevant. Nevertheless, while for EXAM++ the grading cutof of = 5 obtains slightly better correlations than a cutof of = 4 , the large error bars for cutof = 5 (cf. Figure 2) suggest that a lower cutof might yield a more useful evaluation measure.
We analyze the set of evaluation approaches that use the manual benchmark, i.e., Manual EXAM++ and methods under QA + Answer Verification. The best correlation is achieved with self-rating methods on EXAM-Cover, obtaining a 0.959 Spearman’s rank correlation coeficient. However, when it is desired to integrate the evaluation into trec_eval, we find that answer verification approaches are strong contenders. Especially fine-tuning the FLAN-T5-large model on SQuAD2, obtains slightly better results than other methods.
Given that many correct answers are missed due to a diferent phrasing, we further explore LLM-based answer verification. However, this adaptation has strong negative efects on leader board correlation, in several cases obtaining a rank correlation of less than 0.5. We suspect that this assigns a relevant grade to too many non-relevant passages, resulting in a degradation of the leaderboard.
We illustrate our EXAM++ method on an example from the TREC CAR Y3 dataset for query tqa2:L_0384. The passage presented below was retrieved at rank 1 by the dangnt-nlp system and was assessed by TREC judges as ’MUST be mentioned’.
Passage: ID: b95bf325b7fdacac183b1daf7c118be407f52a3a The skin is the largest organ in the human body. Skin is made up of three layers, the epidermis, dermis and the fat layer, also called the hypodermis. The epidermis is the outer layer of skin that keeps vital flu ids in and harmful bacteria out of the body. The dermis is the inner layer of skin that contains blood vessels, nerves, hair follicles, oil, and sweat glands. Severe damage to large areas of skin exposes the human organism to dehydration and infections that can result in death.
TREC judgment: 3 (MUST be mentioned)
The TQA question NDQ_007535 “Outer layer of the skin?” was correctly answered as “epidermis” by this passage (highlighted in text). Under the self-rating prompt, FLAN-T5 indicates that this question can be answered in a mostly relevant way but may have minor gaps (self-rated answerability grade of 4).
A generated exam question, “What are the main components of the epidermis and how do they contribute to the structure of the skin?”, was also graded with a self-rating of 4. The corresponding extracted answer is “keeps vital fluids in and harmful bacteria out of the body” (highlighted in text). Other generated questions for this query are: 1. What are the diferent layers of the skin and their respective functions? 2. How does the structure of the skin contribute to its various functions? 3. What is the role of dermal papillae in the structure of the skin? 4. How does the structure of the hypodermis difer from the other layers of the skin? 5. What structural changes occur in the skin due to aging? 6. How does the skin’s structure contribute to its role in temperature regulation? 7. What role does the extracellular matrix play in the structure and function of the skin? 8. How does the structure of the skin influence its ability to prevent water loss and maintain hydration? 9. What structural adaptations exist in the skin of diferent animals and how do they serve their specific needs?
With EXAM++ we are proposing an alternative evaluation approach that does not merely outsource passage-level relevance determination to LLMs (or human judges). Instead, an exam question bank is created as part of topic development, envisioning that each question addresses an essential piece of information content for the query. As a result, whenever such questions are answerable with responses from a retrieval/generation system, we conclude that the system provides relevant information.
Using the TREC Complex Answer data set, we demonstrate that (1) our proposed approach
can reproduce oficial TREC leaderboards nearly perfectly; and (2) we outperform several
strong LLM-based relevance label predictors [
We believe that more research will improve the question bank generation and LLM-based
grading. Future work should study efects on the quality, cost, and satisfaction of human judges
working with the EXAM++ approach in our Autograde software [
We hope that by integrating EXAM++ evaluation metric with trec_eval, we ofer a system that can be easily adopted by future IR evaluation tracks, ofering organizers an avenue to reduce assessment costs, obtain reusable test collections for generative information systems.
Thomas [
Instruction: You are a search quality rater evaluating the relevance of passages. Given a query and a passages, you must provide a score on an integer scale of 0 to 2 with the following meanings: 2 = highly relevant, very helpful for this query 1 = relevant, may be partly helpful but might contain other irrelevant content 0 = not relevant, should never be shown for this query Question: {query_title} Passage: {context}
Answer:
FaggioliB [
Instruction: Indicate if the passage is relevant for the question. Respond with ’Yes’ or ’No’.
Question: {query_title}
HELM [
Question: {query_title} Passage: {context}
Answer:
Sun [
Instruction: Given a passage and a query, predict whether the passage includes an answer to the query by producing either ”Yes” or ”No”.
Question: {query_title} Passage: {context}
Answer:
FaggioliB_few [
Instruction: Indicate if the passage is relevant for the question. Respond with ’Yes’ or ’No’.
Passage: Its 25 drops per ml, you guys are all wrong. If it is water, the standard was changed 15 - 20 years ago to make 20 drops = 1mL. The viscosity of most things is temperature dependent, so this would be at room temperature. Hope this helps.
Question: how many eye drops per ml Answer: Yes Passage: RE: How many eyedrops are there in a 10 ml bottle of Cosopt? My Kaiser pharmacy insists that 2 bottles should last me 100 days but I run out way before that time when I am using 4 drops per day. In the past other pharmacies have given me 3 10-ml bottles for 100 days. E: How many eyedrops are there in a 10 ml bottle of Cosopt? My Kaiser pharmacy insists that 2 bottles should last me 100 days but I run out way before that time when I am using 4 drops per day.
Question: how many eye drops per ml Answer: No Passage: You can transfer money to your checking account from other Wells Fargo. accounts through Wells Fargo Mobile Banking with the mobile app, online, at any. Wells Fargo ATM, or at a Wells Fargo branch. 1 Money in — deposits.
Question: can you open a wells fargo account online Answer: No Passage: You can open a Wells Fargo banking account from your home or even online. It is really easy to do, provided you have all of the appropriate documentation. Wells Fargo has so many bank account options that you will be sure to find one that works for you. They ofer free checking accounts with free online banking.
Question: can you open a wells fargo account online Answer: Yes