Study Finds That Claire AI® Reduces Item-Writing Time by 67% While Maintaining Item Quality

Developing a single nursing test item requires much more than drafting a question and potential answers. Faculty must determine what the item should measure, select an appropriate cognitive level, construct a clinical scenario, develop defensible answer options, and verify that the item aligns with course objectives.

Most educators repeat this process across multiple exams and courses each year. And the impacts are significant.

• Item writing takes considerable time. In a recent survey of 250 faculty, 71% said creating new questions is one of the most time-consuming aspects of exam development.¹
• Item writing requires specialized knowledge that many educators don’t yet have. Only 50% to 60% of new educators receive formal training in assessment design,^2-4 and one study found that 33% lack confidence writing NCLEX-style questions.³
• Item writing adds to an already demanding faculty workload. Creating defensible, NCLEX-style questions is one of the more labor-intensive tasks in nursing education, and it has impacts on stress levels and work-life balance.

Could the appropriate application of artificial intelligence help address these concerns?

In one of the first peer-reviewed studies to directly measure how AI-assisted item writing influences efficiency and question quality, Yoo et al. determined that it can considerably shorten the task while producing quality items for review and revision.¹  Before reviewing these 2026 findings, let’s examine the issues that fueled the research.

Why Item Development Takes So Much Time and Effort

For years, nurse educators have voiced concerns about developing test items and exams. The stress associated with these tasks can increase considerably when added to the many demands on today’s faculty, who experience high rates of burnout and attrition.

Across courses, faculty may need to develop, update and review numerous exams during a single semester. Each requires enough high-quality items to assess the relevant content and cognitive objectives. Despite these significant requirements, many faculty have little to no formal training in item writing and test construction.^2-5  

But no matter what the level of training or preparedness, item writing is a time-consuming drain on educators. That’s because writing a good exam question is a multistep process requiring, at minimum, the 8 complex steps listed below.  

When faculty feel pressed for time, item development can become reactive and stressful. That is precisely where AI-assisted item writing may offer value.

Psychometricians on the Innovative Learning & Assessments Solutions team at Ascend Learning sought to evaluate that potential value in a mixed-methods study published in the Journal of Professional Nursing in September 2026.¹ The researchers sought to determine whether targeted artificial intelligence assistance from Claire AI^® could increase the efficiency of item development without compromising quality.

Claire AI is the AI assistant developed by ATI to support nursing education. In ATI’s Custom Assessment Builder, Claire AI helps faculty generate and fine-tune NCLEX-style items and complete exams.

What the Research Determined When Faculty Used Claire AI

For this study, researchers used a three-phase mixed-methods design that combined an observed item-writing comparison, faculty interviews and a national survey.

In the first phase, nine nurse educators completed item-writing training and created questions manually and with assistance from Claire AI. The participants produced 335 items during two separate 90-minute writing sessions. Researchers measured the time required to create each item and the number of items produced.

Afterward, two nursing education specialists independently evaluated the items using five criteria: clarity, relevance, cognitive level, defensibility, and distractor plausibility.

The results documented a significant difference in the time required to develop items. When writing questions manually, faculty took an average of 10.69 minutes per item. When they used Claire AI, the average dropped to 3.49 minutes.

Using the Claire AI, the volume of questions the educators produced increased markedly: approximately three times as many items during AI-assisted writing sessions as during manual sessions.

The study also identified how AI assistance influenced the time spent on the two essential steps in item generation, drafting and revising:

• Manual: average 9.13 minutes drafting, 1.56 minutes revising
• AI assisted: average 1.46 minutes drafting, 2.03 minutes revising.

Rather than eliminating faculty work, the authors determined, Claire AI shifted more of that work from creating an initial draft to evaluating and refining one. The effect is that the educator’s role changed from creator of a first draft to the more valuable one: reviewer, editor, clinical expert, and quality assurance specialist.

With AI assistance, participants nearly tripled the number of items they generated during the item-writing sessions: an average of 28.2 items with AI assistance and 9.0 when writing manually. Key findings of the study are summarized below.

Gathering Insight Into the Faculty Experience

In phase two of the study, the researchers conducted cognitive interviews with the nine educators to get insights into their experience with manual question writing and the potential use of AI.

Participants described generating the initial idea for a question as one of the more time-consuming parts of item writing. Several expressed that having an AI-generated starting point helped reduce their “blank page” burden by providing a stem or question concept they could evaluate and improve rather than starting from scratch.

The educators described Claire AI as easy to use and identified a specific potential value for faculty who are new to item writing. They suggested that an AI-assisted tool could provide novice educators with a starting point for developing those skills.

Surveying Educators About Item-Writing Burden

The third phase of the research evaluated 250 responses to a nationwide survey about exam development and time management. The results provided new data on how time-intensive item writing can be:

• 75% of educators reported that manually writing a single exam question typically takes 15 minutes to 1 hour.
• 56% of respondents said an AI item-generation tool could help them generate an item in less than 15 minutes.
• 77% identified automated test-generation tools as a resource that could help reduce the time they spend writing entire exams.

Why Faculty Expertise Remains Essential

In addition to documenting efficacy and quality, the study highlighted an important limitation of AI-assisted item writing: the need for a human in the loop. Faculty expertise and review remain essential.

One area where this was evident is question design. Distractor plausibility is an area that requires faculty judgment. In the study, some AI-generated distractors were too obvious or implausible, and participants reported spending much of their revision time strengthening answer options.

In terms of quality scores, the AI-assisted items generated by two study participants who made little or no edits received slightly lower scores. This finding reinforces the role of AI as a support tool rather than a replacement for educator expertise.

The bottom line, this research showed, is that faculty should remain responsible for reviewing AI-generated items for content accuracy, appropriateness, cognitive complexity, defensibility, and alignment with course and program expectations.

Reclaiming Time for the Work Only Faculty Can Do

To the widespread discussions about AI in nursing education, this study contributes the finding that AI’s value is not producing a greater quantity of questions. Rather, it is creating more space for activities that require uniquely human skills. Time saved on item development potentially gives faculty more bandwidth for student feedback, curriculum improvement, clinical instruction, and other responsibilities that depend on their professional judgment.

AI can shorten the first-draft process while faculty hold the reins to ensure accuracy, relevance and quality. In that supporting role, Claire AI has the potential to help educators spend less time confronting a blank page and more time applying the clinical and educational judgment that makes an assessment defensible.

“The message from this research is not that AI replaces faculty expertise. It reinforces the opposite,” said co-author Beth Phillips, PhD, RN, CNE, a former nurse educator and program director who is now Strategic Nursing Advisor for ATI. “Tools like Claire AI can help educators create assessment content more efficiently, but nurse educators remain essential to ensuring quality, clinical relevance, and sound judgment. The real opportunity is using AI to elevate the work of faculty, not replace it.”

References

1. Yoo H, Lin Y, Attenweiler R, Hodge KJ, Miller JE, Phillips BC. Effectiveness of AI-Assisted Item Writing in Nursing Education: A Mixed-Methods Evaluation. Journal of Professional Nursing. 2026;67:284–293. https://doi.org/10.1016/j.profnurs.2026.09.001
2. Moran V, Wade H, Moore L, Israel H, Bultas M. Preparedness to Write Items for Nursing Education Examinations: A National Survey of Nurse Educators. Nurse Educator. 2022;47(2):63-68. doi: 10.1097/NNE.0000000000001102
3. Hensel D, Moorman M, Stuffle ME, Holtel EA. Faculty Development Needs and Approaches to Support Course Examination Development in Nursing Programs. Nurse Educator. 2024;49(6):315-320. doi: 10.1097/NNE.0000000000001706
4. Palazzo SJ, Levey J. Shaping the Future of Nursing Education: Next Generation NCLEX Question Writing and the Power of Psychometrics. Guest Editorial. Nursing Education Perspectives. 2024;45(2):69-70. doi: 10.1097/01.NEP.0000000000001243
5. Moore WL. Does faculty experience count? A quantitative analysis of evidence-based testing practices in baccalaureate nursing education. Nursing Education Perspectives. 2021;42(1):17-21. DOI: 10.1097/01.NEP.0000000000000754