Objectives
McDonough School of Business instructor Aaron Brown’s objectives were twofold in his implementation of AI in the classroom: to create a comprehensive student assessment tool and to establish an AI equity framework. The assessment tool aimed to identify disparities in students’ ability to engage with AI chatbots and their comfort with the topics covered. A series of intentionally challenging questions were employed to gauge student responses before and after interacting with various AI platforms.
After engaging with the AI, students completed the second half of the assessment, which enabled a comparative analysis of their comfort levels and query effectiveness. Students then worked together to develop an AI equity framework that identifies potential biases of AI tools used within higher education, considering socioeconomic status, race, gender, and geographical factors.
Outcomes
The findings of this project highlighted significant qualitative and quantitative shifts in students’ engagement with AI tools. Those who reported higher comfort levels in the pre-assessment returned more relevant results from the AI tool, demonstrating the importance of familiarity with both the material and the technology. Students unfamiliar with AI chat clients who were more comfortable with the content were able to formulate higher-quality, directed questions after receiving guidance.
The project also found that students’ comfort levels and the quality of their queries improved markedly with structured support. Participants noted an increase in confidence and positivity after their assessments, particularly when they learned to navigate AI tools effectively. A pivotal gain from the project was students’ enhanced ability to ask clear and concise questions, leading to more accurate and useful responses from the AI that facilitated quicker research and problem-solving.
Engaging with AI literacy fostered a deeper understanding of material and concepts among students. By comparing a generic AI client with a specialized one, participants were able to discern the nuanced differences in the quality of responses, leading to improved selection skills for the appropriate AI tool. On average, students posed approximately four queries per quiz question, illustrating their evolving approach as they become more adept at crafting precise questions to AI tools.
As AI continues to advance, the integration of these tools in education signals a shift in how students learn. The ability to formulate informed queries for AI will soon become as critical as traditional research skills. This project underscores the necessity of equipping students with the competencies to navigate AI technologies effectively, preparing them for a future where such tools are integral to the learning process.
Team
Aaron Brown
McDonough School of Business
