Vernacular Language-Based LLM for Foundational Skills Assessment in Primary School Children

Developing an AI-driven assessment tool to evaluate primary school students’ literacy and arithmetic skills in the Odia language, enhancing educational outcomes and streamlining assessment processes. In this 8-week challenge, you will join a collaborative team of 50 AI engineers from all around the world.

The problem

In the context of primary education, traditional methods of student assessment typically involve manual processes, where teachers evaluate students through written tests. This conventional approach entails manually capturing student assessments based on teacher feedback. Such practices are not only time-consuming but also inherently subjective, leading to potential inconsistencies in evaluating student performance. Specifically, the assessment of descriptive and arithmetic answers in the Odia language presents significant scalability challenges, as it relies heavily on the availability and interpretative skills of teachers. This manual method restricts the ability to efficiently scale the assessment processes as the number of students or geographical reach increases.

Additionally, while digital platforms offer potential for automating assessments, primary school children often struggle with these technologies, necessitating the continuation of physical paper-based assessments. This situation creates a complex challenge of maintaining the traditional assessment’s benefits while enhancing its efficiency and scalability through automation.

Impact of the Problem:

• Inefficiency and Resource Intensiveness: Manual assessment processes are labor-intensive and time-consuming, requiring significant teacher involvement, which could otherwise be directed towards teaching and interacting with students.
• Subjectivity and Inconsistency: The reliance on teacher judgments can introduce subjectivity into the assessment results, leading to inconsistencies and potential biases in student evaluations.
• Scalability Constraints: Manual processes are not easily scalable, posing challenges in extending the educational assessment infrastructure to accommodate more students or integrate additional educational contexts, particularly in rural or underserved areas.
• Accessibility and Inclusivity Issues: Relying on digital solutions alone may alienate younger students or those from regions with limited tech accessibility, thereby reducing the inclusivity of educational assessments.
• Barriers to Educational Insights: Inefficiencies and inaccuracies in the assessment process can hinder the accurate tracking of educational outcomes and student progress, impacting educational planning and resource allocation.

The project aims to address these challenges by developing an automated AI-driven assessment tool that evaluates the foundational skills of primary school children in the Odia language, covering literacy and arithmetic competencies. This tool will digitally capture handwritten student responses, process these through an AI model, and automatically provide feedback on the correctness of the answers. By automating the assessment process, this solution seeks to reduce the manual burden on teachers, enhance the accuracy and objectivity of assessments by minimizing human error and bias, and improve the scalability of assessment processes.

This initiative will help streamline the assessment process, ensure consistent and objective evaluation of student competencies, and support scalable educational practices, ultimately leading to better educational outcomes and more informed educational strategies.

The goals

The ultimate objective of this project is to develop and deploy an advanced AI-driven assessment tool to evaluate the foundational skills of primary school children in the Odia language, specifically targeting literacy and arithmetic. This initiative will involve the integration of data collection, AI modeling, and user interface design to create a system that not only assesses handwritten responses but also provides a scalable and accurate method for feedback. The project will unfold over several key milestones, each planned to ensure the successful development and deployment of this transformative technology:

• Data Collection and Preparation: Collect sample papers, question sets, and government learning standards. Use a pre-trained OCR model (or finetune it) to accurately recognize Odia handwriting, establishing a robust foundation for further model development.
• AI Model Development (Arithmetic): Build and test AI models for recognizing and assessing arithmetic problems. This phase focuses on ensuring accuracy in detecting correct and partially correct answers, optimizing the models to handle various arithmetic challenges presented in student responses.
• AI Model Development (Descriptive Answers): Develop a separate model for evaluating descriptive text responses. Train this model to assess grammar, structure, and factual correctness, tailoring it to effectively evaluate more subjective aspects of student answers.
• Deployment: Build a user interface for testing and evaluating the system, integrating the AI models into a functional prototype that allows for real-time interaction and assessment. Test the full system with actual student responses to validate performance in practical scenarios.
• Final Testing and Wrap-up: Conduct final validation and accuracy testing to ensure the system meets all expected standards and requirements. Deliver the project with all necessary documentation, providing a comprehensive overview of methodologies, technologies, and outcomes.
• Stakeholder Presentation and Planning for Future Development: Conclude the project with a final review and stakeholder presentation, gathering feedback for potential improvements. 

Thus, this project aims to deliver a sophisticated AI-driven solution that revolutionizes the assessment process for primary school education in the Odia language. By providing a more efficient, accurate, and user-friendly system, this initiative promises substantial benefits in enhancing educational outcomes and operational efficiency, ultimately contributing to more informed educational strategies and improved student learning experiences.

Why join? The uniqueness of Omdena AI Innovation Challenges

A collaborative experience you never had in your working life! For the next eight weeks, you will build AI solutions to make a real-world impact and go through an entire data science project lifecycle. This covers problem scoping, data collection, and preparation, as well as modeling for deployment.

And the best part is that you will join a global and collaborative team of changemakers. Omdena AI Challenges are not a competition or hackathon but a real-world project that will take your experience of what is possible through collaboration to a new level.

Find more information on how an Omdena project works