AI Tool for Annotation of Meeting Recordings and Training of ML Models

Local Chapter Hyderabad, India Chapter

Coordinated byIndia ,

Mohammad Yahiya

Status: Completed

Project Duration: 25 Mar 2023 - 29 Apr 2023

Open Source resources available from this project

Github Repository

Project background.

We live in the knowledge economy and skills are your card. Some estimates state that 50% of the jobs will be affected due to automation by 2030. Non-routine cognitive skills, often described as “soft skills”, have been increasing in importance since the first industrial revolution. Soft skills have the potential to provide the only long-term competitive advantage in the job market of the future, and open up equal opportunities to culturally-diverse remote employees. At the same time, soft skills are the hardest to learn due to their abstract nature and context-dependency. The traditional coaching platforms are unscalable by design. Online learning marketplaces and MOOCs provide content that does not stick. Self-improvement apps add up to daily distractions and fail to maintain engagement.

Edtech Startup is a tech-enabled skill development tool for companies, aimed to help engineers get skills, like self-confidence, self-awareness, empathy, communication, influencing, taking ownership, with the use of technology. Edtech is going to connect all the learning tools & knowledge sources by automatically fetching user’s content, recommendations and insights into the Edtech learning pipeline. Then, it will help users act on them in the context of their life through a customizable mix of tools e.g., spaced-out notifications, self-reflection, community Q&As, speed-dating with peers. To put it in more context, we provide one source of truth around soft skills with a repository of resources, definitions and recipes on achieving goals.

The problem.

The Omdena Hyderabad Local Chapter is working on a cutting-edge AI tool for linguistic and psychometric assessment of soft skills from meeting recordings. The goal of this project is to provide personalized and quantitative feedback on the soft skills of each participant in a meeting, along with recommendations for skill development throughout their professional career.

The team has already developed a model for assessing communication skills from audio files and a framework for extracting bite-sized recommendations for various soft skills from online forums. However, to achieve a more comprehensive evaluation of a person’s abilities, it is necessary to take into consideration other important soft skills such as interpersonal skills, leadership skills, adaptability, collaboration, empathy, critical thinking, conflict resolution, and time management.

The assessment of these soft skills from meeting recordings requires the input of experts in linguistics and psychology to identify the key features that can be extracted from audio signals and then map them to the relevant soft skills for integration into the machine learning pipeline

Project goals.

- Enhance the model's capabilities by incorporating the essential soft skills with input from linguistics and psychology experts - Annotate a meeting recordings dataset with ratings for various soft skills - Train machine learning models on the annotated dataset to assess the aforementioned soft skills - Integrate the assessment pipeline with the recommendation pipeline to create a unified product.

Project plan.

  • Week 1

    Audio Signal Processing & Feature Extraction

  • Week 2

    Data Annotation

  • Week 3

    Machine Learning Techniques

  • Week 4

    Model Deployment

Learning outcomes.

– Audio based linguistic and psychometric features for evaluation of soft skills.
– Meeting recording corpus annotated with ratings for soft skills.
– Integrated soft skill assessment pipeline and recommendation pipeline.

Share project on: