About
A First-of-its-kind advanced Natural Language Processing (NLP) solution in the field of personalized employee learning and development (L&D). The solution analyses unstructured employee text data, enabling precise grouping of feedback into relevant topics. This pioneering approach was achieved through integration of Latent Dirichlet Allocation (LDA) & Non-negative Matrix Factorization (NMF) techniques
Innovation presentation
Background
The introduction of this innovation was prompted by the HR department's possession of extensive textual data contained in annual appraisal survey forms. Analysing and creating a scalable solution to extract insights from this text data proved challenging. The traditional approach involved manual analysis of a sample data subset, which was far from exhaustive.
Innovative Solution
Consequently, the decision was made to embark on a pilot project aimed at developing an NLP engine capable of accurately classifying free-form text into distinct topic clusters. The primary objective was to utilize this technology for personalizing employee L&D goals based on comments by both appraisers and employees.
Cutting edge technologies
Techniques:
At the core of the innovation were two powerful techniques: LDA and NMF. By applying LDA, hidden topics within the feedback could be uncovered, identifying specific areas for improvement. NMF further refined this analysis, enabling extraction of actionable insights with remarkable precision.
Infrastructure:
Open-source Python libraries were employed on a powerful on-prem server (512 GB of RAM and 32 CPU cores) in a multi-threaded environment to execute this advanced NLP model.
Uniqueness of the project
Following are some of the USPs of the project:
Unparalleled Personalization: Through meticulous analysis of employee feedback, personalized learning recommendations were provided that cater to each employee's unique needs and aspirations. It revolutionized L&D by effectively tailoring offerings for 95% of the employees (65% attributed to employees and 30% courtesy appraisers). This solution outbids the conventional word cloud or bar plot analysis in terms of accuracy.
Optimal utilization of L&D budget: The solution provided invaluable guidance to the HR team in terms of allocating budget and resources to L&D programs that were most likely to yield the highest return on investment.
Integration with Employee Learning Platform: The output generated by this model served as an input for our newly launched MPower LMX platform. This integration allowed us to personalize the landing page of our L&D website according to each employee's specific needs, further enhancing the user experience and engagement.
Enhanced Efficiency: By automating the analysis of vast amounts of unstructured data, this solution saved valuable time and resources (TAT reduced from 2 months to a day) while ensuring that personalized growth opportunities were identified promptly.
Scalability: The scalability of the solution is noteworthy, as it can extract relevant topics from millions of records in just a matter of minutes.
