Enhancing Project Management Support through Retrieval-Augmented Generation: A PMBOK-Aligned Evaluation for Knowledge Assistance (#2723)

Authors

Jones, Eduardo

Ormeno-Arriagada, Pablo

Abstract

Accessing relevant, context-specific project management guidance in real time remains challenging, even with comprehensive frameworks like the PMBOK Guide. Traditional documentation and training resources lack adaptability under dynamic conditions, a limitation particularly critical where consistency, traceability, and standards compliance are essential. This study develops and evaluates a Retrieval Augmented Generation (RAG) system that delivers standards aligned decision support grounded in the ten PMBOK Knowledge Areas. The knowledge base integrates PMBOK summaries, ISO 21500 documentation, case studies, and open educational resources. Using Langchain, we segmented the documents into overlapping text chunks and generated embeddings with multilingual models from HuggingFace and OpenAI. The retrieval system employed FAISS (Facebook AI Similarity Search) for vector indexing, while the generation component used DeepSeek V3 0324, while generation is performed by a large language model using structured, PMBOK-aligned prompts. Evaluation combines Precision@k retrieval metrics, expert Likert-scale ratings for clarity, relevance, and alignment, and scenario-based assessments. Results indicate strong performance in stakeholder and cost management tasks, with slightly reduced completeness in complex multi-step scenarios. The findings demonstrate the system’s potential as a real-time decision support tool for project managers and highlight its applicability in both certification training and live project environments.