A behind-the-scenes look at our pilot portfolio design and management online course (Part 2)
PORTFOLIO DEVELOPMENT, STRATEGIC INNOVATION
Andreas Pawelke, Learning, Systems Thinking & Digital Specialist and Aditi Soni, Service Designer, UNDP IRH
In this two-part blog series, the Innovation Team at UNDP Europe and Central Asia shares their reflections on the recent user-testing for a new online course focused on portfolio design and management. Part 1 provided an overview of the course’s structure, format, and content. In part 2, we delve into our experience with the first cohort of learners and how we’re thinking about modifying the course for future groups.
We recently finished the initial round of our portfolio design and management course and gained useful insights from it. Reviewing the user-testing phase that consisted of participant surveys and interviews, we highlight three key aspects of our first group’s experience: a major accomplishment, a significant challenge, and a critical request.
One accomplishment
In our pre- and post-course assessments, we asked participants the same set of questions during the first live session and immediately after they had completed the course. With this we wanted to get a sense of what they already knew about the content, but also measure any improvements and the overall impact of the course. We found the majority of participants improved their understanding of portfolio design and management, as evidenced by self-reported scores going from 2.3 on average at the start to 4.6 by the end (on a scale of 1-5). Overall the course was effective in helping participants better understand the value of thinking in systems, learn what the portfolio approach is about, and how to design and manage portfolios of learning options.
The user interviews we conducted with a select group of participants confirmed the results from the surveys. Colleagues praised the course for being rich in content, engaging, and interesting, making the often difficult concepts, methods, and processes accessible to those largely unfamiliar with portfolio design and management. They also appreciated the balance between theory and examples from Country Offices which provided a practical context.
What’s more, participants valued the overall structure of the course which took them from the broad conceptual understanding, encompassing the theory and mindsets behind portfolios, to more focused lessons around dynamic management, sense-making and MEL. What helped them understand and retain the learnings, according to some of our colleagues, was the mix of videos, exercises, and readings we provided for the different learning modules.
One challenge
What came as a surprise to us is that the feedback from the survey and interviews indicated a need not only to rethink the design of the three live sessions we ran over the five-week course, but also whether we need live sessions in the first place. Some colleagues expressed difficulties in fully participating in the live sessions due to the time constraints of their busy work schedules and their varying learning speeds. A few participants proposed more flexibility in the format with a mix of asynchronous and synchronous learning.
We are now considering approaches such as additional pre-recorded short video lectures and downloadable materials, and more customized quizzes to cater to different preferences and needs. These changes aim to accommodate varying schedules and learning paces of future course participants. Additionally, to maximize interaction and engagement, we’re exploring the use of peer collaboration and discussion forums. Such platforms might provide a space for participants to share ideas and feedback, thereby enriching the learning experience.
One request
From the beginning the course was designed for colleagues new to portfolios. It provides foundational knowledge on portfolio design and management, covering concepts, terms, methods, and tools to design, manage, and monitor portfolios. At the same time, it offers practical portfolio experiences from different UNDP teams.
Yet, a recurrent request we received was to go beyond the foundational knowledge and offer more advanced modules. Colleagues expressed interest in more in-depth content. Some felt the need for a deeper theoretical perspective while others were interested in more practical cases to gain a deeper understanding of what engaging with portfolios looks like in a real-world context. So, for the redesign, we’re looking into developing advanced, but optional, modules while at the same time designing customized learning journeys for different teams that might include the online course but also in-person bootcamps like the SIU’s Transforma and self-study materials like the Portfolio Primer.
Looking ahead
As we move forward, we’re enhancing the practical learning experience of the course while maintaining its introductory nature. Our plans include more tailored content, interactive exercises, quizzes, case studies, and narratives. In addition, together with the Tech Cell of UNDP Serbia, we’re building a customized AI-powered companion to provide tailored learning experiences. We have now started the redesign process and aim to release the new version in April 2024.
Contact Aditi Soni (aditi.soni@undp.org) or Andreas Pawelke (andreas.pawelke@undp.org) from the UNDP IRH Innovation Team to learn more. We’re particularly interested in hearing from people working on similar learning experiences!
—
The successful first run of the course owes much to the support and expertise of the UNDP IRH Innovation Team. We thank the first cohort participants for their valuable inputs. A special thanks to our colleagues from Country Offices, the Istanbul Regional Hub, the Talent Development Unit, and the Strategic Innovation Unit for their contributions to the course development and production.
Everyone has the right to live in cities that are comfortable, accessible, and environmentally clean, where public spaces prioritize human well-being and inclusivity. Indeed, many […]
Urban Talks is an interactive webinar series that spotlights cities and initiatives that are building new inclusive visions across Europe and Central Asia and wider, […]