John Lawrence Dennis & Dolly Predovic
Internationalization of higher education promotes global citizenship via international educational experiences and it is an impressive cultural, institutional, and financial effort, and yet we still don’t have accurate metrics to measure the impact of these efforts.
Better metrics are needed so that governments, transnational organizations, higher education institutions, and NGO’s have the means to improve their planning policies for internationalization of higher education programs.
Why should we care about this? The three analytics levels.
Reason #1 – Descriptive Analytics – What happened? We must get a better understand regarding what has happened. Investments of time and money should be evaluated by the relationship between the inputs and the outputs and assessment needs to be at various levels – from the individual, institution, local, regional, country and larger transnational level. At each level we have to investigate short and long-term impact – if we had the right input and output measures. Descriptive analytics accurately describe what has happened in the past.
Reason #2 – Predictive Analytics – What will happen? Better metrics for internationalization will help motivate further investment. Essentially, with better metrics we can get a better estimate regarding the likelihood that some aspect of internationalization of higher education can produce a future outcome. Without this reliable information, the ability to predict, control, and understand the positive (or even negative) impact internationalization of higher education is limited.
Reason #3 – Prescriptive Analytics – What could possibly happen? Better metrics could also help with a much broader issue – i.e., predict multiple possible futures and allow decision makers to assess possible outcomes based on future action. Such analytics could help countries with a very concrete problem – i.e., the rapid transformation of global societies- both in terms of ethnic origin as well as religion. This rapid transformation has been coupled with a rise of populism, anti-migration and anti-Muslim rhetoric – and internationalization of higher education could be a much more integral part of a response at a prescriptive level.
In this article, we’re going to discuss how the research we are doing is part of a better descriptive, predictive and prescriptive analytics on the impact of internationalization at the individual student level.
Assumptions need to be questioned.
Research on internationalization of higher education is based on a simple assumption – it’s a net positive on the individual level. Most research therefore, sets out to prove how positive internationalization is in terms of things like intercultural competencies, some specific set of skills – most of which are social in scope (e.g., social intelligence, consensus building, open mindedness, teamwork, etc.), as well as employability.
What if internationalization – in terms of its impact at the individual level isn’t always positive? What if the impact on internationalization, in terms of its influence on specific skill sets – weren’t social, but rather cognitive?
Our research, (See this, and this) which uses game-based analytics software to measure employability by gaining insight into how students transform skills acquired during international experiences into behaviors. This software measures “employee fit” by mapping gaming performance onto those underlying psychological processes that guide behavior, thoughts and emotions. This research, while still in its beginnings is starting to shed some light on how international experiences are not always a positive. Our research is consistently demonstrating that some international experiences have a negative impact on men for some underlying psychological processes that guide behavior, thoughts and emotions – and a positive impact for women. For example, in a recent study, our data is demonstrating that for men, international internships has a negative impact on cognitive factors like learning agility (i.e., learn more easily from their mistakes), and processing speed (i.e., ability to process information quickly), but a positive impact for women.
Better metrics are needed so that governments, transnational organizations, higher education institutions, and NGO’s have the means to improve their planning policies for internationalization of higher education programs.
Our research demonstrates that:
We need to stop only looking for positive effects of internationalization. We need to accept that internationalization could have negative effects. That’s ok. Negative isn’t always bad. It could be that, with our data, the negative impact that study abroad has for women on learning agility – could very well be part of the “cause” for the positive impact study abroad has for women sensitivity to loss. Future research could very well tease that apart with snapshots taken before/after study abroad for these cognitive and social factors.
We need to stop only looking for the social influences of internationalization. Internationalization of higher education, with its various tools for assessing intercultural and global competence has a significant bent towards measuring the social impact of internationalization. For example, the Global Perspectives Inventory, which has three dimensions – i.e., cognitive, intrapersonal, and interpersonal – only measures cognitive knowing and knowledge of cultural differences. Therefore, this questionnaire doesn’t really measure the more general impact that internationalization can have on cognition. Similar criticisms could be directed towards other similar measures – from the Intercultural Development Inventory to the Global Competencies Inventory.
To do better analytics we need better inputs.
Beyond the above criticisms – we believe that we need much richer data on the inputs and then examine how they interact with the outputs associated with internationalization. For example, more needs to be known about:
Origin and destination country.
International experience duration.
International experience immersion level.
Prior international experiences.
Rich demographic information.
Without better data on the inputs and feeding those inputs into better analytical models, we will never know: What has happened? What could happen? and What should we do? Policy makers need to close the circle and report on the benefits of national and transnational higher education internationalization programs and make more informed future investments on these programs. If internationalization of higher education is to truly make a meaningful contribution to society, better data and better analytics is of paramount importance.
The above metrics needs to move beyond the level of averages of the type – 80% studied in Western Europe, but rather we need to know things like – 80% of the engineering students from Germany who had a work-related internship in Poland, increased their learning agility as compared to just 60% of similar students who had a similar internship in Austria. Richer inputs make for richer analysis – and that analysis needs to move from descriptive to predictive to prescriptive.
Consider the differences between origin and destination country. Obviously, not all international mobile experiences are the same. Germans studying in Austria are not the same as Germans studying in Croatia. The differences and similarities between origin and destination country at a cultural level could be factored into the analysis. We did just that with a recent research project where we used Hofstede’s model of cultural differences to look at say the differences between origin and destination country in terms of, for example, uncertainty avoidance – or the level of stress individuals in a culture experience in the face of an unknown future. Our research demonstrates that differences between the origin and destination country in terms of uncertainty avoidance negatively impact’s cognitive factors like processing capacity and learning agility while positively impaction intrapersonal factors like self-monitoring, need for structure and ownership and responsibility.
Similar analysis could be done regarding duration, immersion level, prior experiences as well as rich demographic information. For example, in recent research, we’ve found a net positive impact of international experiences for cognitive factors – like learning agility or quick thinking for those experiences that last longer as well as for older students. While we haven’t been able to get an excellent measure on international experience immersion level, we’re looking forward to learning more about how things like community-based learning or immersive housing experiences students can influence those underlying psychological processes that influence behavior thought and emotions.
Better descriptive, predictive and prescriptive analytics regarding the impact of internationalization requires questioning our assumptions, and it requires having a better handle on the inputs and then better analysis to look at how those inputs interact with the outputs.
Without better data on the inputs and feeding those inputs into better analytical models, we will never know:
What has happened? (Descriptive Analytics)
What could happen? (Predictive Analytics)
What should we do? (Prescriptive Analytics)
Policymakers, like the European Commission, need to close the circle and report on the benefits of national and transnational higher education internationalization programs and make more informed future investments on these programs. If internationalization of higher education is to truly make a meaningful contribution to society, better data and better analytics is of paramount importance.
A post-script word about some concepts here - i.e., outcomes and outputs. After talking with colleague Fiona Hunter, Associate Director of the Centre for Higher Education Internationalisation, it's clear that a quick word about the difference between the two is needed. Several points need to be made:
Outcomes are goal-related and are functionally the desired end-state, while outputs can (and should) be results of actions that lead to outcomes.
Outcomes can be both qualitative and quantitative, outputs are almost always quantitative.
Desired outputs can be achieved that don't necessarily lead to desired outcomes.