The Austrian Public Employment Service (AMS) commissioned the design of a prediction model that assesses jobseekers and the likelihood of their successful integration in the job market. However, the so called AMS algorithm has been met with criticism.
Is this only the result of a fear of the unknown or a lack of understanding for innovation? Could this be a step towards fairness and equal opportunities? Why shouldn’t such an automated assessment process minimize the negative effects that come with the subjective and intuitive nature of human decisions? The key question regarding automated decisions in general is whether unbiased automated decisions are possible at all. Decision algorithms are trained on the basis of data. It is not always known nor easily determinable whether the data at hand is unbiased. Moreover, the quality of a decision algorithm is often evaluated through comparison of the algorithmic results against human results. Thus, considering human results the goal to be reached. Consequently, more often than not human bias is inherent in the allegedly objective algorithms.
The AMS Prediction Model
The AMS would like to predict the likelihood of a successful integration of their jobseekers in the job market by means of a model. On the basis of this prediction short-term and long-term integration prognoses are obtained to divide jobseekers into the three categories A, B and C. In this regard, a positive short-term integration means at least 90 days of employment within the first seven months after the report of unemployment. 180 days of employment within two years after the report of unemployment is considered a positive long-term integration.
For a person in category A a successful short-term integration is predicted to be highly likely with chances of more than 66%. A person with low chances for a long-term integration in the job market (less than 25%) is classified as category C, the rest of the jobseekers are classified as category B.
The following characteristics were used for the creation of this prediction model:
- gender
- age
- nationality
- education
- caregiving duties
- health-related restrictions
- previous job
- extent of employment
- frequency and duration of previous AMS cases
- participation in AMS measures
- regional job market situation (principal residence)
Logistic Regression
The integration chances are computed using logistic regression that is dependent on the characteristics of the group of persons concerned. Logistic regression is a well-known tool in market research where it is used to define models that allow the prediction of a probability of occurrence considering the concrete characteristic of one or more independent variables. A logistic function may assume values between 0 and 1 which indicate the probability of occurrence.
The coefficients of the regression function for a concrete model are determined on the basis of a dataset. The coefficients of the regression function reflect the impact of certain independent variables on the probability of occurrence.
What would an application example look like?
A trading company would like to know with which likelihood each of their customers might buy a certain product XY in the future. To predict this likelihood the customers’ data and past purchases are analysed. The following characteristics are known for customers: place of residence, age, gender, average monthly spending at the trading company, previously purchased products, information regarding personal interests (film, music and/or literature).
Logistic regression can be employed on the basis of the customer data to predict the likelihood with which a certain customer will buy product XY in the future. In this example, the customer characteristics known constitute the independent variables with varying degrees of influence on the probability of occurrence. The regression function of this example could show that the place of residence has less impact on the probability with which product XY is bought by a customer, while age and gender influence this probability strongly. The resulting ideal customer (the customer with the highest likelihood of buying the product) is within the following target group: female, between 40 and 50 years old, interested in literature.
The AMS prediction model was built using a similar technique: AMS cases of the years 2015 and 2016 and the characteristic values of the respective jobseekers of these years were used to determine a logistic regression function. The base group consists of jobsekers with the following characteristics:
- young men,
- holding a compulsory school leaving certificate,
- Austrian citizenship,
- no caregiving duties,
- no health-related restrictions,
- registered with the AMS in a job market region of type 1 (this means in a region promising the best job opportunities),
- previously employed in a service industry,
- more than 1,028 days of employment during the past four years,
- no previous AMS case.
For a person within this base group the integration chances are predict as 52%. For characteristics deviating from the base group the integration chances are increased or decreased accordingly. The integration chances are for example increased to 59% for a person within the base group but with a higher education – holding an apprenticeship certificate instead of the compulsory school leaving certificate.
Is this Artificial Intelligence
No. Although this method aims at the automated classification of jobseekers into predefined categories, the technology employed may not be ascribed to artificial intelligence (AI). The basic principle of artificial intelligence is to derive automated decisions that are not based on an explicitly programmed decision tree. Whereas the AMS algorithm employs such a predefined sequence of decisions.
No matter if the employed technology can be considered AI or not, automated decisions may be governed by the GDPR, in particular by Article 22 of the GDPR.
Article 22 of the GDPR states:
” The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.”
This article entitles a person to not be subjected solely to a fully automated decision in case this decision legally or similarly significantly affects the person. Access to employment and thus the sphere of action of the AMS algorithm is explicitly mentioned as a similarly significant scenario to a legal effect in discussions and interpretations of this article.
Computer says no!
The AMS announced that automated decisions using their prediction model will be checked and corrected by the AMS employees. However, studies (for example conducted by the Harvard Business School [5]) showed that we humans have I high tendency to trust automated decisions and we are easily influenced by them. This means an AMS employee would need to have a sceptical view on the algorithm already, in order to question its decisions.
Biased Data
We like to assume that decisions made by machines are more objective than human decisions. Automated decisions are made on the basis of algorithms that were trained on certain data. However, we can only expect an objective decision process, if the data that builds the basis of the decision process is free of bias. Since the data is provided by us humans, we tend to (most often subconsciously) pass on our own biases together with the data.
For the AMS algorithm the base group is defined as male. This fact itself does not raise any concerns. However, by changing only the attribute gender from male to female in the characteristics of a jobseeker, the predicted score is reduced, and the chances of integration deteriorate. In an interview with the German newspaper “Die Zeit” Johannes Kopf the executive manager of the AMS commented on this fact as follows:
„The algorithm does not discriminate; it solely represents reality. “
But what if reality discriminates? Studies show that female job seekers are less likely to be successfully placed in a job than male job seekers, even when having the same qualification. The data used to determine the prediction model of the AMS algorithms most likely contains such prejudice. In this regard the AMS algorithm may indeed represent reality – including existing and well-known prejudice. Amazon observed a similar outcome when implementing an AI-based system to assess job applicants: The system was trained using data from past applications that lead to a successful employment within Amazon. However, the majority of Amazon employees is male, in particular when considering senior positions. The AI-based systems adapted to this reality and preferred male applicants over female applicants.
In general, trying to derive more objective decisions through the employment of AI technology is a very welcome aim. Nevertheless, without just as objective data as a basis for AI systems we will not reach this goal. Whenever we represent existing bias in an automated system, we do not weaken this bias, on the contrary, it then acts as basis of decision-making. In this way, antiquated beliefs are set in stone (or in this case bits and bytes) and remain in force for future generations.
References
[1] Die Zeit | Digitale Verwaltung: Das Amt und meine Daten
[2] Der Standard | Jobchancen-Berechnung: Testen Sie einen der 96 AMS-Algorithmen
[3] Der Standard | Leseanleitung zum AMS-Algorithmus
[4] Hochschule Luzern | Ressourcen für empirische Methoden: Logistische Regression
[5] Harvard Business School | Algorithm Appreciation: People Prefer Algorithmic To Human Judgment
[6] Focus | Künstliche Intelligenz erachtet Bewerbungen von Frauen als minderwertig – Amazon muss reagieren