Blog Layout

Algorithms are as reliable as you make them

Irma Doze • Feb 25, 2020

The criticism of algorithms is growing. Irma Doze explains why data is our salvation, and how you should use algorithms.

Do you have the guts to use artificial intelligence to separate the good from the bad in your recruitment process or for talent management? Quite scary, but without that data, we definitely won't do better.

The results were clear when the American marketing company Dialog Direct used algorithms to recruit new employees. The organisation saved 75% on interview time and the retention increased by 7 percentage points. This way the organization saved millions of dollars. The French glass producer Saint-Gobain (with 180,000 employees worldwide) also reaped the benefits of the use of algorithms: with the help of an algorithm they spotted various internal talents that would otherwise have been left out of the picture.

No matter how beautiful those examples may sound, algorithms have stirred much discussion lately. Because how honest and fair are they? What if candidates are incorrectly rejected or promoted, based solely on data? We read more and more often that algorithms would be unreliable. Research shows that HR officials want to use HR analytics, but at the same time have huge reserves. Are algorithms actually so much better than we humans? Or worse?

The wrong decision

I understand if you find it difficult to trust data. And rightly so, because an algorithm is never 100% reliable. But neither are our brains. On the contrary, ask a group of 25 people how likely it is that 2 of them have their birthday on the same day. They will estimate that this chance is very small, but in reality, it is almost 60%. In the field of statistics, our intuition often fails us. This is demonstrated by Nobel Prize winner Daniel Kahneman in his book "Thinking, fast and slow".

When making a decision, we are partly guided by prejudices. Suppose an applicant has a C on his diploma. You know that a list of grades does not say much about someone's talent as an employee, but your brain nevertheless records this as something negative. Whether you like it or not, during the job interview, the C haunts you. Your brain automatically searches for confirmation. You see what you expect to see, and you ignore all signals that contradict that feeling.

What do you put in it?

An algorithm would not have that problem. Data does not suffer from self-overestimation or emotions. Data is neutral. It is the combination with human action that makes technology good or bad. There are 2 characteristics that you have to take into account with an algorithm:
  1. What you don't put won’t come
  2. What you put will come out
Let's start with what you put in. Suppose you are looking for a new programmer and you have algorithms search for the right candidate. You do not find age and gender relevant, so you don’t include those variables. What do you put in? You are looking for talent and you want to know how good the candidate is at work. That is why you have algorithms analyse pieces of program that the candidate has written.

Even though after this exercise you know nothing about gender, age or diploma’s of the candidate rolling out of this, you know one thing for sure: you have a programming talent! So, you hire him. But what appears after a while: this colleague does not fit into the team at all. The algorithm has not taken this into account, because: what you do not put won’t come out. You should therefore have taken that variable (match with the team) into account.

The analysis process therefore starts with an important piece of human action: ensuring that the system starts with the right variables. Sit around the table together and brainstorm freely about all the variables that could be important. Think broadly and creatively, it can be hundreds of variables. Then it's the turn of the data: it analyses which variables have the most impact on what you want to predict with the algorithm, based on statistics.

The past predicts the future

But also, at the end of the ride, human action comes into play. Because even with the data that 'comes out', you run into a problem. Algorithms always base their predictions on data from the past. That old data was generated by people. And people are prejudiced.

Take the programmer in question. Perhaps women have a different programming style than men. And that in the past you employed more men than women. Then it becomes a self-fulfilling prophecy: the programming style that evaluates the data as "good" is mainly based on the style of men. That means that the data unknowingly discriminates against gender. The data builds on human choices from the past.

Fine tuning

What should we do with that knowledge? Consider the automatic pilot of an aircraft - all algorithms. In principle, the pilot has to trust them blindly. But if his intuition says the meters are broken, he will really have to take the wheel himself.

We will have to do that too. It is therefore important to keep in mind: do not automate fully immediately, but keep checking yourself. Be critical. Analyse the data, test the algorithms for integrity. Evaluate the results, also view the candidates who did not pass the algorithm. Do you find out that the data unconsciously still discriminates? Then find out why. Then you can adjust the algorithm so that this no longer occurs in the future. Through frequent use of algorithms and analyses, we can fine tune them further and further. In this way they become even better, fairer and more reliable.

Already, algorithms select a lot fairer and fairer than the human brain. We are very aware of the few discrimination that is still in it. We evaluate, analyse, check and test. Something that is often not even done with human decisions. Previously, we were unconsciously unable. Now we are at least aware, and most of the times also competent.
body content of your post goes here. To edit this text, click on it and delete this default text and start typing your own or paste your own from a different source.

Originally published on CHRO.nl (Dutch version)

Updates

By Irma Doze 07 Mar, 2020
Our brain needs a comparison to 'evaluate' a number, to establish whether the result is good or bad. Are we supposed to be happy about spending on average 1.200 euro to recruit an employee? We don't know. That is why we love benchmarking. But benchmarking has its limits. The main disadvantage is the risk of comparing apples and pears. Of course, if you and your team are top players, the results will be motivating. If that is not the case, the results should motivate you to learn and improve. In practice, however, this situation often leads to defensive arguments as to why the benchmark is incorrect. And often rightly so. Therefore, I recommend never to use benchmark data in your internal management reports as 'targets'. The best benchmark is your own, personalized goal. So, abandon benchmarks completely? Not at all. A regular benchlearning exercise will help you learn from others and improve. Do not benchmark against 'the market' though but define exactly who you want to watch and why. And, because no two organizations are alike, adopting best practices without further ado is no guarantee of success. Only further analysis will reliably provide insight into the success factors at play within your own organization.body content of your post goes here. To edit this text, click on it and delete this default text and start typing your own or paste your own from a different source.
By Irma Doze 25 Feb, 2020
Do you have the guts to use artificial intelligence to separate the good from the bad in your recruitment process or for talent management? Quite scary, but without that data, we definitely won't do better. The results were clear when the American marketing company Dialog Direct used algorithms to recruit new employees. The organisation saved 75% on interview time and the retention increased by 7 percentage points. This way the organization saved millions of dollars. The French glass producer Saint-Gobain (with 180,000 employees worldwide) also reaped the benefits of the use of algorithms: with the help of an algorithm they spotted various internal talents that would otherwise have been left out of the picture. No matter how beautiful those examples may sound, algorithms have stirred much discussion lately. Because how honest and fair are they? What if candidates are incorrectly rejected or promoted, based solely on data? We read more and more often that algorithms would be unreliable. Research shows that HR officials want to use HR analytics, but at the same time have huge reserves. Are algorithms actually so much better than we humans? Or worse? The wrong decision I understand if you find it difficult to trust data. And rightly so, because an algorithm is never 100% reliable. But neither are our brains. On the contrary, ask a group of 25 people how likely it is that 2 of them have their birthday on the same day. They will estimate that this chance is very small, but in reality, it is almost 60%. In the field of statistics, our intuition often fails us. This is demonstrated by Nobel Prize winner Daniel Kahneman in his book "Thinking, fast and slow". When making a decision, we are partly guided by prejudices. Suppose an applicant has a C on his diploma. You know that a list of grades does not say much about someone's talent as an employee, but your brain nevertheless records this as something negative. Whether you like it or not, during the job interview, the C haunts you. Your brain automatically searches for confirmation. You see what you expect to see, and you ignore all signals that contradict that feeling. What do you put in it? An algorithm would not have that problem. Data does not suffer from self-overestimation or emotions. Data is neutral. It is the combination with human action that makes technology good or bad. There are 2 characteristics that you have to take into account with an algorithm: What you don't put won’t come What you put will come out Let's start with what you put in. Suppose you are looking for a new programmer and you have algorithms search for the right candidate. You do not find age and gender relevant, so you don’t include those variables. What do you put in? You are looking for talent and you want to know how good the candidate is at work. That is why you have algorithms analyse pieces of program that the candidate has written. Even though after this exercise you know nothing about gender, age or diploma’s of the candidate rolling out of this, you know one thing for sure: you have a programming talent! So, you hire him. But what appears after a while: this colleague does not fit into the team at all. The algorithm has not taken this into account, because: what you do not put won’t come out. You should therefore have taken that variable (match with the team) into account. The analysis process therefore starts with an important piece of human action: ensuring that the system starts with the right variables. Sit around the table together and brainstorm freely about all the variables that could be important. Think broadly and creatively, it can be hundreds of variables. Then it's the turn of the data: it analyses which variables have the most impact on what you want to predict with the algorithm, based on statistics. The past predicts the future But also, at the end of the ride, human action comes into play. Because even with the data that 'comes out', you run into a problem. Algorithms always base their predictions on data from the past. That old data was generated by people. And people are prejudiced. Take the programmer in question. Perhaps women have a different programming style than men. And that in the past you employed more men than women. Then it becomes a self-fulfilling prophecy: the programming style that evaluates the data as "good" is mainly based on the style of men. That means that the data unknowingly discriminates against gender. The data builds on human choices from the past. Fine tuning What should we do with that knowledge? Consider the automatic pilot of an aircraft - all algorithms. In principle, the pilot has to trust them blindly. But if his intuition says the meters are broken, he will really have to take the wheel himself. We will have to do that too. It is therefore important to keep in mind: do not automate fully immediately, but keep checking yourself. Be critical. Analyse the data, test the algorithms for integrity. Evaluate the results, also view the candidates who did not pass the algorithm. Do you find out that the data unconsciously still discriminates? Then find out why. Then you can adjust the algorithm so that this no longer occurs in the future. Through frequent use of algorithms and analyses, we can fine tune them further and further. In this way they become even better, fairer and more reliable. Already, algorithms select a lot fairer and fairer than the human brain. We are very aware of the few discrimination that is still in it. We evaluate, analyse, check and test. Something that is often not even done with human decisions. Previously, we were unconsciously unable. Now we are at least aware, and most of the times also competent. body content of your post goes here. To edit this text, click on it and delete this default text and start typing your own or paste your own from a different source. Originally published on CHRO.nl (Dutch version)
By Irma Doze 23 Feb, 2020
Reports often contain aggregated data. Aggregated data is data that has been merged in order to create numbers. Examples are metrics that summarize individual data, such as the average age of the employees of an organization or one of its departments. While individual, raw, data is usually data about individuals, such as a list of employees who have called in sick, all individual responses to a survey, or a list of all visitors to a website. Analytics requires individual data. The more individual your data is, the more possibilities there are for analyzing it. This is quite simple, because then you have the choice of how you’re going to aggregate the data and can draw more conclusions from it. Another reason is that with individual data, you simply have more data and can draw conclusions, such as correlations between variables, faster. Another important reason, however, to use 'raw' data instead of metrics, is the possibility of an aggregation bias. What is true for a group is not always true for an individual. The use of aggregated data can lead to erroneous conclusions. Therefore, while you use aggregated data in reports, to perform analyses it’s always preferable to have individual data if possible.
By Irma Doze 29 Jan, 2020
Happy employees make for happy customers. We see companies shifting their focus to employee experience. But how do you track the employee experience? It certainly deserves more attention than an annual measurement—one that often ends up on the shelf. Structural and data-driven improvement of the employee experience requires redesigning the way we listen to the voice of the employee. Traditionally companies focus on collecting aggregate data. This approach seems to make sense – it’s statistically accurate, high-level, and shows trending data. But it's not working! My recommendation? Start a dialogue and introduce a 'closed-loop-employee-feedback-program'! With managers actively asking their employees to provide personal feedback focusing on questions that can be acted on. While the manager takes appropriate action, the 'case' is also logged and used (together with other cases) to analyse and identify 'root causes', so the company can fix the internal issues that are causing the problems. It's not about tracking; it’s about transformation! The continuous and structured nature of actively listening to and improving the experience of employees can be decisive for the success of HR.
By Irma Doze 19 Dec, 2019
A good report or presentation is the crown jewel of an analyst’s work—as long as it communicates a clear, persuasive message. Only then HR managers, directors, and the like can easily process the insights and put them straight to use. What you need: 1. A clear structure I recommend the pyramid principles of Barbara Minto. Her method is based on the principle that, in order to understand data, our brains automatically sort it into distinctive groups of pyramids. Ideas are easier to understand if they’re structured as a pyramid of ideas so the information can be consumed more easily and rapidly. 2. Visualizations Ninety-three percent of human communication is nonverbal. I don’t mean to say that you should simply create as many graphics as possible and just add some comments. The creation of a chart or table only makes sense if it supports the story—and if the #data #visualization is good. Clever, well-thought-out charts and tables help to communicate ideas faster and more clearly though. 3. And some magic! Connect with your audience by being personal and sharing your passion for the topic. Show your proof and present clear and sharp conclusions. And, finally, don't be afraid to ask for the next step!
By Irma Doze 17 Dec, 2019
Data-driven decision-making is a new way of thinking and working for many people. Implementing change is no easy task, and for that very reason change requires clearly defined actions. To materially improve decision-making using HR analytics, you first need to make others want it. The absence of a reason for change can lead to resistance. The management must be on board and this will only happen if sufficient noise is made and the initial ambassadors demonstrate the potential value. In most cases, then, it’s advisable to begin with a few well-defined (pilot) projects with modest ambitions. An important advantage of starting small is that you can quickly demonstrate the impact of a given data-driven HR policy. This gets management’s attention and builds the trust needed to gain executive support, which is essential for HR analytics to gain a foothold in the organization. You can then expand upon the knowledge and experience acquired and create a longer-term program, which can lead to a structural data and analytics function.body content of your post goes here. To edit this text, click on it and delete this default text and start typing your own or paste your own from a different source.
By Irma Doze 11 Dec, 2019
An HR director expressed this fear to explain his hesitation concerning HR analytics. “HR is about people,” he said. “You can use numbers to analyze anything—but any discussion on this subject today places too much emphasis on analytics at the cost of the human side of #HR. You always need a ‘people person’. We need to think carefully about what we want to use HR analytics for.” Many HR directors and HR managers will recognize themselves in his words. If you have the same doubts, let me just say: this is not about choosing between HR analytics and the human touch. It’s about combining the two—because ideally, HR analytics and the human touch reinforce each other. Let me refer to the discussion that arose after the Deep Blue II chess supercomputer defeated grand-master Garry Kasparov in six matches. While the media focused on the defeat of man by machine, Kasparov himself said that the combination of a human player and a chess computer would produce a truly unbeatable team. HR-related technologies will continue to develop, and more and more often HR will be asked to implement them where possible. It’s important to strike a good balance between HR #analytics and a personal, human touch. The body content of your post goes here. To edit this text, click on it and delete this default text and start typing your own or paste your own from a different source.
By Irma Doze 07 Dec, 2019
There are many different analytical methods you can use. This can make it difficult to determine which one will best help you answer your analysis question. The hallmark of a hypothesis-driven analysis is that a statistical model is first devised based on a theory. The advantage of this sort of approach is that the results obtained are irrefutable—at least within the context of the analysis. However, “What you don’t put in won’t come out.” This is where data mining comes into play. Data mining is a way to find statistical relationships within large data sets. The analyst builds a mathematical or statistical model on the basis of insights derived from the data and creates a new theory from the ground up. It’s certainly possible to combine methods within the same analysis (that is, in parallel), but it’s also fine to do them in series. One thing to keep in mind-always- is to put predictions and correlations into perspective and to continue to think critically. Existing theories and/or logic can help you decide whether the links you find support a causal relationship or whether you should ignore it.body content of your post goes here.
By Irma Doze 04 Dec, 2019
I read a lot about Employee Experience and Employee Value (Proposition), both aimed at creating value FOR the employee. An important topic that is missing in the whole discussion, at least according to me, is the value OF the employee. Although employees are often seen as the organization’s most important resource, their value is seldom measured, and little is driven by that factor. I'm not so much interested in employees’ explicit economic value (the “balance sheet value”) as I am in the more implicit value analogous to the concept of customer value. What’s even more interesting is the worth of an individual employee and how this combines with the employee life cycle into the employee lifetime value (ELV). The use of the ELV metric offers HR the chance to make the (future) value of personnel—and thereby the personnel policy—more visible, such as by comparing the ELV before and after the implementation of a personnel tool such as training or by analyzing the impact of turnover on ELV. This can be used to guide strategic policy decisions.he body content of your post goes here. In our book you can read how to calculate ELV!
By Irma Doze 29 Nov, 2019
Just like the rest of us, managers and other leaders often tend to make decisions based on their knowledge, experience, judgment, and intuition. Most of us assume that we are capable of making consistent, balanced decisions and think that we can rationally weigh the options. But human decision-making turns out to be far less rational than we think—and organizations have the same blind spot. It’s important to realize that unconscious processes make our behavior inconsistent, random, and unpredictable. In any event, unconscious processes account for the majority of our brain activity. The sluggishness of our conscious thinking means that we often have to think about complex topics or decisions for a long time. We would rather not do that—and that often comes at the expense of the quality of the decision. Simply put, we’re wrong a lot—and that applies both to simple, everyday decisions and complex, life-changing decisions. Many experiments have shown that careful consideration leads to better answers than quick, intuitive decision-making. An example. You are asked: what is the probability that two people will have the same date of birth in a group of 25 people? When you answer this question intuitively, you soon think: less than 20%. However, if you calculate it statistically, this probability appears to be close to 60%. It should seem obvious that managers should take a different approach to assessment and decision-making. Data and analyses can help minimize the influence that our subjective, unconscious, and irrational sides have on decision-making within the organization. We can use data analysis to evaluate the actual behavior of employees in an organization, regardless of whether that behavior is rational or irrational. We can also make processes insightful, measurable, and manageable. Data analysis shows which factors, such as HR initiatives, will be influential. Organizational processes will be less subject to chance. In our book Data-Driven HR you can read more about how HR analytics will enable you to optimize your contribution to strategic HR policymaking.
More Posts
Share by: