Following our previous article, where we took a deep dive into Data Protection Impact Assessments (“DPIA”), it is crucial to recognise that any meaningful discussion on DPIA cannot be undertaken in totality without also understanding Automated Decision-Making and Profiling (“ADMP”) because ADMP is one of the key considerations in determining whether a processing activity may pose a high risk to data subjects and therefore trigger the need for a DPIA.
With the release of the Automated Decision-Making and Profiling Guideline (“ADMP Guideline”), ADMP has now emerged as a particularly important area within Malaysia’s evolving personal data protection framework. While DPIA may already be a familiar concept to many organisations, ADMP remains a relatively new concept that warrants close attention, especially given the increasing adoption of AI across business operations. As organisations increasingly use AI tools and automated systems in their operations, organisations should now also pay very close attention to ADMP, as AI is often used to analyse personal data, generate predictions or inferences, and support decisions concerning individuals, making ADMP highly relevant to many AI-enabled processing activities.
Against this backdrop, this article continues from our earlier discussion on DPIA and aims to set out the top 7 key takeaways that organisations, in-house counsel, compliance teams and data teams should take note of in relation to ADMP. In particular, we will explore how ADMP is related to DPIA, when ADMP may pose a high risk to data subjects, and why the distinction between AI-enabled ADMP and ordinary rule-based automation matters.
Key Takeaway 1: Understanding What ADMP Is
For the first key takeaway, it is important to understand what ADMP actually means.
ADMP is essentially a combination of two concepts: first, automated decision-making (“ADM”); and second, profiling (“P”). When these two concepts are considered together, they are then commonly referred to as ADMP.
Automated decision-making refers to a situation where a processing activity makes a decision through automated means, with no human involvement or only minimal human involvement.
Profiling, on the other hand, refers to a situation where a processing activity uses the personal data of a data subject to analyse, profile, and make inferences about that data subject through predictive and inferential analysis.
In simple terms, ADMP is about a system using personal data to understand, assess, predict, or infer something about the data subject, and then using that prediction or inference to make a decision concerning the data subject, without or with just minimal human involvement.
Key Takeaway 2: Automated Decision-Making and Profiling Often Go Hand in Hand
The second takeaway is that while automated decision-making and profiling are technically two different concepts, in practice, they often go hand in hand because normally the system will first use the personal data of the data subject to profile, evaluate, or make an inference about that individual, and then use that prediction or inference to make an automated decision, either without human involvement or with only very limited human involvement.
This is where the risk becomes much more significant, because if the profiling is wrong, biased, or based on inaccurate assumptions, the automated decision that follows may also be wrong or unfair.
An example is an automated university admissions system. The automated university admissions system may assess an applicant’s academic results, extracurricular activities, personal statement, and demographic information. It may then use this information to rank the applicant or decide whether the applicant should be admitted. The risk is that the system may wrongly assume that students from lower-income families, or families with lower levels of education, are less likely to succeed, and may even treat indicators of socioeconomic disadvantage as negative predictors of future performance. This could result in capable students being scored lower or rejected, not because of their actual ability, but because of assumptions made based on their background.
This illustrates the real concern with ADMP, as the issue is not only that the decision is automated – the deeper concern that we simply cannot ignore and discount is that the decision may be based on prediction, inference or profiling that is inaccurate, unfair or simply biased.
Key Takeaway 3: ADMP Can Trigger the Requirement to Conduct a DPIA
The third takeaway is to understand how ADMP is related to DPIA, as ADMP is one of the key factors in assessing whether a DPIA should be conducted.
A company is under a legal obligation to conduct a DPIA where the processing activity carried out by the company is likely to result in a high risk to the personal data protection of the data subject.
In assessing whether that high-risk threshold is met, one of the key factors to consider is whether the processing activity involves ADMP, and more specifically, if the ADMP poses a high risk to the data subject, then a DPIA would need to be conducted.
This is why ADMP can be understood as a legal extension of DPIA, as it is not merely a standalone concept, but one of the important qualitative factors that organisations must consider when assessing whether a particular processing activity should be subject to a DPIA.
Key Takeaway 4: ADMP May Pose a High Risk Where It Results in Legal Effects or Significantly Affects the Data Subject
The fourth takeaway is to understand when ADMP may pose a high risk to the data subject, thereby triggering the requirement to conduct a DPIA.
ADMP will pose a high risk to the data subject, which will then trigger the need to conduct a DPIA, where the outcome of the ADMP may:
- i) result in legal effects concerning the data subject; or
- ii) significantly affect the data subject.
Put simply, the real concern here is not merely that a machine has made an automated decision without meaningful human involvement, but the deeper concern is that such automated decision may materially alter the opportunities, rights, and actual lived reality of the data subjects. Hence, where an ADMP activity is capable of producing at least one of these outcomes, (i) either resulting in legal effects concerning the data subject, or (ii) significantly affecting the data subject, it may pose a high risk to the data subject and trigger the requirement to conduct a DPIA.
Key Takeaway 5: ADMP May Result in Legal Effects Where It Affects the Legal Status or Legal Rights of the Data Subject
The fifth takeaway concerns the first limb, which is where the ADMP may result in legal effects concerning the data subject.
In simple terms, this refers to a situation where the outcome of the ADMP may affect the legal status or legal rights of the data subject. In other words, the automated decision is not merely informational or administrative in nature, but it has a real legal consequence for the individual.
An example would be an automated credit approval system used by a bank or financial institution.
A bank may use an automated system to analyse a customer’s personal data, such as income, repayment history, existing debts, and credit records, to decide whether the customer qualifies for a loan or other credit facility. If the system automatically approves or rejects the application without meaningful human involvement, the decision may have a legal effect because it determines whether the customer can enter into a financing agreement with the bank. This can be problematic and risky if the system rejects an application based on other information, such as the applicant’s postal or residential address, by predicting that people living in a particular area have poor creditworthiness or a higher risk of default. Such assumptions may be inaccurate and unfair, especially where the applicant’s actual financial position and ability to repay do not support that conclusion. As a result, the applicant may be wrongly denied financing, which can seriously affect the applicant’s legal financing status or rights.
Key Takeaway 6: ADMP May Also Pose a High Risk Where It Has a Significant Effect on the Data Subject
The sixth takeaway concerns the second limb, which is where ADMP may pose a high risk to the data subject if it has a significant effect on the data subject.
This may include situations where the outcome of the ADMP has the potential to:
- i) significantly affect the circumstances, behaviour, or choices of the data subject;
- ii) have a prolonged or permanent impact on the data subject; or
- iii) at its most extreme, lead to the exclusion or discrimination of the data subject.
An illustration of this is a university admissions system. The system may assess an applicant’s academic results, extracurricular activities, personal statement, demographic information, and family background. It may then use this information to predict the applicant’s likelihood of academic success and decide whether the applicant should be admitted. This can be problematic and risky if the system generates predictions or inferences that students from lower-income families, or families with lower levels of education, are less likely to succeed academically. Such predictions may not accurately reflect the individual applicant’s actual abilities, potential, or circumstances, and this may significantly affect the data subject because the applicant may lose the opportunity to enter the university, which directly affects the applicant’s education pathway, future choices, and life circumstances.
The second example, which may have a prolonged or permanent impact on the data subject, is a recruitment screening system. The system may analyse a candidate’s education background, work experience, employment gaps, and other personal data to predict whether the candidate is suitable for the role. Based on that prediction, the system may automatically reject the candidate or rank the candidate lower without meaningful human review. This can be problematic and risky if the system automatically treat candidates with career breaks as less suitable for employment, even where there are legitimate reasons for the break, such as caregiving responsibilities, medical issues, or further studies, and this may have a long-term impact on the data subject because repeated automated rejections can affect the person’s job opportunities, income, career growth, and financial position.
A third example, which illustrates a situation where ADMP may lead to the exclusion or discrimination of the data subject, is an insurance underwriting system. The system may analyse a person’s personal data to predict the person’s risk profile. Based on that prediction, the system may automatically refuse coverage, charge a higher premium, or exclude certain benefits. The risk is that the system may rely on inferences that indirectly disadvantage certain groups, such as persons from particular socioeconomic profiles, for example, the system may assume that individuals from certain socioeconomic backgrounds have higher health risks and classify them as higher-risk applicants. This may lead to exclusion or discrimination because the data subject may be denied insurance, offered less favourable coverage, or excluded from certain benefits based on inferred risk factors rather than an assessment of their actual circumstances.
Key Takeaway 7: The ADMP Guideline Is Focused on AI-Enabled ADMP, Not Every Rule-Based Automated Process
The seventh and final takeaway is that not all automated decision-making or profiling activities necessarily involve AI.
From the illustrations above, the relevant ADMP activities involve the use of AI to carry out profiling, prediction, inference, and subsequent automated decision-making. However, it is important to note that some automated decision-making or profiling activities may operate purely on a rule-based basis, without any AI-driven prediction, inference, or model-based assessment.
For example, an insurance company may have a fixed internal rule that automatically rejects an applicant if the applicant has a prior cancer record. In that situation, the system is not using AI to predict the applicant’s future health condition or infer the applicant’s risk profile, but it is simply applying a pre-determined rule to the information provided and generating an automatic outcome. This is still ADMP in a general sense, but it is not AI-driven ADMP.
This distinction is very important because the ADMP Guideline makes it clear that it applies only where AI is used for the processing of personal data involving ADMP. In other words, the focus of the Guideline is not every form of ADMP, but AI-enabled ADMP, particularly where AI is used to analyse personal data, generate predictions or inferences, and then automate the decision-making without human involvement.
Closing Thoughts
In short, if your organisation uses AI in any processing activity involving personal data, understanding the ADMP Guideline is no longer optional, it becomes essential, as the ADMP Guideline is specifically focused on AI-enabled ADMP, particularly where AI is used to analyse personal data, generate predictions or inferences, and support or make decisions concerning individuals.
As AI becomes increasingly embedded in business processes, organisations should assess whether their use of AI falls within the scope of the Guideline and whether a DPIA may be required.
If you have any questions on personal data protection, ADMP, DPIA or AI governance, please feel free to reach out to the partners in our Technology Practice Group, Ong Johnson and Lo Khai Yi, for a consultation. We have extensive experience in assisting organisations with personal data protection, data governance, AI-related compliance and data security matters, and would be happy to assist organisations in navigating the evolving personal data protection framework in Malaysia.
The Technology Practice Group of Halim Hong & Quek continues to be recognised by leading legal directories and industry benchmarks. Recent accolades include FinTech Law Firm of the Year at the ALB Malaysia Law Awards, Law Firm of the Year for Technology, Media and Telecommunications by the In-House Community, FinTech Law Firm of the Year by the Asia Business Law Journal, a Band 2 ranking for FinTech by Chambers and Partners, and a Tier 3 ranking by Legal 500. The strength of the practice is further reflected in the individual recognition of its partners, including a Band 1 ranking for FinTech by Chambers and Partners within the Technology Practice Group.
About the authors
Ong Johnson
Partner
Head of Technology Practice Group
Fintech, Data Protection,
Technology, Media & Telecommunications (“TMT”),
IP and Competition Law
johnson.ong@hhq.com.my
◦
Lo Khai Yi
Partner
Co-Head of Technology Practice Group
Technology, Media & Telecommunications (“TMT”), Technology
Acquisition and Outsourcing, Telecommunication Licensing and
Acquisition, Cybersecurity
ky.lo@hhq.com.my.
More of our Tech articles that you should read:
