Detection of Credit Card Fraud with an Autoencoder
Detection of Credit Card Fraud with an Autoencoder
A guide for the implementation of an anomaly detector
Do you want to know how to create an anomaly detector using Python and TensorFlow? Then this article is for you. Credit card companies use anomaly detectors to detect fraudulent transactions. It is important to identify fraudulent transactions so that customers do not have to pay for something they did not buy.
Many credit card transactions take place every day, but very few transactions are fraudulent. The fraudulent transactions are anomalies. The article presents an implementation of an autoencoder model to detect these fraudulent transactions. First, we define an anomaly and introduce different types of anomalies. Then we describe the implementation of the anomaly detector for credit card fraud detection. Let’s begin!
Anomaly detection in general
An anomaly detection algorithm identifies novel and unexpected structures in acquired datasets. There are many definitions of an anomaly in the literature. We derive a definition for our use case.
Anomaly definition
Chandola et al [1] describe anomalies as patterns in data that do not conform to a well-defined notion of normal behaviour. Another widely used definition comes from Hawkins. Hawkins [2] describes an outlier as an observation that deviates from other observations to such an extent that it is suspected to have been generated by some other mechanism. Concerning the definitions presented, two essential aspects should be noted (cf. [3]):
The distribution of the anomalies deviates strongly from the general distribution of the data.
The majority of the data are normal observations, and the anomalies are only a small part.
We define an anomaly as follows:
An anomaly is an observation or a sequence of observations that differ significantly from the majority of the data in distribution.
Types of anomalies
We can basically distinguish three types of anomalies.
A punctual anomaly or point anomaly is when an observation deviates significantly from the rest of the data [3] and only lasts for a short time [4]. Fraudulent transactions can lead to point anomalies.
A collective anomaly is a collection of observations that are abnormal compared to the rest of the data. Individual observations can appear as abnormal or as normal, only the occurrence in a group makes them appear abnormal [4]. You can only detect collective anomalies in data where the individual observations are related.
A contextual anomaly describes an observation or several observations that appear abnormal in a specific context [3]. These anomalies, when considered globally, lie within the range of values valid for this variable [4].
In this article, we develop an autoencoder model that can only detect point anomalies. There are also more advanced Autoencoder models, such as GRU or LSTM Autoencoders, which include the temporal component in the data.
Anomaly detection
There are two options for the output of anomaly detection methods:
Anomaly Score: Deviation of an observation from the expected value.
Binary Label: Normal or abnormal observation (Label: 0 or 1).
Some algorithms directly have a binary label as output, and others calculate the label based on the anomaly score over a certain threshold. Thus, you can derive the label from the anomaly score. [4]
In the following, you see the function for the anomaly score (cf. [3]):
In the equation, γ denotes the anomaly score, x_t an observation at time t. n is the number of observations, and p is the number of variables/features. You can convert the anomaly score into a binary label (normal or abnormal) by defining a threshold value δ ∈ R.
The equation shows that you can adjust the binary label according to the threshold value δ. The implementation in this article uses a binary label (0: no fraud and 1: fraud).