Anomaly Detection using Transformers
Ali Rıza SARAL
Anomaly, an unexpected irregularity in the continuous stream of data, holds paramount significance in error detection across diverse datasets.
Transformers, akin to language translators, convert input into output. Leveraging autoencoding, transformers can discern anomalies within expansive datasets. They possess the capability to autonomously learn patterns and signal the presence of anomalies.
1- Transformers are trained with data to replicate each input as output for every record within the vast dataset.
2- However, when encountering an irregularity, transformers fail to reproduce the same output as the irregular input due to lack of training for such occurrences.
3- Consequently, upon inability to generate identical input-output pairs, the program logs relevant data and proceeds with the remaining dataset.
To validate
these assertions, I conducted an experiment:
1-
Training involved arrays of
10 numbers, all commencing with either 1, 2, or 3.
2-
Subsequently, prediction
was executed using arrays of 10 numbers that did not commence with 1, 2, or 3.
3-
My input
training data is 5000 records of:
3 4 3 10 7 2
2 5 10 7 3 4 3 10 7 2 2 5 10 7
3 10 5 3 8 6
7 7 5 9 3 10 5 3 8 6 7 7 5 9
2 2 3 5 4 7
4 2 7 5 2 2 3 5 4 7 4 2 7 5
3 10 10 8 4
9 7 6 4 10 3 10 10 8 4 9 7 6 4 10
3 4 9 6 3 9
3 7 6 7 3 4 9 6 3 9 3 7 6 7
3 2 8 6 2 8
7 3 5 3 3 2 8 6 2 8 7 3 5 3
1 8 8 8 4 2
7 5 4 9 1 8 8 8 4 2 7 5 4 9
1 9 8 7 10 3
5 7 10 7 1 9 8 7 10 3 5 7 10 7
3 7 3 9 8 6
5 2 7 6 3 7 3 9 8 6 5 2 7 6
2 9 9 3 6 7
3 5 6 9 2 9 9 3 6 7 3 5 6 9
... etc.
My test data
is 20 records of:
9 8 3 6 3 2
4 4 10 2 9 8 3 6 3 2 4 4 10 2
7 10 10 10 8
7 8 2 8 3 7 10 10 10 8 7 8 2 8 3
9 2 2 3 8 4
6 10 8 2 9 2 2 3 8 4 6 10 8 2
10 4 5 5 4 8
10 9 6 9 10 4 5 5 4 8 10 9 6 9
10 10 6 10 9
7 5 7 8 3 10 10 6 10 9 7 5 7 8 3
9 2 2 7 7 4
8 7 10 4 9 2 2 7 7 4 8 7 10 4
10 8 5 9 2 8
6 4 9 8 10 8 5 9 2 8 6 4 9 8
10 7 3 3 4 8
5 6 6 2 10 7 3 3 4 8 5 6 6 2
7 6 3 9 4 3
9 10 4 5 7 6 3 9 4 3 9 10 4 5
7 3 4 5 2 3
2 9 8 10 7 3 4 5 2 3 2 9 8 10
8 4 2 6 3 2
6 9 4 5 8 4 2 6 3 2 6 9 4 5
OUTPUT is:
Test
0:
8 3 8
9 6 9 4 6 9 8
==
[start] 8 3 8 9 6 9 4 6 9 8 [end]
->
[start] 1 3 8 9 6 9 4 6 9 8 ANOMALY
Test
1:
7 10
10 10 8 7 8 2 8 3
==
[start] 7 10 10 10 8 7 8 2 8 3 [end]
->
[start] 1 10 10 10 8 7 8 2 8 3 ANOMALY
Test
2:
6 5 4
7 7 2 8 7 5 7
==
[start] 6 5 4 7 7 2 8 7 5 7 [end]
->
[start] 1 5 4 7 7 2 8 7 5 7 ANOMALY
Test
3:
10 4
5 5 4 8 10 9 6 9
==
[start] 10 4 5 5 4 8 10 9 6 9 [end]
->
[start] 1 4 5 5 4 8 10 9 6 9 ANOMALY
Test
4:
9 2 2
7 7 4 8 7 10 4
==
[start] 9 2 2 7 7 4 8 7 10 4 [end]
->
[start] 1 2 2 7 7 4 8 7 10 4 ANOMALY
Test
5:
7 6 3
9 4 3 9 10 4 5
==
[start] 7 6 3 9 4 3 9 10 4 5 [end]
->
[start] 1 6 3 9 4 3 9 10 4 5 ANOMALY
Test
6:
7 10
10 10 8 7 8 2 8 3
==
[start] 7 10 10 10 8 7 8 2 8 3 [end]
->
[start] 1 10 10 10 8 7 8 2 8 3 ANOMALY
Test
7:
9 2 2
3 8 4 6 10 8 2
==
[start] 9 2 2 3 8 4 6 10 8 2 [end]
->
[start] 1 2 2 3 8 4 6 10 8 2 ANOMALY
Test
8:
7 6 3
9 4 3 9 10 4 5
==
[start] 7 6 3 9 4 3 9 10 4 5 [end]
->
[start] 1 6 3 9 4 3 9 10 4 5 ANOMALY
Test
9:
7 10
10 10 8 7 8 2 8 3
==
[start] 7 10 10 10 8 7 8 2 8 3 [end]
->
[start] 1 10 10 10 8 7 8 2 8 3 ANOMALY
Test
10:
7 8 4
8 5 4 4 8 5 2
==
[start] 7 8 4 8 5 4 4 8 5 2 [end]
->
[start] 1 8 4 8 5 4 4 8 5 2 ANOMALY
Test
11:
7 6 3
9 4 3 9 10 4 5
==
[start] 7 6 3 9 4 3 9 10 4 5 [end]
->
[start] 1 6 3 9 4 3 9 10 4 5 ANOMALY
Test
12:
9 2 2
7 7 4 8 7 10 4
==
[start] 9 2 2 7 7 4 8 7 10 4 [end]
->
[start] 1 2 2 7 7 4 8 7 10 4 ANOMALY
Test
13:
9 9 8
6 4 10 3 2 4 3
==
[start] 9 9 8 6 4 10 3 2 4 3 [end]
->
[start] 1 9 8 6 4 10 3 2 4 3 ANOMALY
Test
14:
7 3 6
4 8 3 8 8 4 3
==
[start] 7 3 6 4 8 3 8 8 4 3 [end]
->
[start] 1 3 6 4 8 3 8 8 4 3 ANOMALY
Test
15:
8 7 7
10 4 8 4 7 4 6
==
[start] 8 7 7 10 4 8 4 7 4 6 [end]
->
[start] 1 7 7 10 4 8 4 7 4 6 ANOMALY
Test
16:
7 6 3
9 4 3 9 10 4 5
==
[start] 7 6 3 9 4 3 9 10 4 5 [end]
->
[start] 1 6 3 9 4 3 9 10 4 5 ANOMALY
Test
17:
6 5 4
7 7 2 8 7 5 7
==
[start] 6 5 4 7 7 2 8 7 5 7 [end]
->
[start] 1 5 4 7 7 2 8 7 5 7 ANOMALY
Test
18:
8 4 2
6 3 2 6 9 4 5
==
[start] 8 4 2 6 3 2 6 9 4 5 [end]
->
[start] 1 4 2 6 3 2 6 9 4 5 ANOMALY
Test
19:
9 9 8
6 4 10 3 2 4 3
==
[start] 9 9 8 6 4 10 3 2 4 3 [end]
->
[start] 1 9 8 6 4 10 3 2 4 3 ANOMALY
