simple parallelism in neural networks
A simple network
with 2 parallel inputs compares two numbers and decides which one is bigger. Inputs from 2 input layers are concatenated
and then passed through a final dense layer.
I have tested the network without this final layer and saw that this
final layer increases accuracy.
Without final
layer:
Test Loss:
0.10025522857904434, Test Accuracy: 0.984000027179718
With final
layer:
Test Loss:
0.08202387392520905, Test Accuracy: 0.9919999837875366
# -*-
coding: utf-8 -*-
"""
Created on
Wed Feb 14 17:37:39 2024
@author: ars
"""
from
keras.layers import Input, Dense, concatenate
from
keras.models import Model
# Define
input shapes
input_shape1
= (1,) # Shape for the first input
input_shape2
= (1,) # Shape for the second input
# Define
input layers
input_layer1
= Input(shape=input_shape1, name='input1')
input_layer2
= Input(shape=input_shape2, name='input2')
# Define
first parallel layer
dense_layer1
= Dense(64, activation='relu')(input_layer1)
# Define
second parallel layer
dense_layer2
= Dense(64, activation='relu')(input_layer2)
#
Concatenate the outputs of the two parallel layers
concatenated_layers
= concatenate([dense_layer1, dense_layer2])
# Add an
extra dense layer
extra_dense_layer
= Dense(32, activation='relu')(concatenated_layers)
# Output
layer
output_layer
= Dense(1, activation='sigmoid', name='output')(extra_dense_layer)
# Create the
model
model =
Model(inputs=[input_layer1, input_layer2], outputs=output_layer)
# Compile
the model
model.compile(optimizer='adam',
loss='binary_crossentropy', metrics=['accuracy'])
# Print
model summary
model.summary()
#%%
import numpy
as np
# Generate
training data
num_samples
= 1000
X1_train =
np.random.rand(num_samples, 1) * 100 #
Random numbers between 0 and 100
X2_train =
np.random.rand(num_samples, 1) * 100 #
Random numbers between 0 and 100
y_train =
(X1_train > X2_train).astype(int) # 1
if first number is greater, 0 otherwise
# Train the
model
model.fit([X1_train,
X2_train], y_train, epochs=10, batch_size=32, validation_split=0.2)
#%%
# Make
predictions for numbers 12 and 34
input_data1
= np.array([[12.0]])
input_data2
= np.array([[34.0]])
predictions_12
= model.predict([input_data1, input_data2])
print("Prediction
for 12 and 34:", predictions_12)
# Make
predictions for numbers 34 and 12 (swapping the order)
input_data1_swapped
= np.array([[34.0]])
input_data2_swapped
= np.array([[12.0]])
predictions_34
= model.predict([input_data1_swapped, input_data2_swapped])
print("Prediction
for 34 and 12:", predictions_34)
#%%
# Generate
test data
num_test_samples
= 500
X1_test =
np.random.rand(num_test_samples, 1) * 100
# Random numbers between 0 and 100
X2_test =
np.random.rand(num_test_samples, 1) * 100
# Random numbers between 0 and 100
y_test =
(X1_test > X2_test).astype(int) # 1
if first number is greater, 0 otherwise
# Evaluate
the model on test data
loss,
accuracy = model.evaluate([X1_test, X2_test], y_test)
print(f"Test
Loss: {loss}, Test Accuracy: {accuracy}")
runfile('C:/Users/ars/ARStensorflow/0parallelARS/parallelLayersAddFinalLayer.py',
wdir='C:/Users/ars/ARStensorflow/0parallelARS')
Model:
"model_3"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input1 (InputLayer) [(None, 1)] 0 []
input2 (InputLayer) [(None, 1)] 0 []
dense_8 (Dense) (None, 64) 128 ['input1[0][0]']
dense_9 (Dense) (None, 64) 128 ['input2[0][0]']
concatenate_3 (Concatenate (None, 128) 0 ['dense_8[0][0]',
)
'dense_9[0][0]']
dense_10 (Dense) (None, 32) 4128 ['concatenate_3[0][0]']
output (Dense)
(None, 1) 33 ['dense_10[0][0]']
==================================================================================================
Total
params: 4417 (17.25 KB)
Trainable
params: 4417 (17.25 KB)
Non-trainable
params: 0 (0.00 Byte)
__________________________________________________________________________________________________
Epoch 1/10
25/25
[==============================] - 2s 14ms/step - loss: 0.6836 - accuracy:
0.8263 - val_loss: 0.1272 - val_accuracy: 0.9800
Epoch 2/10
25/25
[==============================] - 0s 4ms/step - loss: 0.1112 - accuracy:
0.9625 - val_loss: 0.0992 - val_accuracy: 0.9850
Epoch 3/10
25/25
[==============================] - 0s 4ms/step - loss: 0.0861 - accuracy:
0.9850 - val_loss: 0.0839 - val_accuracy: 0.9800
Epoch 4/10
25/25
[==============================] - 0s 4ms/step - loss: 0.0717 - accuracy:
0.9900 - val_loss: 0.0709 - val_accuracy: 1.0000
Epoch 5/10
25/25
[==============================] - 0s 4ms/step - loss: 0.0671 - accuracy:
0.9862 - val_loss: 0.0758 - val_accuracy: 0.9700
Epoch 6/10
25/25
[==============================] - 0s 5ms/step - loss: 0.0609 - accuracy:
0.9850 - val_loss: 0.0604 - val_accuracy: 0.9950
Epoch 7/10
25/25
[==============================] - 0s 4ms/step - loss: 0.0566 - accuracy:
0.9950 - val_loss: 0.0572 - val_accuracy: 0.9850
Epoch 8/10
25/25
[==============================] - 0s 4ms/step - loss: 0.0536 - accuracy:
0.9912 - val_loss: 0.0528 - val_accuracy: 1.0000
Epoch 9/10
25/25
[==============================] - 0s 4ms/step - loss: 0.0553 - accuracy:
0.9862 - val_loss: 0.0504 - val_accuracy: 0.9950
Epoch 10/10
25/25
[==============================] - 0s 4ms/step - loss: 0.0485 - accuracy:
0.9887 - val_loss: 0.0497 - val_accuracy: 0.9850
1/1
[==============================] - 0s 105ms/step
Prediction
for 12 and 34: [[0.00312641]]
1/1
[==============================] - 0s 34ms/step
Prediction
for 34 and 12: [[0.99910635]]
16/16
[==============================] - 0s 2ms/step - loss: 0.0456 - accuracy:
0.9960
Test Loss:
0.045580483973026276, Test Accuracy: 0.9959999918937683
