딥러닝 Tensorflow | fully-connected

<fully-connected 구조>

 

(1) input layer : 1차원으로 펼쳐줘야한다.

(2) hidden layer : input layer 노드와 hiddenlayer 노드를 하나하나 전부 연결한다.

(3) output layer : 마지막 hidden layer 노드에서 output layer 노드로 연결해준다.

* 모든 연결에는 weight가 할당되어있다. 

* 모든 단계에는 bias node가 추가로 할당된다.

* 수식 : y = wx + b (w:weight, b:bias)

 

---

* flatten : 1차원으로 쫙 펼쳐주는 방식 { tf.keras.layers.Flatten() }

---

 

 

 

 

 

Flatten¶

In [38]:

import tensorflow as tf

In [39]:

layer = tf.keras.layers.Flatten()
flatten = layer(output)

In [42]:

output.shape, flatten.shape
#28x28x5 = 3920 이므로, flatten 연산자를 통해 1차원으로 펼쳐진 것을 확인할 수 있다.
#배치사이즈 = 1, 이미지를 1개만 넣었기때문에 1로 설정함.
#각 이미지에 대해 flatten을 실행하므로 배치사이즈는 변하면 안된다.

Out[42]:

(TensorShape([1, 28, 28, 5]), TensorShape([1, 3920]))

 

시각화¶

In [45]:

plt.figure(figsize=(10,5))

plt.subplot(211)
plt.hist(flatten.numpy().ravel())

plt.subplot(212)
plt.imshow(flatten[:,:100], 'jet')

plt.show()
#숫자가 0에 몰려있어서 두번째 그래프는 바닥에 누워있다.

 

 

Dense¶

In [46]:

layer = tf.keras.layers.Dense(32, activation='relu')
#unit = 32 : layer에 총 32개의 노드를 만든다.

In [47]:

output = layer(flatten)
output.shape
# 3920 -> 32로 감소하였다.

Out[47]:

TensorShape([1, 32])

In [48]:

layer_2 = tf.keras.layers.Dense(10, activation='relu')
output_2 = layer_2(output)
output_2.shape
# 32 -> 10으로 감소하였다.

Out[48]:

TensorShape([1, 10])

 

Drop out¶

• 노드를 전부 연결하지 않고, 일부만 연결해서 overfitting 자제
• 학습할 때만 drop out 하고, evaluation에서는 안한다.

In [50]:

layer = tf.keras.layers.Dropout(0.7) #쓰는비율입력
output = layer(output)
output.shape
#크기변화없다.

Out[50]:

TensorShape([1, 32])

 

Build Model (실전)¶

 

In [51]:

from tensorflow.keras import layers

In [52]:

input_shape = (28,28,1)
num_classes = 10

In [55]:

inputs = layers.Input(shape=input_shape)

#1st conv block
net = layers.Conv2D(32,3,padding='SAME')(inputs)
net = layers.Activation('relu')(net)
net = layers.Conv2D(32,3,padding='SAME')(net)
net = layers.Activation('relu')(net)
net = layers.MaxPool2D((2,2))(net)
net = layers.Dropout(0.25)(net)
#2nd conv block
net = layers.Conv2D(64,3,padding='SAME')(net)
net = layers.Activation('relu')(net)
net = layers.Conv2D(64,3,padding='SAME')(net)
net = layers.Activation('relu')(net)
net = layers.MaxPool2D((2,2))(net)
net = layers.Dropout(0.25)(net)

#Fully connected
net = layers.Flatten()(net)
net = layers.Dense(512)(net)
net = layers.Activation('relu')(net)
net = layers.Dropout(0.25)(net)
net = layers.Dense(num_classes)(net) #마지막은 시작 클래스 개수와 동일하게 노드 생성
net = layers.Activation('softmax')(net)

model = tf.keras.Model(inputs=inputs, outputs=net, name='Basic_CNN')#이름은 안넣어도 됨

In [59]:

#keras 장점: 각 layer 별 summary 가능
model.summary()

 

Model: "Basic_CNN"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_2 (InputLayer)         [(None, 28, 28, 1)]       0         
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 28, 28, 32)        320       
_________________________________________________________________
activation_6 (Activation)    (None, 28, 28, 32)        0         
_________________________________________________________________
conv2d_8 (Conv2D)            (None, 28, 28, 32)        9248      
_________________________________________________________________
activation_7 (Activation)    (None, 28, 28, 32)        0         
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 14, 14, 32)        0         
_________________________________________________________________
dropout_5 (Dropout)          (None, 14, 14, 32)        0         
_________________________________________________________________
conv2d_9 (Conv2D)            (None, 14, 14, 64)        18496     
_________________________________________________________________
activation_8 (Activation)    (None, 14, 14, 64)        0         
_________________________________________________________________
conv2d_10 (Conv2D)           (None, 14, 14, 64)        36928     
_________________________________________________________________
activation_9 (Activation)    (None, 14, 14, 64)        0         
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 7, 7, 64)          0         
_________________________________________________________________
dropout_6 (Dropout)          (None, 7, 7, 64)          0         
_________________________________________________________________
flatten_4 (Flatten)          (None, 3136)              0         
_________________________________________________________________
dense_4 (Dense)              (None, 512)               1606144   
_________________________________________________________________
activation_10 (Activation)   (None, 512)               0         
_________________________________________________________________
dropout_7 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_5 (Dense)              (None, 10)                5130      
_________________________________________________________________
activation_11 (Activation)   (None, 10)                0         
=================================================================
Total params: 1,676,266
Trainable params: 1,676,266
Non-trainable params: 0
_________________________________________________________________