neural network - keras loss jumps to zero randomly at the start of a new epoch -


i'm training network has multiple losses , both creating , feeding data network using generator.

i've checked structure of data , looks fine , trains pretty expected majority of time, @ random epoch every time, the training loss every prediction jumps say

# end of epoch 3 loss: 2.8845  

to

# beginning of epoch 4 loss: 1.1921e-07 

i thought data, however, can tell data fine , it's more suspicious because happen @ random epoch (could because of random data point chosen during sgd?) persist throughout rest of training. in if @ epoch 3, training loss decreases 1.1921e-07 continue way in epoch 4, epoch 5, etc.

however, there times when reaches epoch 5 , hasn't done yet , might @ epoch 6 or 7.

is there viable reason outside of data cause this? happen few fudgy data points causes fast?

thanks

edit:

results:

300/300 [==============================] - 339s - loss: 3.2912 - loss_1: 1.8683 - loss_2: 9.1352 - loss_3: 5.9845 -  val_loss: 1.1921e-07 - val_loss_1: 1.1921e-07 - val_loss_2: 1.1921e-07 - val_loss_3: 1.1921e-07 

the next epochs after have trainig loss 1.1921e-07

not entirely sure how satisfactory answer findings seem show using multiple categorical_crossentropy loss's seems result in super unstable network? swapping out other loss functions fixes problem data remaining unchanged.


Comments

Popular posts from this blog

python - Selenium remoteWebDriver (& SauceLabs) Firefox moseMoveTo action exception -

html - How to custom Bootstrap grid height? -

transpose - Maple isnt executing function but prints function term -