System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Google Colab Environment
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
• TensorFlow installed from (source or binary): Binary (Colab Pre-installed)
• TensorFlow version (use command below): 2.4.0-nightly
• Python version: 3.6.9
• Bazel version (if compiling from source): N/A
• GCC/Compiler version (if compiling from source): N/A
• CUDA/cuDNN version: N/A (Colab CPU Environment used)
• GPU model and memory: N/A

Describe the current behavior
I am trying to use MobileNetV2 pretrained model for a simple classification task. Here I want to add regularization loss for MobileNetV2 - base model along with added FC layers. I have conducted 3 experiments as per attached colab notebook, where I get different results for the computation of regularization loss. The MobileNetV2 regularizer loss is computed twice in the model. And I dont know why it happens.

Describe the expected behavior
The computation of the loss in all the experiments should be same.

Standalone code to reproduce the issue
Colab Link : https://colab.research.google.com/drive/1CeKMIAq_g0AOKdakupKIeUjEhkL_TtI1?usp=sharing

Other info / logs

1. I have tried the same on TF2.2 but got the same behaviour.
2. I have got the reference of this: issue-37511 for adding regularization loss into pretrained model.