My 34th place solution and writeup for the Bengali Character Classification Competition hosted on Kaggle by Bengali.ai

DISCLAIMER: This repo does not contain the code for the 34th place solution. Reason being that solution was a pipeline I made a month before the competition ended. So I don't have any of the code for it. I will however summarize what I can remember the solution being as well as talk about the final solution that got 201st on the private leaderboard.

Overall this competition was extremely enjoyable. I learned how Pytorch[4] works, learned how to augment data, and even teamed up for the first time! I am very glad to get my first ever medal and have a solution that scored so high on the leaderboard. At the end of this competition I learned a whole lot about public vs private leaderboard scores and learned a lot from reading the amazing summaries and solution by the top teams! Now on to my solution!

34th place solution:
This solution used a pretrained seresnext50 model with a complicated head. Trained on 128x128 preprocessed images[1] with Affine augmentations[2] and mixup/cutmix[3]. Adam optimizer with ReduceLROnPlateau.

201st place solution:
This solution was a blend of many different models trained by my teammates and I. It included 4 of my seresnext50 models, some efficientnet b4 models and some others. I won't go into detail about my teammates models.

34th place model:
All my models are very simple: pretrained seresnext50 with 3 heads for grapheme root, vowel, and consonant. For this solution view the below figure:

201st place model:
This solution used 4 seresnext50 models trained via the same pipeline on different folds. They have simple heads: Seresnext50 -> AdapdiveAvgPool2d -> Flatten -> Dropout -> Linear.

All seresnext50 pretrained weights were loaded from Pytorchcv[4]. All models loss functions depended on whether mixup or cutmix was applied to the batch (see utils/MixupCutmix.py).

Input for this competition was pretty simple. I only used the given data converted to images. I used no external data. The tough part of this competition was augmentation.

34th place input:
This solution used the 128px by 128px preprocessed images made by @lafoss[1]. For augmentation I used Affine augmentations by @corochann[2] as well as the mixup and cutmix implementation by @MachineLP[3].

201st place input:
This solution used just 137px by 236px original images. Albumentations[4] for augmentations which included only SSR, and hard Cutout (64x64 single hole). This solution also used mixup and cutmix.

I experimented a lot with training over the coarse of this competition. Different optimizers, intital learning rates, batch size, schedulers, mixup/cutmix, etc.

34th place training:
For this solution I used AdamW with initial learning rate of 0.001, ReduceLROnPlateau as scheduler, 50% of batches use mixup, 50% use cutmix, and trained for around 50 epochs. This solution was trained on Google Colab.

201st place training:
For this solution I used Adam with initial learning rate of 0.00016, ReduceLROnPlateau as scheduler, same 50/50 split of mixup/cutmix, and trained for 100 epochs. I also applied loss weights per class: [.4, .3, .3], credit to @Robin Smits for that idea (his kernal). This solution was trained on my own RTX 2080 I bought midway through this competition.

For both solutions I split the data using MultilabelStratifiedKFold from Sklearn.

For final submission my team choose a blend of many different models. This blend include 4 of my seresnext50 models.

Public LB: 0.9864
Private LB: 0.9311

However the 34th place solution was not chosen which was a single seresnext50 I trained a month before the competition ended.

Public LB: 0.9661
Private LB: 0.9407

What I should have done:

• Pay more attention to unseen graphemes
• Played with external data and fonts

This competition was a blast! I learned a lot and cannot wait to bring this knowledge to my next competition!
MANY thanks to my teammates who helped me in the final week: Corey Levinson, Parker Wang, Rob Mulla, and Balaji Selvaraj!
I am very excited to get my first competition medal and can't wait to obtain more!
Now on to the next competition and happy modeling!

My previous competition: 6DoF Car Detection Competition

My next competition: Deepfake Detection Challenge

• [1] Preprocessed 128x128 images kernal by lafoss https://www.kaggle.com/iafoss/image-preprocessing-128x128

• [2] Affine Augmentation implementation and kernal by corochann https://www.kaggle.com/corochann/bengali-seresnext-training-with-pytorch

• [3] Mixup/Cutmix with OHEM loss implementation by MachineLP https://www.kaggle.com/c/bengaliai-cv19/discussion/128637

• [4] My thanks to the contributers and makers of Pytorch, PytorchCV, and Albumentations