My stock app stopped providing notifications with the conversation open upon update to Android Lillipop. This is probably what's happening to you as well. I agree that this would be a great fix for Chomp to do. I suggest we consolidate our votes in one place - here is the link with the most votes:
https://chomp.uservoice.com/forums/221270-feedback/suggestions/9935013-when-in-conversation-there-s-not-a-notification-s