If your phone is set to use Chat Bubbles, supported apps will open a conversation in them. If that does not happen, the app is not supported. Facebook Messenger in particular has its own feature ...

Messaging "Bubbles" on Android allows conversations to float in popped-out windows that overlay your current activity. This is a convenient feature, but it can be very annoying if you don't want it.