Ok, I got helped, it worked with a VPN.
As expected it had nothing to do with IPv6, the server is just missing vapid keys. So here is a PSA for all Mastodon admins: Make sure your server has vapid keys set, or Tusky 19.0+ (and likely all other apps that use push notifications) will not work with it. How to generate those keys is explained in the Mastodon documentation: https://docs.joinmastodon.org/admin/config/
@lanodan I don't know I never ran an instance
@lanodan @ConnyDuck I don't know what they are but I have 'em set up apparently so I guess mastodon warns you somewhere
@ConnyDuck I can't tell if vapid keys are an actual thing or if you're just making fun of them.
@evilroda @ConnyDuck they exist