I found the source.
The firewalld default options was changed, but not noted in the release notes 
After set NftablesTableOwner=no in /etc/firewalld/firewalld.conf, now the flush will works again.
The external flush will be needed, because firewall-cmd can only add or remove one entry from set, but not flush the set.