First of all, why playing with the SharePoint groups that way? You have 3 main ways to provide users with access to the site, using Owners, Members and Visitors (When you click on SharePoint). I wouldn’t go in the advanced mode and create more groups the way we used to do with SharePoint server, rather I’d add people I want to an Office 365 group (or Azure security group) and then invite them to my site and provide them permissions through the Share link.
This is the better way to do it, remember when you want to use more advanced stuff in SharePoint, such as Audience Targeting, it will work only with M365 Groups or Security Groups.
Keep it simple, and feel free to add these users to other M365 groups as needed (or if you prefer, Security Groups) then add them back to your site using the Share link.