No opportunity attack happens.
I agree with the rules interpretation of this answer. An opportunity attack occurs “…when a hostile creature that you can see moves out of your reach.” The Boggle’s withdrawal after an attack does not involve moving out of the target’s reach, so no opportunity attack is made on that basis.
Nor is this simply a technicality.
I disagree with the other answer that this is merely a technicality and that opportunity attacks could be allowed for this use of dimensional rift.
First, I observe that an opportunity attack is not just about moving out of reach. That is a necessary but not sufficient condition. The creature moving also must do so without taking the Disengage action, and without having any other feature that would limit or eliminate opportunity attacks against them (e.g. Barbarian’s third-level Eagle Spirit totem, which imposes disadvantage to opportunity attacks against the Barbarian).
There are clearly a number of specifics in the rules that offset the “moving out of reach” aspect that would normally allow an opportunity attack.
Second, it is not a logical inconsistency that an opportunity attack depends on moving out of reach but must take place before the creature is actually out of reach.
Indeed, the presence of the Disengage action makes more clear why an opportunity attack happens. It is not the movement itself, but the nature of the movement. I.e. to move away from a hostile creature in an unguarded way. The entire movement is done in an unguarded way and thus invites the opportunity attack.
The risk to the moving creature is present as soon as they try to move, thus the attack can occur before they actually move away.
The action economy is used to balance this risk, by allowing a creature to spend an action to eliminate it. Narratively, this movement is done more carefully, in a way that prevents the opportunity attack. It doesn’t mean there’s an inconsistency between the rules and the narrative movement.
Likewise, it’s easy to see that the Boggle is not really moving away per se, and does not have the unguarded aspect of their movement. They are no more moving away than a fighter is moving away after they attack with a sword when they bring their sword back close to their body. Even taking into account that the Boggle ultimately winds up inaccessible by the target of their attack, there’s no inconsistency in disallowing an opportunity attack on the Boggle as they finish their attack and move back to the other side of the rift.
See also e.g. Boggle Tactics for another unofficial interpretation along these lines:
A boggle may open a Dimensional Rift (bonus action) that allows it to reach a victim and pummel him or her through the rift (Attack action), then run away (movement). It can do so without incurring an opportunity attack, because the target can’t attack back through the rift!