c# – Unity Runtime Surface Snapping (Like Shift+Cntrl in Editor)

I am trying to allow the user of my VR game to move objects around using a pointer, then when holding down a button, snap to a grid and align to the highest surface, essentially replicating the Editor behavior or holding down shift+control and moving around a transform gizmo. (Try it, it’s fun!)

I have the grid part down, but can’t wrap my head around how to do the surface snapping.
Here is the code I have so far. I would appreciate any and all help!

 
// offsetPos is where the VR pointer is.
Vector3 offsetPos = pointer.objectControlPoint.transform.position + cursorOffset;
Vector3 newPos;
 
if (isSnapping) // Snap to ground code.
{
// I read previously to do this up then down thing, but It's not working as expected
    RaycastHit groundHit = new RaycastHit();
    if (Physics.Raycast(selectedObject.transform.position, Vector3.down, out groundHit))
    {
         RaycastHit objectHit = new RaycastHit();
         if (Physics.Raycast(groundHit.point, Vector3.up, out objectHit))
         {
                Vector3 snapDiff = groundHit.point - objectHit.point;
                snapYPos = snapDiff.y + (selectedObject.collider.bounds.extents.y);
          }
    }
      // worldGrid is a monoBehavior on another object, and gridCellSize is just a float
      float gridPosX = Mathf.Floor(offsetPos.x / worldGrid.gridCellSize) * worldGrid.gridCellSize;
      float gridPosZ = Mathf.Floor(offsetPos.z / worldGrid.gridCellSize) * worldGrid.gridCellSize;
      newPos = new Vector3(gridPosX, snapYPos, gridPosZ); // Sets the target position to the nearest grid cell, with a Y of the snap position.
}
else // If not in snap mode, set target position to just the VR cursor.
{
       newPos = new Vector3(offsetPos.x, offsetPos.y, offsetPos.z);
}
 
// I know lerp might not be the most efficient, but I like the smooth effect, and it looks good when snapping to the grid...
selectedObject.transform.position = Vector3.Lerp(selectedObject.transform.position, newPos, movementLerpSpeed * Time.deltaTime);

The objects I need to be snapping all are different sizes and have their origins in different places, but all do have appropriate box colliders.

Thanks so much!