Can Lightning work without the “network” part?

Let’s imagine a minimal version of the Lightning network with a single “central” lightning node (say a service provider which we can call a Server) surrounded by thin clients (the actual users potentially simply using open-source client software on their phone).

The network part in this scenario is non-existant as communication simply happens between Users <-> Server. Obviously to pay anybody in this system both users have to use the same Server.

I would imagine then each User simply having a channel open with the Server and whenever users Alice & Bob need to make payments with each other they simply use their single open channel with Server, and Server adjusts the 2 channels accordingly to follow on the payment. (I imagine some issues here with the payment not being possible if the channels are not funded in the direction needed for the payment).

Alice & Bob then each get their confirmation instantly validated by their open-source thin client (hence no trust with Server is needed).

I know there’s has been lots of research and complexity around the “network” part of Lightning (such as routing mechanisms, funding channels, knowledge & discovery of network typography etc.) hence I was curious whether a simpler Client<->Server approach as mentioned above would be an easier and more user-friendly implementation strategy for simpler use-cases (such as regular users using a non-trusted central server like

Are there any downsides of this approach apart from the clear fact that in order to make payments users have to use (but not trust!) the same Server?