Memory upgrade options for iMac Pro?

I plan to get an iMac Pro. I would love to maximize the crazy amount of memory but beyond my budget. You plan to get 64GB of RAM. If I wanted to upgrade after the purchase, would I need 3 more 64GB units to fill it or does not it work like this for this machine?

Note: I understand that upgrading is not easy and I will take it to a certified place to upgrade it – hoping to save a few dollars this way.