A new feature with the latest release is the ability to redirect users to a 'Under Construction' page and prevent anyone from accessing the site.
This feature can be found under the PowerApps Portals admin center, Portal Actions. Now finding this option is a little tricky as I could not find this from the PowerApps admin center.
Instead, make your way to the Dynamics 365 Administration Center - this is the previous interface for managing Dynamics 365 tenants as compared to the current 'Admin Centers'.
From here, click Applications, select the portal to manage, then click the Manage button to get to the PowerApps Portals admin center.
I would assume this 'find the right admin page' issue will be quickly resolved as Microsoft's continues to port the interface...
Showcase of Dynamics 365 design and technology