Presumably you want to reply to DNS requests for a hostname for your captive portal. You might try to just use a raw ip address, but then you can't use https. So then you have the problem that you can't just reply with a fake answer for other domains due to caching. E.g. Windows caches negative responses for 5 minutes, which would be a pretty bad experience for your customers.
I guess you might be able to just fail to reply to DNS requests for domains outside you captive portal, I have no idea if anyone has tried that or there might be other complications.
Edit: Actually not replying wouldn't work great either because then the user can't be redirected to the captive portal. This might be less of an issue today since most devices have standardized a way to detect captive portals using a small set of hostnames.
Aside from custom root certs being installed, https is out either way, not only with IP address, right? HSTS makes that even more of a problem, and pinning makes even the custom root cert a problem. Captive portals seem like an increasingly fragile idea, except that OSes increasingly intervene appropriately.
...and bring up UI (sandboxed browser) if anything else than expected short 2xx was received.
Step 2) Mobile device reuse same cookies for the same portal second time around, so portal can recognize returning user and let them in without annoying with login prompt each time.
Assuming mobile device has a separate cookie jar for this, and captive portal is HTTPS with proper certificate, and doesn't try to break other people's SSL, that is pretty secure. Sure, this is not the most efficient protocol, but up-side is - it's open-ended. You can fit here anything from "press here if you agree to behave" to complete walled garden like hotspot on a train that shows you interactive map, schedule and transport connections available on the next stop along with small banner asking $1 for full internet access.
You just reply to any DNS request with your own server's IP, which accepts any HTTP requests with a redirection to the captive portal. The replies can have low TTLs to avoid the caching problem.
I guess you might be able to just fail to reply to DNS requests for domains outside you captive portal, I have no idea if anyone has tried that or there might be other complications.
Edit: Actually not replying wouldn't work great either because then the user can't be redirected to the captive portal. This might be less of an issue today since most devices have standardized a way to detect captive portals using a small set of hostnames.