Even though I told the installer to keep the default config (which it did) but over-written /etc/nginx/conf.d/default.conf The only change I had to make was port number, since I am using varnish changed my port to 8080 for nginx. Restarted nginx.
Another note: Packages provided by Ubuntu use /etc/nginx/sites-available/ and /etc/nginx/sites-available/ for storing site configs but the package directly from Nginx stores configs in /etc/nginx/conf.d/.
Not sure what the schedule looks like in general, but last night I waited ~10 minutes after the advisory watching the packages show up one by one. Still faster than any other source, of course.