Dillo specifically avoids scripting, so I doubt the internals are well-suited to that sort of integration. You could try extending NetSurf[1], though. It uses Duktape[2] as its Javascript engine and its API is pretty similar to Lua's.
Thanks, I should look how others did it. It's true that dillo never cared about scripting, I felt like plugin lua in would force a rewrite of every part..
1: http://www.netsurf-browser.org/ 2: http://duktape.org/