> Where you can request it looks at hotel options in Montreal, and it starts answering in half a second

Yet the answers will get outdated quickly whilst the silicon is fixed.

>Yet the answers will get outdated quickly whilst the silicon is fixed.

Bro is living in 2020 before rag was widely introduced.

I was told updates require replacing at least two layers of metal, though not whole thing. Was that not accurate? Can you say more?

You can change the prompt, and give the agent tools, and that keeps the agent up to date.

The agent doesn't know the date, or know what hotels there are in Montreal, it sees:

> System: You are an AI agent. The date is 11th August 2026. Your knowledge cut-off is March 2023. User is based in <date>. If you need to search for something to support the user say {search:<term>} and a list of options will be provided along with instructions on how to access. Or say {help} for a full list of commands.

> User: Can you help me find hotels in Montreal for next weekend?

The AI then interacts with the tools given in the base prompt, which can obviously be updated. So it then goes:

> AI: Of course, let me search for that. {search: hotels in montreal for 16th August}

> System: [Provides list of websites]. Say {read[n]} to read option or say {start subagent:<goal>) to register subagent.

> AI: {start subagent: List hotels on booking.com available on 16th August}.

[etc etc, then eventually]

> AI: Yes I have searched for you and I found a few options!

While you can't change embedded knowledge, a good model knowing that the date is 5th January 2040 can infer certain things (e.g. while it might not have been trained on certain deaths, it can probably guess that it should search before answering if it means a person would be 102 and their last information is from 2024)

You are talking about two different things.

Yes, to update the blueprint for new models two layers will be updated. That is the NN.

To instead update the data on which to operate you could use a RAG to query.

(As in "the Pathfinder 2.0 NN is on the chip; the geodata is in the OpenGeoMaps dump-DB-nightly" - not really overlapping with LLM+RAG but may give an idea in a different scenario.)

> before rag was widely introduced

And when did RAG start to work properly as a mature, reliable technology?

I mean, it's pretty damn reliable now. Has been for simple high contrast Q&A for a while now.

I've just been doing research and experiments for work related stuff.

Typically we've used plain embeddings for a lot of high contrast documents aka discrete facts.

However I've been working with a >1000 page document of complex procedures with incredibly low contrast where embedding falls flat.

There's top down/graph searching, bottom up/embedded; alts like colbert, reranking, reasoning, search agents and now (though seemingly quite new) specific search agent models.

Ultimately I found that a reasoning enabled search agent doing a hybrid of bottom up (with reranking) followed by top down, gave the absolute best results. Paired with Luna for cheaper and faster tokens it benchmarks pretty well even for vague references to procedures.

I would imagine that search specific models just coming out are even better and I'll have to evaluate using these but for now the above works well for us.

Having an agent get vector search results to use as anchors and then being able to explore the sections and subsections above that, then eventually digesting as much as is relevant (big context, cheap tokens) is amazing.

Very interesting and very good (though, let us say, much more recent that ancient): only, it paradoxically does suggest that the computational cost (when compared either to a simple "search by vectors distance" or to the task failures that we can have witnessed by chief implementations*) strongly suggest the benefit of 1000x speed boosting, energy conservative Taalas board.

*(Of course it has "always" worked well for «simple high contrast Q&A», ever since the base embeddings technology worked properly: that is almost by definition; it is on real world use cases, where the nuances of reality are present, that it failed miserably.)

And outside of idiotic demos, who exactly is going to ask an LLM to look at hotels in Montreal for them? This usecase has never made sense to me in the slightest

In the slightest? The user goes to their computer and types in "Montreal hotels" and Google comes up with no shortage of results, including a bit from their LLM. So that's already happening, but how do you narrow down the results from that initial search? Click around on Expedia for an hour? You probably know what you care about, just tell the LLM that you have dogs or are a vegan or whatever instead of wasting a bunch of time doing it by hand yourself.

> who exactly is going to ask an LLM to look at hotels in Montreal for them

Anybody who has a specific informal query ("SELECT ... FROM ... WHERE has_carpark AND ... ORDER BY score(has_jacuzzi , walk_distance(...) ...) DESC") but does not want to research and cross the different scattered info himself (does not want to build the virtual DB himself).

I mean it was just an example, but I get AI to do things like this all the time... ChatGPT planned my latest work trip by looking through flights, hotels and pulled together 3 options of itineraries for me, and wrote an email I could send to my client with estimated costs etc.