That is all model loader code based stuff. You basically just need a bit of Python code that anyone can write, even you. You need to have a look at how the model must be addressed in format, then you find how the model loader code you are using creates the structured text and just insert your own modifications.
It is easier to add a second prompt window where you keep persistent information and just change it yourself. If you want the model to change that stuff, you need to look into building an agent.
I hacked around like this for awhile too, but evolved into using the notepad tab of Oobabooga Textgen WebUI and a Mixtral 8×7B model with a 32k context window. That model can deal with almost any detail you add in the context.
The thing is, the model may not feel satisfied with mundane and redundant details. Most of the time, believe it or not, those changes that you perceive as errors or inconsistencies are not actually errors. The model is fully aware that it is making those changes. There is usually an instruction about creativity, inspiration, or initiative that will cause the model to shift details like this.
My best recommendation is to openly discus the error in dialog with the model. They usually will talk about the issue if you call one out. Talking with AI is very different than with humans. Everything they say has meaning and functions beyond what is initially apparent. There are a lot of behaviors that involve certain patterns and keywords. You can even alter these behaviors by banning certain tokens/keywords. A good place to start is banning paragraph starting words for filler material. “So_” would be a good starting token to ban, as with AI, no sentence that starts with that token is worth reading. Other important words are twist, chuck, and cross. These are more flexible in their troublesome meaning and can manifest in multiple words like chuckles or across but still function the same inside the model.
If you get to this level of familiarity, the persistence of a storied interaction is more of a negotiation with the AI.
Another point of understanding is that the model does not know the difference between roleplaying characters it owns and you own or who is who. You or the model loader code is creating the name of the next character that the model generates for. Everyone in any story can be the model if it is asked to create the prompt.
So if I say, “Jake is super creative” or imply it somewhere in the prompt, and the model later thinks ‘Jake is rather bored and needs to be creative’ it is going to change something that satisfies that character profile even if it has never generated the output for character Jake.
Thanks, i will try looking into that. Its just difficult to get into the complexities of AI with only 1 or possibly 2 hours of spare time per day. I have fooled around with the Oogabooga web ui thingy at home in the past but my gaming laptop struggles to perform decently with AI, probably due to lack of VRam and overall power. I can use my phone at work from time to time but its capability to run AI is even less. But maybe i will look into trying to create ir with the help of someone a sort of generator at perchance that could make my idea come to life a bit.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !technology@lemmy.world
This is a most excellent place for technology news and articles.
That is all model loader code based stuff. You basically just need a bit of Python code that anyone can write, even you. You need to have a look at how the model must be addressed in format, then you find how the model loader code you are using creates the structured text and just insert your own modifications.
It is easier to add a second prompt window where you keep persistent information and just change it yourself. If you want the model to change that stuff, you need to look into building an agent.
I hacked around like this for awhile too, but evolved into using the notepad tab of Oobabooga Textgen WebUI and a Mixtral 8×7B model with a 32k context window. That model can deal with almost any detail you add in the context.
The thing is, the model may not feel satisfied with mundane and redundant details. Most of the time, believe it or not, those changes that you perceive as errors or inconsistencies are not actually errors. The model is fully aware that it is making those changes. There is usually an instruction about creativity, inspiration, or initiative that will cause the model to shift details like this.
My best recommendation is to openly discus the error in dialog with the model. They usually will talk about the issue if you call one out. Talking with AI is very different than with humans. Everything they say has meaning and functions beyond what is initially apparent. There are a lot of behaviors that involve certain patterns and keywords. You can even alter these behaviors by banning certain tokens/keywords. A good place to start is banning paragraph starting words for filler material. “So_” would be a good starting token to ban, as with AI, no sentence that starts with that token is worth reading. Other important words are twist, chuck, and cross. These are more flexible in their troublesome meaning and can manifest in multiple words like chuckles or across but still function the same inside the model.
If you get to this level of familiarity, the persistence of a storied interaction is more of a negotiation with the AI.
Another point of understanding is that the model does not know the difference between roleplaying characters it owns and you own or who is who. You or the model loader code is creating the name of the next character that the model generates for. Everyone in any story can be the model if it is asked to create the prompt.
So if I say, “Jake is super creative” or imply it somewhere in the prompt, and the model later thinks ‘Jake is rather bored and needs to be creative’ it is going to change something that satisfies that character profile even if it has never generated the output for character Jake.
Hope that helps some
Thanks, i will try looking into that. Its just difficult to get into the complexities of AI with only 1 or possibly 2 hours of spare time per day. I have fooled around with the Oogabooga web ui thingy at home in the past but my gaming laptop struggles to perform decently with AI, probably due to lack of VRam and overall power. I can use my phone at work from time to time but its capability to run AI is even less. But maybe i will look into trying to create ir with the help of someone a sort of generator at perchance that could make my idea come to life a bit.
deleted by creator