Ironically that’s literally what can happen with a genie, you will forbid them from something with guardrails but still find a way to give you what you did not want
Is not the whole schtick with the genie wish that it technically follows the wish but distorts the intention? LLMs aren’t even guaranteed to do that. You could say “don’t delete this file” and it may still end up doing exactly that.
Ironically that’s literally what can happen with a genie, you will forbid them from something with guardrails but still find a way to give you what you did not want
Is not the whole schtick with the genie wish that it technically follows the wish but distorts the intention? LLMs aren’t even guaranteed to do that. You could say “don’t delete this file” and it may still end up doing exactly that.