A recently disclosed flaw in a widely used AI assistant offers an uncomfortable lesson. Specialists got the system to talk about its own protective measures. The reasons given for refused answers were enough to derive the architecture, until an undocumented parameter came to light. After that, a single click was enough for the assistant to read connected services with the user's permissions.
The pattern is well known and tops the OWASP risk list for language models: the instruction and the processed content end up in the same context. A web page or a document can therefore act as a command line. This is not a teething problem but a property of the design.
For companies the conclusion is not a ban but a question of architecture. An assistant must never see more than the role it supports. Every connection needs a justification, external content counts as data and never as instruction, and persistent memory belongs in the review cycle. Bans merely shift usage into private accounts.