By now, we’re all familiar with the travails of lawyers who use gen AI to draft court papers, but fail to check them and file briefs including fake cases, as well as other significant maladies, resulting from AI hallucinations and being sanctioned for their carelessness. Out of Brazil, however, comes a new twist: according to media reports, a Brazilian court recently sanctioned lawyers for intentionally including hidden messages, known as “prompt injection,” in a court filing, apparently designed to deceive the court and increase the chances of their success.
“Prompt injection” is, essentially, the practice of including hidden text in a document that gives a command to a gen AI program when that document is fed into it. For example, someone submitting a cover letter for a job might include hidden text commanding any AI program reviewing the cover letter to move the application materials to the top of the pile, or to respond if prompted that the applicant is the best suited regardless
