• 0 Posts
  • 17 Comments
Joined 1 year ago
cake
Cake day: June 7th, 2023

help-circle











  • LLMs choose words based on probabilities, i.e. given the word “blue”, it will have a list of words and probabilities that those words should follow “blue”. So “sky” would be a high probability, “car” might also be quite high, as well as a long list of other words. The LLM chooses the words not by selecting whatever has the highest probability, but with a degree of randomness. This has been found to make the text sound more natural.

    To watermark, you essentially make this randomness happen in a predefined way, at least for cases where many different words could fit. So (to use a flawed example), you might make it so that “blue” is followed by “car” rather than “sky”. You do this throughout the text, and in a way that doesn’t affect the meaning of the text. It is then possible to write a simple algorithm to detect whether this text was written by an AI, because of the probability of different words appearing in particular sequences. Because its spread throughout the text, it’s quite difficult to remove the watermark completely (although not impossible).

    Here’s an article that explains it better than I can: https://www.kdnuggets.com/2023/03/watermarking-help-mitigate-potential-risks-llms.html





  • You might want to look up the law of unintended consequences.

    The bigger the intervention, the bigger the potential unintended consequence.

    By far the easiest solution to climate change is not emitting greenhouse gasses in the first place. It is still a monumental challenge but if we don’t do that, we’re just treating the symptoms not the cause