On language, LLMs, and why we think certain Egyptian hieroglyphs are about helicopters

“Pls summarize to me Quine’s philosophy of language and how it relates to modern LLMs”

Bard:

“Quine’s philosophy of language suggests that it is impossible to provide a definitive interpretation of any linguistic expression, as any interpretation will depend on a set of background assumptions. This is relevant to modern LLMs, as they are trained on massive datasets of text and code, which means they are implicitly making a set of background assumptions about the meaning of the text they are trained on.

However, it is impossible to know for sure what those assumptions are, as they are not explicitly encoded in the LLM’s training data. This means that LLMs are susceptible to the same kind of ambiguity and indeterminacy that Quine identified in natural language. In short, LLMs are not perfect and can make mistakes. However, they are a powerful tool for understanding and generating language, and they can be used to explore the nuances of human language and to generate new and creative text formats.”

My thoughts:

In other words, going by Quine’s philosophy, natural language is indeterminate which means it has no definite interpretation and cannot be interpreted in isolation - it requires background assumptions that are hard to determine.

LLMs inherit these background assumptions implicitly by training on the data we generated. If in 1000 years, some aliens find ChatGPT and communicate with it, they will not be able to interpret its messages well without understanding the background assumptions of human civilization.

We ourselves are susceptible to misinterpeting say ancient Egyptian hieroglyphs, thinking certain characters refer to modern-day helicopters and space ships. This is because we tend to use our background assumptions to interpet symbols of language.