Lambda Symbolics

Symbolic AI for fun and profit

Black-and-white portrait of a man with glasses and a beard.

Howdy!

I figured I would start giving explanations to the more nerdy things in Autolith, namely, its symbolic AI features.

Symbolic AI is an approach to AI in which a system represents aspects of a problem using symbols and manipulates those representations according to some operations with those symbols.

A symbol is e.g. an object, a category, a relation, or maybe an action. Then, Expressions combine symbols into structures that can state facts, describe rules, or propose questions.

The term represents a whole bunch of things in the world of gigachad Lisp era of AI. Here's a small list:

Representing knowledge

For classical AI, the thing we need to do first is to have a way to represent knowledge.

A program might represent an individual animal by a name and associate it with a category. These are facts.

Facts are statements recorded in a system.

A fact such as mammal(moby) can be read as “Moby is a mammal.” Here mammal is a predicate, or property, and moby is an individual. I hope you learned Prolog lol.

Other predicates can represent relations between multiple individuals, imagine a parent(a, b) expression which means that a is a parent of b.

Collection of facts or other statements are generally referred to as knowledge bases. Their contents are represented as structures the program can inspect and manipulate.

We can have representations that are good and representations that are less good. There is a whole spectrum of approaches between having attribute-value pairs, having logical languages (like Prolog), and having a semantic network which represents and entities and relations as nodes and edges.

Rules and inference

Rules specify relationships between conditions and conclusions/actions.

In logic-oriented systems, a rule states that a conclusion follows when its conditions hold. For example:

If x is a mammal, then x is an animal.

The variable x stands for an individual.

This rule applies to an individual for which the stated condition (is a mammal) is established. If we are discussing so-called production-rule systems, rules may instead specify actions to perform when conditions match the current state.

Inference in classical AI means something a lil bit different from LLM inference, hahaha. It is the process of deriving conclusions from statements according to rules of reasoning.

In the classic approach of forward chaining, a system will:

  1. start with available facts
  2. discover which rules have conditions that match
  3. and derive further facts

It sibling, backward chaining, works like this:

  1. starts with a question
  2. discovers rules and facts that could establish an answer
  3. attempts to find an answer

Life is great, so both approaches can be used in rule-based systems

An example

Consider a knowledge base containing the following fact and rule:

Fact:  Moby is a mammal.
Rule:  If x is a mammal, then x is an animal.

Moby is a name for an individual whale.

The fact states that this individual is a mammal.

The rule states that all individuals satisfying this condition are also animals.

The inference proceeds as follows:

  1. The inference system takes in the fact that Moby is a mammal
  2. it matches Moby to the variable x in the rule’s condition
  3. It applies the rule with x bound to Moby
  4. It derives the conclusion that Moby is an animal
The fact that Moby is a mammal and the rule that every mammal is an animal lead to the conclusion that Moby is an animal.

Now, we can record the conclusion plus the facts and rules used to derive it and save it as a trace.

That recorded trace explains the inference in terms of the knowledge base and the system’s rules, and is incredibly relevant when keeping track of whether facts still hold.

A derivation connecting the fact that Moby is a mammal and the rule that every mammal is an animal to the conclusion that Moby is an animal.

The example also shows the role of representation.

Merely stating that Moby is a whale does not help us, since our system can't derive that Moby is a mammal from this one rule alone.

Therefore, we would also need another statement which connects whales to mammals.

Questions or changes or uncertainty (?)

Symbolic systems can answer questions (without the involvement of LLMs or neural networks, mind you!). As implied above, they do this searching the knowledge base and applying rules.

For instance, if we remember the previous example, the system can answer whether Moby is an animal.

Knowledge isn't immutable and it can change no problem. Whenever a stored fact is removed or corrected, the conclusions that depend on it might have to be reconsidered. This, as we will see in the next part, this is particularly important for Autolith.

Systems can record dependencies between statements to help determine which conclusions require revision. Conclusion can have more than one justification each, so removing one premise does not have to invalidate it per se, there is a lot of leeway in how you implement your systems.

Ordinary logical rules very often express categorical relationships, but not all information is certain, we can be wrong. Some systems attach probabilities, confidence measures, or other qualifications to statements and use corresponding methods of inference.

Relation to neural models

This approach to artificial intelligence is more machine-like than how we perceive neural networks, especially LLMs.

Neural models usually learn numerical params from data. The internal representation is distributed across numerical values rather, and not stored as individually readable facts and rules.

Symbolic and neural methods differ in representation and processing, but they can be combined.

For a very auspicious example, neural models or LLMs can convert data into candidate structured facts, while a symbolic component checks these structured facts or uses them for inference.

Also, a symbolic system can supply constraints or maybe retrieve structured info for a neural model.

I think it's a shame that we are kinda forgetting symbolic AI in the LLM hype, as it is incredibly useful... and cheap!

Anyways, I am trying to keep this short, more soon.

Lukáš Hozda