A BDD-Based Algorithm for Learning from Interpretation Transition

Abstract

In recent years, there has been an extensive interest in learning the dynamics of systems. For this purpose, a new learning method called learning from interpretation transition has been proposed recently [1]. However, both the run time and the memory space of this algorithm are exponential, so a better data structure and an efficient algorithm have been awaited. In this paper, we propose a new learning algorithm of this method utilizing an efficient data structure inspired from Ordered Binary Decision Diagrams. We show empirically that using this representation we can perform the same learning task faster with less memory space.

This research was supported in part by the NII research project on “Dynamic Constraint Networks” and by the “Systems Resilience” project at Research Organization of Information and Systems, Japan. We would like to thank Earl Belinger for its help to improve the english quality of the paper.

A Appendix

1.1 A.1 Proof of Theorem 1

Proof

Let $n$ be the size of the Herbrand base $|B|$. This $n$ is also the number of possible heads of rules. Furthermore, $n$ is also the maximum size of a rule, i.e. the number of literals in the body; a literal can appear at most one time in the body of a rule. For each head there are $3^n$ possible bodies: each literal can either be positive, negative or absent of the body. From these preliminaries we conclude that the size of an NLP $|P|$ learned by LF1T is at most $n\cdot 3^n$. But thanks to ground resolution, $|P|$ cannot exceed $n\cdot 2^n$; in the worst case, $P$ contains only rules of size $n$ where all literals appear and there is only $n\cdot 2^n$ such rules. If $P$ contains a rule with $m$ literals ($m<n$), this rule subsumes $2^{n-m}$ rules which cannot appear in $P$. Finally, ground resolution also ensures that $P$ does not contain any pair of complementary rules, so that the complexity is further divided by $n$; that is, $|P|$ is bounded by $O(\frac{n\cdot 2^n}{n})=O(2^n)$.

In our approach, a BDD represents all rules of $P$ that have the same head, so that we have $n$ BDD structures. When $|P|=2^n$, each BDD represents $2^n/n$ rules of size $n$ and are bound by $O(2^n/n)$, which is the upper bound size of a BDD for any Boolean function [21]. Because BDD merges common parts of rules, it is possible that a BDD that represents $2^n/n$ rules needs less than $2^n/n$ memory space. In the previous approach, in the worst case $|P|=2^n$, whereas in our approach $|P| \le 2^n$. Our new algorithm still remains in the same order of complexity regarding memory size: $O(2^n)$.

Regarding learning, each operation has its own complexity. Let $k$ be the place of a literal in the variable ordering so that for the starting node literal of a BDD $k=0$. In our BDD, a node has at most $2\cdot ((n-k)-1)$ children: $(n-k)-1$ positive and negative links to all literals which are superior to $k$ in the ordering. Insertion of a rule is done in polynomial time; in the worst case, we insert a rule where only one literal that differs from the BDD. Because we follow only the first common literals, we have to check at most $2\cdot ((n-k)-1)$ links on $n-1$ nodes, which belongs to $O(n^2)$.

Subsumption as well as generalization checks require exponential time. In the case of subsumption, in the worst case the BDD contains $2^n/n$ rules and the rule is not subsumed by any of them.

That means that we have to check every rule, and each check belongs to $O(n^2)$ so that the whole subsumption operation belongs to $O(n^2\cdot 2^n/n)=O(2^n)$. To clear the BDD we have to perform the inverse operation. We always have to check the whole BDD, so if the size of the BDD is $2^n$ then the complexity of the whole clear check also belongs to $O(2^n)$.

To generalize the new rule we have to check if the BDD subsumes one of its complementary rules. Like for subsumption, in the worst case we have to check every rule. A rule can be generalized at most $n$ times; for each generalization we have to check at most $n$ complementary rules, so the complexity of a complete generalization belongs to $O(n^2\cdot 2^n/n)=O(2^n)$. For the complexity of generalization of BDD rules we consider the inverse problem. In the worst case, every rule of the BDD can be generalized by the new one. Because the new rule does not cover any rules of the BDD, it can generalize each rule of the BDD at most one time. Then, we have at most $2^n/n$ possible direct generalizations on the whole BDD. In the worst case, each of them can be generalized at most $n-1$ times, and like before, for each generalization we have to check at most $n$ complementary rules. If a rule is generalized $n$ times it means that its body becomes empty, i.e. the rule is a fact, and it will subsume and clear the whole BDD. Then, the complexity of a complete generalization of the BDD belongs to $O(2^n/n\cdot (n-1)\cdot n)=O(2^n)$.

Each time we learn a rule from a step transition we have to perform these four checks which have a complexity of $O(n^2+2^n+2^n+2^n)=O(2^n)$. From $2^n$ state transitions, LF1T can directly infer $n\cdot 2^n$ rules. Learning the dynamics of the entire input implies in the worst case $2^n\cdot 2^n$ operations which belong to $O(4^n)$. Using our dedicated BDD structure the memory complexity as well as the computational complexity of LF1T remains the same order as the previous algorithm based on ground resolution: respectively $O(2^n)$ and $O(4^n)$.