Property-Directed Inference of Universal Invariants or Proving Their Absence

Abstract

We present Universal Property Directed Reachability ($\mathsf PDR ^{\forall }$), a property-directed procedure for automatic inference of invariants in a universal fragment of first-order logic. $\mathsf PDR ^{\forall }$ is an extension of Bradley’s PDR/IC3 algorithm for inference of propositional invariants. $\mathsf PDR ^{\forall }$ terminates when it either discovers a concrete counterexample, infers an inductive universal invariant strong enough to establish the desired safety property, or finds a proof that such an invariant does not exist. We implemented an analyzer based on $\mathsf PDR ^{\forall }$, and applied it to a collection of list-manipulating programs. Our analyzer was able to automatically infer universal invariants strong enough to establish memory safety and certain functional correctness properties, show the absence of such invariants for certain natural programs and specifications, and detect bugs. All this, without the need for user-supplied abstraction predicates.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

We present Universal Property Directed Reachability ($\mathsf PDR ^{\forall }$), a procedure for automatic inference of quantified inductive invariants, and its application for the analysis of programs that manipulate unbounded data structures such as singly-linked and doubly-linked list data structures. For a correct program, the inductive invariant generated ensures that the program satisfies its specification. For an erroneous program, $\mathsf PDR ^{\forall }$ produces a concrete counterexample. Historically, this has been addressed by abstract interpretation [17] algorithms, which automatically infer sound inductive invariants, and bounded model checking algorithms, which explore a limited number of loop iterations in order to systematically look for bugs [6, 13]. We continue the line of recent works [2, 32] which simultaneously search for invariants and counterexamples. We follow Bradley’s PDR/IC3 algorithm [9] by repeatedly strengthening a candidate invariant until it either becomes inductive, or a counterexample is found.

In our experience, the correctness of many programs can be proven using universal invariants. Hence, we simplify matters by focusing on inferring universal first-order invariants. When $\mathsf PDR ^{\forall }$ terminates, it yields one of the following outcomes: (i) a universal inductive invariant strong enough to show that the program respects the property, (ii) a concrete counterexample which shows that the program violates the desired safety property, or (iii) a proof that the program cannot be proven correct using a universal invariant in a given vocabulary.

Diagram Based Abstraction. Unlike previous work [2, 32], we neither assume that the predicates which constitute the invariants are known, nor apriori bound the number of universal quantifiers. Instead, we rely on first-order theories with a finite model property: for such theories, SMT-based tools are able to either return UNSAT, indicating that the negation of a formula $\varphi$ is valid, or construct a finite model $\sigma$ of $\varphi$. We then translate $\sigma$ into a diagram [10]—a formula describing the set of models that extend $\sigma$—and use the diagram to construct a universal clause to strengthen a candidate invariant.

Fig. 1.

Motivating examples. $n^{*}(x,y)$ means a (possibly empty) path of n-fields from x to y.

Property-Directed Invariant Inference. Similarly to IC3, $\mathsf PDR ^{\forall }$ iteratively constructs an increasing sequence of candidate inductive invariants $F_{0}\cdots F_{N}$. Every $F_{i}$ over-approximates the set $\mathcal {R}_i$ of states that can be reached by up to i execution steps from a given set of initial states. In every iteration, $\mathsf PDR ^{\forall }$ uses SMT to check whether one of the candidate invariants became inductive. If so, then the program respects the desired property. If not, $\mathsf PDR ^{\forall }$ iteratively strengthens the candidate invariants and adds new ones, guided by the considered property. Specifically, it checks if there exists a bad state $\sigma$ which satisfies $F_{N}$ but not the property. If so, we use SMT again to check whether there is a state $\sigma _a$ in $F_{N-1}$ that can lead to a state in the diagram $\varphi$ of $\sigma$ in one execution step. If no such state exists, the candidate invariant $F_{N}$ can be strengthened by conjoining it with the negation of $\varphi$. Otherwise, we recursively strengthen $F_{i-1}$ to exclude $\sigma _a$ from its over-approximation of $\mathcal {R}_{i-1}$. If the recursive process tries to strengthen $F_{0}$, we stop and use a bounded model checker to look for a counterexample of length N. If no counterexample is found, $\mathsf PDR ^{\forall }$ determines that no universal invariant strong enough to prove the desired property exists (see Lemma 1). We note that $\mathsf PDR ^{\forall }$ is not guaranteed to terminate, although in our experience it often does.

Example 1

Procedure split(), shown in Fig. 1(a), moves the elements not satisfying the condition C from the list pointed to by h to the list pointed by g. $\mathsf PDR ^{\forall }$ can infer tricky inductive invariants strong enough to prove several interesting properties: (i) memory safety, i.e., no null dereference and no memory leaks; (ii) all the elements satisfying C are kept in h; (iii) all the elements which do not satisfy C are moved to g; (iv) no new elements are introduced; and (v) stability, i.e., the reachability order between the elements satisfying C is not changed. Our implementation verified that split() satisfies all the above properties fully automatically by inferring an inductive loop invariant consisting of 36 clauses (among them 19 are universal formulae) in 206 sec.

Example 2

Procedure filter(), shown in Fig. 1(b), removes and deallocates the elements not satisfying the condition C from the list pointed to by h. The figure also shows the loop invariant inferred by $\mathsf PDR ^{\forall }$ when it was asked to verify a simplified version of property (iii): all the elements which do not satisfy C are removed from h. The invariant highlights certain interesting properties of filter(). For example, clause $L_4$ says that if the head element of the list was processed and kept in the list (this is the only way $i \ne h$ can hold), then j becomes an immediate predecessor of i. Clause $L_7$ says that all the elements $x_3$ reachable from h and not satisfying C must occur after j.

Experimental Evaluation. We implemented $\mathsf PDR ^{\forall }$ on top of the decision procedure of [32], and applied it to a collection of procedures that manipulate (possibly sorted) singly linked lists, doubly-linked lists, and multi-linked lists. Our analysis successfully verified interesting specifications, detected bugs in incorrect programs, and established the absence of universal invariants for certain correct programs.

Main Contributions. The main contributions of this work can be summarized as follows.

• We present $\mathsf PDR ^{\forall }$, a pleasantly simple, yet surprisingly powerful, combination of PDR [9] with a strengthening technique based on diagrams [10]. $\mathsf PDR ^{\forall }$ enjoys a high-degree of automation because it does not require pre-defined abstraction predicates.

• The diagram-based abstraction is particularly interesting as it is determined “on-the-fly” according to the structural properties of the bad states discovered in PDR’s traversal of the state space.

• We prove that the diagram-based abstraction is precise in the sense that if $\mathsf PDR ^{\forall }$ finds a spurious counterexample then the program cannot be proven correct using a universal invariant. We believe that this is a unique feature of our approach.

• We implemented $\mathsf PDR ^{\forall }$ on top of a decision procedure for logic $AE^{R}$ [31], and applied it successfully to verify a collection of list-manipulating programs, detect bug, and prove the absence of universal invariants. We show that our technique outperforms an existing state-of-the-art less-automatic PDR-based verification technique [32] which uses the same decision procedure.

2 Preliminaries

Programs. We handle single loop programs, i.e., we assume that a program has the form while $Cond$ do $Cmd$, where $Cmd$ is loop-free. We encode more complicated control structures, e.g., nested or multiple loops, by explicitly recording the program counter. For clarity, in our examples we allow for a sequence of instructions preceding the loop. Technically, we encode their effect in the loop’s pre-condition.

From Programs to Transition Systems. The semantics of a program is described by a transition system, which consists of a set of states and transitions between states.

Program States. We consider the states of the program at the beginning of each iteration of the loop. A program state is represented by a first-order model $\sigma = (\mathcal {U}, \mathcal {I})$ over a vocabulary $\mathcal {V}$ which consists of constants and relation symbols, where $\mathcal {U}$ is the universe of the model, and $\mathcal {I}$ is the interpretation function of the symbols in $\mathcal {V}$. For example, to represent memory states of list manipulating programs, we use a vocabulary $\mathcal {V}$ which associates every program variable $\mathtt {x}$ with a constant x, every boolean field $\mathtt {C}$ with a unary predicate $C(\cdot )$, and every pointer field n with a binary predicate $n^*(\cdot ,\cdot )$ which represents its reflexive transitive closure.¹ We use a special constant $null$ to denote the null value. We depict memory states $\sigma = (\mathcal {U}, \mathcal {I})$ as directed graphs (see Fig. 2). Individuals in $\mathcal {U}$, representing heap locations, are depicted as circles labeled by their name. We draw an edge from the name of constant x and of a unary predicate C to an individual v if $\sigma \models x=v$ or $\sigma \models C(v)$, respectively. We draw an $n^*$-annotated edge between v and u if $\sigma \models n^*(v,u)$. For clarity, we do not show edges that can be inferred from the reflexive and transitive nature of $n^*$.

Transition Relation. The set of transitions of a program is defined using a transition relation. A transition relation is a set of models of a double vocabulary $\hat{\mathcal {V}} = \mathcal {V}\uplus \mathcal {V}'$, where vocabulary $\mathcal {V}$ is used to describe the source state of the transition and vocabulary $\mathcal {V}' = \{ v' \mid v\in \mathcal {V}\}$ is used to describe its target state: A model $\sigma ' = (\mathcal {U},\mathcal {I}')$ over $\mathcal {V}'$ describes a program state $\sigma = (\mathcal {U},\mathcal {I})$, where $\mathcal {I}(v)=\mathcal {I}'(v')$ for every symbol $v\in \mathcal {V}$.

Definition 1

(Reduct). Let $\hat{\sigma }= (\mathcal {U},\mathcal {I})$ be a model of $\hat{\mathcal {V}}$, and let $\Sigma \subseteq \hat{\mathcal {V}}$. The reduct of $\hat{\sigma }$ to $\Sigma$ is the model $(\mathcal {U},\mathcal {I}_i)$ of $\Sigma$ where for every symbol $v \in \Sigma$, $\mathcal {I}_i(v) = \mathcal {I}(v)$.

We often write a transition $\hat{\sigma }$ as a pair of states $(\sigma _1,\sigma _2)$, such that $\sigma _1$ is the reduct of $\hat{\sigma }$ to vocabulary $\mathcal {V}$, and $\sigma _2$ is the state described by the reduct to $\mathcal {V}'$. Each transition $(\sigma _1,\sigma _2)$ describes one possible execution of the loop body, $Cmd$, i.e., it relates the state $\sigma _1$ at the beginning of an iteration of the loop to the state $\sigma _2$ at the end of the iteration. We say that $\sigma _2$ is a successor of $\sigma _1$, and $\sigma _1$ is a predecessor of $\sigma _2$.

Properties and Assertions. Properties are sets of states. We express properties using logical formulae over $\mathcal {V}$. For example, we express properties of list-manipulation programs, e.g., their pre- and post-conditions, Pre and Post, respectively, using assertions written in a fragment of first-order logic with transitive closure. In our analysis, these assertions are translated into equisatisfiable first-order logic formulae [31]. We use $(\varphi )'$ to denote the formula obtained by replacing every constant and relation symbol in formula $\varphi$ with its primed version.

Verification Problem. The transition system of a program is represented by a pair $TS= ( Init ,\rho )$, where $Init$ is a first-order formula over $\mathcal {V}$ used to denote the initial states of the program, and $\rho$ is a formula over $\hat{\mathcal {V}}$ used to denote its transition relation. A state $\sigma$ is initial if $\sigma \models Init$, and a pair of states $(\sigma _1, \sigma _2)$ is a transition if $(\sigma _1, \sigma _2) \models \rho$. We say that a state is reachable by at most i steps of $\rho$ (or i-reachable for short, when $\rho$ is clear from the context) if it can be reached by at most i applications of $\rho$ starting from some initial state. We denote the set of i-reachable states by $\mathcal {R}_i$. We say that a state is reachable if it is i-reachable for some i. We say that $TS$ satisfies a safety property $\mathcal {P}$ if all reachable states satisfy $\mathcal {P}$. We often define ${ Bad}\mathop {=}\limits ^{\mathrm{def}}\lnot \mathcal {P}$, and refer to states satisfying Bad as bad states. We define $\rho \mathop {=}\limits ^{\mathrm{def}} Cond \wedge { wlp}( Cmd , { Id})$, where ${ wlp}({ Cmd}, { Id})$ denotes the weakest liberal precondition of the loop body and ${ Id}$ is a conjunction of equalities between $\mathcal {V}$ and $\mathcal {V}'$ (see [31] for more details). We define $Init$ and ${ Bad}$ using the programs pre- and post- conditions: $Init \mathop {=}\limits ^{\mathrm{def}}{ Pre}$ and ${ Bad}\mathop {=}\limits ^{\mathrm{def}}\lnot Cond \wedge \lnot { Post}$. That is, a state is initial if it satisfies the pre-condition, and it is bad if it satisfies the negation of the loop condition (which indicates termination of the loop) but does not satisfy the post-condition. This captures the requirement that when the loop terminates the post-condition needs to hold.

Example 3

In Example 2, $Init \mathop {=}\limits ^{\mathrm{def}}(i=h) \wedge (j = null )$ and $Bad \mathop {=}\limits ^{\mathrm{def}}(i = null ) \wedge \lnot (h \ne null \rightarrow (\forall v.\, n^{*}(h,v)\rightarrow C(v)))$. Note that these refer to the pre- and post-conditions that should hold right before the loop begins and right after it terminates, respectively. Here, a state is bad if $i = null$ (i.e., it occurs when the loop terminates) and h points to a non-empty list that contains an element not having the property C.

Invariants. An invariant of a program is a property that should hold for all reachable states. It is inductive if it is closed under application of $\rho$.

Definition 2

(Invariants). Let $TS= ({ Init}, \rho )$ be a transition system and $\mathcal {P}$ a safety property over $\mathcal {V}$. A formula $\mathcal {I}$ is a safety inductive invariant (invariant, in short) for $TS$ and $\mathcal {P}$ if (i) ${ Init}\Rightarrow \mathcal {I}$, and (ii) $\mathcal {I}\wedge \rho \Rightarrow (\mathcal {I})'$, and (iii) $\mathcal {I}\Rightarrow \mathcal {P}$.

If there exists an invariant for $TS$ and $\mathcal {P}$, then $TS$ satisfies $\mathcal {P}$. An invariant is universal if it is equivalent to a universal formula in prenex normal form. We note that the invariants inferred by $\mathsf PDR ^{\forall }$ are conjunctions of universal clauses, where a universal clause is a universally quantified disjunction of literals (positive or negative atomic formulae).

3 Universal-Property-Directed Reachability

In this section, we present Universal Property Directed Reachability ($\mathsf PDR ^{\forall }$), an algorithm for checking if a transition system $TS$ satisfies a safety property $\mathcal {P}$. $\mathsf PDR ^{\forall }$ is an adaptation of Bradley’s property-directed reachability (IC3) algorithm [9] that uses universal formulae instead of propositional predicates [9, 22, 29] or predicate abstraction [32]. We use Example 2 as a running example throughout this section.

Requirements. We require that the transition relation $\rho$, as well as the ${ Init}$ and ${ Bad}$ conditions, are expressible in a first-order logic $\mathcal {L}$ (We can partly handle transitive closure using the approach of [31]. See Sect. 5.) We require that every satisfiable formula in $\mathcal {L}$ has a finite model, and assume to have a decision procedure $SAT (\psi )$, which checks if a formula $\psi$ in $\mathcal {L}$ is satisfiable, and a function $model (\psi )$, which returns a finite model $\sigma$ of $\psi$ if such a model exists and None otherwise.

Fig. 2.

Graphical depiction of models found during the analysis of the running example.

3.1 Diagrams as Structural Abstractions

$\mathsf PDR ^{\forall }$ iteratively strengthens a candidate invariant by retrieving program states that lead to bad states and checking whether the retrieved states are reachable. In that sense, $\mathsf PDR ^{\forall }$ is similar to IC3. The novel aspect of our approach is the use of diagrams [10] to generalize individual states into sets of states before checking for reachability. Diagrams provide a structural abstraction of states by existential formulae: The diagram of a finite model $\sigma$, denoted by $\textit{Diag}(\sigma )$, is an existential cube which describes explicitly the relations between all the elements of the model.²

Definition 3

(Diagrams). Given a finite model $\sigma = (\mathcal {U},\mathcal {I})$ over alphabet $\mathcal {V}$, the diagram of $\sigma$, denoted by $\textit{Diag}(\sigma )$, is a formula over alphabet $\mathcal {V}$ which denotes the set of models in which $\sigma$ can be isomorphically embedded. $\textit{Diag}(\sigma )$ is constructed as follows.

• For every element $e_i \in \mathcal {U}$, a fresh variable $x_{e_i}$ is introduced.

• $\varphi _{distinct}$ is a conjunction of inequalities of the form $x_{e_i} \ne x_{e_j}$ for every pair of distinct elements $e_i \ne e_j$ in the model.

• $\varphi _{constants}$ is a conjunction of equalities of the form $c = x_{e}$ for every constant symbol c such that $\sigma \models c = e$.

• $\varphi _{atomic}$ is a conjunction of atomic formulae which include for every predicate $p \in \mathcal {V}$ the atomic formula $p(\bar{x_e})$ if $\sigma \models p(\bar{e})$, and $\lnot p(\bar{x_e})$ otherwise.

Then: $\textit{Diag}(\sigma ) \mathop {=}\limits ^{\mathrm{def}}\exists x_{e_1} \ldots x_{e_{|\mathcal {U}|}}. \varphi _{distinct} \wedge \varphi _{constants} \wedge \varphi _{atomic}$ .

Intuitively, one can think of $\textit{Diag}(\sigma )$ as the formula produced by treating individuals in $\sigma$ as existentially quantified variables and explicitly encoding the interpretation of every constant and every predicate using a conjunction of equalities, inequalities, and atomic formulae. For example, the diagram of $\sigma _b$, depicted in Fig. 2($\sigma _b$), is

$$ \begin{array}{rcl} \textit{Diag}(\sigma _b) &{} \mathop {=}\limits ^{\mathrm{def}}&{} \exists x_0,x_1,x_2.\, x_0 \ne x_1 \wedge x_0 \ne x_2 \wedge x_1 \ne x_2 \wedge \\ &{} &{} h=x_0 \wedge j=x_1 \wedge i=x_2 \wedge null = x_2 \wedge \\ &{} &{} \lnot C(x_0) \wedge \lnot C(x_1) \wedge \lnot C(x_2) \wedge \\ &{} &{} n^{*}{x_0}{x_0} \wedge n^{*}{x_1}{x_1} \wedge n^{*}{x_2}{x_2} \wedge n^{*}{x_0}{x_1} \wedge \\ &{} &{} \lnot n^{*}{x_0}{x_2} \wedge \lnot n^{*}{x_1}{x_0} \wedge \lnot n^{*}{x_1}{x_2} \wedge \lnot n^{*}{x_2}{x_0} \wedge \lnot n^{*}{x_2}{x_1}\ . \end{array} $$

The first line records the fact that the universe of $\sigma _b$ consists of three elements. The second line characterizes the interpretations of all the constant symbols in $\sigma _b$. The other lines capture precisely the interpretation of predicates C and $n^{*}$ in $\sigma _b$.

Lemma 1

Let $\sigma$ be a model over $\mathcal {V}$, and let $\phi$ be a closed existential first-order formula over $\mathcal {V}$. If $\sigma \models \phi$ then $\textit{Diag}(\sigma ) \Rightarrow \phi$.

Semantically, Lemma 1 means that for any models $\sigma$ and $\sigma _i$ such that $\sigma _i \models \textit{Diag}(\sigma )$ if $\sigma \models \phi$ then $\sigma _i \models \phi$. This implies that if a bad state is reachable from $\sigma$ and the program can be proven correct using an inductive universal invariant $\mathcal {I}$ then all the states in $\sigma$’s diagram are unreachable too: $\mathcal {I}$ is an inductive invariant, thus any state $\sigma$ leading to a bad state must satisfy (closed existential) formula $\lnot \mathcal {I}$. Hence, $\textit{Diag}(\sigma ) \Rightarrow \lnot \mathcal {I}$, which means that all states satisfying $\textit{Diag}(\sigma )$ are unreachable. In this sense, the abstraction based on diagrams is precise for programs with universal invariants.

3.2 Data Structures and Frames

$\mathsf PDR ^{\forall }$ is shown in Algorithm 1. It uses procedures $\textit{block}()$ and $\textit{analyzeCEX}()$, shown in Algorithms 2 and 3, respectively, as subroutines. The algorithm uses an array $F$ of frames, where a frame is a conjunction of universal clauses. For clarity, we refer to the ith entry of the array using subscript notation, i.e., $F_{i}$ instead of $F[i]$. Intuitively, frame $F_{i}$ over-approximates $\mathcal {R}_i$, the set of i-reachable states. The algorithm also maintains a frame counter N which records the number of frames it developed. We refer to $F_{0}$ as the initial frame, to $F_{N}$ as the frontier frame, and to any $F_{i}$, where $0 \le i < N$, as a back frame.

$\mathsf PDR ^{\forall }$ maintains several invariants which ensure that every frame $F_{i}$ is an over-approximation of $\mathcal {R}_i$, and hence that the sequence of developed frames is an over-approximation of all the traces of the program of length $N+1$ or less. Technically, this means that the algorithm constructs an approximate reachability sequence.

Definition 4

Let $TS= ({ Init}, \rho )$ be a transition system and $\mathcal {P}$ a safety property. A sequence $\langle F_0, F_1,\ldots ,F_{N}\rangle$ is an approximate reachability sequence for $TS$ and $\mathcal {P}$ if:

• (i) ${ Init}\Rightarrow F_0$.

• (ii) $F_i \Rightarrow F_{i+1}$, for all $0 \le i < N$, i.e., for every state $\sigma$, if $\sigma \models F_{i}$ then $\sigma \models F_{i+1}$.

• (iii) $F_i \wedge \rho \Rightarrow (F_{i+1})'$, for all $0 \le i < N$, i.e., for every transition $(\sigma _1,\sigma _2) \models \rho$, if $\sigma _1 \models F_{i}$ then $\sigma _2 \models F_{i+1}$.

• (iv) $F_i \Rightarrow \mathcal {P}$, for all $0 \le i \le N$.

Items (ii) and (iii) ensure that every frame includes the states of the previous frame and their successors, respectively. Together with item (i), it follows by induction that for every $0 < i \le N$ the set of states (models) that satisfy $F_i$ is a superset of the set $\mathcal {R}_{i}$. Furthermore, by item (iv) no frame includes a bad state.

3.3 Iterative Construction of an Approximate Reachability Sequence

$\mathsf PDR ^{\forall }$ is an iterative algorithm. At every iteration, the algorithm either strengthens the Nth frame, if it contains a bad state, or starts to develop the $N\!+\!1$th frame, otherwise. In addition, in every iteration, it might also strengthen some of the back frames. Each strengthening of frame $F_{i}$ is performed by determining a universal clause $\varphi _i$ which holds for any i-reachable state, and then conjoining $F_{i}$ with $\varphi _i$.

Initialization. The algorithm first checks that the initial states and the bad states do not intersect. If so, it exits and returns the state that satisfies both $Init$ and $Bad$ as a counterexample (line 2). Otherwise, it sets $F_{0}$ to represent the set of initial states (line 3), $F_{1}$ to represent all possible states (line 4), and the frame counter to 1. Note that at this point, $F_{1}$ is a trivial over-approximation of the set of initial states and their successors, but it might contain bad states.

Iterative Construction. The algorithm then starts its iterative search for an inductive invariant (line 6). Recall that when the algorithm develops the Nth frame, it has already managed to determine an approximate reachability sequence $\langle {F_0,\ldots ,F_{N-1}\rangle }$. Hence, every iteration starts by checking whether a fixpoint has been reached (line 7).If true, then an inductive invariant proving unreachability of ${ Bad}$ has been found, and the algorithm returns valid (line 8). Otherwise, the algorithm keeps on strengthening the frontier frame $F_{N}$ by searching for a bad witness, a bad state in the frontier frame (line 9). If no such state exists, it means that no bad state is N-reachable. Moreover, at this point $\langle F_0,\ldots ,F_{N}\rangle$ is an approximate reachability sequence. Thus, the iterative strengthening of $F_{N}$ terminates and a new frontier frame is initialized to $true$ (line 10 and 11).

If the frontier frame contains a bad witness, i.e. $F_{N} \wedge Bad$ is satisfiable, then there might be an N-reachable bad state. Due to our requirement for finite satisfiability of the logic, the bad witness is a finite model. Given a bad witness $\sigma _b$ (line 13), the algorithm tries to determine whether it is indeed reachable, and thus the program does not satisfy its specification, or whether $\sigma _b$ was discovered due to some over-approximation in one of the back frames. This check is done by invoking procedure $\textit{block}()$ with the index of the frontier frame and $\sigma _b$ as parameters (line 14). The latter either returns a counterexample, determines that it is impossible to prove the specification using a universal invariant (in the given logic and vocabulary), or strengthens the frontier frame to exclude the set of states in the diagram of $\sigma _b$, and possibly strengthens some back frames too (see below). The iterative construction and strengthening of the frames continues until reaching a fixpoint, finding a counterexample, or determining the absence of a universal invariant.³

Example 4

When analyzing the running example, our algorithm discovers that state $\sigma _b$, shown in Fig. 2, is a bad witness when $F_{1}= true$, and thus it invokes $\textit{block}(1,\sigma _b)$. In this example, $\textit{block}()$ succeeds to block $\sigma _b$. Unfortunately, the strengthened frame $F^1_1$ (see below) still has bad models. Therefore, the iterative strengthening continues and the next iterations find $\sigma _b'$, depicted in Fig. 2, as a bad witness model for $F^1_1$, $\sigma _b''$ as a bad witness model of $F^2_1$ and $\sigma _b'''$ as a bad witness model of $F^3_1$. At that point, however, the algorithm determines that the strengthened frame $F^4_1$ does not have a bad witness. $\langle F_{0},F^4_1 \rangle$ is now an approximate reachability sequence and $\mathsf PDR ^{\forall }$ goes on and initializes a new frame, $F_{2}$, to $true$, and the search for an inductive invariant continues.

Diagram-Based Abstract Blocking. Procedure $\textit{block}(j,\sigma )$, shown in Algorithm 2, gets an index of a frame $j= 0 \cdots N$ and a state $\sigma$ which is included in the jth frame, i.e., $\sigma \models F_{j}$, and tries to determine whether $\sigma$ is j-reachable. The unique aspect of our approach is the way in which it abstracts $\sigma$ to a set of states in order to accelerate the strengthening routine. Namely, the use of diagrams. More specifically, $\mathsf PDR ^{\forall }$ computes the diagram $\varphi$ of $\sigma$ (line 21) and then checks whether there is a j-reachable state satisfying $\varphi$. Importantly, due to Lemma 1, if a universal invariant exists then the generalization of $\sigma$ to its diagram will not include any reachable state, hence the abstraction is precise in the sense that it maintains unreachability. In this case the strengthening of $F_{j}$ is also guaranteed to succeed, excluding not only $\sigma$, but its entire diagram.

The check if the diagram $\varphi$ of $\sigma$ includes a j-reachable state is done conservatively by determining whether some state of $\varphi$ is an initial state or has a predecessor in $F_{j-1}$. (Recall that $F_{j-1}$ over-approximates $\mathcal {R}_{j-1}$.) The former is equivalent to checking if $\varphi \wedge Init$ is satisfiable. Note that if we reached the initial frame, i.e., if $j=0$, then $\sigma \models Init$, hence the above formula is guaranteed to be satisfiable. Explicitly checking that $\varphi \wedge Init$ is satisfiable is required only at the second frame, i.e., if $j=1$:

Lemma 2

For every $1 < j \le N$, when $\textit{block}(j,\sigma )$ is called, $F_{i} \Rightarrow \lnot \textit{Diag}(\sigma )$ for every $i \le j-1$. In particular, $Init \Rightarrow \lnot \textit{Diag}(\sigma )$.

If the algorithm finds an adverse initial state, i.e., an initial state satisfying $\varphi$, (line 22),⁴ it invokes procedure $\textit{analyzeCEX}()$ for further analysis (see below). Otherwise, the algorithm checks if the formula $\delta = F_{j-1} \wedge \rho \wedge (\varphi )'$ is satisfiable (line 24),⁵ i.e., whether some state of $\varphi$ has a predecessor in $F_{j-1}$. There can be two cases:

Case I. If $\delta$ is unsatisfiable then no state represented by $\varphi$ is j-reachable. Hence, $F_{j}$ remains an over-approximation of $\mathcal {R}_{j}$ even if any state of $\varphi$ is excluded. The exclusion is done by conjoining the jth frame with the universal formula $\lnot \varphi$ (line 28), and results in a strengthening of $F_{j}$. In fact, $\lnot \varphi$ is conjoined to any back frame (line 27). We refer to the exclusion of the states of $\varphi$ as the blocking of (the diagram of) $\sigma$ from frame $F_{j}$.

Example 5

In our running example, in the first iteration $\textit{block}(1,\sigma _b)$ updates $F^0_1$ to $F^1_1= true \wedge \lnot \textit{Diag}(\sigma _b)$. This excludes $\sigma _b$, but also all states where $i=null$, C is empty, and j is n-reachable from h in any (nonzero) number of steps. In later iterations $\textit{block}$ updates $F^2_1 = F^1_1 \wedge \lnot \textit{Diag}(\sigma _b')$, $F^3_1 = F^2_1 \wedge \lnot \textit{Diag}(\sigma _b'')$, and $F^4_1 = F^3_1 \wedge \lnot \textit{Diag}(\sigma _b''')$.

Case II. If $\delta$ is satisfiable, then there exists an adverse state $\sigma _a$ in frame $F_{j-1}$, a state which is the predecessor of some state of the diagram of $\sigma$ that we try to block at frame $F_{j}$. Note that $\sigma _a$ is not necessarily a predecessor of $\sigma$ itself. The adverse state $\sigma _a$ is found by taking the reduct of a (finite) model of $\delta$ (line 25). If an adverse model $\sigma _a$ exists then the algorithm recursively tries to block it from $F_{j-1}$ (line 26). The recursive procedure continues until the adverse state is either blocked or the algorithm finds an adverse initial state (line 22). Note that blocking an adverse state during the development of the Nth frame leads to a strengthening of some back frame $F_{i}$, and thus tightens its over-approximation of $\mathcal {R}_{i}$.

Finding Concrete Counterexamples and Proving the Absence of Universal Invariants. Procedure $\textit{analyzeCEX}()$, shown in Algorithm 3, is called when an adverse initial state is found. Such a state indicates that an abstract counterexample exists:

Definition 5

(Abstract and Spurious Counterexamples). A sequence of formulae $\langle \phi _j, \phi _{j+1} \cdots \phi _N\rangle$ is an abstract counterexample if the formulae $\phi _j \wedge { Init}$, $\phi _N \wedge { Bad}$, and $\varphi _i \wedge \rho \wedge (\phi _{i+1})'$, for every $i=j\cdots N-1$, are all satisfiable. The abstract counterexample is spurious if there exists no sequence of states $\langle \sigma _j, \sigma _{j+1} \cdots \sigma _N\rangle$ such that $\sigma _j \models { Init}$, $\sigma _N \models Bad$, and for every $j \le i < N$, $(\sigma _{i}, \sigma _{i+1}) \models \rho$.

An abstract counterexample does not necessarily describe a real counterexample. In fact, if $j \ne 0$, the counterexample is necessarily spurious (as, if a real counterexample shorter than N had existed, the algorithm would have already terminated during the development of the $N-1$th frame). However, when $j=0$, the algorithm determines if the abstract counterexample is real or spurious by checking whether a bad state can be reached by N applications of the transition relation (line 31). Technically, $\textit{analyzeCEX}()$ can be implemented using a symbolic bounded model checker [5]. If a real counterexample is found, the algorithm reports it (line 35). Otherwise, the obtained counterexample is spurious. Technically, this means that the property is neither verified nor falsified. In our case, the algorithm can determine that the verification effort is doomed: The spurious counterexample is in fact a proof for the absence of a universal invariant (see Proposition 1).

Generalization of Blocked Diagrams. Rather than blocking a diagram $\phi$ from frames $0 \cdots j$ by conjoining them with the clause $\lnot \phi$ (line 28), our implementation uses a minimal unsat core of $\psi = (( Init )' \vee (F_{j-1} \wedge \rho )) \wedge (\varphi )'$ to define a clause L which implies $\lnot \phi$ and is also disjoint from $Init$ and unreachable from $F_{j-1}$. Blocking is done by conjoining L with $F_{i}$ for every $i \le j$.⁶

4 Correctness

In this section we formalize the correctness guarantees of $\mathsf PDR ^{\forall }$. We recall that if $\mathsf PDR ^{\forall }$ terminates it reports that either the program is safe, the program is not safe, providing a counterexample, or the program cannot be verified using a universal inductive invariant.

Lemma 3

Let $TS= ({ Init}, \rho )$ be a transition system and let $\mathcal {P}$ be a safety property. If $\mathsf PDR ^{\forall }$ returns valid then $TS$ satisfies $\mathcal {P}$. Further, if $\mathsf PDR ^{\forall }$ returns a counterexample, then $TS$ does not satisfy $\mathcal {P}$.

Proof

$\mathsf PDR ^{\forall }$ returns valid if there exists i such that $F_{i+1}\Rightarrow F_i$. Therefore, $F_i \wedge \rho \Rightarrow (F_{i+1})' \Rightarrow (F_i)'$. Recall that, by the properties of an approximate reachability sequence, ${ Init}\Rightarrow F_0 \Rightarrow F_i$ and $F_i \Rightarrow \mathcal {P}$. Therefore, $F_i$ is an inductive invariant, which ensures that $TS$ satisfies $\mathcal {P}$. The second part of the claim follows immediately from the definition of a counterexample.    $\Box$

Proposition 1

Let $TS= ({ Init}, \rho )$ be a transition system and let $\mathcal {P}$ be a safety property. If $\mathsf PDR ^{\forall }$ obtains a spurious counterexample $\langle \phi _j \cdots \phi _N\rangle$ then there exists no universal safety inductive invariant $\mathcal {I}$ for $TS$ and $\mathcal {P}$.

Proof

Assume that there exists a universal safety inductive invariant $\mathcal {I}$ over $\mathcal {V}$. We show by induction on the distance $N-i=0 \cdots N$, of $F_{i}$ from $F_N$ that every state $\sigma _i$ generated by $\mathsf PDR ^{\forall }$ at frame $F_i$ is such that $\sigma _i \models \lnot \mathcal {I}$. This implies, by Lemma 1, that every diagram $\phi _i$ generated by $\mathsf PDR ^{\forall }$ at frame $F_i$ is such that $\phi _i \Rightarrow \lnot \mathcal {I}$, and hence $\phi _i \Rightarrow \lnot { Init}$. (Recall that by definition ${ Init}\Rightarrow \mathcal {I}$, i.e., $\lnot \mathcal {I}\Rightarrow \lnot { Init}$). This contradicts the existence of a spurious counterexample, where $\phi _j \wedge { Init}$ is satisfiable.

The base case of the induction pertains to $F_{N}$. It follows immediately from the property that a state $\sigma _N$ generated at frame $F_N$ is a model of the formula $F_N \wedge { Bad}$, and in particular is a model of ${ Bad}= \lnot \mathcal {P}$, i.e., $\sigma _N \models \lnot \mathcal {P}$. Since $\mathcal {I}\Rightarrow \mathcal {P}$, or equivalently $\lnot \mathcal {P}\Rightarrow \lnot \mathcal {I}$, we conclude that $\sigma _N \models \lnot \mathcal {I}$.

Consider a state generated at frame $F_i$. Then $\sigma _i$ is the reduct of a model of the formula $F_i \wedge \rho \wedge (\textit{Diag}(\sigma _{i+1}))'$ to $\mathcal {V}$. Moreover, by the induction hypothesis, $\sigma _{i+1} \models \lnot \mathcal {I}$. Since $\lnot \mathcal {I}$ is an existential formula, this means by Lemma 1 that $\textit{Diag}(\sigma _{i+1}) \Rightarrow \lnot \mathcal {I}$. We conclude that $F_i \wedge \rho \wedge (\textit{Diag}(\sigma _{i+1}))' \Rightarrow F_i \wedge \rho \wedge (\lnot \mathcal {I})'$. Therefore, $\sigma _{i}$ is also (a reduct of) a model of the formula $F_i \wedge \rho \wedge (\lnot \mathcal {I})'$. If we assume that $\sigma _i \models \mathcal {I}$, we would get that $\mathcal {I}\wedge \rho \wedge (\lnot \mathcal {I})'$ is satisfiable, in contradiction $\mathcal {I}$ being inductive. Hence, $\sigma _i \models \lnot \mathcal {I}$.    $\Box$

Example 6

Procedure traverseTwo(), presented in Figure 3 together with its pre- and post-condition, traverses two lists until it finds their last elements. If the lists have a shared tail then p and q should point to the same element when the traversal terminates. The program indeed satisfies this property. However, this cannot be proven correct using an inductive universal invariant: Take, as usual, $Init$ to be the procedure’s precondition and $\mathcal {P}$ to be the safety property whose negation is $Bad = (i = null \wedge j = null ) \wedge \lnot post$, where $post$ is the procedure’s postcondition. Consider the state $\sigma _0$ depicted in Figure 4. Clearly, this model satisfies ${ Init}$. Therefore, if $\mathcal {I}$ exists, $\sigma _0 \models \mathcal {I}$. $\sigma _0$ is a predecessor of $\sigma ^t_1$ and hence it should be the case that $\sigma ^t_1 \models \mathcal {I}$. Now consider $\sigma _1$, which is a submodel of $\sigma ^t_1$ and interprets all constants as in $\sigma _1$. If $\mathcal {I}$ is universal, then $\sigma _1 \models \mathcal {I}$ as well. However, $\sigma _1 \not \models \mathcal {P}$, in contradiction to the property of a safety invariant. Indeed, when using $\mathsf PDR ^{\forall }$, the spurious counterexample $\langle {\sigma _0,\sigma _1,\sigma _2}\rangle$ presented in Figure 4 is obtained. This indicates that no universal invariant for $\mathcal {P}$ exists. Note that state $\sigma _1$ is a predecessor of $\sigma _2$ and recall that $\sigma _0$ is a predecessor of $\sigma ^t_1$. The spurious counterexample was obtained because $\sigma ^t_1$ satisfies the diagram of state $\sigma _1$.

Fig. 3.

A procedure that finds the last elements of two non-empty acyclic lists.

Fig. 4.

A spurious counterexample found for procedure traverseTwo(), shown in Fig. 3.

5 Implementation and Empirical Evaluation

$\mathsf PDR ^{\forall }$ is parametric in the vocabulary, and can be implemented on top of any decision procedure for finite satisfiability of first-order logic formulae. The language of these formulae should be expressive enough to capture the assertions, transition system, and space of candidate invariants. Our algorithm is not guaranteed to terminate, thus the underlying logic does not have to be decidable. Our implementation, however, uses $EA^{R}$ which is a decidable logic [31].

$EA^{R}$ allows for relational first-order formulae with a quantifier prefix of the form $\exists ^*\forall ^*$ and a deterministic transitive-closure operator $^*$, but forbids functional symbols. We use $n^*$ to construct reachability constraints over the pointer field n, e.g., in Examples1 and 2, and to define the “next” relation n [31] using a universal formula. We note that The latter can be done only when the prefix of the resulting formula is of the form $\exists ^*\forall ^*$.

$EA^{R}$ satisfiability is reducible to effectively-propositional (EPR) satisfiability, also known as the Bernays-Schönfinkel-Ramsey class, and hence is decidable and enjoys the small model property, i.e., every satisfiable formula in $EA^{R}$ is guaranteed to have a finite model. Technically, the reduction introduces axioms (EPR formulae) that capture the reflexivity, transitivity, acyclicity and linearity properties of the $^*$ operator [31].

Table 1. Experimental results. Running time is measured in seconds. N denotes the highest index for a developed frame $F_i$. “# Z3” denotes the number of calls to Z3. AF denotes “Abstraction Failure” of [32]. TO means timeout ($>$ 1 hr). (a) Correct programs; “# Cl. ($\forall$)” = number of ($\forall$-)clauses in the inferred invariant. (b) Correct programs for which there is no universal inductive invariant. (c) Incorrect program; “C.e. size”is the maximal number of elements in a model that arises in the counterexample.

Benchmarks. We implemented $\mathsf PDR ^{\forall }$ and applied it to a collection of procedures that manipulate singly-linked lists, doubly-linked lists, multi-linked lists, and implementations of an insertion-sort algorithm [16], and a union-find algorithm [16]. Our experiments were conducted using a 3.6GHz Intel Core i7 machine with 32GB of RAM, running Ubuntu 14.04. We used the 64bit version of Z3 4.4 [19] with the default settings to check satisfiability of EPR formulae. Table 1 summarizes our experimental results.

(a) Verification. Our analyzer successfully verified memory safety, i.e., the absence of null-dereferences and of memory leaks, preservation of data-structure integrity, meaning that the procedure never creates cycles in the list, and functional correctness of several singly- and doubly-linked list manipulating procedures. The precondition says that the expected input is a (possibly empty) acyclic list, and the post-condition is the one expected from the procedure’s name. For example, the post-condition of reverse() is that it returns a list comprised of the same elements as in its input, but in reversed order. To verify the absence of memory leaks, we used a unary predicate $alloc(\cdot )$ to record whether a node is allocated. To verify the other properties, we used auxiliary predicates to mark the elements of the input list and record the reachability order between them.

We also verified the correctness of several procedures that manipulate sorted lists: sorted-insert() inserts an element into its appropriate place in a sorted list, sorted-merge() creates a sorted list by merging two sorted ones, and bubble-sort() and insertion-sort() sort their input lists. We represented the order on data elements by a binary predicate together with the appropriate axioms.

In addition, we verified several procedures that manipulate multi-linked lists: overlaid-delete() takes an overlaid list and deletes a given element. (Overlaid lists use multiple pointer fields to index the same set of elements in different orders.) nested-split() moves all the elements not satisfying C into a sublist. flatten() takes a nested list and flattens it by concatenating its sublists. ladder() creates a copy t of a list h and places a pointer p from every element in h to its counterpart in list t. We then verify that the p field of every element in h points to a distinct element in list t. This property indicates, indirectly, that both lists have the same length. Finally, we verify the union-find algorithm. E.g., for compressing find() operation, we prove the it maintains the reachability between every node and its root and preserves the elements.

We compared our results to [32], where $EA^{R}$ was used to verify properties of list-manipulating programs with PDR, using human-supplied (universally-quantified) abstraction predicates as templates. We note that [32] can also establish certain functional correctness properties, but theirs are strictly weaker than ours. For example, they do not verify that a reversed list does not contain more elements than in its input list.

(b) Verifying the Absence of Universal Invariants. Our tool was also able to show that certain properties cannot be verified with a universal invariant. It proved that procedure shared-tail(), described in Example 6, does not have a universal invariant. We applied our tool to procedure comb(), which is a simplified version of ladder() where the newly allocated elements are not linked together, hence resulting in a heap shaped like a comb. The tool discovered that it is not possible to use a universal invariant to prove that when comb() terminates there is no null-valued p-field in the input list.

(c) Bug Finding. We also ran our analysis on programs containing deliberate bugs. In all of the cases, the method was able to detect the bug and generate a concrete trace in which the safety or correctness properties are violated.

6 Related Work

Synthesizing quantified invariants has received significant attention. Several works have considered discovery of quantified predicates, e.g., based on counterexamples [18] or by extension of predicate abstraction to support free variables [24, 33]. Our inferred invariants are comprised of universally quantified predicates, but unlike these approaches, our computation of the predicates is property directed and does not employ predicate abstraction. Additional works for generation of quantified invariants include using abstract domains of quantified data automata [25, 26] or ones tailored to Presburger arithmetic with arrays [20], instantiating quantifier templates [8, 38], applying symbolic proof techniques [30], or using abstractions based on separation logic [4, 21].

Other works aim to identify loop invariants given a set of predicates as candidate ingredients. Houdini [23] is the first such algorithm of which we are aware. Santini [39, 40] is a recent algorithm which is based on full predicate abstraction. In the context of IC3, predicate abstraction was used in  [7, 12, 32], the last of which specifically targeting shape analysis. In contrast to previous work, our algorithm does not require a pre-defined set of predicates, and is therefore more automatic: The diagrams provide an “on-the-fly” abstraction mechanism.

PDR has been shown to work extremely well in other domains, such as hardware verification [9, 22]. Subsequently, it was generalized to software model checking for program models that use linear real arithmetic [29] and linear rational arithmetic [11]. The latter employs a quantifier-elimination procedure for linear rational arithmetic to provide an approximate pre-image operation. In contrast, our use of diagrams allows us to obtain a natural approximation which is precise for programs that can be verified using universal invariants.

The reduction we use into EPR creates a parametrized array-based system (where the range of the arrays are Booleans). A number of tools have been developed for general array-based systems. The SAFARI [3] system is relevant. It is related to MCMT and Cubicle [14, 15, 27, 28], SAFARI uses symbolic pre-conditions to propagate symbolic states in the form of cubes that are conjunctions of literals over array constraints, and uses interpolants to synthesize universal invariants. Our method for propagating and inductively generalizing diagrams differs by being based on PDR.

The logic used by our implementation has limited capabilities to express properties of list segments that that are not pointed to by variables [32]. This is similar to the self-imposed limitations on expressibility used in a number of shape analysis algorithms [4, 21, 34–37, 41]. Past experience, as well as our own, has shown that despite these limitations it is still possible to successfully analyze a rich set of programs and properties.

7 Conclusions

$\mathsf PDR ^{\forall }$ is a combination of PDR/IC3 [9] with the model-theoretic notion of diagrams [10]. The latter provide PDR an aggressive strengthening scheme in which the structural properties of a bad state are abstracted “on-the-fly” by a formula describing all of its possible extensions, which are then blocked together within the same iteration of PDR’s main refinement loop. This obviates the need for user-supplied abstraction predicates. This form of automation is particularly important when one tries to verify tricky programs, e.g., programs that manipulate unbounded data structures, against a variety (of possibly changing) specifications. Indeed, our implementation successfully analyzed multiple specifications of tricky list-manipulating programs, discovered counterexamples, and, uniquely to our approach, showed that certain programs cannot be proven correct using a universal invariant. Interestingly, we noticed that sometimes the tool had to work harder to verify simple properties than when it was asked to verify complicated ones. In particular, verifying partial correctness properties was done faster when verified together with memory safety than without. In hindsight, this might not be surprising due to the property guided nature of the analysis.

We are very pleased with the simplicity of our approach and believe that the notion of diagram-based abstractions is particularly useful for the verification of programs that manipulate unbounded state. In the future, we plan to apply it in other contexts too, e.g., for the verification of network programs [1].