GENERATIVE MODELS FOR GRAPH-BASED PROTEIN DESIGN

The task of finding viable design is referred to as inverse protein finding problem. Focus on long-range in sequence but local in 3D space.

1 Intro

Top-down framework By structured self-attention layers, this model can capture higher-order, interaction-based dependencies between sequences and structures. (Previous method can only solve first-order effects) A well-finding evidence: the long-range dependencies are generally short-range in 3D space. One advantage is to facilitate adaptation to specific parts of the 3D structure space.

Our method is similar to Graph Attention Networks, but augmented with edge features and an autoregressive decoder.

2 Methods

2.1 Representing Structure

Edge formulates sequence.

3D Considerations Expected to satisfy two properties: (1) Invariance. (2) Locally Informative.

Structural Encoding Define in terms of the geometry encoding. The backbone geometry: $$Oi = [b_i n_i b_i \times n_i]$$ $$b_i$$ is the negative bisector of angle between the rays $$(x{i-1} - xi)$$, $$(x{i+1} - x_i)$$ $$n_i$$ is a unit vector normal to that plane

$$ ui = \frac{x_i- x{i-1}}{ | xi- x{i-1} |}, bi = \frac{u_i - u{i-1}}{| ui - u{i-1} |}, ni = \frac{u_i \times u{i+1}}{| ui \times u{i+1} |}

Then get the spatial edge feature. It related reference frame $$(x_i, O_i)$$ to reference frame $$(x_j, O_j)$$. We decompose it into features for distance, direction, and orientation as:

$$ e_{ij} = Concat( r(| x_j - x_i |), O_i^T \frac{x_j - x_i}{| x_i - x_j|}, q(O_i^T O_j))

Positional encodings: We need to model the position of each neighbor j relative to the node under consideration i. Therefore we get the position embedding as a sinusoidal function of the gap i-j.

Node and edge features We get an aggregate edge encoding vector $$e{ij}$$ by concatenating structural encoding $$e^{(s)}{ij}$$ and positional encoding $$e^{(p)}_{ij}$$.

For node features, compute the three dihedral angles of the protein backbone $$(\phi_i, \psi_i, \omega_i)$$ and embed these on the 3-torus as $${sin, cos} \times (\phi_i, \psi_i, \omega_i)$$.

2.2 Structured Transformer

Introduce the Structured Transformer model that stems from the self-attention based Transformer model.

We restrict each node by considering only k-nearest neighbors in 3D space.

Autoregressive decomposition

$$p(s|x) = \prodi p(s_i | x, s{< i})$$

Such conditionals are parameterized by two sub-networks:

an encoder that computes refined node embeddings from structure-based node features $$V(x)$$ and edge features $$\epsilon (x)$$
a decoder that autoregressively predicts letter $$s_i$$ given the preceding sequence and structural embeddings from the encoder

GENERATIVE MODELS FOR GRAPH-BASED PROTEIN DESIGN

GENERATIVE MODELS FOR GRAPH-BASED PROTEIN DESIGN