Back to the room

Pass 3 of 3 · Deep dive of PHAROSPass 1 · OverviewPass 2 · How it works

Code Observatory / method

encode_raw

codeintel.embeddings.OnnxEmbedder.encode_raw

PINNED SOURCE Source and edges from the index at this commit. An execution appears only where one was recorded.

PHAROS at 975a38dMIT license

Around this symbol

Search all symbols

Loading

Calls (2)
SymbolFileResolved by
_padembeddings.pysame file
clipbase.pyunique name
Inside (1)
SymbolFileResolved by
OnnxEmbedderembeddings.pysame file
codeintel.embeddings.OnnxEmbedder.encode_rawbackend/app/codeintel/embeddings.py, lines 344–364Open source
    def encode_raw(self, texts: list[str]) -> list[list[float]]:        import numpy as np        self._load()        out: list[list[float]] = []        for i in range(0, len(texts), self.batch_size):            batch = self._tokenizer.encode_batch(texts[i : i + self.batch_size])            ids = np.array([e.ids for e in batch], dtype=np.int64)            mask = np.array([e.attention_mask for e in batch], dtype=np.int64)            feeds = {"input_ids": ids, "attention_mask": mask}            if "token_type_ids" in self._input_names:                feeds["token_type_ids"] = np.zeros_like(ids)            hidden = self._session.run(None, feeds)[0]  # (batch, seq, dim)            if self.pooling == "cls":                pooled = hidden[:, 0, :].astype(np.float32)            else:                m = mask[..., None].astype(np.float32)                pooled = (hidden * m).sum(axis=1) / np.clip(m.sum(axis=1), 1e-9, None)            pooled /= np.clip(np.linalg.norm(pooled, axis=1, keepdims=True), 1e-12, None)            out.extend(_pad(v.tolist(), self.dim) for v in pooled)        return out

What the index recorded

Signature
def encode_raw(self, texts: list[str]) -> list[list[float]]:
File
backend/app/codeintel/embeddings.py
Lines
344–364
Parser
python-ast
Git blob
df7f8202f23ca6ad2c57aca5f09b435eb6486d79
Call sites
14 named, 2 resolved, 12 left unlinked

Read it on GitHub at this commit