Pass 3 of 3 · Deep dive of PHAROSPass 1 · OverviewPass 2 · How it works
Code Observatory / class
SentenceTransformerEmbedder
codeintel.embeddings.SentenceTransformerEmbedder
PINNED SOURCE Source and edges from the index at this commit. An execution appears only where one was recorded.
PHAROS at 975a38dMIT license
Around this symbol
Search all symbolsLoading
| Symbol | File | Resolved by |
|---|---|---|
| get_embedder | embeddings.py | same file |
| Symbol | File | Resolved by |
|---|---|---|
| __init__ | embeddings.py | same file |
| _encode | embeddings.py | same file |
| _load | embeddings.py | same file |
| embed_documents | embeddings.py | same file |
| embed_query | embeddings.py | same file |
| encode_raw | embeddings.py | same file |
| Symbol | File | Resolved by |
|---|---|---|
| embeddings | embeddings.py | same file |
codeintel.embeddings.SentenceTransformerEmbedderbackend/app/codeintel/embeddings.py, lines 167–226Open source
class SentenceTransformerEmbedder(_PrefixMixin, Embedder): def __init__(self, model_name: str, device: str | None = None, batch_size: int = 32): self.model_name = model_name self.prefixes = _use_prefixes(model_name) self.model_id = model_name + ("+prefix" if self.prefixes else "") self.batch_size = batch_size self._device = device self._model = None self._lock = threading.Lock() def _load(self): if self._model is not None: return self._model with self._lock: if self._model is None: try: from sentence_transformers import SentenceTransformer except ImportError as exc: raise EmbeddingUnavailable( "sentence-transformers is not installed " "(pip install -r requirements-edge.txt)" ) from exc device = self._device if device is None: try: import torch device = "cuda" if torch.cuda.is_available() else "cpu" except ImportError: device = "cpu" logger.info("Loading embedding model %s on %s", self.model_name, device) try: self._model = SentenceTransformer( self.model_name, device=device, trust_remote_code=True, # Avoids a native crash seen with pytorch_model.bin on # Windows + Python 3.13 (see project history). model_kwargs={"use_safetensors": True}, ) except Exception as exc: raise EmbeddingUnavailable(f"failed to load {self.model_name}: {exc}") from exc return self._model def _encode(self, texts: list[str]) -> list[list[float]]: model = self._load() vectors = model.encode( texts, batch_size=self.batch_size, normalize_embeddings=True, show_progress_bar=False, convert_to_numpy=True, ) return [_pad(v.tolist(), self.dim) for v in vectors] def embed_documents(self, texts: list[str]) -> list[list[float]]: return self._encode([self._doc(t) for t in texts]) def embed_query(self, text: str) -> list[float]: return self._encode([self._query(text)])[0]Shows the first 60 of 63 lines. The rest is on GitHub at this commit.
What the index recorded
- Signature
class SentenceTransformerEmbedder(_PrefixMixin, Embedder):- File
backend/app/codeintel/embeddings.py- Lines
- 167–229
- Parser
- python-ast
- Git blob
df7f8202f23ca6ad2c57aca5f09b435eb6486d79
