---
title: 'Indexing code by function, class and method'
canonical: 'https://www.dealmobile.co.uk/writing/indexing-code-by-function-class-and-method'
url: 'https://www.dealmobile.co.uk/writing/indexing-code-by-function-class-and-method'
type: Post
published: '2026-10-21'
summary: 'How Trove splits source code into functions, classes and methods with tree-sitter, re-indexes only the files that changed, and where its extractors fall short.'
---

# Indexing code by function, class and method

Published 2026-10-21

![Diagram of one source file split into four chunks: a function, a class summary (highlighted), and two methods, each labelled with its name.](https://www.dealmobile.co.uk/sites/default/files/styles/almanac_shot_md/public/content/images/posts/indexing-code-by-function-class-and-method/hero.png.webp?itok=C9QRtQj9)

[Trove](https://www.dealmobile.co.uk/projects/trove.md) indexes code as well as documents, so an AI coding tool can ask where something happens and get the code back. Splitting code by size, as you might a document, works badly: a chunk that starts halfway through one function and ends halfway through the next is a poor answer to any question. So Trove parses each source file with tree-sitter and makes one chunk for each function, class or method, labelled with its name, file and lines. This is how that works, and where it falls short. Trove is archived, so this is the public code as it stood.

## One extractor per language

`core/code/parser.py` picks an extractor by file extension, from a map in `core/code/extractors/__init__.py`. There are ten: Python, PHP, JavaScript, TypeScript, HTML, CSS, YAML, JSON, Gherkin and Dart. Some extensions share one: `.vue`, `.svelte` and `.mdx` go to the HTML extractor, and `.scss` and `.less` to the CSS one.

The grammars come ready-built in the tree-sitter-languages package, and each extractor loads its parser the first time it’s needed:

```python
@property
def parser(self):
    """Lazy-load the tree-sitter parser."""
    if self._parser is None:
        import tree_sitter_languages
        self._parser = tree_sitter_languages.get_parser(self.tree_sitter_language)
    return self._parser

```

The rest is each extractor walking the syntax tree for the node types it knows. A file that can’t be read or parsed gives no chunks and a warning, and the run carries on.

## What a chunk holds

Each extractor returns `CodeChunk` objects, defined in `extractors/base.py`. The content is the source text of the node, and the metadata says what it is: the language, the kind of symbol, its name and lines, and where they exist, its signature, docstring, parent class and visibility. Chunks from a git repository also get its name, remote and branch. Every code chunk is marked `chunk_type: code`, which is what the code searches filter on.

The Python extractor shows the approach. A top-level function becomes one chunk holding the whole function, with its parameters and return type as the signature and its docstring pulled out. Each method becomes a chunk of its own, with its class recorded as the parent. Visibility comes from the name: one leading underscore is protected and two is private. Module-level assignments are kept too, when the name is in capitals or starts with an underscore.

A class is the interesting case. Indexing the whole class would repeat every method a second time, and a long class would become one big chunk that matches everything a little. So the class chunk is a summary: the definition line, the docstring and a list of the methods.

```python
class_content = class_def_line
if docstring:
    class_content += f'\n    """{docstring}"""'
if method_names:
    class_content += f"\n\n    # Methods: {', '.join(method_names)}"

```

A question about what a class is for finds the summary, and a question about how it does something finds the method. The summary still records the class’s full line range, so a tool that wants the whole class can read it from the file.

JavaScript works the same way, with more shapes to find: function declarations, arrow functions assigned to a variable, such as `const load = async (id) => ...`, and exports. It remembers the byte range of everything it has taken, so a function inside a class or an export isn’t indexed twice. The TypeScript extractor runs the JavaScript one and then adds interfaces, type aliases and enums. `.tsx` files get the TSX grammar, because, as a comment in the code says, the TypeScript grammar hangs on JSX.

## Re-indexing only what changed

Code changes all day, and embedding a whole repository again for one edited file is slow. So `core/code/incremental.py` keeps a state file with each file’s modification time, a SHA-256 hash of its contents and the IDs of the chunks it produced. On the next run, a file whose modification time hasn’t changed is skipped without being read. A file whose time has changed is hashed, and only a different hash sends it back through the parser.

The chunks of every changed or deleted file are removed from ChromaDB by ID before the new ones go in. The IDs are stable, built from the file, the kind and name of the symbol, its first line and a hash of its content:

```python
key = f"{source}:{symbol_type}:{symbol}:{line_start}:{content_hash}"
return hashlib.sha256(key.encode()).hexdigest()[:12]

```

Inside a repository, the file list comes from `git ls-files --cached --others --exclude-standard`, which reads git’s index instead of walking the disk and leaves out whatever `.gitignore` excludes. Files over 500 KB are skipped by default. With more than ten files, parsing runs in separate processes, each building its own parser, because tree-sitter’s language objects can’t be passed between processes.

## Searching it

The MCP server has two tools just for code. `trove_search_code` runs the usual keyword and vector search, keeps only code chunks, and can narrow them by language and kind of symbol. `trove_find_symbol` searches for a name, then prefers an exact match:

```python
exact_matches = [
    r for r in results
    if r.get("metadata", {}).get("symbol_name") == symbol_name
]

```

If nothing matches exactly, it says so and shows the five closest results instead. Names survive the keyword index well. The tokenizer lowercases text and keeps runs of letters, digits and underscores, so `fetch_user` stays one word and `fetchUser` becomes `fetchuser`, and a search for the exact name matches it by keyword.

A client can also switch on cross-referencing in its search settings. For each of the first five document chunks in the results, Trove pulls likely code names out of the text (CamelCase words, snake\_case words and anything in backticks) and runs a quick search for matching code. The idea is that documentation goes out of date and the code doesn’t.

## What it misses

Running the extractors over a few short sample files for this post turned up two gaps.

The Python extractor looks only for `function_definition` and `class_definition` nodes directly inside the module or the class body. tree-sitter wraps anything with a decorator in a `decorated_definition` node, so decorated code is skipped: a `@property`, a `@staticmethod`, a `@dataclass` class, a function under `@lru_cache`. It’s missing from the class summary’s list of methods too. There’s code to record a function’s decorators, but since decorated functions are never reached, no chunk ever gets any. Given a plain function, a decorated one, a class with a plain method, a `@property` and a `@staticmethod`, and a `@dataclass`, it found the plain function, the class and one method.

The TypeScript extractor gets its classes from the JavaScript one, which looks for the class name as an `identifier`. The TypeScript grammar calls it a `type_identifier`, so TypeScript classes, and every method in them, aren’t indexed at all. Functions, arrow functions, interfaces, types and enums are.

There are smaller problems around search. The `code_first` search mode filters the results to code, adds 0.5 to every code chunk’s score and switches on cross-referencing. The filter runs first, so cross-referencing finds no document chunks to start from, and adding the same amount to every score changes no order: in practice it’s a code-only search. The cross-referencing’s pattern for file names has a capturing group around the extension, and `re.findall` returns only the group, so `src/api/client.ts` comes back as `ts` and is dropped for being too short. And the setup wizard writes the file extensions you choose to `code.extensions`, which the indexer never reads. It reads `code.languages`, and without that, it indexes every extension it supports.

Trove is archived, so these are written down here rather than fixed.

## Using it

Code indexing is switched on for each client in its `client.yaml`, with the extensions listed under `languages`, which is where the indexer reads them:

```yaml
code:
  enabled: true
  paths: ["src/"]
  languages:
    python: {extensions: [".py"]}
    typescript: {extensions: [".ts", ".tsx"]}
  exclude_patterns: ["**/node_modules/**", "**/dist/**"]

```

Then index the code on its own:

```bash
python scripts/ingest.py --client myproject --code-only

```

Later runs re-parse only the files that changed, and `--rebuild-code` starts from scratch. From Claude Code, `trove_find_symbol` finds a function or class by name, and `trove_search_code` answers a question such as “where are retries handled” with the functions themselves.

The code is on [GitHub](https://github.com/tomgould/trove) under the Business Source License 1.1.

- **Projects:** [Trove](https://www.dealmobile.co.uk/projects/trove.md)
- **Tags:** [Python](https://www.dealmobile.co.uk/writing/tag/python.md), [Search](https://www.dealmobile.co.uk/writing/tag/search.md), [AI](https://www.dealmobile.co.uk/writing/tag/ai.md), [Trove](https://www.dealmobile.co.uk/writing/tag/trove.md)

---

Index of this site: [llms.txt](https://www.dealmobile.co.uk/llms.txt)