# Find topics

> Group a collection's cards into named topics, written as notes you can see in Graph view.

Topic Map groups the cards into broad **families** and finer **topics**, names each one, and writes them as notes. Because topics are notes that link to their sources, Graph view shows the clusters.

## Before you start

The collection is indexed.

## Steps

1. Run **Topic Map: Collection: Find topics…** and pick the collection. Or select **Find topics…** in the collection's dialog.
2. Optionally add **seeds**: subjects you want a topic for whatever the clustering finds. One per line, as `Name = what to search for`.
3. Read what a run would do. No model has been called.
4. Select **Dry run** to see the clusters' sizes and each seed's hits. This spends only the seeds' embeddings.
5. Select **Find topics**. Naming the topics runs as a job, with its cost.
6. Read the list of notes it would write, then select **Write**.

Clustering runs inside Obsidian, spread over your computer's cores: a few seconds for hundreds of cards, about half a minute for thirteen thousand on a recent Mac.

Runs are fastest with Obsidian visible. While clustering runs, Topic Map asks Obsidian's window to stay at full speed even when it's hidden behind other windows, and sets it back afterwards. When that isn't possible, the run summary says so, and clustering slows down while Obsidian is hidden.

## What you get

Under `Topic Map/Topics/<collection>/`, with **Topic note layout** (Settings → Topic Map → Folders) on its default, *One note per family*:

- a note per family: its description, a list of its topics, then each topic as a `##` section holding its description and every card with its quote, linking the book's note or the source note, and a **passage** link that opens the passage (with **Open in book** for a book);
- a `Seeded` folder with a note per seed.

A topic's heading is its name, with the characters a heading link can't hold (`# | ^ : [ ] % / \`) turned into spaces; two topics in one family with the same name get ` 2`, ` 3`. So a topic is linked as `[[<Family name>#<Topic name>]]`, and the panel's **Open in family note** goes straight to its section. Topics that belong to no family are sections of a note called `No family`.

Set the layout to *One note per topic* and you get the older shape: a folder per family, with the family's note linking its topics, and a note per topic holding what its section holds. The next **Write** after a change replaces the notes in the other layout. The dry run says how many notes it writes, replaces and removes (to the trash), and **Undo a run** brings the old ones back.

A term that has several versions in a topic is one line in the topic note, with its versions folded under it (see [Repeated terms](repeated-terms.md)). Topic notes pick this up at a Find topics run, never at an index run. When the groups file can’t be read, the notes are written without groups and the run says "Repeated terms aren’t grouped" and why. When it is out of date, the run says "Groups are out of date" instead; **Group again** on the panel’s **Groups** tab fixes both.

With **Color the Graph view** on (Settings → Topic Map → Folders; off by default), the Graph view gets a color group per family, beside your own groups.

A family or topic note you have edited since Topic Map wrote it is kept as it is on the next run, also when you switch layout: it stays where it is, and the dry run and the result say so.

## Names are paid for once

Topic Map keeps every name it has paid for, beside the index. A later run reuses a topic's name when the topic has the same cards, the lens's naming prompt is the same and the naming task asks the same model first. Only the topics that changed are named again. So a second run after a small change, or a step between **Fewer**, **Auto** and **More**, costs little.

- A family is named again when any of its topics was renamed. When any family's first name changes, every family is named again with its siblings in view.
- A loose topic gets its second try as before. The second try's name is kept too.
- The dry run says how many names it would reuse, for example "18 of 22 names reused from earlier runs, 4 to name". The finished run says how many it reused.
- Names paid for before a run stopped at its spend limit, or was canceled, are kept. The next run asks only for the rest.
- **More options → Name topics again** asks for every name afresh.
- **A reply with no name is asked for once more.** If a naming reply comes back with a description but no name (empty, only spaces, or "unnamed cluster"), Topic Map asks again with the same prompt. If the second reply has no name either, the topic or family keeps the name "unnamed cluster" and the run says so ("the reply had no name") in its summary and dialog. Such a reply is never kept.
- **Topics already called "unnamed cluster"** (from a run before this check) need no **Name topics again**. The next Find topics run asks again for each kept name that is empty or "unnamed cluster" and reuses all the others. That is one model call per such topic, plus a call for each family and sibling pass whose names changed as a result.

Undo a run leaves the kept names alone: one is reused only for the same cards, prompt and model.

## How topics check themselves

Every run tunes and checks its own topics, with no question asked, and says what it did when it finishes:

- **The strongest directions are removed first.** On a large collection, the strongest patterns in the card vectors are about how a card is written (a short term or a long claim), not what it is about. Find topics removes the three strongest before clustering. It keeps any that belongs mostly to one book.
- **The number of families and topics is measured.** The run tries a few sizes on a sample and keeps the steadiest one that passes every check:
  - few topics come from a single book;
  - no handful of families holds most of the passages;
  - no two families are nearly the same.

  Numbers you type in the dialog are used as they are.
- **Families are named against each other.** Each family is named a second time with the other families' names in view, leaving out words most of them share.
- **Overlapping families.** Two families with close names and close cards, which the namer also says are the same, are merged. The family note says so. Pairs where only some of those agree wait for you on the panel's **Topics** tab, folded above the families (nothing shows when there is nothing to decide).
- **Mostly from one book.** A topic whose cards come 80% or more from one book says so, and is listed last in its family.
- **Loose topics get a second try.** A topic whose cards don't share a subject is named once more from cards its first naming didn't see. Only then is it flagged loose.
- **Probes.** Short terms from the collection's context are searched. A term whose cards spread over more than two families is named in the run's summary.

Each run's numbers are kept in `topics.json`, at the end of `topics.md`, and on the run's record beside its cost. The panel's **Runs** tab shows only what changed since the last Find topics, with the full numbers folded beneath. The numbers are:
- the sizes and how they were chosen;
- the closest pair of families;
- the share of single-book topics;
- the loose topics;
- the merges;
- the probes.

### Answer overlapping families

On the panel's **Topics** tab, open *pairs of families may overlap*:

- **Merge** joins the two families.
- **Keep apart** leaves them as they are. On a family merged on its own, it splits the merge again.

Each answer shows what Write would change first. **Write** rewrites the topic notes and keeps your answer, so later runs follow it without asking again. **Undo a run** takes back both.

### Fewer or more topics, and pinned families

In the Find topics dialog:

- **Fewer · Auto · More** sets how many families and topics a run aims for.
- **More options** holds exact counts, a spend limit and **Name topics again**.
- **Pinned families** lists the families the collection always wants. Add one as a name and what to search for, or remove one.

A change to the size or the pins is written to the collection note when you choose **Write**, and the proposal lists it first.

### Settings on the collection note

All optional:

| Property | What it does |
|---|---|
| `families: [Name = what to search for]` | Families you always want. Their cards are set aside before clustering, so a small theme isn't absorbed into a larger one. |
| `topics: {probes: [term, term]}` | Terms to check for spreading, in place of phrases from the context. |
| `topics: {size: fewer}` | `fewer`, `auto` (the default) or `more` families and topics. |
| `topics: {remove_directions: 0}` | How many strongest directions to remove (3 by default; 0 clusters the vectors as they are). |
| `topics: {auto_merge: false}` | Never merge on its own: every overlapping pair waits for you. |

## See a topic in the graph

Open a topic, family or seed note and run **Open local graph** (a topic note’s right-click menu, or **Topic Map: Setup: Commands…** → This topic). The local graph opens pinned to that note, in the collection's colors when **Color the Graph view** is on. Graph views already open show new color groups once reopened.

## Related

- [Draft a write-up](draft-a-write-up.md)
- [Undo a run](undo-a-run.md)