Repository navigation
Incremental refresh #23
Description
Hey, for https://github.com/louis030195/obsidian-ava, I'm trying to implement increment refresh of the state of the vault.
Concretely, I build sentence embeddings of the whole vault and would like to re-compute embeddings every time a note is updated/deleted/created.
Do you see any way of doing this incrementally rather than reloading the vault and recomputing everything every time? (It takes ~1 min on mps device on my 500k words vault)
Ideally, I'd see maybe an API that let me listen to vault changes with callback(s) in this library?
Thanks 🚀😃
Activity
mfarragher commentedon Dec 9, 2022
These are a few API changes that I think can enable this:
gather()to be called withoutconnect()I can see how incremental refresh would be useful for NLP analytics. With that there would only be the dictionaries of text to be updated. Incremental refresh to update info populated via
connect()seems complex in contrast.When looking at libraries that monitor file changes in a directory, there are a few out there, but they don't seem to be cross-platform. They seem Linux-focused. I won't add functionality for this in obsidiantools via a new method or class, but think it'll be interesting to see what recipes can be made for this.
mfarragher commentedon Dec 14, 2022
So far I've added this to the code: 5b73e6f
I have also been prototyping some code to listen to file changes with another library. More work is needed to play around with async for the attr updates.
mfarragher commentedon Dec 28, 2022
More attr setters are available since this commit: 6880842