This adds History.Helpers.get_text_data() helper to fetch contents of a
"text" message. This removes the need to know internals of message
structure outside of history module.
It is unwieldy to handle fields in opt table one by one. Luckily there
is vim.bl_extend("force", ...) that can help us here.
Also move annotation for opts from parameter to a separate class to make
it easier to work with.
- Refactor TODOs handling: move logic to `context_messages` instead of using `template_opts`
- Improve todo message filtering by using more precise tags (`<todos>...</todos>`)
- Append TODOs after history messages
- Remove the TODOs section from `_task-guidelines.avanterules` template
Signed-off-by: doodleEsc <cinuor@gmail.com>
Commit a9b01e7 ("chores: use stopinsert (#1745)") replaced uses of
vim.api.nvim_feedkeys(...) with proper commands to exit and enter insert
mode. However "stopinsert" does not help when editor is in visual mode.
Unfortunately there is no dedicated command to exit visual mode so we
have to go back to simulating keystrokes again.
Now that we have a table listing all containers and can iterate over it
we no longer need a separate table with window IDs that we need to
maintain. Remove it.
Maintaining secondary table of window IDs is cumbersome and is prone to
getting out of sync with the true state of the sidebar. In preparation
for removal of winids table move all containers (sub-windows of the
sidebar) into "containers" table.
The change is mostly mechanical rename with following exceptions:
- Sidebar:reifresh_winids() and other places where the code scanned
entire Sidebar object looking for tables with specific fields, such
as "winid", or "mount" needed to be adjusted for the new structure
- Sidebar:new() and Sidebar:reset() have been adjusted to make better
use of the new sub-table.
Under the hood vim.list_extend() does the same loop over the source
table and uses table.insert() to append the values to the destination
table, so using a temporary table makes little sense.
Switch to directly inserting (appending) values into history_lines
table.
When updating chat history to be used in LLM request there are several
instances where we do O(n^2) operations: scanning all messages to locate
a tool "use" and for each use scan messages again to locate
corresponding "result".
Refactor the code to scan messages once collecting tool uses and results
together, and then do 2nd scan to drop incomplete tool invocations and
refresh "view" and "edit" results with the latest content.
Also reduce number of pre-scan loops (where we discard partially
generated messages or messages that are not interesting or too-old) by
combining them when possible.
This reduces time to scan initial 417 messages on my system (which
result in 576 final messages) from 0.32 to 0.12 seconds.
Rate limit handling seems to be broken: it starts an one-shot timer
with callback that reschedules the timer each second and prints/updates
messages. Simultaneously it schedules a deferred by 1 second function
that cancels the timer and resumes the stream. This results in the timer
executing at most once, and stream resume happening way too early.
Fix this by switching to use repeating timer with first instance
executing immediately. All message handling is moved into the timer
callback. Once countdown is complete the same callback will stop and
destroy the timer and resume the stream.
Move code related to converting history messages into UI representation
from utils into history/render.lua for better code organization and
clean up the implementation.
Code dealing with scanning history messages and extracting some data or
state belongs to avante/history/ so move it there. Along with the move
update the implementation to make use of get_tool_use_data() and
get_tool_result_data() helpers to simplify it.
Also rename the function to History.get_pending_tools() to better
reflect its purpose: "uncalled" means something that was done without
request, unsolicited, not something that has not completed yet.
Simplify the code locating viewed files as well as code locating last
completion attempt by using these new helpers.
Also avoid using "goto" in simple loops.
Tool results are not useful without their "use" messages, so delete them
when deleting "use" ones.
Makes use of the new get_tool_use_data() and get_tool_result_data()
helpers.
There are many places in the code that wants to work with content of
"tool use" and "tool result" messages. Currently such code uses
is_tool_use_message() and is_tool_result() message to check if message
is of right kind, and then pokes into message internals. This is not
very efficient.
Introduce get_tool_use_data() and get_tool_result_data() that would
return contents of the message if it is of right kind, or nil otherwise.
Also introduce get_tool_result() that attempts to locate result of a
tool execution by its invocation ID.
The utils module has grown too big and contains unrelated functionality.
Start moving code related to managing history messages comprising chat
history into lua/avante/history module to keep the code more manageable.
Sorting function in FileSelector:get_filepaths() is not very efficient
since it has to hit the filesystem each time it is called to determine
if it is dealing with a directory or not.
Optimize performance by figuring this data in get_project_filepaths()
and returning a list of structures containing paths and directory flags,
and use this data in get_filepaths(). Do the absolute->relative
conversion once.
Also use vim.fn.isdirectory() instead of vim.loop.fs_stat() everywhere
in the file.
The FileSelector:get_filepaths() function incorrectly filters selected
files due to a mismatch in path formats (relative vs. absolute).
Fix the issue by converting relative file paths to absolute paths before
doing the comparison. Use a set instead of re-scanning the table when
filtering out duplicates.
get_project_filepath() converts candidate file paths to relative form
twice, once in the first iterator, and then in vim.tbl_map() call while.
It also iterates the same list twice for no apparent reason.
Combine both scans into one and use dedicated vim.fn.isdirectory().
