adds zsh support to waveterm. big change, lots going on here. lots of other improvements and bug fixes added while debugging and building out the feature.
Commits:
* refactor shexec parser.go into new package shellenv. separate out bash specific parsing from generic functions
* checkpoint
* work on refactoring shexec. created two new packages shellapi (for bash/zsh specific stuff), and shellutil (shared between shellapi and shexec)
* more refactoring
* create shellapi interface to abstract bash specific functionality
* more refactoring, move bash shell state parsing to shellapi
* move makeRcFile to shellapi. remove all of the 'client' options CLI options from waveshell
* get shellType passed through to server/single paths for waveshell
* add a local shelltype detector
* mock out a zshapi
* move shelltype through more of the code
* get a command to run via zsh
* zsh can now switch directories. poc, needs cleanup
* working on ShellState encoding differences between zsh/bash. Working on parsing zsh decls. move utilfn package into waveshell (shouldn't have been in wavesrv)
* switch to use []byte for vardecl serialization + diffs
* progress on zsh environment. still have issues reconciling init environment with trap environment
* fix typeset argument parsing
* parse promptvars, more zsh specific ignores
* fix bug with promptvar not getting set (wrong check in FeState func)
* add sdk (issue #188) to list of rtnstate commands
* more zsh compatibility -- working with a larger ohmyzsh environment. ignore more variables, handle exit trap better. unique path/fpath. add a processtype variable to base.
* must return a value
* zsh alias parsing/restoring. diff changes (and rtnstate changes). introduces linediff v1.
* force zmodload of zsh/parameter
* starting work on zsh functions
* need a v1 of mapdiff as well (to handle null chars)
* pack/unpack of ints was wrong (one used int and one use uint). turned out we only ever encoded '0' so it worked. that also means it is safe to change unpack to unpackUInt
* reworking for binary encoding of aliases and functions (because of zsh allows any character, including nulls, in names and values)
* fixes, working on functions, issue with line endings
* zsh functions. lots of ugliness here around dealing with line dicipline and cooked stty. new runcommand function to grab output from a non-tty fd. note that we still to run the actual command in a stty to get the proper output.
* write uuid tempdir, cleanup with tmprcfilename code
* hack in some simple zsh function declaration finding code for rtnstate. create function diff for rtnstate that supports zsh
* make sure key order is constant so shell hashes are consistent
* fix problems with state diffs to support new zsh formats. add diff/apply code to shellapi (moved from shellenv), that is now specific to zsh or bash
* add log packet and new shellstate packets
* switch to shellstate map that's also keyed by shelltype
* add shelltype to remoteinstance
* remove shell argument from waveshell
* added new shelltype statemap to remote.go (msh), deal with fallout
* move shellstate out of init packet, and move to an explicit reinit call. try to initialize all of the active shell states
* change dont always store init state (only store on demand). initialize shell states on demand (if not already initialized). allow reset to change shells
* add shellpref field to remote table. use to drive the default shell choice for new tabs
* show shelltag on cmdinput, pass through ri and remote (defaultshellstate)
* bump mshell version to v0.4
* better version validation for shellstate. also relax compatibility requirements for diffing states (shelltype + major version need to match)
* better error handling, check shellstate compatibility during run (on waveshell server)
* add extra separator for bash shellstate processing to deal with spurious output from rc files
* special migration for v30 -- flag invalid bash shell states and show special button in UI to fix
* format
* remove zsh-decls (unused)
* remove test code
* remove debug print
* fix typo
* first draft at ui
* first draft connecting openai to cmd info ui
* added more updates and added prompt engineering
* focus textarea on mount
* fix aichat layout issue
* added focus back to input after closing cmd info chat
* added ability to clear chat history, closes on command enter, better initial greeting message, and small things like user can refocus the help textarea by pressing ctrl + space again
* added a bunch of small upgrades
* added context for chatgpt info command
* added context to chatgpt cmd info command
* added ability to use arrow keys to grab code markdown
* added error handling as well as fixed the text area outline bug
* slightly changed the ui for selecting
* revamped ui and added correct scroll behavior
* changed ui, fixed up code block selection, some other small changes
* deep copied cmd info chat when we return
* added error checking and removed logs
* made deep copy code more clear
* prettier with the right version
* more prettier changes
* removed extraneous comment
* init
* integrate showShellPrompt flag
* renive debugging code
* remove debugging code
* run gofmt. add migration files.
* remove debugging code
* remove migrations and adjust code. show prompt on import ssh configs as well.
* fix show/hide logic
* reset mmap.go
* use resolveBool and utilfn.ContainsStr
* make AlertModal take a generic 'confirmkey' instead of hard coding hideShellPrompt
* rename confirmkey to confirmflag (to be consistent). move confirmflag checking into the alertmodal. short circuit with Promise.resolve(true) if noConfirm checked.
* disable buttons while status is 'connecting'
* minor refactor
The ssh_config library only loaded the config data the first time
certain functions are used. After that, cached values are used instead.
This library has been updated in a fork to add a method that allows the
configuration files to be reloaded. This update takes advantage of that
change so the config files can be updated without restarting waveterm.
* add an alert modal for the sshconfig import button
When the sshconfig import button is pressed, there currently is no
visual indicator of what changed. This adds an alert modal to pop up
only in the case where the gui button is used.
* improve alert modal for sshconfig imports
The previous message for SSH configuration imports was vague and did not
provide detailed information as what happened during the import. This
clarifies that by specifying which remotes were deleted, created, and
updated. Updates are only ran and recorded if they would actually change
something.
* fix port value limiting
The SSH config import port limiting was correct but set off a warning in
linters. It has been updated to do the same behavior in a different way.
Also, port limiting was never added to manually adding a new remote.
This change adds it there as well.
* change user-facing term to connection
Previously, the ssh configuration alert modal used to use the word
"remote" to describe connections. "Remote" is the internal name but it
isn't consistent with what is being displayed to users. So it has been
replaced with "Connection" instead to match.
* change remote to connection for ssh import buttons
Like the previous change, the word "remote" was used instead of
"connection." This was for the tooltips added to connections that had
been imported in the connections menu.
* update one more remote -> connection
* work on cmd-d to delete a selected line
* call stoppropagation when code.tsx captures keyboard input
* finish up with line delete. add a trash icon to line. prevent delete when cmd is running (show error msg)
* remove byte sanitization for user commands
When serializing jsonBytes in packet.go::MarshalPacket, a step existed
that attempted to manually sanitize the bytes before sending them. This
was initially done to avoid invalid characters in json; however, go
should handle this for us. But this sanitization broke
internationalization because it excluded characters required for unicode
in other languages. Because of that, it has been removed.
* properly decode non-ascii on frontend
The functions atob and btoa do not convert base 64 to strings in the
expected way. The base64ToArray function handles it properly but other
cases do not. These other cases have been replaced with a helper
function that makes use of the base64-js package. This package has
already been included as a dependency of another package we use, but it
was added to the package.json file to make the inclusion explicit.
* automatically set/share LANG var with waveshell
Waveterm previously did not set the LANG environment variable which
caused problems for international users. On Linux, this is done
automatically, but it needs to be done manually on macos.
Even on linux, the wavesrv LANG variable is shared with waveshell to
ensure the same one is used on remotes.
* only set the lang var if not previously set
In order to prevent waveterm from overriding the lang var entirely, this
ensures that it is only manually determined if it hasn't previously been
set.
* use envMap instead of os to determine var
This is slightly more performant and relies more directly on our code
instead of external code.
* allow numerical start and allow dots in ssh alias
We previously restricted the ssh alias to start with an
alphabetic character and did not allow dots within it. This caused
problems with users wanting to use an ip address as an alias. This lifts
that restriction so both can freely be used.
Note that while it is common to use the hostname as the alias, hostnames
are not allowed to use the underscore character. However, we do allow
the alias to contain an underscore. I do not think we can remove this
from aliases now but it is something to watch out for in the future.
* allow backslash in username
This involved converting the regexp string into a raw string to able to
use the backslash character. this is important.
* Revert "allow backslash in username"
This reverts commit cafe281241. Upon
reflection, more time is needed to evaluate that this actually corrects
the issue. It will be performed with proper diligence at a later time.
* add port to end of canoncial names when not 22
The canonical name is the identifying key in the database, so it causes
problems if another remote entry has the same canonical name. By adding
the port number to the end of this, it is possible to differentiate the
two.
* add db migrations for adding port to canonicalname
The up migration adds the port to the existing canonical id if it exists
and is not 22. The down migration strips the port off the canonical name
and deletes extra remotes with the same canonical name. If all remotes
with that canonical name have been archived, it keeps the first one
added to the database. If any have not been archived, it keeps the first
added to the database from the non-archived group.
* remove ability to edit port number via ssh import
Previously, ssh imports could edit the port number since it was possible
to change them in the config file without changin the canonical name.
Now that the canonical name contains the port, a change in the port will
simply create a new database entry. For this reason, the ability to
modify the ssh port is dead code and has been removed.
* allow backslash in username
This involved converting the regexp string into a raw string to able to
use the backslash character. this is important.
* simplify up migration logic
The previous up migration was suboptimal because it was accounting for a
corner case not found in production. That case no longer needs to be
considered, so the procedure was simplified.
* fix commands that require quoted paths
Several commands did not wrap the path in quotes which caused problems
when attempting to store the waveterm installation in a place that had a
space in the path. This corrects this in the particular case where the
username does not have spaces but the path to the executable does.
Note: the case of a user name having spaces has not been tested but
likely does not work.
* fix logging problem for wavesrv
A previous fix replaced the getWaveSrvCmd with getWaveSrvPath. This
needs to be getWaveSrvCmd to enable logging.
* fix variable name
* create migrations for required database change
This is a first attempt that does not appear to be working properly. It
requires review.
* fix errors in db migrations
The previous commit had an extra json call that broke the update and did
not remove the imported interies during a downgrade.
* change migrations to use column instead of json
It makes more sense to associate the source of a config with the remote
type than the sshopts type. This change makes that clear in the database
structure.
* ensure adding a remote manually tags correctly
Using the usual way of adding a remote should result in a sshconfigsrc
of "waveterm-manual". This will be important for filtering out remotes
installed manually and remotes installed via import
* create basic structure for parsing ssh config
This entails creating a new command, making it possible to query only
the imported remotes from the database, and implementing the logic to
handle all of the updates needed.
This needs improvements in a few areas:
- the /etc/ssh/config needs to be parsed as well
- the logic for editing exisiting imported remotes needs to be written
- error handling needs to be improved
- update packet responses need to be provided
* add sshkey support and implement editing
We now search for the ssh identity keyfile and add it if it is found.
Additionally, the logic to edit previously imported ssh hosts has been
added.
* combine hosts from user and system ssh config
We now check both the user ~/.ssh/config as well as the /etc/ssh/config
for hosts. This loops through each file starting with the user one. For
each host, it selects the first pattern without a wildcard and chooses
that to be the alias. If any future hosts are found to have the same
alias, they are skipped. Errors are raised if neither config file can be
opened or no aliases were found.
* improve logging and error reporting
Error reporting is now shortcircuited in cases of individual remotes in
order to allow the other remotes to continue. These errors are now
printed to logs instead.
* allow imports to edit ssh port
Previously, ssh ports could not be edited after the fact. Unfortunately,
this can cause problems since the port can be changed in an ssh config
file. To address this, we allow imports to change the port if a host
with the same canonical name had previously been imported.
* fix response to parse command
* fix error handline for alias parsing
Small mistake of checking for equality instead of inequality
* fix the ability to overwrite hostName with alias
if ssh_config does not find Hostname, it won't output an error. Now we
compare against the result instead of looking for an error.
* fix the error catching for User and Port
This fixes the same problem where parsing the config doesn't give an
error in the case when nothing is found. As before, this checks for a
blank result instead.
* remove unused code
* remove repeated canonical name check
The logic that checks for an existing canonical name already exists in
the AddRemote function, so it is not needed here.
Secondly, we now only allow edits of previously created remotes if they
have not been archived. If they have, the usual logic for creating a new
remote takes precedence.
Lastly, there is no need to archive a remote that has already been
archived so an additional check has been added.
* allow archives to preserve the SSHConfigSrc
* add log message for archiving of imported remotes
* create variables for string variants
Matches existing code style
* add cleanup for opened files
* move migration 25 to migration 26 (already merged a migration 25)
* fix RemoteRuntimeState in ModelUpdate by moving type to sstore.go. Fix some bugs in remote:parse. Fix key/identityfile, return value, and remote editing (should go through msh). remote sudo. add info messages around parse status
* fix issue with archiving the sshconfigsrc
A bug in RemoteType's FromMap caused the loss of sshconfigsrc during the
conversion. This has been corrected and the schema has been updated.
* fix order of archiving removed imported remotes
Previously, if the canonical name changed, the code would try to create
a new remote before archiving the old one. This did not work if the
alias didn't change. Now we archive first and add a new remote after.
* fix ability to change port when importing config
Importing from sshconfig needs to allow the port to change. This was not
happening because of a bug that has been corrected.
* always use host in place of hostname
Since host is the key actually searched for in the ssh config file,
searching for user@hostName may not actually work. To avoid this, we now
always use user@host instead.
* automatically determine ConnectMode
This aims to select a connection mode based off what is provided in the
ssh config file. It aims for auto connections when possible but will
fall back to manual if we can't easily support it
* remove sshkeysource migration number confilict
Previously had conflicting migration numbers of 26. The change not in
the main branch has been moved to 27 to remove the conflict.
* move sshkeysource migration to migration 28
* add WaveOptions flag parsing for ssh config
This is currently being used to allow users to force manual connect mode
if desired. It will also be used to force skipping options in the future
but that is not complete in this commit.
* implement ignore flag for ssh config parsing
The ignore flag will now archive an imported remote if it previously
existed and not create a new remote in its place.
* fix discovery of identity file
Previously, a ~ in the identity file's path was not expanded to the home
dir. Because of this, files with a ~ were previously identified as
invalid files. By expanding it during the search, this is no longer the
case.
* disable frontend edit button for imported remotes
Imported Remotes should not be editable in waveterm by users. This edit
makes it clear that the button will not work for those cases. Further
edits may be needed to explain why it doesn't work and what to do
instead.
* add backend rejection of updating imported remote
As before, we don't want manual editing of an imported remote inside the
app. This ensures that it can't happen on the backend.
* create tooltips for sshconfig edit/delete buttons
For remotes that are imported, edits are not allowed. This adds a
tooltip that explains what to do instead.
Deleting remotes that are imported is allowed, but they will come back
if the user imports again. The tooltip explains a way to avoid this.
* add logo after name for imported remotes
In the connections screen, there previously was not a way to tell
imported connections from manually created connections. This change adds
a logo after the imported ones to differentiate them.
* small formatting updates
* add import tooltip to connection modal
Added the logo for an imported config to the connection modal. It also
provides a short description when it the mouse hovers over it.
* add button to import ssh config
Make the command into a button for a simple gui interface.
Also ran prettier to clean up some syntax.
* remove strict casing on WaveOptions
WaveOptions was previously very specific about the casing of the ignore
and connectmode subcommands. With this update, the casing is
automatically converted to lowercase and can be ignored.
* add status dot before name in connections screen
* add space and tooltip to connection imported icon
* re-prettier
* sync schema.sql with running schema
* remove incognito field from history table. also don't add empty FeState vars
* history updates, add festate, durationms, exitcode, status, and tags into history table
* update screen/session delete, and clear to no longer purge history items. move deleted screens/sessions into a tombstone table.
* update schema
* fix alias -> fn
* quiet the security warning about an unchecked byte conversion. no real security issue here, but add a range check for good measure.
* working on an in-memory store for screen information
* nostrpos sentinel
* textareainput now tracks selection (to update backend)
* make websocket connections much safer. add a defer/panic handler for each ws message handled on backend. don't allow client to reconnect to backend ws handler more than once per second (handles issue with lots of fast fails)
* use onSelect to have frontend textarea sync state to backend ScreenMem store
* restore cmdline when switching screens
* prettier
* take a stab at fixing #99. ignore invalid output before we see a real packet. the complication here was ensuring we always output a real packet in every flow so we don't actually lose valid errors.
* add ping packets to prime the parser (when in ignoreUntilValid mode)
* work on basic sidebar layout
* fix more golang warnings
* sidebar open/close
* add ability to set width of split
* sidebar add and remove, set width, etc.
* almost working sidebar implementation -- still needs height/width, input control, and bug with initial add, but getting there
* add isSidebarOpen() method
* fix resize jump -- must set width in error handler as well (before window is loaded)
* sidebar UI touchups and help
* more sidebar progress, render more like regular lines, just in the right column
* merge
* move migration to 26
* simplify sidebar types
* checkpoint
* proxy things through parent screen object for sidebar
* checkpoint, add/remove from sidebar
* work on add/remove icons for sidebar
* fix height calculation, remove close button
* bring back close button when no line is selected
* add sidebar flag to run command to run new command output in sidebar
* implement 'sidebar' kwarg in eval. this lets sidebar work for slashcommands as well that produce lines (codeedit, mdview, etc.)
* prettier
* minor fixes
* working on resizing. must exclude sidebar entries and send separate resize events based on size of sidebar (implement exclude / include for resize)
* fix sidebar terminal command resizing
* add sidebar header (toggles for half/partial width and close). add hotkey to open/close sidebar (Cmd-Ctrl-S). more robust calculation for sidebar width. add width validation. minimum sidebar width is 200px. other fixes, etc.
* wrote client code for communicating with lambda cloud
* Added timeout functionality, added check for telemetry enabled for clouod completion, added capability to unset token, other small fixes
* removed stale prints and comments, readded non stream completion for now
* changed json encode to json marshal, also testing my new commit author
* added no telemetry error message and removed check for model in cloud completion
* added defer conn.close() to doOpenAIStreamCompletion, so websocket is always closed
* made a constant for the long telemetry error message
* added endpoint getter, made errors better
* updated scripthaus file to include dev ws endpoint
* added error check for open ai errors
* changed bool condition for better readability
* update some error messages (use error message from server if returned)
* dont blow up the whole response if the server times out. just write a timeout message
* render streaming errors with a new prompt in openai.tsx (show content and error). render cmd status 'error' with red x as well. show exitcode in tooltip of 'x'
* set hadError for errors. update timeout error to work with new frontend code
* bump client timeout to 5 minutes (longer than server timeout)
---------
Co-authored-by: sawka
* back-end implementation. initial ui implementation
* rough implementation of drag and drop
* wrap with AnimatePresence
* persist screen order
* cleanup
* return all screens when updating indices
* remove y axis anitation
* remove debugging code
* chain filtering and sorting logic
* remove unused var
* fix issue where tabs shift to left on hover
* fix tabElem scroll into view regression
* clear scrollIntoViewTimeout when component unmounts
* remove borken style prop
* completely remove animation related props
* scroll into view only when there's a new tab
* minor comment fix
* resolvePosInt() to resolve index
* move tab width const to magiclayout.ts
* move back scroll into view code as it was before
* clear timout
* refactor setTimeout
* remov debugging codes
* format cmdrunner.go
* move clearTimeout
* remove wheel event listener
* switch emain to using the bash from PATH, not defaulting to /bin/bash. not convinced this fixes#132 yet because the PATH set for MacOS applications is not set from .bash_profile or .zprofile. those seem to be set with launchctl setenv.
* move MacUserShell() func to shexec. use dscl UserShell output as bash path on MacOS when present. fixes#132
* fix ability to log into ssh with key and password
A previous refactor to the ssh password system broke the ability to use
key+password to log in. This change handles key+password as a special
case allowing it to be handled separately.
* clean up unnecessary print
My last change left in a debug print that wasn't necessary in the code.
This has been removed.
* allow @ symbol in user for ssh connections
Previously, the @ symbol was only used as a way to either:
- separate the user from the host
- separate "sudo" from the user
This change expands this to allow the username part of
sudo@username@host or username@host to contain any number of @ symbols
in addition to the ones previously allowed. Host is not allowed to
contain an @ symbol as per the usual definition.
* clean up regex changes
Moved the dash in the regex pattern to the end to make it explicitly
clear that it isn't part of a range.
Removed the hostNameRe regex as it is unused.
* fix clear so it doesn't archive running commands
Clear previously archived every command that existed in the current tab.
This change alters this behavior so the commands with a status of running
or detached are not archived by clear. As things currently stand,
detached is not used so the only immediate effect will be with running
commands. As before, the clear command only affects the current tab.
* remove unnecessary print
A print statement for debug still existed in the ArchiveScreenLines
function. It has been removed.
* remove isWebShare from ArchiveScreenLines
The isWebShare feature is currently unused and there is not a plan to
add it back soon. For this reason, it has been removed from the
ArchiveScreenLines function.
* clean up query formatting
