I just wanted Emacs to look nice – Using 24-bit color in terminals

I want to make sure that people understand that you do not have to use a terminal to interact with Emacs: instead, you can use one of Emacs's "graphical" front ends for MacOS, Windows, X.org or Wayland.

Visually, graphical Emacs looks so much like Emacs running in a terminal that if the tool bar has been turned off (and most users decide to turn it off) most non-users of Emacs will assume (incorrectly) that a graphical Emacs window they see running on a colleague's computer is a terminal window (unless perhaps the window is currently displaying an image). (Graphical Emacs is an image viewer among other things.)

In a survey of Emacs users, 80% said that they use one of the graphical front ends.

Right. There's also the sophomorically named "TRAMP" for connecting from a locally hosted emacs session to files across a wide range of protocols including ssh.

TRAMP also exhibits a wide range of performance problems. Getting it not to hang the display of the buffer list or lock up Emacs completely for minutes at a time requires a deep dive in to a decade of Stack Overflow breadcrumbs. I've done this, and I'm still not completely satisfied with the performance. I do however love the concept.

No, it requires only one thing: turn on persistent SSH connections. This means that new SSH connections to the same server do not actually require opening a new TCP connection, do not require negotiating new session keys, and do not require additional authentication. To turn this on, add these settings to your ~/.ssh/config file:

    ControlMaster auto
    ControlPersist yes
    ControlPath ~/.ssh/control/%C

You should also use key–based authentication so that TRAMP never has to ask for a password, but this is less important once you are using persistent connections. Add something like this to your ssh config:

    Host orod-na-thon
    Hostname 192.168.2.133
    User db48x
    IdentityFile ~/.ssh/id_ed25519_your_private_key

Now go to Emacs and open up /ssh:orod-na-thon:~/.ssh/authorized_keys. This is the TRAMP path for your authorized_keys file on the machine named orod-na-thon. It will ask you for your password. Paste in your public key (which you can get by opening up the .pub file that goes alongside the private key in ~/ssh), and save it. Now future connections to orod-na-thon will be able to authenticate automatically using this key pair, and there will be no password prompt when you open a file there.

One problem that I looked briefly into, but have not found a solution is that often I'm browsing the file system (in a terminal) on the remote box and want to open up a file in a certain location (often just for a quick edit). The process of typing pwd, and copying the location into emacs (similar applies to Neovim) already is too much friction, that it's easier to just open the barebones Vim on the remote.

Anyone aware of a solution of how to start the local editor from the remote to open the remote file?

Best solution to that is to use Eshell. Run `M-x eshell` to open the shell inside Emacs (on your local machine), then type `cd /ssh:orod-na-thon:path/to/files`. This will transparently ssh to the server and change to the path to your files. The `ls` command will now show the files that exist on the remote server rather than the local computer. You might now expect that there would be an eshell command that you could type to open a remote file in your local Emacs, but there isn’t.

No, in Emacs you always open a file by typing `C-x C-f`. (Unless you rebound it to some other key, in which case make the obvious substitution.)

Any time you hit `C-x C-f` to open a file it defaults to opening files from the working directory of the current buffer. The working directory of the eshell buffer is on the remote server, so the default list of files that you see are all the ones you were already looking at with `ls`. You can start typing a filename and autocomplete will do the rest.

This is an even deeper and more convenient composition of the shell and the editor than having special commands for tasks like opening a remote file in the local editor.

You might now expect that there would be an eshell command that you could type to open a remote file in your local Emacs, but there isn’t.

All Emacs Lisp functions are Eshell commands. Say 'find-file <filename>' at Eshell and you're in.

Technically correct, but it is way easier to hit `C-x C-f` just like you would when you are opening a file at any other time. No need for special cases, you just open the file.

`vterm` beats the crap out of `eshell` as a functional terminal, unless you've put a lot of effort into customizing it.

https://www.masteringemacs.org/article/running-shells-in-ema...

That’s because eshell is not a terminal. It’s a shell. You don’t need a terminal to explore the files on the other server.

You might now expect that there would be an eshell command that you could type to open a remote file in your local Emacs, but there isn’t. No, in Emacs you always open a file by typing `C-x C-f`

`C-x C-f` calls the `find-file` function which can be called directly from eshell

Probably not the answer You are looking for...

Try some variant of this in your remote .bash_profile or what-not:

  if [ "$PS1" ]; then
      export PS1='\h:\w\$ '
      if [[ "x${TERM}" = "xeterm" || "x${TERM}" = "xeterm-color" ]]; then
       function set-eterm-dir {
         echo -e "\033AnSiTu" $(whoami)
         echo -e "\033AnSiTc" $(pwd)
         echo -e "\033AnSiTh" $(hostname -f)
       }
      PROMPT_COMMAND=set-eterm-dir
    fi
  fi

Now, locally, invoke M-x ansi-term, then within the ansi-term ssh to the remote machine. Change dirs, hit C-x C-f to open a file in the current (remote) directory.

See also the ansi-term hints on emacswiki

https://www.emacswiki.org/emacs/AnsiTermHints#h5o-5

or the comments in /usr/local/share/emacs/*/lisp/term.el

Edit: formatting

Apologies, I meant to add more saying I could not get a remote machine to invoke the local emacs reliably. I got close with hacking up emacsclient to ssh back, but was eventually stymied by new network policies at $JOB and eventually settled for the above.

db48x has a great answer below.

I don't have a specific answer for you, but what I normally do is open

   /remote:/the/path/<tab>

and autocomplete the file name.

Additionally, you can just do dired:

   /remote:/the/path  <return>

and scroll to the remote file I want to edit and press enter

The process of typing pwd, and copying the location into emacs (similar applies to Neovim) already is too much friction, that it's easier to just open the barebones Vim on the remote.

Vim has had the ability to open files remotely over ssh, sftp, etc. for a long time using the built-in plugin netwr [1].

Neovim takes it to another level [2]:

    Nvim's RPC functionality allows clients to programmatically control Nvim.
    Nvim itself takes command-line arguments that cause it to become a client to
    another Nvim running as a server. These arguments match those provided by
    Vim's clientserver option.

[1]: https://vonheikemen.github.io/devlog/tools/using-netrw-vim-b...

[2]: https://neovim.io/doc/user/remote.html

Yes, that's great, and it makes things extremely snappy while the socket connection remains open. Sleep your computer overnight, experience some network stormy weather, leave Emacs buffers open for weeks (as one does) and eventually one of those ControlMaster sockets are going to become wedged, and when that happens you're going to go through some things.

If that happens and you don’t want to suffer through the default 30–second (or whatever it is) timeout, then shorten it. Add `ConnectTimeout 5` to your config.

Good idea fine tuning the timeouts. I took your suggestion and dropped the SSH ConnectTimeout to 5, I was still using the default. I also peeked at the Tramp timeouts, and I realized that I had the variable as tramp-connect-timeout and not tramp-connection-timeout. Thanks!

I had this trouble when I upgraded emacs. I'm now stuck on 27.2 because later versions broke tramp in some way, and I haven't taken the time to debug it.

Counterpoint: If you want a terminal w/o the bloat of a full GUI, but still most of the advantages, `xterm-mouse-mode` will get you 80% of the way there:

    (require 'mouse)
    (xterm-mouse-mode t)
    (mouse-wheel-mode t)
    (setq mouse-wheel-scroll-amount '(1 ((shift) . 1))) ;; scroll one line at a time
    (setq mouse-wheel-progressive-speed nil) ;; don't accelerate scrolling
    (setq mouse-wheel-follow-mouse 't) ;; scroll window under mouse
    (setq scroll-step 1) ;; scroll smoothly

Also, I should point out that I use `(setq auto-save-default nil)` - as God intended.

“without the bloat of a gui”? Aren’t you missing the fact that the terminal emulator is itself a gui?

Unsaid in all of this is that Emacs isn't my main text editor - that is the job of BBEdit where the GUI is both expected and rather good - instead, it is the "editor of last resort" (Vim is inscrutable to me).

The emacs GUI inside a terminal window - invoked w/ xterm-mouse-mode - is still rather minimal. I'm thinking more about emacs-gtk or Aquamacs. You could argue that the GUI gives you access to more system capabilities in a straightforward way - buttons/controls v keybindings - but this isn't a dealbreaker for me.

As long as I have a readily available reference for keybindings, that's good enough for me.

w/o the bloat of a full GUI,

I remember that logic making sense when I tried to run Emacs on an 8MB 486... but that was about the last time it did.

w/o the bloat of a full GUI,

People are going to be using Emacs with a monitor. It's more bloat to have to have a GUI application that emulates a terminal and then run Emacs inside that emulator instead of just having Emacs be the GUI itself.

I don't run emacs in a terminal emulator. Emacs is my terminal emulator. All my terminal programs get run in a (graphical) emacs shell-mode buffer. The rare program that needs full screen control uses term-mode.

It's not the most featureful or fastest terminal emulator, but I can jump around the buffer like any other buffer and seamlessly access it along with the other content I'm editing.

On a remote computer where I can't run a graphical emacs, I either remotely edit files via tramp, or run a headless emacs server process and connect remotely with a graphical emacsclient.

I also use emacs graphically, somehow I managed to switch after years of using it in the terminal and now I prefer it. However, I've never managed to settle on any of the terminal emulation modes. Just too many keyboard shortcut conflicts and complicated magic to remember how to copy and paste between different types of buffers.

These days I've pretty much settled on having and emacs window and GNOME Terminal side by side as my two main work windows, and it's pretty comfortable. I like that I can mouse select and copy from the terminal window and then C-y into emacs and vice versa.

If I really have some need I might still open a shell or ansi-term session but it's pretty rare now. Sometimes it's convenient when I find myself in a remote shell session and want to have a text editor and shell vertically tiled, I'll just run emacs with ansiterm because for the life of me I can't ever remember the screen keyboard shortcuts for splitting windows.

I do that, too, for the reason you describe, except instead of shell mode, I use a mode I wrote myself.

Could you explain the part about the headless emacs server? I thought emacsclient only worked locally.

I've always drifted back to the terminal based emacs for two reasons.

While I've been using X11 off and on since the 90s, I still do not understand how to deal with fonts beyond helplessly googling for a constructive command I eventually stumble on something that works and then I have to leave it alone.

The display performance of most terminal programs is still noticeably faster than the X11 setup I end up with in a corporate environment.

The font stuff is 50x better than it used to be and there's even a reasonable font selector built right into the GUI with the "Set Default Font" in the context menu (ctrl-right-click). And then you can "save" that from the GUI and get a line like this in your .emacs:

   '(default ((t (:family "FantasqueSansM Nerd Font Mono" :foundry "PfEd" :slant normal :weight regular :height 98 :width normal)))))

Which emacs knows how to modify, too.

I agree. I use X based emacs and the protocol is efficient. The experience seems as lightweight as using a terminal, and NOT like using VNC or similar.

advantages include fonts (even different ones at the same time), menus, mouse, colors, multiple frames/windows, and more...

You can also run the graphics locally use tramp to access remote systems. I can edit config files on a remote linux-based router which basically has busybox, with all the bells and whistles.

a much better experience.

Good point. I never considered that people might think I run Emacs in a terminal. Graphical Emacs is a lot better, but it's still great that 90% of stuff works and looks the same in a terminal.

Digging around in terminfo is like holding a flickering, flaming torch up to some hieroglyphics in some deep catacomb. It's weird to think just how much complexity goes into a terminal application. Makes me even more grateful to the devs behind iTerm and Kitty (and all the others!)

EDIT: for iTerm+zsh, I collected some functions for setting tab colors: https://gist.github.com/aclarknexient/84ebe33c1879f921685304...

EDIT2: Contrast terminfo with the old-school method of connecting to HTTP or SMTP with telnet. Yeah, it's simple text but that text is being displayed by a complex application that supports all sorts of wild stuff!

I use my own terminal, and early made the decision not to give a shit about terminfo, pretend to be rxvt, and deal with the fallout (now, admittedly, the only user I care about for my terminal is me - at least for now). Modern terminals supports enough of a shared subset that it's less painful doing that than having to deal with applications get all confused because they don't recognize your TERM variable, to the point where I've typically found the closest match whenever I've used a less common terminal in the past too. (This matters less if you don't regularly ssh into new systems, of course)

Absolutely agreed. The main problem with that is providing new features that xterm doesn't support, but terminfo pretty much refuses to add new features anyway, so modern terminals have had to work around that with some combination of autodetection, separate environment variables, or YOLO approaches. So, at this point, I'm not sure it's worth supporting terminfo at all for most applications, either as a terminal emulator or as a CLI program. (The article makes a similar point about the negative value terminfo is providing for modern applications.)

If you're building a program that takes over the terminal screen, you probably should use one of the established libraries that abstracts over terminfo and over old non-ANSI Windows consoles. But if you're doing light terminal handling in an application that doesn't take over the screen, such as emitting colors or doing simple cursor control, forget about terminfo and just handle two or three cases: ANSI, optionally old pre-ANSI Windows consoles, and files/terminals with no support for anything. Rounding ancient terminals or terminal emulators to "no support" and giving them the same thing you'd give a redirection to a file seems entirely reasonable for a new program.

Just... please please please call isatty() and don't barf ANSI cruft into my buffers.

I'd frankly far prefer people not to call isatty(), and require switches for machine-readable output. The kind of tools that "barf ANSI cruft" are usually the tools I want to do so even when I'm piping their output into something else. It's a real nuisance to e.g. deal with options to preserve colours through a pipe for every element. There might well be exceptions where I'd prefer to default to checking isatty(), but they are few.

If there was any remotely standard way of signaling the preferred default, and a remotely standard command line option to toggle that default, it'd make things a lot nicer, because I totally understand where you're coming from; I get equally annoyed just in the opposite scenarios, so the situation is a nuisance for both of us.

You could write a wrapper around something like https://github.com/cvolny/faketty, which can manipulate isatty() calls when used with LD_PRELOAD on Linux (and presumably DYLD_INSERT_LIBRARIES on macOS except on Apple's binaries).

Why use an LD_PRELOAD wrapper rather than running the program with a real pty?

I'd love to have a standard command-line tool for "run the following program with its output set to a pty, and then pipe the output to stdout". That's the most reliable way to get programs to display color and similar.

Any tool whose output is ever parsed, including "parsing" as simple as "tool | grep xyz", shouldn't emit terminal escapes to a pipe by default. If `xyz` has an embedded color sequence in it, that grep will fail. Or worse, produce unexpected results. (The standard color sequences end in `m`; a grep for 'msomething' could match 'something' preceded by a color sequence.)

Yes, absolutely.

Also, as a pet peeve: always read your input from stdin, and if stdin isn't readable, do not assume that if stdout/stderr is a tty you're allowed to use that for input. (This assumption is broken in batch/CI/etc systems where stdout/stderr may be a tty so that a program emits color/etc, while making stdin /dev/null because there's no user interaction possible.)

TTY queries are written to stdout but read from stdin. That's not user interaction. E.g. if you're system doesn't have ioctl for window size (or you're over a remote serial etc), setting the cursor to the bottom right and asking it's position. Those programs break with no stdin because a tty is inherently bidirectional communication!

Such queries are not universally supported, and a program using them has to be prepared to expect a lack of response to the query. (This is one of many issues with such queries.)

I’d like to know more of the motivations for building your own terminal. Are you using a library or written all from scratch?

See a lengthier reply elsewhere in this thread, re: motivations etc. but it's almost all from scratch. Frankly, a simple terminal is not a huge amount of work, and you can get something working "well enough" to start with with very little code.

The huge amount of work comes from nailing all the quirks you will have to deal with if you let a bunch of other users loose on it, with their expectations of running all kinds of applications I've never tested that does weird stuff.

Yeah, the last 10%. I saw your reply after posting mine, thanks.

Famously the last 90% of the project :(

pretend to be rxvt, and deal with the fallout (now, admittedly, the only user I care about for my terminal is me - at least for now

It's fine when it's just you. Unfortunately, there are some widely used terminals that default to claiming to be xterm, but are not xterm compatible.

Frankly, I find most work well enough with xterm or rxvt termcaps to prefer that over having apps just refuse to even try to work until you've changed settings.

But conversely, if distributed like that, I'd also feel that this implicitly means any failure to act like xterm is a bug they've implicitly suggested it is reasonable to report or complain about (and if the version number doesn't imply it's an early stage release, and it still doesn't work well with its defaults, I'd get cranky)

If you send feature-detection queries it doesn't matter.

The nasty part is that RXVT violates ISO 2022 structure in weird ways. It's not alone in that, but most of the other-program violations are much more easily fixed.

That's fascinating! Did you have a specific itch you wanted to scratch by writing your own terminal? Or was it a learning project that you found useful?

A combination of a procrastination project, wanting a pure Ruby library I can use for output from hybrid applications (e.g. my editor) to be able to have a mostly terminal UI but being able to add in limited graphics when available, and wanting a testbed for other terminal changes I want to experiment with in terms of creating more of a hybrid UI.

A basic terminal can be fairly small - mine is ~1800 lines of Ruby at the moment, which I consider disappointingly large. For comparison st is ~8k lines, I think, and xterm is ~88k. But you can do a working terminal in far less than my 1800, even in more verbose languages, so it's a nice space to play in where you can get something that works in very little (set TERM to a dumber terminal than rxvt, dump any escape sequences your terminal doesn't yet understand to a log, and run the apps you need, then iterate...), and build up in whichever direction you want to something quite usable very quickly. Then you can spend a lifetime polishing quirky little details nobody with you will ever care about... ;)

When I was on ITS, I just implemented SUPDUP in my terminal. It made Emacs usable at 300 baud. That protocol is a nice piece of engineering.

I got a big dose of Telecoms standards early in my career, including some X.25 and networking stuff. After the clarity and simplicity of SUPDUP, my reaction to Big Standards could best be described as "allergic".

[Marshall Rose and Michael Padlipski are good reading on this subject. To this day I use the word "octet" as an epithet]

use standards

not too many

mostly plain ones

Any chance you’d be willing to share some screenshots of the tab colours in action?

Here you go: https://gist.github.com/aclarknexient/84ebe33c1879f921685304...

Good point, I'll add some tomorrow.

iTerm is the software I miss most from Mac. Is not the fastest terminal emulator but it’s the best looking by miles.

I suspect the author would have had a better time had they known to do "export COLORTERM=truecolor". This is mentioned in a footnote in the linked Emacs document about detecting truecolor support, and it appears in their Konsole screenshot, but I suspect it got lost inside their layers of terminal multiplexers (as, I find, it often does). It lets you skip past all this hard stuff straight to the part where the app understands that you want true color. A terminfo database isn't even required.

Alternatively, just use your favorite GUI Emacs with either `tramp` or `sshfs`.

I have tried to love TRAMP, but I can't figure out how to get eglot to work properly. I use it for C++ in a large codebase, so eglot is mandatory.

The other thing is that I like to close my laptop, reopen it the next day, and have access to the same remote buffers. (Maybe this is possible with TRAMP.)

Ultimately I just use emacs in tmux over ssh, with CLICOLOR defined appropriately, and this gets me 95% of what I want.

I wish I could get rid of those last <50ms of keypress latency. And variable-sized fonts for eglot inlay type annotations. They are quite distracting when they're the same size as the code.

TRAMP remote buffers aren’t a huge issue on disconnection because the buffer contents are stored locally: TRAMP’s magic mostly occurs on open and save. Unfortunately, your modew like eglot have to be TRAMP-aware and execute over a ssh connection somehow. Many LSP servers use stdio communication, so it should be possible to put together a mode hook that detects if the buffer file name is a TRAMP file name and then does something like `ssh remotehost -- cd /project/root ‘&&’ clangd --stdio`

Eglot does that automatically though. It detects when you're in TRAMP and launches the language server in a remote. Of course, it assumes that the language server exists at the remote, but that should be the case.

COLORTERM is also a very weird thing. Most of the time I don't need it to be defined, but when using a screen session it suddenly looks very crappy when COLORTERM is set to truecolor. Removing the env var makes it look all normal again.

Tmux supports 24-bit color. I bit the bullet and switched a few years ago, and my muscle memory remains exactly the same. I'm incredibly happy with the power, flexibility, configurability, and reliability of Tmux.

Does your screen even support 24bit colors?

If unsetting COLORTERM fixes the display problem then it is a program issuing 24bit colors but the terminal (in your case screen) not understanding the escape sequences.

You're right: I should have included COLORTERM in this history, and a couple of the other fits and starts, like the `RGB` terminfo capability.

The issue I had with COLORTERM is that not every program respects it. In fact, I've had this in my .bashrc for a while:

  case $TERM in
    linux | screen*)
      unset COLORTERM
      ;;
    *)
      export COLORTERM=truecolor
      ;;
  esac

(Screen does support 24-bit color now, so that case probably came from Ubuntu 18.)

Agreed - COLORTERM has its own problem. I prefer your approach, it looks very sound!

A fork of terminfo may be needed if the description of modern terminal capabilities can't be added -- or if old and deprecated attributes repurposed for that job (like in your padding example): if you're automating the correction/creation of terminfos in ~/, IMHO, it may be better to piggyback on tic as much as possible.

Anyway, to backport modern terminal descriptions to legacy programs, creating correct binary terminfos in ~/.terminfo seems the best practice. You can also invent new TERM. When I wanted to have italics etc about everywhere, personally that's just what I did for sixel-tmux: https://github.com/csdvrx/sixel-tmux/?tab=readme-ov-file#ste... : just declare a new $TERM you know to be right, and use that in the apps that let you use a little logic in their configuration file

I do that in my .vimrc:

   " If Vim doesn't know the escape codes to switch to italic
   let &t_ZH="\e[3m"
   let &t_ZR="\e[23m"
   " Italics pseudo-auto toggle: force italics if we recognize it's supported from TERM
   if match($TERM, "xterm-256color-italic")==0
    highlight Comment cterm=italic
   endif

BTW you should package your emacs config file with the emacs APE from https://cosmo.zip/pub/cosmos/bin/emacs to do a cuteemacs, to carry your config files and everything in a portable binary: https://github.com/csdvrx/CuteVim

Another idea: have you considered that the "program that would be run by .bashrc" to generate the "unambiguous 24-bit color capabilities" could set environment variables to communicate them? Like, if you can't stuff them in terminfo, stuff them in environment variables! Env var work well on Windows, and the configuration files of terminal programs could just use these when available to override.

It's be like adding friends to your proposed TERMVERSION: TERMSETF24 etc where you've have a finer control than COLORTERM

A terminfo database isn't even required.

As of Emacs 28. You still need the terminfo database in older versions of Emacs.

Indeed. Going down the terminfo road lies madness only.

This is partly due to the maintainer of ncurses being somewhat ... conservative and the new fangled terminal devs wanna move things fast.

The best thing about Emacs compared to vim/neovim is that it has first-class GUI support that's not simply a terminal emulator in a window.

I know about gVim and hundreds of half-baked neovim frontends, but the default way of running emacs is through a bonafide GUI, and I love that.

Terminals are alright, but they have too much historic cruft and limitations that actually do not make any sense in this day and age.

If your issue is running Emacs on a remote server, there's TRAMP and emacsclient (does it work over TCP?)

The great thing about running everything in a terminal is you can copy anything that is shown in the terminal. Frustrating when a GUI app won't let you copy some text it's showing and you have to retype it somewhere else by hand

Pasting into a terminal is such a minefield, though.

On net I'm not sure it's an improvement. Being a pure ncurses-esque console app limits the richness of what you're able to display. Native controls and dialogs are in fact a good thing and not a cardinal sin.

Pasting into a terminal is such a minefield, though.

Huh? Just use Ctrl+Shift+V or even Ctrl+V if your terminal emulator of choice allows that? I don't see where the minefield is

Well, god help you if what you're pasting contains unicode for a start.

Many dialogs that look uncopyable actually support ctrl-c, sometimes even giving some funny ascii art of the buttons.

Emacs allows to copy anything into anything.

This is not an issue with Emacs though, GUI or otherwise, because everything's just a buffer

This depends heavily on your workflow. For example I use vim almost exclusively for editing files, and drop back into the shell to use external tools (e.g., git, linters, formatters, debuggers, file tree manipulation, etc.) So my terminal emulator is already the main application. For this workflow the lack of a dedicated GUI is almost immaterial.

this 10,000,000,000%!! (dr stone ref.) I love modal editing and vim a ton, but I couldn't stand the half baked solutions terminal editors gave me. We have an entire rich graphical platform, why not use it?

The best thing about Emacs compared to vim/neovim is that it has first-class GUI support that's not simply a terminal emulator in a window.

Not true. There are lots of native front-ends for Neovim; Neovim was architected from the start to enable them [1]:

[1]: https://github.com/neovim/neovim/wiki/Related-projects#gui

My guess is that widespread 24-bit support in terminal apps is still held back by the fact that the macOS terminal still doesn't support it. As far as I can tell nearly everything else does :(

Is that really a good reason to not support it?

It seems possible to offer graceful degradation for non-supporting terminals, and I'd be surprised if the number of Terminal.app users that would even notice the difference would be that high:

Surely everybody who cares about a good terminal experience uses some third-party client already? Then again, maybe I'm underestimating managed/corporate devices that don't allow third-party terminal emulators, since they do have quite critical access in the end.

It's the default, which means it has a lot of people use it. And IME the alternatives on mac aren't very good.

IME, this is like the golden age of terminal apps in general and macOS-compatible ones in particular. There are several really good terminals for macOS:

iTerm2: (https://iterm2.com/)

Kitty terminal (https://sw.kovidgoyal.net/kitty/)

WezTerm termina (https://wezfurlong.org/wezterm/index.html)

Alacritty (https://github.com/alacritty/alacritty)

My daily driver is WezTerm…

- Runs on Linux, macOS, Windows 10 and FreeBSD

- Multiplex terminal panes, tabs and windows on local and remote hosts, with native mouse and scrollback (https://wezfurlong.org/wezterm/multiplexing.html)

- Ligatures (https://github.com/tonsky/FiraCode#fira-code-monospaced-font...), Color Emoji and font fallback, with true color and dynamic color schemes (https://wezfurlong.org/wezterm/config/appearance.html#colors).

- Hyperlinks (https://wezfurlong.org/wezterm/hyperlinks.html)

- Searchable Scrollback (https://wezfurlong.org/wezterm/scrollback.html) (use mouse wheel and `Shift-PageUp` and `Shift PageDown` to navigate, Ctrl-Shift-F to activate search mode)

- xterm style selection of text with mouse; paste selection via `Shift-Insert` (bracketed paste is supported!)

- SGR style mouse reporting (works in vim and tmux)

- Render underline, double-underline, italic, bold, strikethrough (most other terminal emulators do not support as many render attributes)

- Configuration via a configuration file (https://wezfurlong.org/wezterm/config/files.html) with hot reloading

- Multiple Windows (Hotkey: `Super-N`)

- Splits/Panes (Split horizontally/vertically: `Ctrl-Shift-Alt-%` and `Ctrl-Shift-Alt-"`, move between panes: `Ctrl-Shift-ArrowKey`)

- Tabs (Hotkey: `Super-T`, next/prev: `Super-Shift-[` and `Super-Shift-]`, go-to: `Super-[1-9]`)

- SSH client with native tabs (https://wezfurlong.org/wezterm/ssh.html)

- Connect to serial ports for embedded/Arduino work (https://wezfurlong.org/wezterm/serial.html)

- Connect to a local multiplexer server over unix domain sockets

- Connect to a remote multiplexer using SSH or TLS over TCP/IP

- iTerm2 compatible image protocol support, and built-in imgcat command (https://wezfurlong.org/wezterm/imgcat.html)

- Kitty graphics support

- Sixel graphics support (experimental: starting in `20200620-160318-e00b076c`)

Surely everybody who cares about a good terminal experience uses some third-party client already?

Unclear. The built-in Terminal app is pretty good otherwise, and true color isn't necessary for a lot (most?) of common terminal usage. It's not so poor that all command line users would definitely have a third party app.

If you are not using iTerm2 on macOS, it's your own fault.

Tried it. Slow, uses lots of memory.

Yeah, that's my experience as well, I find ConEmu on Windows to also be a bit slow.

I wonder if this is terminal.app or the terminfo entry.

My system seems to have TERM=xterm-256color

I wonder what would happen with a 24-bit color terminfo file.

That's a very interesting investigation. However, perhaps computing has lost its way, when a production system uses emulators of 1970s terminals to run a 1970s editor on a version of a 1970s operating system. I really think that we're stuck in a local maximum and there are probably much better ways to do things. Our computers are millions of times more powerful now, but we're still doing things the same way as literally half a century ago.

People may be stuck in what they think is a local maximum. I ditched emacs close to 20 years ago (for jEdit, later Sublime and finally IntelliJ) and have never regretted it in the slightest.

I don't want my editor to read usenet or play solitaire. I want it to edit text and surface useful contextual information, and otherwise stay out of the way and let me devote my mental energy to the task at hand and not futzing with the damn editor config.

surface useful contextual information

Emacs with Org-mode it's far more than an editor or a mail/usenet/elisp interpreter.

It's literally what you want.

No, that's exactly what I don't want.

I want to edit code. Nothing else.

Org-mode has a 'babel' programming mode. You can code and append snippets, docummentations, charts, output and anything you want.

https://orgmode.org/worg/org-contrib/babel/intro.html

By contextual data I’m really just talking about things like being able to auto complete from it only the code base, but associated sql schemas as well. Or presenting code navigation as a sorted hierarchy of classes, functions, etc. Lazy human want computer do work.

https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-...

With Org you can export that into a PDF, html5 files, whatever.

I really want to use Orgmode (or maybe something like Neorg/Vimviki for vim), but the biggest issue with all of those things is that somehow, there still isn't a proper mobile app for any of them. No, I don't want a simple TODO app that uses Orgmode under the hood. I want an actual app for Orgmode, something which works like Obsidian or Logseq. (In fact, Logseq supports orgmode, but that support is very limited and sometimes incompatible with actual Orgmode)

I don't understand how there are so many passionate users of Orgmode yet apparently not a single one of them is a mobile developer who sometimes takes notes on their phone.

I ditched emacs close to 20 years ago (for jEdit, later Sublime and finally IntelliJ) and have never regretted it in the slightest.

In that time I've carried on using Emacs, and have regretted it several times ;)

However, Emacs has come on a lot. It's much easier to configure than it used to be, and LSP has brought its semantic features back into the range of other editors. Its text processing abilities have always been stellar and far outshone anything else I've tried.

I'm very happy I stuck with it. I've kept many years of muscle memory and tricks and avoided having to reconfigure 3 different editors. Plus there are killer features like magit and org-mode.

I tried using kitty recently. Its really neat, until you try to sshing somewhere. If the remote system doesn't have kitty in its terminfo, nothing works. Kitty's "solution" to this is to use a custom ssh wrapper, which I'm sorry, is not an acceptable solution. A simple solution would be to let users override what TERM gets set to, but they refuse to allow this on some moral ground.

Maybe I'll try it again in a few years when its in more terminfo packages.

Why not just install Kitty’s terminfo file on the remote server? This is quick and easy to do, and only needs to be done once.

That is what kitty's "custom SSH wrapper" does, if I understand correctly. Each time you ssh into a server, it will copy kitty's terminfo file if it doesn't already exist.

I use the ssh kitten, it works fine.

Oh, that’s nice. I don’t see why anyone would be against using that.

Servers in workplaces are often ephemeral. It's easy enough to write provisioning code that updates the system terminfo database, but it needs to be written, tested, reviewed, and deployed. Speaking of deploys, how are your Q1 OKRs shaping up?

The distros I use (debian, arch) have a kitty-terminfo package. First thing I install when I ssh into a machine. My terminal settings just work and I don't have to do archeology on decades of obscure terminal escape sequences.

I just type 'export TERM=xterm' whenever I log onto one of these systems (I use tmux-direct as my $TERM). It seems like Debian 12 has a lot of newer terminfos these days in the default install, and I've tried to upgrade most places I ssh to that. Like our CI system can take a Docker image as the place where your code runs; so instead of using the default one from 2018 or whatever, I just made my own that has all these niceties.

(I use kitty on Mac and Windows Terminal on Windows, but it ends up not mattering because I'm always in tmux, so that's the terminfo that remote machines need. I never bothered setting up 24 bit color though, so I have no idea if that would work with my setup. I switched to TERM=tmux-direct to get italic fonts in Emacs running on Linux inside tmux inside an SSH session from Windows Terminal. It works!)

Yeah that's silly. TERM is as broken as web user-agents. You just have to accept that and `export TERM='xterm-compatible, like xterm-256color, like gnome-terminal'` or whatever :-D

One of the most frustrating things about terminal Emacs is figuring out how to get nice colors in tmux. By the time I get it working finally, I have no idea which combination of the tens of things I did made it work.

What features aside from session persistence do you use tmux for? I partially moved to emacs because I was annoyed with tmux/screen.

Windows, panes, a nice bar with the time, scrolling.

You can get all of that with just Emacs tho

I guess but tmux is installed on every system I connect to, and emacs is not.

Some systems only have ancient versions of emacs available in repos, limiting packages I could install.

You don't need emacs on the other end to get those features, connect over SSH with TRAMP.

But emacs running remotely in tmux will far outline the emacs on my laptop.

I really want to use TRAMP. Basic editing works but I had a lot of trouble getting lsp based programming modes, like rustic, working.

Maybe I should try again. Terminal mode works but requires a lot of workarounds and copy/paste integration isn't perfect.

I found it easier to tune the 16 color palette in each of my various terminal apps than to try and get application-side terminal codes just right.

In my ~/.Xdefaults for rxvt-unicode:

    ! yellow
    Rxvt*color3:rgb:F5/79/00
    ! magenta
    Rxvt*color5:rgb:AD/7F/A8
    ! cyan
    Rxvt*color6:rgb:88/8A/85
    ! white
    Rxvt*color7:rgb:BA/BD/B6
    ! ...

And in my custom theme .el:

    (if (display-graphic-p)
        (setq color-yellow "#f57900"
              color-bright-yellow "#fce94f"
              color-red "#ff6464"
              color-bright-red "#ef2929"
              ...
              color-cyan "#888a85")
      (setq color-black "black"
            color-white "white"
            ...
            color-bright-magenta "brightmagenta"))

I then use these to set various faces

     `(minibuffer-prompt
       ((t (:foreground ,color-blue))))
     `(highlight
       ((t (:background ,color-bright-yellow))))
     `(region
       ((t (:background ,color-bright-blue))))
     `(shadow
       ((t (:foreground ,color-bright-cyan))))
     `(secondary-selection
       ((t (:background ,color-bright-magenta))))

etc.

As a bonus, colored `ls` output etc. is nicer too. The raw 16 color palette is too saturated and often has bad contrast.

(if (display-graphic-p)

This right here. Emacs actually supports graphical output (X11, Cocoa, Win32). Additional bonuses include: Esc is actually Esc; you can bind Ctrl+Shift or Cmd; you can use more than one font at a time; you can display a picture; change the shape of the cursor... It's like a time machine that takes you 20 years forward - into 1989.

There is, however, considerable upside in having all the same key bindings whether you're in local UI (maybe using tramp) or ssh to a remote server. Tramp has limitations and is a bit clunky so I use it rarely.

Clipboard integration and mouse cursor support are the two reasons I generally break out emacs-ui (my shell script for it), otherwise it's emacs-nox, generally.

`ls` actually supports 24 bit color if you put the sequences in DIRCOLORS

Yep. I just tune the 16-color palette in Terminology (it's an INI file) to my preferred theme for the same styling in the console and terminal applications (vim).

Yah, I do pretty much the same thing. The default colours are far too dark for the dark background, so I just increase the luminance of all the colours.

It was saturated because older TTL displays did not allow for anything else!

The red wire was a lot of red, and only red, or nothing and when the intensity wire is added, 100 percent red happened. Wash, rinse, repeat

Just discovered today standard ANSI escape sequences work on Windows 10 command prompt. Even 24-bit color works fine!

Try (not possible to copy paste, see [1]):

  echo ^[[48;2;255;0;0mHello^[[0m

Hello is printed with a red (RGB(255,0,0)) background. "^[[0m" resets back to normal colors.

[1]: You can type ^[ (escape) by pressing CTRL-'['. Don't type ^ and [ as separate chars, that won't work.

Just don’t forget to add

  DEVICE=C:\DOS\ANSI.SYS

to your C:\CONFIG.SYS file.

You haven't needed to do that for like, 7 years now. RGB color support was added to the windows console in 2016: https://devblogs.microsoft.com/commandline/24-bit-color-in-t...

[1]: You can type ^[ (escape) by pressing CTRL-'['. Don't type ^ and [ as separate chars, that won't work.

You can also use the \x1b escape code(or the \e alias) together with a command that accepts escape code in order to avoid nonprintable characters on the command-line:

  echo -e '\e[48;2;255;0;0mHello\e[0m'

For scripts, always use printf though since echo -e is nonstandard and even plain echo differs in edge cases:

  printf '\e[48;2;255;0;0mHello\e[0m\n'

Please note I was talking about Windows 10/11 command prompt. Your tip works on Linux.

Interesting, as a Vim user I'm surprised Emacs doesn't have better theme support. I always assumed it had everything. In modern Vim there's even hex support.

I'm not sure why you say that. https://www.gnu.org/software/emacs/manual/html_node/emacs/Cu... and https://emacsthemes.com/ exists.

I got that impression from the article because of the length he went with ANSI escape codes. I wouldn't touch that other than for my PS1. But I see now there are sane options and the author probably just wanted some fun.

Emacs does support hex colors in themes as well

This is a cool deep dive into terminal standards and their footguns but... not sure why?

Unless you're showing pictures in your terminal, why would you want true color and the overhead that introduces? If you're just doing a status bar on a black background, your eye won't really see the difference between FF0000 and FF1515 (which I picked because the next 16 bit color is FF3333 which you can already barely tell the difference from).

I'm just saying I'm not sure I would want that much color choice, it would just give me anxiety about picking the right color!

I'm just saying I'm not sure I would want that much color choice, it would just give me anxiety about picking the right color!

Developers of themes for Emacs often target the GUI first, where 24-bit color is the default. If you're a hardcore terminal Emacs user (and there are many legit reasons for this) you're stuck by default with 256 colors, and the themes just don't look right due to having to map the carefully chosen colors to the nearest color in the 256 color palette.

Exactly correct. Loading solarized in 256-color emacs shows just how poorly the (carefully-selected) Solarized colors map onto the xterm-256color palette.

It clearly depends on what you do but having more colors is nice.

Even if 256 colors are available it's not enough. The palette of the 256 colors is gruesome. You can clearly see that a technical person without any knowledge of color theory designed it.

Meanwhile I want my emacs to look more like this https://imgur.com/a/h0jA1ro

(in case anyone thinks that's serious, it's a joke. I only use Cool Retro Term for presentations)

edit: apparently my emacs works with 24 bit colour out of the box: https://imgur.com/a/BM5OTxp. The syntax highlighting is a bit annoying though.

https://muppetlabs.com/~mikeh/crt_emacs.png

Cool Retro Term is great, I want to use it in a presentation as well.

I actually use a light customized crt effect with the windows terminal.

I have a deep love for cool retro term, I am sad that the project is mostly abandoned and has multiple issues on MacOS.

IMHO none of the other retro terms look nearly as good.

Hmm... My Emacs has just worked with full colour in terminal for years. I didn't know it was a thing that needed to be configured. Skimming through the article I hope I never have to know this stuff.

I also have icons in my terminal by using nerd icons.

The article is a deep dive on the evolution on terminal color, mentioning some of the "colorful" characters in the scene. This is specifically about the ultimate 24-bit expression of gawdy terminal bling, aka COLORTERM=truecolor. Perhaps you're not the kind of person who would notice if their Emacs color theme looked wrong because the theme designer's carefully chosen 24-bit color was mapped to the nearest color in the 256 color palette of the stock terminal.

Oh, I am that kind of person, believe me. Mine looks identical in the terminal. I use zenburn theme. The only thing I ever set in my bashrc is `TERM=xterm-256color`. I'm not sure why. I do have `COLORTERM=truecolor` set, but I didn't do it myself. It just works for me with or without those variables set in both xterm and alacritty.

The easiest way to make Emacs look nice is to install VSCode.

Not sure that's a good answer. There's plenty of stuff that Emacs don't that VSCode doesn't do very easily.

I mean you're right: installing VSC and forcing yourself to use it for a while will make Emacs seem truly amazing by comparison.

This is why I just use VSCode now.

I enjoyed tinkering for years. One day my Vim colors wouldn’t work in Tmux for like the 5th time in my life. I said, you know what? I’m done with all of this.

I appreciate that what you did is fun. I just thought I’d share a perspective that sometimes we’re sick of tinkering.

Can you se vscode in the terminal now?

I really love emacs in the terminal. I created a docker image that looks beautiful (and identical) whether you're editing locally or remotely. It's configured properly for 24 bit color and unicode, and it also has all the underlying binaries for native compilation, lsp, ripgrep, fast json and so on, all ready for both amd and arm. Caveat -- I haven't used it on windows yet. https://hub.docker.com/r/wwarner/emacs-gopy

I get that the post is about terminal witchcraft, but regarding the Emacs part I believe you, author, ought to look into TRAMP mode.

> It turns out, for years, popular terminals have supported 24-bit color. And yet they’re rarely used.

Hard to say how rarely. I've been using 24-bit color in bash prompts and neovim scheme for a while. Can't go back to 16 or 256 color ones, they are just too limited.

I wish tty terminal also supported 24-bit color.

For setting the color in bash I use something like:

    printf "\x1b[38;2;${r};${g};${b}m"

    printf "\x1b[${f};38;2;${r};${g};${b}m"

If I need to set extra bits like bold.

Wow, this looks so much harder than simply running GUI Emacs.

I wrote the patch for 24 bit color in xterm, but I have to admit I was today years old when I learned how to make ncurses work with xterm-direct. I was already using my own terminal library in Lisp, but it's still useful to be able to fall back to using ncurses. Thanks to Mr. Dickey for answering a stack overflow question, and the author Chad for bringing it up.

Great article ! This is exactly how I use my Emacs, only in tty. Take that VS :)

That timeline is very enlightening and those references are extremely useful. Must have taken a lot of time to find and make sense of all that material. I'm especially thankful for the explanation of terminfo files.

We used semicolon (like other SGR parameters) for separating the R/G/B values in the escape sequence, since a copy of ITU T.416 (ISO-8613-6) which presumably clarified the use of colon for this feature was costly.

Non-free standards are self-defeating. What's the point of the standard if the people who need them to implement stuff can't read them?

Protesilius has changed the game on theming, to the extent that modus-operandi and modus-vivendi are now baked into emacs.

They are fucking incredible and you only need to look at their readme to see the the intense duty of care. It’s not just aesthetics it’s accessibility.

My blog is built by emacs and I use modus-operandi for syntax highlighting.

If only there was a way to magically pass extended modifier keys to Emacs over a terminal…

It's the only thing stopping me from using Emacs remotely over a terminal: I really do need my Super modifier. Hyper I can live without, but being able to do shift-super-f for example is fundamental.

Really shameful that they lock away standards under such high prices that people have to go to such lengths to not buy them, with potential results like this. If it's a standard it should be available. It maybe made sense to charge a printing fee back when they were only available as books but I can't believe a crappy pdf was $200.

Or, I don't know, install the latest Emacs, open a terminal that supports 24bit xterm, and do:

    emacs -nw

If it looks good when you apply a theme, great.

If not add the following to .bashrc, .zshrc, .fishrc (you know the drill):

    export COLORTERM=truecolor

(reload the shell, or just execute this line and start emacs -nw again, voila.)

For some reason, this article makes it look like you need to dig into terminfo, generally you won't need to, maybe 10 years ago, not now.

(I quickly retested just now, to be sure, on Macos iTerm & Linux, Alacritty, Kitty, Gnome Terminal. 24Bit xterm is commonly supported these days.)

I run emacs with minimal colour, so will probably never use this but I very much enjoyed the read. I'm rather used to the garish green of tmux and a small amount of rusty red brown from the amounts of emacs colour left in my life.

This is fantastic! Thank you for writing this and sharing it.

In my case, I don't care. Give me plain black and white all day long--and I do. When you have that many colors on the screen it no longer is helpful and only becomes a blur of clown colors.

If you must, use just a couple of colors--or even just shades of gray or bold--to highlight important things like functions.

But that's what indentation is for. You indent/format your code to help find things. It seems to work well in books. And that works everywhere without any work involved to make it happen.

The first edition of ECMA-48 is lost to time,

Apparently there is a copy in Germany https://search.worldcat.org/title/1069189730