return to table of content

Ruby: A great language for shell scripts

codesnik
117 replies
17h2m

I sometimes wonder why we don't see ruby used for shell stuff more often. It inherited most of the good stuff for shell scripting from Perl, and Perl took a lot of it's syntax from sh and sed and awk, so almost anything you can do in shell script you can do in ruby, but with an option of making it gradually less terse and more readable, while having sane variables and data handling from the start.

Also ruby is great in allowing complexity to grow smoothly, no sudden hiccups. You start with just one line (everything goes into module main implicitly), extend it to a single-file script, require some built-in libraries, then add a module or helper class in the same file, and only then maybe extract those files to required files, add gems, whatever. No boilerplate whatsoever, no jumps, no big rewrites.

meanwhile, a lot of tooling nowadays is written in Go, and I have no idea why, it's not friendly for os manipulation at all, and number crunching power is not needed in many, many tasks of that sort.

seabrookmx
32 replies
16h59m

I think golang is used because you can easily create a single static binary, which is incredibly easy to distribute. I often find non-trivial CLI tools written in Python cumbersome because of the dependency wrangling necessary.

derefr
6 replies
16h37m

This complaint comes up enough that I'm surprised nobody's created the Ruby equivalent of GraalVM, to compile a Ruby script, all its deps, and a WPOed subset of the Ruby runtime, into a native executable.

pansa2
2 replies
16h36m

WPO?

derefr
1 replies
16h7m

While Program Optimization, in this case mostly meaning dead-code elimination for any runtime code not called by the Ruby code.

bongobingo1
0 replies
13h30m

I think you mean Whole Program Optimization.

lolinder
2 replies
16h2m

It's not quite what you're describing, but TruffleRuby is Ruby on GraalVM:

https://github.com/oracle/truffleruby

Unlike GraalVM Java, as far as I can tell TruffleRuby doesn't provide a bundler that can create a single executable out of everything, but in principle I don't see why it couldn't.

mike_hearn
0 replies
12m

Worth noting that the GraalPython implementation does support creating a single binary.

https://www.graalvm.org/latest/reference-manual/python/stand...

I'm not sure I'd try replacing shell scripts with natively compiled Python binaries. That said, I use a Kotlin Scripting based bash replacement in my own work that has many useful features for shell scripting and is generally much more pleasant. You have to "install" it in the sense of having it extracted somewhere, but it runs on Win/Mac/Linux and can be used without root etc.

kaba0
0 replies
9h41m

Graal can create an executable from a ruby program as well with TruffleRuby and native image (both part of the general graal project).

np_tedious
5 replies
16h49m

Most shell-ish scripts probably use no dependencies and will not be picky about exact version

mmh0000
3 replies
16h44m

Mmm. I’d argue that all shell scripts use a ton of dependencies that are different across Unix/Linux.

I. E. ‘sed -i’ is only in GNU sed. Same with ‘grep -P’.

passthejoe
1 replies
16h29m

This has bitten me many times

EasyMark
0 replies
14h43m

It’s why if my bash turns into more than a page or two I start re-evaluating it and turn it into python

throwaway7ahgb
0 replies
5h4m

Correct but if you're in a situation where this is a issue you probably know about it and can use POSIX versions that are more portable.

Otherwise nobody thinks of it because most likely it is not being distributed.

nunez
0 replies
3h36m

Tons of scripts rely on coreutils (sed, awk, grep, head) to manipulate data.

All of those have wildly different behavior depending on their "flavors" (GNU vs Busybox vs BSD) and almost all of them depend on libc being installed.

danmur
5 replies
15h34m

Bundle it into a pex and distribute that. Its still way large but its easy to distribute.

mikepurvis
4 replies
15h14m

Pex was also the solution I landed on after evaluating several non-container options for distributing a Python project to arbitrary Linux hosts.

It works well but with one huge caveat: although you bring the stuff required to reconstitute the venv with you, you’re actually still using the system’s python executable and stdlib!! So for example if you want to make a project targeting all supported Ubuntu LTS versions, you have to include the wheels for every possible python version you might hit.

Ultimately this boils down to there not really being a story for statically compiled python, so in most normal cases you end up wanting a chroot and at that point you’re in a container anyway.

networked
2 replies
8h5m

I wish an easy cross-platform PEX or shiv [1] were a thing. Binary dependencies are the biggest reason I prefer the new inline script metadata spec (https://packaging.python.org/en/latest/specifications/inline...) and `pipx run`. Luckily, they're pretty great. They have changed how I write Python scripts.

The way inline script metadata works is that your script declares arbitrary dependencies in a structured top comment, and a compliant script runner must provide them. Here is an example from a real script:

  #! /usr/bin/env -S pipx run
  # /// script
  # dependencies = [
  #   "click==8.*",
  #   "Jinja2==3.*",
  #   "tomli==2.*",
  # ]
  # requires-python = ">=3.8"
  # ///
pipx implements the spec with cached per-script virtual environments. It will download the dependencies, create a venv for your script, and install the dependencies in the venv the first time you invoke the script. The idea isn't new: you could do more or less the same with https://github.com/PyAr/fades (2014) and https://github.com/jaraco/pip-run (2015). However, I only adopted it after I saw https://peps.python.org/pep-0722/, which PEP 723 replaced and became the current standard. It is nice to have it standardized and part of pipx.

For really arbitrary hosts with no guarantee of recent pipx, there is https://pip.wtf and my venv version https://github.com/dbohdan/pip-wtenv. Personally, I'd go with `pipx run` instead whenever possible.

[1] I recommend shiv over PEX for pure-Python dependencies because shiv builds faster. Have a look at https://shiv.readthedocs.io/en/stable/history.html.

mikepurvis
1 replies
1h58m

All else being equal I’d probably prefer poetry for the broader project structure, but that would definitely be compelling single script use cases.

networked
0 replies
1h34m

You can also combine the two. Something I have done is script dependencies in inline script metadata and dev dependencies (Pyright and Ruff) managed by Poetry.

danmur
0 replies
12h19m

Nuitka has worked for me for everything Ive tried (in house dev tools). I didnt end up using it for work because I can rely on a pristine system Python with the right version so pex makes more sense.

There are other options I didnt look too much into, e.g. Beeware

MikeTheGreat
3 replies
14h57m

I often find non-trivial CLI tools written in Python cumbersome because of the dependency wrangling necessary.

I'm thinking of trying out Mojo in large part because they say they're aiming for Python compatibility, and they produce single-file executables.

Previous to that I was using PyInstaller but it was always a little fragile (I had to run the build script a couple of times before it would successfully complete).

Currently I'm using pipx and Poetry, which seems pretty good (100% success rate on builds, and when my 5-line build script fails it's because of an actual error on my part).

Which is a round-about way of asking everyone:

Does anyone have any other good way(s) to build single-file executables with Python?

teleforce
0 replies
13h57m

Fun fact, you can use D language as compiled scripting using rdmd with powerful and modern programming features although it has much faster compilation than comparable C++ and Rust [1]. The default GC make it intuitive and Pythonic for quick scripting more than Go. Its recent native support for OS lingua franca C is the icing on the cake [2].

From the website, "D's blazingly fast compilation allows it to be used as a high level, productive scripting language, but with the advantages of static type checking" [3].

[1]Why I use the D programming language for scripting (2021):

https://news.ycombinator.com/item?id=36928485

[2]Adding ANSI C11 C compiler to D so it can import and compile C files directly:

https://news.ycombinator.com/item?id=27102584

[3] https://dlang.org/areas-of-d-usage.html#academia

qznc
0 replies
6h32m

Scriptisto is an underrated tool: https://github.com/igor-petruk/scriptisto

It can do the Python venv stuff behind the scenes for you and it just looks like a single Python file.

mmebane
0 replies
14h48m

You could try Nuitka [1], but I don't have enough experience with it to say if it's any less brittle than PyInstaller.

[1]: https://nuitka.net/

noisy_boy
2 replies
15h51m

I think one of the advantages of a script is that you can quickly check what it is doing by simply opening it - an executable won't afford that.

m00x
1 replies
12h57m

Plus it can be run on any machine, while golang needs to be compiled for the specific architecture you'll be running it on. No messing about trying to get the right build.

type0
0 replies
2h57m

I actually think its a less of a problem than many imagine. If you have different architectures it actually is better and more predictable because it's compiled, also it's incredibly easy to compile even for noobs

passthejoe
1 replies
16h28m

Golang's not so secret weapon

intelVISA
0 replies
15h9m

Hard to miss those >100 meg 'single binaries'

theshrike79
0 replies
11h43m

I spent a weekend going through all my old python scripts with Gemini and ChatGPT, rewriting them to Go just because of this.

Most of them were so old that I would have had to skip like 3 generations of package managers to get to the one that's used this year (dunno about next year) if I wanted to upgrade or add dependencies.

With Go I can just develop on my own computer, (cross)compile and scp to the destination and it'll keep working.

shitlord
0 replies
13h20m

The ability to type check and unit test your code is also valuable. This is possible with many languages but with Go it requires basically zero configuration.

schneems
0 replies
14h40m

Bootstrapping and different behavior for different versions and not being able to use the dependency ecosystem really make it a lot more difficult than people realize if you’re trying to “script” at scale.

I’ve used rust for this task but people get mad that I’m calling it a “script”. “That’s not a script that’s a program” which…sure. But so maybe we need another term for it? “Production-scripts” or something.

My experience is rewriting Ruby and bash buildpacks for the open spec CNCF Cloud Native Buildpack project (CNB) https://github.com/heroku/buildpacks

I agree that Ruby is easier to start and grow complexity, that would be a good place to start.

aae42
0 replies
16h26m

I've been waiting for a single executable interpreter for Ruby for a while now

like deno or bun, but for Ruby

artichoke ruby is the closest we've got

asa400
21 replies
14h20m

I sometimes wonder why we don't see ruby used for shell stuff more often.

The reason we don't see Ruby used more for shell stuff is because Python won this particular war. It's already installed on basically every Linux distribution out there, and this simple fact outweighs all other language considerations for probably >95% of people who are writing shell scripts in something that isn't Bash.

Personally, I don't much like Python, and even though Ruby is not my favorite language either, I find it much better than Python for this kind of work. But I don't get to decide how Debian builds their infrastructure, so in the end, I tend to use Python.

shmerl
8 replies
14h3m

How hard is to install it though? That doesn't sound like a reason not to use it.

throwaway7ahgb
1 replies
5h7m

How hard is it to install anything? That really isn't the point.

shmerl
0 replies
2h49m

Seems like a lot below and around bring this as the main point for not using it. Which doesn't make sense to me.

throwaway55533
1 replies
14h1m

Mitigating the risk of downloading a script from the internet and executing it -- even from a "trusted" website or package manager -- is absolutely a good reason not to use it.

shmerl
0 replies
13h42m

Any decent distro has it. So you don't need to execute any random scripts, just install it or prepare the image with it for your OS install. That's it.

I don't really get this whole defaults being a blocker for tools choice.

theshrike79
0 replies
11h45m

If a client has certified a specific Linux distro as an approved platform, that's what we use.

We can either deliver a single executable (Go) or a Python script, as python is preinstalled on their distro.

If we'd want to use Ruby, it'd be a huge hassle of re-certifying crap and bureauracy and approvals and in that time we'd have the Python solution already running.

paulddraper
0 replies
13h58m

Significantly harder than doing nothing

oopsallmagic
0 replies
9h24m

It's not. This is a non-issue. Every web shop is writing bash to twiddle a build script on servers they also manage, which includes the ability to install any package they want.

asa400
0 replies
13h57m

Depends on you, your team, your target hardware/os, your project, and many other factors. Considering all of those things, the hurdle of installation might just be too large for it to be worth it.

tayo42
6 replies
13h34m

It's already installed on basically every Linux distribution out there,

PEP 668 pretty much negates this though. To do anything you need a python environment set up per script/project w/e

sgarland
2 replies
5h36m

Python ships with venv support. It’s not that difficult to bootstrap a venv before running your script, and that’s only if you actually need tooling other than stdlib, which you probably don’t.

tayo42
1 replies
3h34m

It's definitely clunky and tedious to switch between every projects or scripts environment.

Idk why people are pretending there aren't tons of useful libraries out there. Like if you want to script anything with aws, use yaml

sgarland
0 replies
2h20m

There are plenty of ways to have the venv automatically activate (and de-activate) when you enter/leave the directory for the project. direnv [0], mise [1], or various shell hooks.

There are useful libraries, I’m not saying there aren’t. I just dislike it when people include one as a dependency when they really didn’t need it.

[0]: https://github.com/direnv/direnv

[1]: https://github.com/jdx/mise

tayo42
0 replies
3h38m

You need to introduce a build and release process then to do this then which still detracts from it being simple or the selling point being it's already installed.

kristjansson
0 replies
13h16m

Iff you’re going beyond stdlib. Which lots of useful python programs don’t need to do.

oopsallmagic
1 replies
9h25m

And of course, it is impossible to install additional interpreters on the computer.

carstenhag
0 replies
3h34m

I never started using python, ruby or node because all of them were a pain to use for me - this was 7-8 years ago, so maybe it has changed a lot. But even 2-3 years ago I had lots of issues just running one python project. Module, not module, pip or not...

Way too confusing, compared to go for example. Or hell, even Java/Kotlin when you use an IDE and it autoconfigures most things.

nevdka
0 replies
13h44m

This is also the reason perl was used before python began to dominate. It was installed everywhere before python was installed everywhere.

mu53
0 replies
14h0m

this whole argument is silly. In my time on this site, I have seen someone suggest that every language is good for shell scripting including C.

Python and bash are used in the real world most often because convincing your sysadmin/infra/boss guy to install ruby for one script is a hard sell when you already have good-enough tools built into the system that don't add risk/complexity.

lwrtac
0 replies
6h59m

Yes, Python won the war, which is a pity. Linux distributions started getting bloated at the same time they switched to Python for everything. Yum hanging inexplicably and such things never occurred before.

The BSDs do not have this problem (yet!). I hope they stay sane and keep using Perl/sh.

NullInvictus
19 replies
15h49m

I think the quality of a language for shell scripting is often secondary. What’s of greater significance is where it is at. I.e., does it have it already installed? The answer with Linux and Bash is almost always “yes”. Not so with ruby.

The moment you start asking the user to install things, you’ve opened up the possibility for writing a program rather than a shell script. The lifecycle of a piece of software is almost always one of growing responsibility. This cycle is devastating when it happens to shell scripts. What was once a simple script slowly becomes creaking mass of untestable, poorly understood code playing in the traffic of swimming environments (which grep you got, buddy?).

I guess I’m saying that once you open up the possibility of writing a program, you generally take that option and are usually happier for it. In the “write a program” world, ruby is still good, but it becomes a far harder question to answer whether ruby is still the right choice. There are a lot of languages with a lot of features engineers like.

kqr
12 replies
13h25m

This is indeed why I use Perl over Ruby. As long as it's not for a Window machine, a Perl script is deployed by copying it over and that's it.

brightball
4 replies
5h52m

Perl is deeply underappreciated and needs a lot more love. One of the keynotes at the polyglot conference that I run is going to be Perl talk and I'm really looking forward to it.

pdimitar
3 replies
3h37m

Does it still require global library / module installations for your script's dependencies? If so, hard pass.

kqr
2 replies
1h56m

It does not, and has not for at least a decade!

pdimitar
1 replies
1h53m

So any guides on how to make a self-contained Perl script that needs dependencies?

pmontra
2 replies
9h16m

That's true of Python and Perl as long as you keep using only the features built in in the core language (standard lib or whatever they call it.) The same applies to Ruby.

My scripting language is bash in at least 99% of cases. I used to program in Perl when I need some complex logic. I stopped using it some 10 or 15 years ago when I switched to Ruby for two reasons: I became more familiar with it than with Perl and it's easier to manage data structures whenever I need something complex or classes. That doesn't happen often in scripts but as I wrote, I use bash for all the normal stuff.

I use Python for the scripts that start an HTTP server because it has the http.server module in the standard lib and it's very simple to write handlers for GET, POST and all the other HTTP verbs. The last example was a script to test callbacks from an API. I just implemented two POST and PUT methods that print the request data and return 200 and a {} JSON. I think that to do the same in Ruby I would need to install the webrick gem.

kqr
0 replies
1h54m

The same applies to Ruby.

With a big difference -- Perl and Python will always be installed on these machines, whereas Ruby might need two deployment steps: (1) copy file, (2) install Ruby!

BiteCode_dev
0 replies
7h43m

In fact, that's true for Python if you use a zipapp and no c extension: https://docs.python.org/3/library/zipapp.html

You can happily copy the zip of your scripts and all deps in the server.

You still do have to mind your versions, as always with python.

noisy_boy
1 replies
13h8m

That was the reason Perl was what I switched too from bash when I was working on Solaris boxes; it was miles ahead of what was possible with bash AND it was already present. If I remember an older version of Python was also installed but by then Perl had already got me reeled in and I felt Python to be too "verbose" compared to Perl (I eventually changed my opinion when I got a bit more experience under my belt).

kqr
0 replies
12h25m

Interesting! I still find Python too verbose to stand in for shell scripts when Perl is available, with what I think is a decent chunk of experience.

petre
0 replies
12h2m

One usually needs modules to easily do something more advanced, but yes, Perl is almost always installed. Although I find Ruby much more ergonomic, I still reach for Perl as well because I know it better and don’t have to open the documentation so often.

MaxBarraclough
0 replies
6h54m

Even on Windows there's a good chance. The Git for Windows project bundles Perl, but not Ruby.

oopsallmagic
3 replies
9h26m

Not once have I worked anywhere where the people writing shell scripts didn't also control all of the boxen those scripts ran on.

vault
2 replies
8h33m

I'm glad you never worked at a bank or an insurance company!

vsuperpower2020
1 replies
8h3m

Why do you choose to write in a snarky way? Why does that make you glad? Why does this make you energetic?

throwaway7ahgb
0 replies
5h7m

It's tongue in cheek, and he's right. I am a old man Sunos/VMS/Linux admin. Having root used to be my god given right.

However I haven't worked at a company in years that gives anyone access to root anywhere except your own local machine or maybe in rare cases a dev box that is destroyed and rebuilt at will.

shlant
0 replies
5h34m

yea as soon as I read through the post, I ssh'd into one of my many Ubuntu servers, ran `ruby -v` and then noped out. From past experience I want nothing to do with trying to wrangle RVM or rbenv and then making sure the paths work properly.

pdimitar
0 replies
3h43m

I.e., does it have it already installed? The answer with Linux and Bash is almost always “yes”. Not so with ruby.

True, not true for Ruby, but with Golang and Rust you have an almost-no-dependencies final binary so the argument there does not apply.

which grep you got, buddy?

For dev machines it's not such a tall order to require `rg` be installed these days.

solidsnack9000
11 replies
16h20m

Instability. Ruby has not been the same language for very long. Migrating to 1.9 was a huge hassle for many firms. This may seem like a long time ago in tech years; but then there was Ruby 2.0; and shell scripts, meanwhile, have stayed the same the whole time.

A secondary reason is that Ruby has been very slow for much of its life, which means that for situations where you need to run a huge stack of scripts -- init systems, for example -- it would be punishing.

Ruby does have a terse and intuitive syntax that would make for a good system shell. Although it has some magic, it is less magical and confusing than shell itself. Ruby provides many basic data types that experience has proven are useful for shell scripting -- like arrays and dictionaries -- and they are integrated in a much cleaner and clearer way than they are integrated into widely used shells like Bash.

System tools that are written in Go may still make sense to write in Go, though. Go, it is true, does not have a nice terse syntax for short scripts and one liners; and it doesn't have a default execution model where everything is in main and so on; but that is because it is not a scripting language. Other languages used to write system tools and system services -- like C, C++, Java and Rust -- don't have those things either.

cortesoft
4 replies
16h15m

Migrating to 1.9 was a huge hassle for many firms.

This seems contrary to my experience. We took a large project from 1.8 to 1.9 to 2.0 to 3.0, and it was much easier than we expected. It was a lot easier than our Python 2 to 3 conversations were.

lolinder
2 replies
15h59m

It was a lot easier than our Python 2 to 3 conversations were.

Python's is (present tense very much intended) notoriously one of the worst-managed transitions in programming language history, so that's not exactly a ringing endorsement.

KerrAvon
1 replies
14h56m

It is completely relevant, because it’s arguing against the claim that Ruby was unstable once back in 2005.

lolinder
0 replies
14h22m

I didn't say it wasn't relevant, I said it wasn't a ringing endorsement. It can have been better than Python and still completely unbearable.

solidsnack9000
0 replies
16h8m

The comparison to Python isn't the relevant one. During that time, what was it like to migrate from shell to...probably the same shell?

KerrAvon
2 replies
14h58m

Your first two points don’t seem valid, in my experience.

The Ruby 2.0 migration wasn’t that interesting from a compatibility perspective; it certainly wasn’t anything like Python 2 -> 3.

And Ruby is __not__ slow compared to bash. I don’t where these myths get started, but someone needs to justify the Ruby-is-slow thing with actual data.

networked
1 replies
6h16m

I don’t where these myths get started, but someone needs to justify the Ruby-is-slow thing with actual data.

As an outside observer of the Ruby world, I have an impression that it was Ruby MRI that was slow. CPU-bound synthetic benchmarks like the much-criticized Benchmarks Game showed Ruby ≤ 1.8 a good deal slower than CPython 2. Here is an illustrative comment from that time: https://news.ycombinator.com/item?id=253310. People also complained about early Rails, and the perception of Ruby's performance got mixed up with that of Rails.

Then YARV came out, and Ruby became several times faster than its MRI former self on different benchmarks (https://en.wikipedia.org/wiki/YARV#Performance). With YARV, Ruby gradually caught up to "fast" interpreted languages. Now interpreted Ruby seems as fast as CPython 3 or faster in synthetic benchmarks (for example, https://github.com/kostya/benchmarks/blob/7bf440499e2b1e81fb...), though still behind the fastest mainstream interpreters like Lua's. Ruby is even faster with YJIT.

chucke
1 replies
9h6m

Mentioning 1.9 migration and ruby being slow? Python 2 to 3 was waaaaay worse and more negatively impactful, and equally slow (slower in most cases).

Ruby never had US market penetrative as perl or python, which were basically invented in the US, and congregated people from the academic realm. These things aren't decided based on meritocracy (no things ever are).

dragonwriter
0 replies
16m

Mentioning 1.9 migration and ruby being slow? Python 2 to 3 was waaaaay worse and more negatively impactful

Python 2-to-3 was mainly worse than Ruby 1.8 to 1.9 because Python had already won, and had a much bigger and more diverse ecosystem.

RulerOf
0 replies
14h37m

Ruby does have a terse and intuitive syntax that would make for a good system shell.

I learned enough PowerShell to be comfortable using it, and then picked up Bash and Ruby a few years later.

I longed for a Ruby shell for a couple years.

yen223
7 replies
16h26m

It wasn't that long ago that all the interesting infrastructure projects (vagrant, chef) were written in Ruby.

CoffeeOnWrite
2 replies
15h49m

fluentd today is a popular (most popular?) log collector in k8s land.

seneca
0 replies
2h53m

A lot of people use fluentbit specifically because fluentd doesn't perform well.

nunez
0 replies
3h34m

Wow didn't know fluentd is a Ruby production. Who said Ruby is slow?

inferiorhuman
1 replies
9h27m

I'd argue that writing Chef in Ruby (and Erlang) was absolutely to its detriment. Yeah, it was popular. It was also a debugging and scaling nightmare (not that Opscode helped that any).

In fact one of the reasons I rage quit megacorp for a second time was that I was required to use an Enterprise Chef instance that would log people out at random every 0-3600 seconds. I could throw plenty of deserved shade at my coworkers but Opscode didn't understand their product any better and I wasted more than enough time on conference calls with them.

codesnik
0 replies
8h13m

I love ruby, and I'm using it for 18 years, but I've spent half a year on chef a decade ago and it was one of the worst wastes of time I had ever. Nothing to do with the language, everything to do with architecture of the thing.

nunez
0 replies
3h35m

Shopify and GitHub are still mostly Ruby, right?

grumpyprole
0 replies
11h7m

You missed out GitHub !

shagie
5 replies
14h44m

... maybe extract those files to required files, add gems, whatever.

CPAN is the killer feature of Perl. It just works. First off, most of the time I don't need a CPAN module for doing shell scripting in perl. Perl itself is rich enough with the file manipulations that are needed for any script of less than 100 lines.

My experiences with Ruby and installing gems have been less pleasant. Different implementations of Ruby. Gems that don't compile / work on certain architectures. Breaking changes going forward where a script that was written 2 years ago doesn't work anymore. Sometimes it's someone was doing something clever in the language that doesn't work anymore. Other times its some gem got updated and can't be used that way anymore. ... which brings us to ...

I believe that Go's advantages come into play when the program gets more complex that that 100 line size and it becomes a "program" rather than a "script" that has complexity to deal with. Furthermore, executables built in Go are most often statically linked which means that someone upgrading the libraries doesn't break what is already working.

SoftTalker
4 replies
14h19m

Does anyone under age 50 or so even know Perl?

kfrzcode
1 replies
14h10m

Yes.

newalexandria
0 replies
13h53m

Do you all order the same thing when you get together at the cafe?

pcwalton
0 replies
13h25m

I'm not even 40 and I remember it well enough.

mekster
0 replies
13h48m

Make it 30 and it's actually a question.

SoftTalker
3 replies
15h16m

why we don't see ruby used for shell stuff more often

Simple, ruby is not installed by default. Even Python, while it is on (almost?) all modern Linux distributions, is not installed on the BSDs.

jiggunjer
1 replies
12h18m

Python's also not installed by default in (most?) official docker images.

shagie
0 replies
4h55m

Docker images fill a different role. They shouldn't have everything installed on them as that broadens the attack footprint. They should be doing one thing, and one thing only. If it's a "run this executable that was built" - then only what is needed should be there.

Installing python and other general purpose tools gives any attacker that gets into a docker container many more tools to work with for getting out.

For docker, the trend isn't "build a general purpose machine" but rather "what can we slim this down to that only has the bare minimum in it?" This can be taken all the way to the distroless images ( https://github.com/GoogleContainerTools/distroless ) and means that the security team won't be asking you to fix that CVE that's in Python that you don't use.

If, however, you do need python in an image because that image's purpose is to do some python, then you can pull a python image that has the proper release.

oopsallmagic
0 replies
9h20m

Can't you just install it?

zarzavat
2 replies
15h8m

What tooling do you use that’s written in Go? I’d have said that Python is the most popular language for tooling, by a country mile.

The only tooling I know that’s written in Go is Docker.

0xCMP
1 replies
14h28m

lots of CLIs are written in Python, absolutely, but many started more recently are almost exclusively Go unless there is serious interest in using Rust. It's almost certainly the ease of cross compilation plus the ability for users to run it without changes to their system.

theshrike79
0 replies
11h41m

This is the key.

I can easily provide precompiled packages for all sane combinations and users can just download one executable, edit the config file and be running.

Instead of having to mess with virtual environments and keeping them updated (they tend to break every time you upgrade the system python version).

SPBS
2 replies
9h34m

meanwhile, a lot of tooling nowadays is written in Go, and I have no idea why

What? Go is used because distributing a static binary without any dependencies is way better than asking each and every user to download an interpreter + libraries.

sgarland
1 replies
5h31m

So stop using 3rd party libraries. Seriously, the number of times I’ve seen people importing requests to do a single HTTP GET, or numpy to use a tiny portion of its library is absurd. You can do a hell of a lot with the stdlib if you bother to read the docs.

p_l
0 replies
4h58m

Not using third party libraries does not help against py2->py3 and changes between 3.x point versions.

It's only relatively recently that I could really expect that the target system would have python3, and then I'd also have to deal with some really annoying errors (like python3 barfing on non-ASCII comments when reading a source file with "C" locale, something that used to work with python2 IIRC, and definitely was an issue with "works on my machine" devs).

venvs are horrible, even compared to bundler.

But the python2 era left imprint on many who think it's just going to be there and work fine.

zarathustreal
0 replies
4h25m

Just because you don’t see it doesn’t mean it’s not the most-used shell scripting language. For example, when I was at AWS it was used for templating in something like 90% of all pipeline tooling

pdimitar
0 replies
3h42m

meanwhile, a lot of tooling nowadays is written in Go, and I have no idea why

No-dependencies final static binary.

it's not friendly for os manipulation at all

If you say so. I'd love to hear how did you get to that conclusion.

and number crunching power is not needed in many, many tasks of that sort.

You are aiming very wrongly, it's about startup time. I got sick of Python's 300+ ms startup time. Golang and Rust programs don't have that problem.

lelanthran
0 replies
3h18m

meanwhile, a lot of tooling nowadays is written in Go, and I have no idea why, it's not friendly for os manipulation at all

I'm not sure where you're going with this: My experience of Ruby and Go is that:

1. Go is a lot easier to do OS manipulation type stuff.

2. Go is a lot easier to modify down the line.

TBH, #2 is not really a consideration for shell-scripts - the majority of the time the shell script is used to kick off and monitor other programs, transforming an exact input into an exact output.

It's glue, basically, and most uses of glue aren't going to require maintenance. If it breaks, it's because the input or the environment changed, and for what shell is used for, the input and the environment change very rarely.

giraffe_lady
0 replies
16h36m

You can kind of figure it out by skimming the comments here. Most mainstream languages have decent-to-great tools built in for scripting, so the difference isn't that huge. So people just prefer to script in the language they already prefer in general, or that the project is written in.

braza
0 replies
10h35m

I sometimes wonder why we don't see ruby used for shell stuff more often

The best piece of code that I worked on was an ETL in pure Ruby. Everything in modules, simply to read, no crazy abstractions, strange things like __main__, abstract clssses or whatever.

Maybe others can chime in, but the main difference that is found in ruby developers is that they really have fun with the language making everything with a higher lever of software craftsmanship that other folks in the data space, e.g. Python of Julia.

WWWMMMWWW
0 replies
7h9m

because there's Python

djbusby
25 replies
17h11m

These are great points, if you already have Ruby in your stack.

For me, when Bash ain't enough I upgrade to the Project language (PHP, Python, JS, etc). For compiled project I reach for LSB language (Perl, Python) before introducing a new dependency.

cortesoft
11 replies
17h7m

Ruby is fantastic for the main project language, too! And is also standard on many Linux distros.

__loam
6 replies
15h17m

Ruby is unsuitable for large projects for the same reason python is, but it also lacks the huge ecosystem and labor pool that python has. When I'm interviewing and a company tells me ruby is the main language, I end the call.

davepeck
4 replies
14h56m

“X is unsuitable for large projects” when there are many readily discovered existence proofs to the contrary (including but not limited to $1B+ businesses, massive communities, deep thoughtful and disciplined engineering spokespeople, etc.) strikes me as a common trope here on HN.

(Which is not to say X is flawless even at scale or a clear best fit along all axes. That is true for no X that I know of.)

justin_oaks
1 replies
14h48m

The fact that a language is used for large, successful businesses/projects doesn't mean another language wouldn't be better. It's just terribly difficult to measure such things.

cortesoft
0 replies
13h58m

Another language being "better" is very different than the language being "unsuitable"

__loam
0 replies
13h29m

Despite ruby, not because of it.

IshKebab
0 replies
11h51m

It's absolutely possible to have a successful product with an unsuitable language. It doesn't mean it doesn't cost you.

sgarland
0 replies
5h26m

I think JavaScript as a backend language was a mistake, and that Node has single-handedly caused more damage to the entire tech industry than any other aspect.

That hasn’t stopped billions of dollars of revenue from being created with it.

At least Ruby is unpopular enough (compared to Node) that people who know it are probably decent at their job.

simoncion
3 replies
16h35m

Ruby's fine for small to medium-sized projects. It's pretty great for small DSLs.

In my professional experience on medium to large-sized projects, its lack of explicit typing, the habit of Rubyists to use its fairly-substantial metaprogramming capabilities at the drop of a hat, and (for projects that include pieces or all of Rails) the system's implicit library loading make such projects a nightmare to reason about and maintain.

I'm sure these aspects are manageable with a small group, or a very, very disciplined group, but when you have large-sized projects that have been continually worked on for ten, fifteen, more years, you're just not going to be able to guarantee that everyone involved will have the combination of knowledge and discipline require to ensure the thing remains easy to understand, extend, and maintain.

cortesoft
0 replies
16h18m

I work on a 12 year old Ruby project at my job, and while some of what you say is true, I still love working on it more than anything else.

amarshall
0 replies
15h49m

Implicit library loading isn’t even the real problem, it’s the global namespace. I think it, more than anything else, makes large projects difficult to manage due to the inability to grok the dependency graph.

IshKebab
0 replies
11h52m

This has been my experience too reading the Gitlab code. It's absolutely impossible to follow - you can't use static typing to follow flow because there isn't any, and you can't even grep for identifiers because half of them are dynamically generated. Every time I've wanted to understand something I've been unable to even find the relevant code.

Contrast that with gitlab-runner (Go) or VSCode (Typescript) both of which I have been able to not only read easily but contribute features to. VSCode is at least as big a codebase as Gitlab.

That experience has made me want to avoid Ruby like the plague!

nightpool
6 replies
16h54m

Many distros come with Ruby standard, it's not very big and I think it has a lot to recommend it over Python or Perl when it comes to lightweight scripting for sysadmins and day to day automation. I wouldn't necessarily pull it into an open source project where they weren't already present, but I would definitely choose it for personal use well before Python or Perl

dymk
5 replies
16h46m

Ubuntu, Debian, Arch, CentOS - none of these, as far as I know, ship with a Ruby interpreter by default. I’d like to be wrong, but I don’t think many do.

mixmastamyk
1 replies
16h26m

It's in the repos, but not installed by default.

Not on my Fedora box either.

gavindean90
0 replies
14h2m

And Python 3 is everywhere

o11c
0 replies
14h37m

It might be installed by default from the graphical CD, but you can't assume everybody uses that, even outside of server environments.

bigstrat2003
0 replies
14h41m

Does Arch even ship with Python by default? Its whole thing is being minimalist by default and letting you customize what is and is not installed.

sestep
5 replies
17h9m

What does LSB stand for?

o11c
1 replies
14h44m

Nobody actually follows the LSB as written. But it remains useful as an idea of "stuff that was installed in old distros so usually has some newer version available".

Assuming you can survive all the incompatible interpreter changes for Python etc., the main annoyance with LSB proper is shared library versions.

nequo
0 replies
2h29m

That is one reason to use Perl I guess where you can request the feature set of a specific Perl version.

wisemang
0 replies
17h8m

Linux Standard Base. Already on the system.

lr4444lr
13 replies
16h25m

Not denying Ruby is good, but other than process forking, why should I prefer it to Python?

katbyte
6 replies
16h1m

Not using white space as a delimiter

ianschmitz
5 replies
11h56m

Ruby is surprisingly picky with white space

manume
3 replies
11h25m

Can you give an example? I can't think of a single situation where whitespace matters in Ruby (unless of course you forget to put a space between two commands or something silly).

codesnik
1 replies
7h49m

if foo is a method then

`foo + bar` and `foo+bar` are `foo()+bar`, but `foo +bar` is `foo(+bar)`

ternary ? : also has some interesting whitespace dependent mixups with symbols, but I cannot remember what. I think that parser has many gotchas like that, but they are really really rare to bite you, because ruby's magic follows human intuition as much as possible.

p_l
0 replies
4h49m

still less annoying than Python's semantic whitespace

ezrast
0 replies
1h58m

It's not really a problem in practice (and I love Ruby), but it's still wild to me that they made the parser do this:

    $ irb
    irb(main):001:0> def foo(x=70) = x
    => :foo
    irb(main):002:0> i = 2
    => 2
    irb(main):003:0> foo / 5/i
    => 7
    irb(main):004:0> foo /5/i
    => /5/i

katbyte
0 replies
1h22m

Yea but it’s still not a delimiter

cyclotron3k
3 replies
9h58m

Personally, I find Ruby's syntax more natural, (I'm going to be heavily biased though, having written Ruby for 10+ years). But for example, let's say I wanted to make a hash (dict) of files, keyed by their size (for some unknown reason), in Ruby it would look like:

  Pathname.glob('*').filter { |f| f.file? }.each_with_object({}) { |f, h| h[f.size] = f }
Whereas the equivalent Python would be:

  result = {}
  for file_path in glob.glob('*'):
    if os.path.isfile(file_path):
      result[os.path.getsize(file_path)] = file_path
Or capitalizing a string in Ruby:

  string.split(' ').map(&:capitalize).join(' ')
And in Python:

  words = string.split(' ')
  capitalized_words = [word.capitalize() for word in words]
  result = ' '.join(capitalized_words)
Python seems to be more convoluted and verbose to me, and requires more explicit variable declarations too. With the Ruby you can literally read left to right and know what it does, but with the Python, I find I have to jump about a bit to grok what's going on. But maybe that's just my lack of Python experience showing.

sgarland
0 replies
5h13m

  from string import capwords 
  result = capwords(string)
This does the split/capitalize/join dance, all in one.

The file example you gave could also be turned into a dict comprehension if desired. I’m on mobile, but I think this would work.

  result = { f:os.path.getsize(f) for f in glob.glob(“*”) if os.path.isfile(f) }

pseudalopex
0 replies
5h30m

    result = {os.path.getsize(f): f for f in os.listdir() if os.path.isfile(f)}

    result = ' '.join(word.capitalize() for word in string.split(' '))
    result = ' '.join(map(str.capitalize, string.split(' ')))
    result = string.title()

ndand
0 replies
5h53m

You can capitalize a string in Python using functional style

  ' '.join(map(str.capitalize, string.split(' ')))
which is similar to the example in Ruby, except the operations are written in reverse order.

manume
0 replies
11h24m

For 99% of use cases (especially when writing shell scripts) it doesn't matter, just pick the one you know better. Both are nicer than Bash though. :)

corytheboyd
0 replies
15h24m

It’s just preference at that point IMO, and I’d base it off “what language are the rest of the scripts using?”.

xavdid
11 replies
13h44m

Ruby's a great language- I've always enjoyed its ergonomics and clarity. But its editor tooling hasn't kept up with its one-time competitor, Python.

I've mostly been in the Python ecosystem for the past few years and the LSP investment from Microsoft has really shown. Rich Python support in VSCode is seamless and simple. Coming back to Ruby after that caught me off guard - it feels like I'm writing syntax-highlighted plain text. There's an LSP extension from Shopify, but it's temperamental and I have trouble getting it working.

Editor support isn't everything (the actual language design is still the most important), but it definitely affects how eager I am to use it. I basically never choose Ruby over Python given the option, which is too bad. Ruby's a cool language!

rafamvc
5 replies
13h36m

It got a lot better recently. You should take a second look.

trevor-e
2 replies
13h32m

I tried last month and it was still a mess. The old Ruby extension used to work fine and the new LSP one from Shopify doesn't want to work for whatever reason.

manume
1 replies
11h51m

... the new LSP one from Shopify doesn't want to work for whatever reason.

Sorry, but calling it "a mess" simply because you can't get it to work is quite unfair. I've been using the LSP from Shopify since it came out, it works great, is very stable and updates come in on a regular basis.

trevor-e
0 replies
2h27m

I would say it's quite fair. It's not just me but several coworkers, other people in this thread, and reviews on the actual VSCode extension itself. I sank several hours trying to fix whatever issue it has with my system and continued to run into problems. I'll give it another shot when I'm back on Ruby projects.

xavdid
1 replies
13h32m

I mean, I was trying to set up editor support for a Ruby script last week. So unless it was improved really recently, it's not where I'd like it to be.

manume
0 replies
11h48m

but it's temperamental and I have trouble getting it working.

I was trying to set up editor support

Not sure what problems you had exactly, but saying that editor tooling is bad, simply because you can't get it to work, is not fair. I've been using the LSP from Shopify since it came out, it works great, is very stable and updates come in on a regular basis.

anothername12
2 replies
13h31m

The “editor tooling” in Jetbrains’ Rubymine is fabulous.

xavdid
1 replies
13h25m

That's true, I've heard good things! I don't use Jetbrains products, but I'm glad they work well here.

anothername12
0 replies
13h7m

Yeah I think they have some special sauce that puts them a bit ahead of what LSP-based. Definitely check it out

barrell
1 replies
10h17m

Funny, I have the complete opposite experience with python. Constantly turned off inline errors and warnings because they were almost always wrong - packages I installed “could not be found” by the LSP, it constantly worried about type issues that were no longer incorrect, it didn’t pick up function changes across files, etc etc.

Then you think “maybe I just have the wrong lsp” only to realize there are half a dozen that all behave differently and nobody can agree on.

I tried them all, they all turned even my simplest of scripts between 10-50% red. I think half of my ire towards python came from the fact that the LSP situation was so awful, I just had to get used to reading code with a bunch of “errors” in my face, or turn them off completely… I could never decide which was worse

globular-toast
0 replies
5h59m

Hmm... I use python-lsp-server and it just works almost every time. The main thing is your editor being aware of your venv. Which editor do you use? I use direnv and the direnv support in Emacs so each project has its own venv, then install the project plus all dev dependencies (like type stubs) in that venv. The LSP server itself is installed globally, though.

kazinator
11 replies
16h43m

It seems like a waste of precious syntax to dedicate backticks to running shell commands.

What's between the backticks is not even portable; the commands rely on an operating-system-specific command interpreter.

puts `ls`.lines.map { |name| name.strip.length } # prints the lengths of the filenames

Fantastic example, except for the commandment violation: "thou shalt not parse the output of 'ls'"!

You really want to read the directory and map over the stat function (or equivalent) to get the length.

  2> (flow "."
       open-directory
       get-lines
       (mapcar [chain stat .size]))
  (727 2466 21 4096 643 16612 5724 163 707 319 352135 140 51 0 4096
   114898 1172428 1328258 4096 4096 4096 29953 4096 4096 0 27 4096
   4096 35585 8450 968 40960 14610 4096 14 755128 1325258 4096 17283
   218 471 104 4096 99707 1255 4096 129 4096 721 9625 401 15658
   4096 235 98 1861 664 23944 4286 4096 1024 0)

Too
7 replies
13h59m

They have inherited the second biggest mistake of shellscripts; Requiring the user to manually check $? after each command.

No thanks. Anything that doesn't have error handling enabled by default goes straight in the trash bin.

oopsallmagic
1 replies
9h17m

How is it supposed to know how you want your errors handled by default? Handling errors is the programmer's job.

Too
0 replies
6h22m

There are lots of techniques to make it obvious that errors can happen and need to be handled.

1. Make it difficult to ignore that a function can return an error. This is the golang approach. Errors are part of the return values and unused return values are compiler errors.

2. Make it impossible to use parts of the return value having an error state. Rust does this with the Result sum type and pattern matching.

3. Tailor for the happy-path and throw exceptions in case there are any errors. With optional means to catch them if recovering errors is desired. This is how most other languages function.

Hiding the error status in another variable, that is super easy to overlook and that the programmer might not even know exists, then continuing despite this variable not being checked will inevitably introduce bugs allowing faulty data in your system.

latexr
1 replies
9h40m

They have inherited the second biggest mistake of shellscripts; Requiring the user to manually check $? after each command.

This isn’t true for Ruby nor shell scripts. In Ruby you have `system` or `Open3`. In shell scripts you:

  if my_command
  then
    on_success
  else
    on_failure
  fi
Shellcheck even warns you of that.

https://www.shellcheck.net/wiki/SC2181

jerska
0 replies
3h10m

You’re checking the return value of the command here. Do you wrap all calls in an if?

I believe the author was talking about set -e (often used with -o pipefail), so that any unhandled error simply exits the script.

tasuki
0 replies
11h13m

It's not required to check $? after each command. It's only when calling out to the shell. How would you propose to handle that? Throw an exception on a shell command which fails?

Some would say calling out to shell is an anti pattern by itself. Others would say exceptions are an anti pattern. (Just use appropriate return type and there's no need for exceptions ever!)

stouset
0 replies
10h22m

This is only true for backticks, which are somewhat intended for non-serious use. If you want exceptions for subprocess failure, `system` does the trick.

There’s really no need for this kind of over-the-top response.

nomilk
0 replies
12h36m

I couldn't spot any way to make something like `ls -j` (j is an illegal option) throw an exception in ruby (as opposed to simply outputting the system error message).

The closest I could find is what you suggest (checking $?), or using something like this [1], which would require changing syntax:

  system('ls -j', exception: true) 
Would be great to know if there's some easy callback or, ideally, a global setting one can make so a ruby exception is thrown if there's an error running system commands using the backtick syntax.

[1] https://stackoverflow.com/a/54132729/5783745

kouteiheika
1 replies
7h26m

You really want to read the directory and map over the stat function (or equivalent) to get the length.

Indeed. Equivalent in Ruby:

    Dir["*"].map { |x| File.size(x) }

kazinator
0 replies
7h18m

I avoided

  (flow (glob "*") (mapcar [chain stat .size]))
because * skips entries starting with dot, which sends us down a certain distracting rabbit hole.

pdntspa
0 replies
2h2m

And what psychopath is trying to parse the output of ls?

None of this makes any sense to me, and I write Ruby for my day job.

aendruk
1 replies
13h44m

Also been loving that Nix can give you this for shell scripts, e.g.:

  #!/usr/bin/env nix-shell
  #!nix-shell -i bash -p cowsay imagemagick

  identify "$1" | cowsay
https://nixos.wiki/wiki/Nix-shell_shebang

quechimba
0 replies
16h4m

Didn't know about this, very useful!

meiraleal
0 replies
17h0m

All browsers implemented it before deno with ESM and importmap.

corytheboyd
0 replies
15h32m

Yay I’m glad someone else knows about this and thinks it’s awesome too :) It’s crazy I didn’t find it until I was digging through bundler docs looking for something completely unrelated

didip
8 replies
16h46m

No it’s not a great language.

Too many ways to do the same thing.

Not packaged by default on most Linux.

Monkey patching makes things even harder to debug.

x3n0ph3n3
4 replies
14h25m

Monkey patching isn't the norm anymore and not really a valid complaint in this decade.

manume
3 replies
11h13m

Hearing people still mention "monkey patching" always makes me chuckle... I haven't "monkey patched" anything in Ruby in > 5 years, and I don't see it in any of the popular libraries/gems anymore either.

p_l
2 replies
4h48m

Especially when the big difference is that Ruby had proper OO system that allowed patching, when necessary, to be done in much saner way...

whereas the origin of monkey patching seems to by Python with its totally broken magical method names and kitbashed object model.

x3n0ph3n3
1 replies
34m

Module#prepend was introduced in Ruby 2.0, in 2013, which was the solution to monkey patching.

dragonwriter
0 replies
20m

Well, that and Module#refine.

ekianjo
0 replies
16h44m

yeah. The not packaged by default is the killer issue.

dwheeler
0 replies
16h14m

Ruby is packaged for all significant Linux distros. It isn't installed by default in many, but installing it is a trivial one-time command, and it takes relatively little space.

alabhyajindal
0 replies
1h9m

You're the one doing monkey patching when you're writing a program. Don't do it if you think it makes things hard.

rednafi
7 replies
16h34m

Ruby is slow and encourages an esoteric convention over configuration style of OO code.

If you enjoy it, more power to you. However, Python is everyone's second favorite or least favorite language, and it runs laps around Ruby any day. Then there's Go if you need some extra oomph!

jakogut
2 replies
16h32m

Go is a good language for when I want binaries an order of magnitude larger than necessary.

simoncion
1 replies
16h24m

Oh but golang's "deployment story" is SO SIMPLE! /s

notnmeyer
0 replies
14h54m

isn’t it though?

webstrand
0 replies
16h32m

Perl is basically available everywhere git is installed, too. Way lower startup time than ruby.

pilaf
0 replies
15h57m

Ruby performance has improved a lot in the last few years, especially with the introduction of the JIT (YJIT), and it keeps improving with every new release. I think the notion of Python being the faster of the two may be outdated. You can see some comparisons in the Benchmarks Game [1] (ignore the reference to PHP and Erlang in the URL, seems to be a typo by the website's maintainer, although my link will break if they fix it).

Also I think the convention over configuration mantra belongs in the Ruby on Rails world rather than Ruby itself. Ruby as a language is far less opinionated about how you do things, especially for small shell scripts.

1: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

pansa2
0 replies
16h1m

Python […] runs laps around Ruby any day

Is today’s Python much faster than, say, 5 years ago? Ruby was definitely quicker than Python back then.

I’ve heard of performance improvements to Python, but they seem to be 10-25% range which wouldn’t be enough to catch up to Ruby.

gwright
0 replies
16h13m

Ruby is slow and encourages an esoteric convention over configuration style of OO code.

I think you are commenting on Rails, not Ruby when you refer to "convention over configuration". Even in Rails I'm not sure what "esoteric" means in your comment and that approach isn't even related to the object-oriented concepts.

nomilk
5 replies
13h39m

If you never coded in ruby before, but use macOS, you already have ruby installed. Just open terminal and type

  irb
to bring up the ruby interpreter and try out the code in the article.

anothername12
3 replies
13h18m

I think the built in one is deprecated though right?

Neywiny
2 replies
12h35m

Are there distributions where this doesn't happen? I feel like I'm often installing or compiling new versions of packages and languages because they're outdated on Ubuntu

medstrom
1 replies
5h40m

A rolling distro like Arch or Void.

Neywiny
0 replies
3h35m

Gotcha, thanks

shagie
0 replies
4h18m

    ~ % irb
    
    WARNING: This version of ruby is included in macOS for compatibility with legacy software. 
    In future versions of macOS the ruby runtime will not be available by 
    default, and may require you to install an additional package.

    irb(main):001:0>

kajika91
4 replies
12h9m

No pipe (and I mean in parallel too) no love for me.

Nice calling syntax though.

copirate
3 replies
8h59m

You can pipe with the `pipeline*` method of open3 which is part of the stdlib:

For example:

    require "open3"
    last_stdout, wait_threads = Open3.pipeline_r("cat /etc/passwd", ["grep", "root"])
    last_stdout.read # => "root:x:0:0::/root:/bin/bash\n"
    wait_threads.map(&:value).map(&:success?) # => [true, true]
https://ruby-doc.org/3.2.2/stdlibs/open3/Open3.html

e12e
2 replies
6h14m

Can you easily chain these, though? (gzcat some.txt|grep foo|sort -u|head -10 etc?). Especially lazily, if the uncompressed stream is of modest size, like a couple of gigabytes?

RulerOf
1 replies
1h27m

I'd suspect you could do that with Open3, but if you are, why not just read the file and process with Ruby instead?

e12e
0 replies
51m

I'm currently working with 150MB worth of gzipped JSON - marshalling the full file from JSON to ruby hash eats up a lot of memory. One tweak that allows for easier lazy iteration over the file (while keeping temporary disk Io reasonable) is to pipe it through zcat, jq in stream mode to convert to ndjson, gzip again - for a temp file that ruby zlib can wrap for a stream convenient for lazy iteration per read_line...).

Generally marshalling a gig or more of JSON (non-lazily) takes a lot of resources in ruby.

floppy-disk
3 replies
10h28m

While I write most of my scripts in Ruby and enjoy doing so, there is one gripe I have with it: its slow start-up time. On my machine, running an empty Ruby script takes about 100ms, compared to <10ms for Python, Perl, Lua, Bash.

One can mitigate the problem somewhat using the `--disable-gems` flag, but that's not a good general solution.

Borg3
2 replies
9h59m

100ms !!!! wtf? :)

[borg@cube] time ruby -e 'nil'

real 0m0.007s

floppy-disk
1 replies
9h42m

Whoa, interesting... What version are you running? Here's my system:

  $ ruby -v
  ruby 3.3.1 (2024-04-23 revision c56cd86388) [x86_64-linux]

  $ time ruby -e ''

  real 0m0.122s
  user 0m0.102s
  sys 0m0.020s
I found an old Reddit thread also hinting at bad start-up times: https://old.reddit.com/r/ruby/comments/aqxepw/rubys_startup_....

Borg3
0 replies
6h20m

Very old one :) Because its reliable. Version 1.8.7 here. I want controllable delays. I want controllable threads :) I want portability.

Of course, if you need more peformance, you need to go for newer stuff.

cyclotron3k
3 replies
16h46m

Totally agree! Other tricks I rely on:

a) put a `binding.irb` (or `binding.pry`) in any rescue block you may have in your script - it'll allow you to jump in and see what went wrong in an interactive way. (You'll need a `require 'irb'` in your script too, ofc)

b) I always use `Pathname` instead of `File` - it's part of the standard library, is a drop in replacement for `File` (and `Dir`) and generally has a much more natural API.

c) Often backticks are all you need, but when you need something a little stronger (e.g. when handling filenames with spaces in them, or potentially hostile user input, etc), Ruby has a plethora of tools in its stdlib to handle any scenario. First step would be `system` which escapes inputs for you (but doesn't return stdout).

d) Threads in Ruby are super easy, but using `Parallel` (not part of the stdlib) can make it even easier! A contrived example: `Parallel.map(url_list) { |url| Benchmark.measure { system('wget', url) }.real }.sum` to download a bunch of files in parallel and get the total time.

MacOS has Ruby 2.6 installed by default which is perfectly serviceable, but it's EOL and there are plenty of features in 3+ that make the jump more than worthwhile.

MatthiasPortzel
2 replies
15h49m

You'll need a `require 'irb'` in your script too, ofc

irb is a part of Ruby core, so this isn’t true. (It may have been at one point? I’m not sure.)

I love binding.irb. I use it all the time.

delichon
0 replies
15h29m

Me too (binding.pry). I think of it as REPL based development.

cyclotron3k
0 replies
14h18m

Huh, TIL! Requiring `irb` has been unnecessary since 2.5 and I never noticed.

corytheboyd
3 replies
15h35m

Hell yeah! I’ll never forget being fooled by HN that you have to use Perl if you want portable scripts that aren’t bash, writing a whole script with it, and having a coworker politely, yet firmly, tell me that I am dumb and it should just be Ruby… and it was a script I was checking into a Rails app! It’s even trivial to include dependencies with bundler inline https://bundler.io/guides/bundler_in_a_single_file_ruby_scri...

notnmeyer
2 replies
14h57m

this is… somewhat horrific, right? would anyone do this without explicit version pinning?

loosely pinned deps installed on execution sounds fucking awful.

manume
0 replies
11h18m

You can pin a gem to a specific version, of course.

`gem "mygem"` installs the latest version. `gem "mygem", "~> 4.0.0"` installs >= 4.0.0 but < 4.1.0, which is what you probably want when using Semantic Versioning, which most gems adhere to, to get the latest patch version. `gem "mygem", "4.0.10"` installs exactly that version.

corytheboyd
0 replies
14h50m

Well, don’t use this then. It’s a neat feature for quickly iterating, and you can quite easily go up from here to a proper Gemfile/Gemfile.lock.

bdcravens
3 replies
14h27m

I spend most of my time writing Rails or other backend Ruby, and I prefer my system-level scripts in bash. Philosophically I don't want to have to manage dependencies or broken gems (though inline deps obviate that, and it's not like I've never had to wrestle with apt)

manume
2 replies
11h27m

What do you mean by "broken gems"?

bdcravens
1 replies
11h19m

Either dependency issues with other gems, or gems that break due to some sort of library in the OS. If you pin all of your versions, and use it in one place, that's less of an issue, but many scripts are designed to have some level of portability (even if it's to a new instance of the server)

manume
0 replies
11h5m

In my experience, Bundler has improved a lot with regard to resolving dependency issues over the years. And OS libs are only really depended on by a few gems, no? 99% of them don't use FFI or call OS libs.

Moreover, how often do you really move a script to a completely different OS, where you don't know which OS libs are installed? And wouldn't those missing OS libs also be a problem when writing the script in Bash or any other language?

pipeline_peak
2 replies
14h15m

Being able to call external commands with backticks alone makes it better suited than Python for shell scripting.

frou_dh
0 replies
8h55m

IMO that feature snatches defeat from the jaws of victory because if the command fails (exit code >0) it doesn't raise an exception. Same issue with the system(...) method by default.

__MatrixMan__
0 replies
13h14m

If you've reached for something with more structure than the string-soup that is bash, why would you then embrace the string-soup that is invoking subprocesses with backticks?

Involving an entirely separate parser just to split your args up feels like a footgun to me.

Python's `sh` handles this rather nicely:

http://sh.readthedocs.io/en/latest/

phendrenad2
2 replies
16h21m

Sadly, 9 out of 10 environments lack a Ruby interpreter out of the box. Are you going to add 5 minutes to your docker build to compile Ruby? Probably not.

Luckily, I've found that Perl has most of the best features of Ruby, and it's installed everywhere. It's time to Make Perl Great Again.

manume
0 replies
11h29m

Sadly, 9 out of 10 environments lack a Ruby interpreter out of the box.

Please name those 10 environments you are talking about. In my experience, a reasonably recent Ruby version is present almost everywhere.

add 5 minutes to your docker build

Why on earth would it take 5 minutes to install anything? If you install Ruby through a package manager (it's present in pretty much all of them: https://www.ruby-lang.org/en/documentation/installation/#pac...) it takes only seconds.

bigstrat2003
0 replies
14h40m

Why on earth would you add the compilation of a Ruby interpreter to your docker build? Just install it through the package manager of whatever distro your image is built on.

easylion
2 replies
16h59m

Java for the win. Have used java for any kind of programming or running shell commands or literally anything since last 5 years professionally and personally.

Language is just a tool, anything and everything works if you love it enough

simoncion
0 replies
16h19m

In fairness, Java is still pretty slow to start. If you're looking for non-traditional "scripting" languages, you might look into Erlang. 'erl_call' is a particularly interesting little helper script that's packed into the standard system.

cozzyd
0 replies
16h16m

java as a bash replacement is certainly a take I've never heard before...

dcchambers
2 replies
16h45m

I work for a company that has a large Rails monolith. Although we use many more languages than just ruby these days, we still have a ton of scripting, config, and tooling that is all in Ruby. It's a joy to work with IMO.

Another common pattern I see is people using embedded ruby in a shell script. It does make it a little harder to read/understand at a glance, but it's nice for being able to do most simple things in sh but drop into ruby for things that sh/bash suck at.

That said, I get a feeling that the people that joined once we'd added stuff outside of the Rails monolith and don't know/use Ruby are...not big fans.

mberning
1 replies
16h19m

I had the same experience. Somebody in our company inherited a Ruby script and was trying to modify it and was stuck. They came to me exasperated. The error message was something really trivial like addition is not defined for some object type. If you don’t understand the base level concepts of the language it’s going to be a very bad time. Sadly people are not that interested in learning about Ruby nowadays and look at it as a huge imposition to deal with. I love it still.

dcchambers
0 replies
16h14m

Yup that tracks with my experience.

Ruby is a wonderful language worth learning (and it's really not difficult to pick up) but I see way more people push against learning it than they do other things (eg python).

JohnMakin
2 replies
15h8m

As a self proclaimed shell advocate I am… intrigued. As a self proclaimed shell advocate, I also have a revulsion to back ticks.

pikelet
0 replies
13h54m

You can also use %x(ls some/dir), which is a syntax used in other places too, like %w[word word word] (array of word strings, notice you can choose the delimiter).

justin_oaks
0 replies
14h52m

Yes, all hail $( )

I haven't used backticks in my shell scripts in years.

oliviergg
1 replies
12h1m

For me, the problem with shell scripting is that I do it only occasionally. I’ve always been tempted by higher-level languages. The arrival of tools like ChatGPT has made me much more comfortable writing scripts of intermediate complexity. I find shell scripting more interesting now.

manume
0 replies
11h35m

Similar case for me, but I think the author says it well:

That is, most of the cases Bash for me is enough, but if the script starts to become complex, I switch to Ruby.

Even if ChatGPT lets you bang out more complex shell scripts easily, if you have to come back to it later on to fix an error or add a new feature, it's really hard to understand it (if you don't deal with such scripts on a daily basis). If you start with Ruby (or Python or similar) from the beginning, it's much easier to understand and extend later on.

fire_lake
1 replies
11h55m

Nothing about dependencies!

A scripting language needs a way to declare dependencies in a locked-down way, inside of the script that requires them. They must be portable across platforms.

derefr
1 replies
16h45m

I love using Ruby for shell scripting, but there are also a ton of little nits I have to fix whenever I'm doing it.

For example: Ruby has no built-in for "call a subprocess and convert a nonzero exit status into an exception", ala bash `set -e`. So in many of my Ruby scripts there lives this little helper:

    def system!(*args, **kwargs)
      r = system(*args, **kwargs)
      fail "subprocess failed" unless $?.success?
      r
    end
And I can't ask "is this command installed" in an efficient built-in way, so I end up throwing this one in frequently too (in this instance, whimsically attached to the metaclass of the ENV object):

    class << ENV
      def path
        @path ||= self['PATH'].split(':').map{ |d| Pathname.new(d) }
      end

      def which(cmd)
        cmd = cmd.to_s
        self.path.lazy.map{ |d| d + cmd }.find{ |e| e.file? && e.executable? }
      end
    end
I have other little snippets like this for:

• implicitly logging process-spawns (ala bash `set -x`)

• High-level wrapper methods like `Pathname#readable_when_elevated?` that run elevated through IO.popen(['sudo', ...]) — the same way you'd use `sudo` in a bash script for least-privilege purposes

• Recursive path helpers, e.g. a `Pathname#collapse_tree` method that recursively deletes empty subdirectories, with the option to consider directories that only contain OS errata files like `.DS_Store` "empty" (in other words, what you'd get back from of a git checkout, if you checked in the directory with a sensible .gitignore in play)

...and so forth. It really does end up adding up, to the point that I feel like what I really want is a Ruby-like language or a Ruby-based standalone DSL processor that's been optimized for sysadmin tasks.

adamnemecek
1 replies
16h31m

I’ve been enjoying nushell

__MatrixMan__
0 replies
13h10m

Nushell is fantastic.

Being able to just paste json into a terminal and immediately start working with it in a structured way feels like a superpower.

The interactive help and well-thought-out error messages feel like a hug.

JohnMakin
1 replies
15h6m

To simulate “types” in complex shell scripting I typically involve a lot of json objects acting as my data structures and the file system for a rudimentary database. They’re horrifying ugly, but they tend to work pretty reliably.

This is probably easier.

kchr
0 replies
6h4m

The `ip` network utility now supports JSON ouput via the `ip --json` invocation since a while back. I would love to see more tools implementing this as an option!

Demiurge
1 replies
16h43m

If Rails could be written in Python, I would probably prefer that. Ruby has too much of Perl. If I didn't choose to write a critical piece of software in Perl, which caused me numerous sleepless nights hunting for a missing quote or other weird character, I might have thought it was cool. By my experience with PHP and Perl is why I prefer Python. Clever is not what I ever want a language to be.

Python > Lua > PHP > Ruby > JavaScript > Perl > Bash

wkirby
0 replies
16h40m

Honestly same, but only because Python has types relatively sorted. While I actually enjoy the language, Ruby is… struggling… and after supporting multiple Rails apps in the wild for close to a decade, what me and my team miss the most is the ability to hint types easily.

softwaredoug
0 replies
4h53m

Shoutout to Hannes Moser at Shopify and our regular pairing on the Disco CLI tool that orchestrated a lot of Elasticsearch config stuff. :)

As a Python guy I found the setup for this sort of CLI too really refreshing!

samatman
0 replies
15h48m

The things which make Ruby good for shell scripts are, to a large degree, things it inherited from Perl. Which was, and is, a great language for scripting.

People use it a lot less these days, for a lot of reasons, some better than others. I myself do simple stuff in bash, and pull out some Python for more complex scripting. My Perl chops have withered, last time I was paid to use it was 21 years ago, but it really is a great scripting language, and you'll find it installed on a great deal more systems than Ruby.

One of these days I'll give Raku a spin, just for old time's sake.

nunez
0 replies
3h41m

I agree. Ruby is a _fantastic_ language for getting things done quickly whose credibility was unfairly maligned by Rails.

Unbelievably easy to read, and, with rspec, it is stupid easy to write tests for. No need to fuss with interfaces like you do with Golang; yes, that is the right thing to do, but when you need to ship _now_, it becomes a pain and generates serious boilerplate quickly.

I've switched to Golang for most things these days, as it is a much safer language overall, but when shell scripts get too hard, Ruby's a great language to turn to.

nightpool
0 replies
16h59m

Ruby is my favorite shell scripting language, I used it last year for a complex ffmpeg automation script (use blackdetect to detect long stretches of power-off and split a video file into much smaller components), and Ruby made it a breeze when I know it would have been a real struggle to get working in bash or powershell

hiAndrewQuinn
0 replies
10h29m

I have what probably sounds like a niche use case, where most of the boxes I work on don't have access to the Internet.

So for me, "is it installed in the base distribution" is the difference between being able to start immediately no matter which box I'm concerned with, and spending months trying to upstream a new program to be installed with our OS image team.

I took a look around a vanilla Debian 12 box, and didn't see Ruby in there [1]. So, sadly, although I really like the way Ruby looks, I'm going to have to stick with Bash and Python 3 for the hard stuff.

[1]: https://hiandrewquinn.github.io/til-site/posts/what-programm...

greenthrow
0 replies
16h8m

By definition a shell script is one that is run by the shell. The term should not be overloaded to include scripts that require another interpreter.

globular-toast
0 replies
6h3m

Cool, I'm sold, added Ruby to my to learn list! I use Python a lot but I don't think it's as good for writing scripts as bash or Perl. Ruby looks like it fits that "better but not much harder" category much better.

drusepth
0 replies
17h7m

Ruby is an amazing language. I've seen some systems it's not already installed on and hop over to something like perl/python in those cases, but Ruby is by far my preferred hammer for small scripts. The code is beautiful.

Small nit: your note in Feature 4 is actually supposed to be in Feature 5, I assume.

corytheboyd
0 replies
14h10m

Overall a nice lite write up! Bash is great, but it occasionally becomes untenable, usually around the time where HTTP and whatnot becomes involved. Same goes for shell exec exit codes, you can use an API like popen3 for this: https://ruby-doc.org/stdlib-2.4.1/libdoc/open3/rdoc/Open3.ht...

You mention using threads and regex match global variables in the same write up. Please use the regex match method response instead of the $1 variables to save yourself the potential awful debugging session. It even lets you access named capture groups in the match response using the already familiar Hash access API. Example: https://stackoverflow.com/a/18825787

In general, just don’t use global variables in Ruby. It’s already SO easy to move “but it has to be global” functionality to static class methods or constants that I’ve encountered exactly zero cases where I have NEEDED global variables. Even if you need a “stateful constant” Ruby had a very convenient Singleton mixin that provides for a quick and easy solution.

Besides, if you actually WERE to take advantage of them being global VARIABLES (reassigning the value) I would confidently bet that your downstream code would break, because I’m guessing said downstream code assumes the global value is constant. Just avoid them, there’s no point, use constants. This applies to any language TBH, but here we’re talking about Ruby :)

blahgeek
0 replies
17h2m

Perl also satisfies all listed features

bingemaker
0 replies
6h30m

My first application of Ruby was to use it for shell auto-completions. I'm so grateful that I learnt Ruby first, and then Rails. Ruby is a great language to get some utility working out real fast. Rails is great for MVP. I fail to understand why people bitch about Ruby/Rails by comparing them to other languages/frameworks.

aorth
0 replies
11h17m

Side note: Firefox is offering to translate this blog post from Portuguese for me though the content is clearly in English. I noticed the `<html>` element has a `lang="pt"` attribute. The site is generated by Jekyll, which I have not used in years, so I'm wondering if this is a site-level setting or could be overridden in frontmatter...