Rye: A Hassle-Free Python Experience

Given that you guys are in charge of both uv and rye, why keep two alive at the same time? Why not just kill rye now to avoid fragmentation, and confusion that comes with the burden of having to choose between the two?

I work on Rye and uv, if you have any questions :)

How do you pronounce "uv"?

How can you generate a universal lock file when some packages will query the running system in arbitrary ways during resolution?

Any ongoing work/plans to follow for cross-platform lock files?

This is one concern that would prevent the team I'm on from moving. We do ML work, so I'll use a pytorch-based project for my example. The desired state is that all dependencies are in pyproject.toml, e.g., from which you can generate a set of lockfiles from an AArch64 Mac or an AMD64 Windows workstation for the following platform configurations:

1. Mac with default (MPS) pytorch

2. Windows with CUDA pytorch

3. AArch64 Linux with CPU pytorch

4. AMD64 Linux with CPU pytorch

5. AMD64 Linux with CUDA pytorch

P.S. Big thanks for making Ruff, we're huge fans of linting and formatting both.

What's the relationship between uv and uvloop?

How does it compare to Pipenv and Poetry? I had some problems every time I used Poetry. I wanted to like it but it hanged or took forever and similar things often.

Does Rye/UV with universal resolution work properly with PyTorch, or does it suffer from the same issue as Poetry?

https://github.com/python-poetry/poetry/issues/6409

https://news.ycombinator.com/item?id=39257501

I assume this "Universal" resolution is enabled by adding the "--universal" cli argument?

It doesn't look like this is currently documented but I found some hints in the release notes.

I like how you ignore the "best practices" for packaging created by PyPA (i.e project.toml and friends) and just do requirements.txt cowboy stuff.

Don't get me wrong, both are hilariously bad, but I like to see more chaos in Python infrastructure. You pushing against PyPA (rightfully) delegitimizes them.

Great work. I've switched to using Rye now as I used to have to occasionally setup a new computer to work on a project and it was always a complete pain (pyenv+venv+pip).

Now its:

* Install Rye,

* Pull from Github,

* Type rye sync

You have a lot of firms that care about predictable and performant building that are shifting to you, and one of the things that comes up is SBOM generation for ingestion into tools like guac.

https://guac.sh/

https://pypi.org/project/sbom4python/ https://github.com/CycloneDX/cyclonedx-python

Your recently added ability to unpin dependencies so devs are more encouraged to stay compatible as they dev, then generate a correct explicit requirements.txt for reproducibility, makes both vuln management and the SBOM step a far easier thing than poetry etc.

Thank you!

For similar reasons, we use https://hatch.pypa.io/latest/why/ and appreciate that it plays nicely with `uv`.

I regularly download the python source code, compile it with standard prod optimizations, then install to /use/local/python${version}. This works extremely consistently since python 3.7 (released in 2018). In my experience, these commands are so stable and consistent they could be automated away. What might the author's issue or underlying protest be?

I'm just guessing, but I imagine the scenario goes like this:

1. Work at a company that runs some ancient version of CentOS or something.

2. The system package manager doesn't have anything newer than Python 3.6.

3. Official binaries from python.org print an inscrutable error about the glibc version when you try to run them.

4. Building Python from source requires some new system dependencies, takes forever, and then produces an even less scrutable error about a missing header file.

5. Googling that error leads to a ./configure flag that works around the issue, which upstream GCC fixed in 2017.

6. Success?

If you haven't seen that error #3 before, or dealt with "manylinux" build environments, you've dodged an entire world of pain. The same goes for scripting installs on Windows, and for the part of that page that's talking about "limiting the CPU instructions that can be used" :')

Agree. Support PSF, don’t advocate a new “right way” of doing it. Weird position.

I do like that Rye's Python builds apparently use libedit instead of readline. One less GPL-licensed thing to worry about for redistribution :)

Python building, packaging and deployment has two extreme states: the king's highway and the hall of a thousand knives. If the portable Python suggestions do not make sense to you, then consider yourself lucky, because you have managed to stick to the highway.

Original author of Rye here: there are no official Python builds outside of macOS and Windows and the macOS builds cannot be installed programmatically. They also cannot be placed in any location other than the dedicated framework location which often causes issues for people who do not expect specific versions to be installed. Quite often installing the macOS builds of Python breaks other tools that expect a different version to be there.

I’m glad regular Python versions work for you, and you can register them with Rye. That’s very intentionally supported.

The goal of rye is to reach higher. It wants to see how close we can get to an experience you can get from node or Rust. That includes getting access to Python itself.

I have been programming Python since 2.2 and have helped countless of people over the years with their Python challenges. From mentoring to tutoring to just helping on IRC and other places. So many times people had a completely broken Python installation or ran into compilation errors. Even pyenv ships patches to make Python compile cleanly in some cases.

The indygreg builds have issues, no doubt about it. In an ideal world the PSF would distribute portable Python builds for all platforms.

No Python Binary Distributions: CPython builds from python.org are completely inadequate. On some platforms you only get an .msi installer, on some you literally only get tarballs.

I'm just guessing, but they could mean that there are no macOS/Windows binaries for security releases of older Python versions. You can't, for example, download an installer for Python 3.10.14. The last Windows installer is for Python 3.10.11 (April 5, 2023).

It doesn’t work consistently across environments

Poetry has two flaws imo:

1. It's written in Python, which makes it slower and prone to bootstrapping issues.

2. It doesn't manage your Python installation, which necessitates the use of a tool like pyenv.

Rye sidesteps both of those by (a) being written in Rust and (b) trying to solve all of the problems solved by poetry and pyenv in one go.

Does it automatically install and setup the required Python version for the project?

If poetry works for you, it's fine. It works great 90+% of the time for me and I found it before rye or hatch or whatever, so I use it too. Eventually it will either have to adopt to the new standards as they firm up or it will probably get left in the dust, but that's fine. Hopefully if/when that happens something like rye will be mature and boring and well supported everywhere instead.

The latest tools are not always the best tools, old and boring, as long as they work for you are perfectly fine.

Poetry struggles with complex packages like PyTorch.

https://iscinumpy.dev/post/poetry-versions/

Neat! A relevant quote: "[rye] has matured to the point where it now does most if not all of what poetry does only faster"

I wrote the `so-many-python-package-managers` article and posted it earlier today. I'd like to apologize for misrepresenting `rye` in the first draft. I did not realize it had made so much progress in the span of the past year, and it really does look like an excellent tool. Certainly, with the `uv` backend, it just about fixes my one major gripe with `poetry`

I'll be giving it a go on my next project!

(Note: I updated my article just now to say more good things about `rye`)

It's "just" a wrapper for the existing tools. If you're already using ASDF for your python versions and know how to use venv, there's not much gain here - saves a couple of simple commands only. It could replace your poetry (https://python-poetry.org/) if you're using it though.

Specifically, it doesn't do anything new/different by itself.

So much free advertising XKCD gets simply by being always relevant... :~P

I was going to complain, but I’ll ask you/yall instead: what do you mean “makes it actually pleasant”? Is it too hard to summarize? Because I don’t think I ever identified anything about Anaconda or Poetry that felt like a particular choice, at least UX-wise. And curation-wise, it seems like a hard sell to trust a smaller org over the larger established group.

In other words: what does it say on the tin?? All I can read is “good package manager is good because good and fast and good”. Maybe there’s a comparison or ethos page…

Python feel like cargo

I am sold. Was thinking of trying out pixie after poetry took whole day and still couldn't resolve deps.

Looks like there are more python package managers that chat apps from Google ?

If like me, you've ignored poetry and friends and stuck with pip-tools (congrats!), uv (used by rye internally) is a drop in replacement.

IMHO pip-tools was always the far nicer design than poetry, pipenv etc as it was orthogonal to both pip and virtualenv (both of which have been baked into Python for many years now). I would argue Rye is the iterative, standards compliant approach winning out.

Beyond the speedups from Rust, it's nice to have some opinionated takes on where to put virtualenvs (.venv) and how to install different Python versions. It sounds small, but since wheels fixed numpy installs, sane defaults for these and a baked in pip-tools is basically all that was missing. Talking of which, what has been the point of anaconda since binary wheels became a thing?

As somebody who tried to pick up Python after hearing there was one way to do everything…the installation and environment management experience was a train wreck.

Glad to hear it’s getting better finally.

No, they're not. Just teaching a beginner what the hell a venv is is a nightmare

I am fortunate enough to not know why you're wrong (having only ever used Python for my own personal projects); but I've seen enough people claim that (in a corporate setting) you are, to be confident that they're right.

Unless you want reproducible installations on multiple platforms.

I don't mind venv and pip — they may be inconvenient but at least they work.

The requirements.txt however is essentially broken for any project that changes in time. If you don't freeze transitive dependencies versions your builds break with no prior notice. If you do, transitive dependencies stay there forever even when not needed anymore.

My philsophy is simple. If the program is intended to be distributable, just use Go. If it does not require port stuff, use docker. If you have an IT team or someone to hand you a computer with OS and Python version installed that everyone else in the org uses, use venv.

If you have to work with ports, you have to distribute programs, or your libraries depend on C or OS stuff, then start consulting where you do not have to manage the codebase or have no committment to it after getting paid.

It's really insane. And, as a user, frustrating that there is still no standard after so many years.

Indeed. Docker solved distributing and running python programs like 10 years ago. You can even run CUDA and pytorch in docker nowadays. And the usual answers you see on HN every time someone brings up "just use docker" on those threads, is "but I don't wanna """learn""" docker". Takes 10 min to get a python container running with 0 experience in Docker.

Yes, however unlike Poetry et al, this one is actually good!

Have you tried conda? How is it compared to Rye?

I haven't had an idea of Rye yet, but conda can do "manage venvs, python versions and my project's dependencies" fine.

I agree. Rye is working great for me. I wanted a Python cargo, and Rye has achieved that from my perspective so far.

It runs every time you build a docker image or build something in your CI

I actually think that Python's tooling should not be written in Python. Because if yes, you end up with at least two version of Python, one to run the tooling, one to run the project.

If you are building a production app that uses python in a containerized way, you may find yourself rebuilding the containers (and reinstalling packages) multiple times per day. For us, this was often the slowest part of rebuilds. UV has dramatically sped it up.

I'm not sure of the answer, but one thing Rust has obviously bought them is native binaries for Mac/Windows/Linux. For a project that purports to be about simplicity, it's very important to have an onboarding process that doesn't replicate the problems of the Python ecosystem.

Rye is a python only thing, and if you're using python without any external dependencies, it probably doesn't have that much benefit (although it does make having different python versions for different projects a lot cleaner if you need that).

I think that it has a huuuuge benefit for applications as well as libraries. Rye generates lock files, which are really handy for pinning down versions of something running in production so that it'll behave predictably.

Some OSes need shims to easily call scripts on the command line. Also, you can define multiple entry points that might live in one big ol file.

For that use-case, you can set up a `virtual project` using Rye. And use it just to create python envs, and sync dependencies.

Honestly, the biggest time-saver for me has been Rye automatically fetching python binaries that work everywhere, and setting up clean venvs.

- https://rye.astral.sh/guide/virtual/

My favorite ecosystem, Ruby, has an order of magnitude less issues in this regard, so you can imagine my sadness to discover my second tech love, Emacs, is just absolutely unreal in this regard. And there's just no way around it, I wish I could hack on Emacs in Ruby, but there appears to be no way around learning elisp. And elisp is... crufty, to say the least. At least the documentation is really good once you get past all the beginners issues. And there's a refreshing self-awareness there, fun seeing 'this function has been around since around Emacs 23' when looking stuff up.

Are you claiming rye came up with the first mature alternative? Tons of people have been using solutions like Poetry for years. Don't discredit that.

Say what you want about npm/node, but it's super easy to bootstrap a project and to understand what dependencies are required.

Huh? JavaScript is legendary for having a near-infinite amount of ever-changing frameworks and "build" systems, with various overlapping abilities.

I think it's just in Python's DNA. I dunno, even going back to the Python 2/3 debacle, it's just one language and ecosystem I've never felt "has it together". Despite being a language for beginners, I've never felt it especially easy to use.

You can virus scan a binary before running it

I remember that you can detect the "curl | bash" server side and serve a different script than what the user would get by downloading using other methods[1]. But yeah, the binary itself already has enough attack surface.

[1] https://news.ycombinator.com/item?id=34145799

It looks really interesting but it is hard to really invest in yet another ecosystem that tells you to curl and pipe into bash and then tells you to eval arbitrary command output.

Thanks for the link. Is it faster than conda?

Funny, I realized within the first few minutes that I’m not interested in Rye, but I couldn’t stop watching because I loved the mechanical keystroke sounds so much. I don’t use a mechanical keyboard for various reasons (I tried), but those sounds…

(Not saying having them in the video for everyone is a good thing—I respect the fact that some people can find them annoying)

I really don't think this is "aggressive". You seem to be saying "I don't like the keyboard sound", which is fine, but overcharacterising isn't necessary. And writing a two paragraph comment about how easy it is to fit in with your preference appears more deliberately aggressive than anything the video does.

Is this a common opinion? After watching the video I couldn't imagine why anyone would be so bothered by the keyboard sounds in it, which is maybe just a little bit too loud? And way less distracting than the bad sound quality of the microphone used (which again, is pretty easy to get over).

Cross-platform applications that rely on binary libraries have always been the beginning of my Python dependency misery. It usually works fine, but when it doesn't, it's miserable.

Try making a portable package that pulls pytorch with CUDA (reliably) with that set up. I don't even think it's possible as you can't add another package index when using python build.

Keep reading the python tutorial you're working through and you'll eventually see they import 3rd party libraries. That's when things get difficult.

- Open piano

- Push keys in the correct order

If you forget what order to push the keys, it's literally written on the sheet in code easy enough for young children to decipher. I don't understand the kerfuffle.

I’ve used both Hatch and Rye extensively. Hatch is great, but currently lacks monorepo support (it is expected to be added by the Autumn). I do like the way that Rye bootstraps itself, the dependency resolver does seem to be much faster, I like that you get updated pip lockfiles on every sync, and it already has opinionated monorepo support. Getting a new developer spun up is as easy as “rye sync”. But I agree that the fact that Hatch is under the auspices of the PyPA is persuasive indeed.

Rye for some reason requires that you put dev dependencies under a non-standard key in pyproject.toml, but other than that, it’s pretty trivial to swap between Hatch and Rye as you like (which was indeed one of the primary goals of PEP621 afaiu). I for one will certainly be checking in on Hatch again when monorepo support is added.

Hatch is very featureful and achieves a lot, but for me, a lot of the defaults and how it's configured is entirely backwards from what I expect. I've concluded it is not for me.

My concrete suggestion is that Hatch would be better for me if it was developed by a team and not a single person. Then its core ideas would be judged by multiple people from different angles. As it is now, it's written to the taste and philosophy of a single author.

That's what I'm worried about. What if we start using Rye, bake it into our projects and then Astral goes "Sike! Now we will collect all information available to the Rye binary and require 1$ for each build"

Anaconda competitor? Many companies in this space start out by releasing new OSS tools and then turn into consultancy sweatshops.

Rye-powered Python deployment platform a la Vercel?

Obligatory indeed. No need to water it down - both are still highly relevant for python

#1987 is actually outdated and way too small. Nowadays it should include build/install/pip/flit/rye/setup/setuptools/setuptools-deprecated/poetry/[insert 50 more build tools].

it uses pyproject.toml which is what Poetry uses and seems to be the standard people are moving to

Yes, PEP 517 is standard now. But, that's not actually a build system and depending on how you configure it, the actual contents can differ wildly. While build-system.build-backend is standard, how one actually configures that backend in tools.<your-build-tool> are not standard at all.

It's more proper to think of it as a standard meta-format for build system configuration.

There is effectively no real portability between build/project tools except in the most absolutely trivial cases.

Well... why are you using Docker?

Do you have problems with Poetry? If not, don't switch

Rye installs as a shell script you pipe to bash

curl -sSf https://rye.astral.sh/get | bash

(I believe this downloads the correct binary for you. Rye is a single binary built in rust but it will also download a version of Python to use for some operations) or you can download a zipped tar file from https://rye.astral.sh/guide/installation/#installing-rye

https://rye.astral.sh/guide/installation/#add-shims-to-path Oh-My-Zsh:

mkdir $ZSH_CUSTOM/plugins/rye rye self completion -s zsh > $ZSH_CUSTOM/plugins/rye/_rye

Then make sure rye plugin is enabled in ~/.zshrc

I think the ZSH_CUSTOM would be your ohmyzsh folder

I am starting to wonder if I’m the only one that just types “brew install NameOfThatThingTheyWantMeToPipeInFromCurl” into my terminal as a reflex.

It’s already available: https://formulae.brew.sh/formula/rye#default

Counterpoint: directly from the article, a relevant quote: "[rye] has matured to the point where it now does most if not all of what poetry does only faster"

So perhaps rye is all we need. I've been happy with pipenv before (letting me keep a list of dependencies and their versions in a file separate from the requirements.txt - so I can upgrade core dependencies rather than sub-dependencies). But I'll give rye a try next time I touch Python packages.

and most notably Rust has Cargo – quite possibly the most widely loved package manager tool in existence.

I thought Rubygems was "the most widely loved package manager" given how often it's been used as inspiration for other languages' package management systems... but maybe I'm just getting old :)

I don't disagree, but the trouble is, which do you axe?

Everyone agrees there's too many, but nobody yet agrees what to converge on.

It's probably made worse by the fact that package management isn't considered part of the python core team's remit.

There's a github issue on the Rye page for "Should Rye exist?" for exactly that reason, and Armin (the original developer) seems pretty consious of the package manager clutter. But in all honesty, it solves a real problen that exists (that other package managers don't solve) so I can't see any reason Rye should do anything differently just because there are lots of package managers that currently exist.

Rye isn't aiming to replace apt. Rye is just aiming to do the same as Go, having one tool to manage dependencies. In current ecosystem there are many many different tools tho manage Python project's dependencies unfortunately.

It's kind of ironic that there isn't a decent business model for folks who build tools that uplift the productivity of MILLIONS of devs.

I'll just say that even though Linux is FOSS, RedHat still became a multi-million dollar company.

Likewise Travis Oliphant was able to build things like Anaconda and Quantsight to monetize some of these essential open source tools.

With wide enough adoption, almost anything can be monetized at some level.

It already is: https://formulae.brew.sh/formula/rye#default

I use it with vscode and I've not had any problems. vscode just sees the environment. Rye puts things in standard places (PEP places) and uses common tools under the hood, so I think that it should generally just work with an editor that understands python environments.

You should use pyproject.toml to specify loose constraints instead.

This would have been not totally insane pre containers and using VirtualBox and just reimaging.

no we won't

If you're interested in the historical rationale for Rye I suggest reading the original README from when it was first released: https://github.com/astral-sh/rye/blob/8e430b0519fe2a125837e1...

And Armin's later post about the vision for the project a year later: https://lucumr.pocoo.org/2024/2/4/rye-a-vision/

It looks like Armin Ronacher answered this question in the issues when he was maintainer (https://github.com/astral-sh/rye/issues/316#issuecomment-158...):

While I understand the desire for XDG, I personally hate it because it encourages common files to be in different locations on different operating system, driving up the complexity for everybody. Once you go down that path, every single documentation has to explain different paths for different platforms and configurations, how different configs can affect where stuff is. It gets gnarly quickly.

I do not believe the benefits outweigh the downsides and XDG is generally not enough of a standard that it prevents people from putting stuff into the home folder. If you want to relocate `rye` there, you can already do by exporting `RYE_HOME`.