return to table of content

Show HN: Wat – Deep inspection of Python objects

wantsanagent
13 replies
22h46m

Re: Insta-load

Please don't do this. Exec'ing arbitrary, obfuscated code you just copied from the internet directly in a running environment is such a bad idea it shouldn't even be suggested.

At the very lease please put huge warnings around this section to let people know that it is a horrendously bad idea and they do it at their own peril.

timdorr
7 replies
21h45m

I'm not a Python dev. Why would they do this? This is giving vibes of malware embedded into npm packages.

skeledrew
4 replies
21h8m

Convenient way to quickly add extra debugging capability without rerunning. It isn't much different from the many `curl example.com/install.sh |bash` you see around. It's up to the user to check things out before running.

etbebl
1 replies
18h13m

Can't you just install it in your environment in another terminal and then import?

skeledrew
0 replies
16h20m

Sure, but it's a few extra steps. It's been proven(tm) that many people prefer a single, simple thing they can just copy, paste and run, so they can get back to their main concern.

And some may not want it actually installed for whatever reason. Such as when there's no proper separation between dev and prod deps. (I'm mostly just guessing at this point though...)

SOLAR_FIELDS
1 replies
18h33m

Yeah, the piping to bash is a tried and true method for various installers. People make a fuss about it, but we don’t see people getting owned that way often. I think with bash installers though it’s pretty trivial to just visit the link and read through the 100 lines of bash. So anything installed this way should be as simple as visiting the link and reading a short amount of code imo

jampekka
0 replies
1h59m

The same people seem to be more OK with installing random .debs and .rpms which can of course arbitrarily run any code with root.

jampekka
0 replies
1h56m

Pip install allows arbitrary code execution.

CornCobs
0 replies
6h48m

Not everywhere that python is run has access to pip. Sshing into some locked down remote machine and needing to debug some script is a use case that comes to mind.

oreilles
1 replies
21h16m

How is it any different than installing the package via pip ? Not only most people won't check the source before running the code, but there is also no way to be sure that the code shipped by pip is the one you read on GitHub...

__MatrixMan__
0 replies
5h49m

gp has a leg to stand on only if they regularly audit the contents of their site packages. Otherwise you're totally right.

tantalor
0 replies
22h44m

This is so bad it should be nsfw-blurred and you have to click 2 buttons to even see it.

slt2021
0 replies
19h8m

the gzipped payload is static, you are free to inspect the payload before eval()

erinaceousjones
0 replies
7h35m

Yes, the big warning disclaimer in that part of the docs is definitely required here.

That idea of exec'ing arbitrary obfuscated (compressed) shell code that's easy to copy-paste into a python shell is very helpful, mind.

I've had to debug issues in production with silently hanging I/O threads and my only access is via `kubectl exec`.

This wat tool and that "insta-load" idea pairs nicely with pyrasite for that very useful !!!DO NOT DO THIS, YOUR CONTAINERS SHOULD BE LOCKED DOWN SO YOU CANNOT DO THIS!!! step of copying gdb and the pyrasite library into a running container when all you have is a python shell to play with.

(This almost feels like an exploit, running shellcode after getting RCE :))

https://pypi.org/project/pyrasite/

enriquto
11 replies
20h23m

from wat import wat

Given the cool nature of this project, I'm surprised they don't offer simply "import wat" with identical usage syntax. Thus inviting curious users to wat/wat in order to discover the trick...

igrek51
10 replies
20h1m

"import wat" would be great, but Python has some restrictions about modules not being callable. That's why I ended up with longer `from wat import wat`. Not sure but maybe this would be more convenient `import wat; wat.wat / object`

Rhapso
2 replies
19h19m

Python has code execution at import time. Just grab the global context, and overwrite the module with a callable.

skitter
1 replies
18h21m

Afaik the import sets the module in the importing module's context only after the code in the imported module is run.

Edit: Oh, you don't mean in the importing module's globals, you mean `sys.modules`. Yeah that works!

Rhapso
0 replies
17h48m

actually, no you were right about what I meant but i'll claim credit for helping you realize there was a better way XD

scrollaway
1 replies
19h10m

Check out the module “q”! It’s callable, its author talked about how great it would be to have a module level __call__ because the way it was made callable is super wonky.

enriquto
0 replies
9h40m

the way it was made callable is super wonky.

why do you say it's wonky? it's just

    sys.modules[__name__] = wat
According to the official python documentation [0] of sys.modules:

This is a dictionary that maps module names to modules which have already been loaded. This can be manipulated to force reloading of modules and other tricks.

Thus, the modules dictionary is being used as intended, and it has the desired effect. Nothing wonky about that.

Of course, a __call__ method would be better because you could still keep the other functions inside the module. But for a single-function import like "wat" it seems quite natural and sane.

[0] https://docs.python.org/fr/3/library/sys.html

igrek51
1 replies
8h55m

Thank you all, I learned from you that it's possible to have a callable module. Although `sys.modules[__name__] = wat` looks like a black magic and I'm a bit afraid of locking out to other importable things in this package, I think I'll go for it.

notpushkin
0 replies
6h55m

It is kinda black magic, and I think it's not something people would expect. You can import other things just fine though:

    import sys

    def wat(): print("wat")
    def bar(): print("bar")

    wat.wat = wat
    wat.bar = bar

    sys.modules[__name__] = wat
...

    >>> import wat
    >>> wat()
    wat
    >>> from wat import bar
    >>> bar()
    bar
Edit: I think a better way would be to instead add __call__ to the current module somehow though.

skeledrew
0 replies
11h15m

See snoop[0], which does just this. It also has an install function that registers it in builtins, making it universally available.

[0] https://pypi.org/project/snoop/

alkh
6 replies
22h51m

It important to remind everyone that you can already do something similar via the build-in function help(). For example, if I run help({1}) I get the documentation for a set, running help(set.add) gives the documentation for the add() method, etc. You can even preview objects that you haven't imported by using strings(ex. running help('numpy.ndarray') will correctly give you the documentation, provided numpy is installed in your current python environment). It's pretty helpful and doesn't require installing anything extra

igrek51
1 replies
20h17m

You're completely right. `dir()` does the same in terms of functionality. In fact, my tool makes use of `dir` under the hood. I just wanted to make it more readable, and to combine `dir`, `type`, `repr`, `len`, `inspect`, etc. into one easily accessible place.

fuzztester
0 replies
13h29m

even just

  print(name.__doc__)
works for many kinds of names in Python.

orbisvicis
0 replies
17h49m

And vars().

guruparan18
0 replies
21h51m

Also most advanced IDE's do this better and lot more to troubleshoot/debug.

fuzztester
0 replies
22h21m

and the dir() built-in Python function.

agumonkey
0 replies
12h20m

help says too much, usually you'd like a quick overview slightly formatted and well classified, which is what wat seems to afford here

Retr0id
6 replies
22h21m

I'm normally a big fan of python operator overload hacks, but why are we doing `wat / foo` instead of just `wat(foo)` ?

jawns
3 replies
17h30m

I get the reasoning behind it, but the choice of the division operator is still a little weird.

I believe it should be equally possible to overload the bitwise OR operator (|), which, for people used to piping in Unix, is probably more intuitive.

Tomte
2 replies
13h6m

Wrong direction: you‘d „pipe“ wat through the object. Although TiddlyWiki users might feel right at home.

igrek51
0 replies
7h41m

Right, that would be a wrong pipe direction. Fun fact: It's already possible to do "wat | foo" or "wat << foo" (if you're more familiar with C++ iostream), it has the same effect as "wat / foo"

auscompgeek
0 replies
8h47m

It could override the reverse bitwise or magic method `__ror__`.

contravariant
0 replies
20h13m

Probably so you can run a big complicated formula in the REPL and easily prepend `wat /` to it to inspect what's going on.

Same reason jupyter notebooks allow you to prepend '?' or '??' to inspect a variable (though not expressions, in that respect this syntax is better)

JulianChastain
0 replies
21h47m

`wat(foo)` also works. The author says he implmented the `wat / foo` syntax to allow you to more quickly use wat by avoiding parentheses

Aeveus
6 replies
23h59m

Looks like a great tool. Will start playing with it whenever I have to dive into an existing project again.

One thing I'd like to note, though, is that most engineers (at least around and including myself) would be triggered by the "Insta-load" example of executing base64 encoded (and thus obfuscated) code.

fuzztester
2 replies
23h24m

just so that it is clear for everyone, including newbies here, does the insta-load code snippet work like this:

the author has encoded the wat module's source code into base64. the snippet shown decodes that back into python code, and then executes it on the fly, thus having the same effect as importing the wat module in the normal python way?

igrek51
1 replies
20h32m

Yes, that has the same effect as installing the package. So if you feel uncomfortable, you can either install the pip package (and of course review the installed code) or review the decoded string before executing it. It's not that obfuscated anyways, it's still quite readable.

fuzztester
0 replies
18h15m

got it, thanks.

tacoooooooo
0 replies
23h45m

not hard to see it

with open('wat_source.py', 'w') as f: f.write(zlib.decompress(base64.b64decode(code.encode())).decode())

kajecounterhack
0 replies
23h50m

I was just about to post that it freaked me out a little, but wasn't sure if I was going to be the only one.

Spivak
0 replies
23h48m

The insta-load is fine. You're supposed to save the snippet, decode and uncompress it, look at it to see what it does and that it's completely self-contained uncontroversial Python code and copy-paste your version that you bookmarked. It's super short.

languagehacker
5 replies
1d

This is interesting. I'm wondering what compelled the author to use the division magic method for arguments instead of the more intuitive and commonly used approach to passing parameters.

igrek51
1 replies
20h38m

It's the parentheses that drove me crazy. As people already noted, it's for faster typing, at the cost of the division magic as you noted. If it's more familiar to you, it works with `wat(object)` syntax as well.

wnolens
0 replies
14h39m

I love it

pvg
0 replies
22h59m

The README suggests for easier typing in interactive use.

lakshayg
0 replies
1d

Interesting choice indeed. It looks like it supports the traditional approach as well

wat object can quickly inspect things by using the division operator (to avoid typing parentheses). A short, no-parentheses syntax wat / object is equivalent to wat(object).
enragedcacti
0 replies
1d

I think it makes a lot of sense in the context of a live interpreter. Wrapping and object in help() or dir() is an annoying set of movements vs. just pressing 5 keys with no modifiers (Home, w,a,t,/). It makes its feel much more like a magic command without actually needing it to be magic. The dot modifiers are also convenient compared to passing kwargs imo.

benrutter
4 replies
22h2m

Ah this looks fun! I use "dir" all the time with python, and find it more useful than official documentation in some cases where documentation is not great.

Surprised there isn't more innovation and new tools like this around python's interactive shell given it's one of the real strong points the language has.

pjot
3 replies
19h54m

There’s the help() function as well! Super helpful!

notpushkin
2 replies
7h6m

And if you use IPython, you can use the ? shorthand, too:

    In [1]: int?
    Init signature: int(self, /, *args, **kwargs)
    Docstring:
    int([x]) -> integer
    int(x, base=10) -> integer

    Convert a number or string to an integer, or return 0 if no arguments
    are given.  If x is a number, return x.__int__().  For floating point
    numbers, this truncates towards zero.

    [...]

shabble
0 replies
4h31m

the double questionmark `foo??` is sometimes also useful, since it'll show full source for the function/method/thing if you do it to a function.

hughesjj
0 replies
1h29m

Favorite way to develop python, so far:

1. Write the program skeleton

2. Put an import pdb; pdb.set_trace() whereever I need to fill out next

3. Once I hit the breakpoint, type 'interact'

4. Start coding in the debugger.

4a. For small functions write in a scratch pad, test against another python console, then paste into the current debugger.

4b. For network calls, save the inputs and outputs. Toss a decorator on some functions to log the inputs and outputs to files, and cool we already have some proto test data.

4c. For documentation, call help() on the variable or method or module.

skeledrew
2 replies
21h41m

Waaat :D. This is so nice. I used to use python-ls[0] for similar, but something about it that I can't recall broke for me and it's no longer maintained. Adding to my debugging arsenal which primarily consists of snoop[1] and pdbpp. Only thing I'd like for wat now is maybe an ipy widget for even easier object exploration in Jupyter.

I'm also really appreciating the base64 exec hack. All my years in Python I never thought of or came across it until now. I'll totally be using it for some things :).

[0] https://github.com/gabrielcnr/python-ls [1] https://pypi.org/project/snoop/

nyoomboom
0 replies
47m

Seems like if you add more "a"s to W[a..]t you slide a scale from disgusted to awed to comical.

revskill
2 replies
10h32m

Is there similar tool for typescript/javascript ?

CornCobs
1 replies
6h51m

Isn’t the browser inspector more than sufficient? You get the interactive object inspector in the console, live expressions, log points etc.

For node you it’s possible to hook it up with the browser inspector as well

jampekka
0 replies
2h4m

Don't know why you're downvoted, but browser devtools have the best object inspection around. I miss them all the time when coding Python.

Hooking devtools with Node is possible but could be a lot nicer.

ilyagr
1 replies
19h54m

I wonder if there's something like this for Lua. It doesn't have Python's built-in conveniences for introspection like `help()`.

nython
0 replies
8h21m

There were various internal Quality of Life modules during my time at amzn, but nothing public. I suspect that given enough time working with Lua, each team will create an ad hoc, informally-specified, bug-ridden, slow implementation of half of python.

heyts
1 replies
2h17m

This looks super useful, but I was wondering if I'm the only one bothered by this recent trend of overloading completely unrelated operators (here the `/` operator) in the name of legibility.

wrboyce
0 replies
1h44m

I do agree overloading / is an odd choice in this instance, but it’s a shame “is” can’t be overloaded! Realistically wat(foo) would be fine though.

blueboo
1 replies
23h23m

rich.inspect has been my go-to for this. Comes pre-installed in a vanilla Colab kernel too

098799
1 replies
8h44m

One might suggest adding:

  try:
      from wat import wat
  except ImportError:
      pass
to your $PYTHONSTARTUP file to avoid the cumbersome import.

BiteCode_dev
0 replies
7h18m

You can even add the inline importer in base64 that is pretty neat.

But I eventually printed that output, and put it in a dir I point my PYTHONPATH to so that I will always have it available.

Let's see if it sticks.

ur-whale
0 replies
10h46m

Given the natural built-in introspection capabilities of the language, I never understood why something like that wasn't also a built-in part Python.

I mean the number of times I've bumped on json not being able to dump the content of an object is just infuriating given how flexible Python is.

I'm definitely giving this a whirl.

timhh
0 replies
23h20m

Maybe it's just me but I just use a proper debugger. Debugpy and VSCode work fantastically.

In a previous company I set things up so that when there's an uncaught exception it will automatically start a VSCode debug session and connect to it.

Here's the extension: https://github.com/Timmmm/autodebug/

Unfortunately the Python part of that is not open source but it was only a few lines of code - should be easy to recreate. That repo does contain a C library that does a similar thing.

You might just say "why not just run your program directly with the debugger?" and yeah that is better when you can do it, but I'm working with complicated silicon verification flows that are often several layers of Python and Make, followed by a simulator (e.g. Questa or VCS) that itself loads the library you want to debug. Very difficult to make debugging work through all those layers, but it's quite easy to write some code at the bottom of the stack that says to a debugger "I'm here, debug me!".

maxmcd
0 replies
21h7m

Is there something similar for Javascript?

lofaszvanitt
0 replies
12h44m

Python's biggest weakness is the lack of a function that shows you how exactly a variable looks like. the var_dump (php, ha!) works like wonders. The best library ever created for Python ;). and now wat is also looking to be a close contender for best helping hand.

fuzztester
0 replies
22h15m

the author is using the Python inspect module of the stdlib under the hood to provide the functionality, of course with a lot of value-add.

see inspection.py in the wat module.

it has this on line 2:

import inspect as std_inspect

frays
0 replies
14h41m

dir(), help() and now adding wat to my toolbox to help debug!

fermigier
0 replies
11h16m

These kinds of tools are useful.

Twenty years ago I wrote an object introspector for Zope ;)

Nowadays, I'm using devtools daily, and icecream and q occasionally. I'll give wat a try.

chis
0 replies
22h52m

This is incredible

blackbear_
0 replies
1d

Looks great! And it would make for a great addition to Jupyter lab. I actually can't believe there is still no decent variable explorer bundled in

AcerbicZero
0 replies
1d

Wow, this is the kind of tool that would have been a game changer when I was learning python; being able to see what is happening under the covers is part of the critical path for me to learn a language, and native python debugging is underwhelming, at best.

Instead I just installed pry and became a rabid ruby fanboy, but this might get me to give it another go.