Here, I found it fun to write a small bot to play automatically. Just paste this JS code in the dev console. It reached 9000 points on my first attempt to let it run. Feel free to tweak the code to make pac-man survive longer :-)
function bot() {
/* direction of the enemy: to our right (1) or to our left (-1) */
dir = (enemy.x > player.x) ? 1 : -1;
/* if pac-man... */
if (
/* ...has no powerup or powerup expires in less than 10 "ticks" */ powerTicks < 10 &&
/* ...is headed toward enemy */ player.vx == dir &&
/* ...is too close to enemy */ abs(player.x - enemy.x) < 25 &&
/* and if enemy's state is not "eyes flying back" */ enemy.eyeVx == 0
) {
// "ArrowUp" or any arrow key reverses the direction of pac-man
document.dispatchEvent(new KeyboardEvent('keydown', {code: 'ArrowUp'}));
document.dispatchEvent(new KeyboardEvent('keyup', {code: 'ArrowUp'}));
}
}
setInterval(bot, 100);
The strategy is ultra simple. Every 100 ms it evaluates the situation and chooses to move away from the enemy (ghost) if it's too close to it, and has no powerup (or if the powerup is expiring very soon).The corner cases where pac-man dies is the game difficulty progressively increases (the ghost becomes faster) until you eat it, so sometimes some pellets are left in the middle and pac-man doesn't have enough time to eat them until the ghost reaches it. The ghost will progressively get faster and faster and death is guaranteed. You could improve the code by tempting the ghost to get close to one edge, then cross over to the other edge and quickly eat the middle pellets.
Also, as soon as new pellets are added, one should prioritize eating the middle pellets.
Also, one could add code to detect the powerup pellets, and chose to NOT move away from the ghost if it calculates it can eat the powerup pellet before the ghost reaches pac-man.
This is super awesome!
The trouble I ran into was after eating the ghost, the spawn point of the new ghost was too close to my current position and simply sped over me.
With this and all the considerations you've noted above, I'd probably throw an ML solution at it - imagine it takes less coding time.
deep reinforcement learning guy here. Youre looking at coding up double dqn, and probably training it for about 5 to 30 minutes. Very likely it would not outperform a simple near perfect algorithm, and would fail in some weird ways that would annoy you a few tens of thousands of times. Youd have issues like it refusing to go left, ever, after dying. So youd need to know about epsilon then epsilon decay etc.
Coding that from nothing (and understanding it) could take days. Weeks to months if you dont know ml stuff.
In this particular case I think basic Q Tabular Learning could play optimally, no neural net required.
I could be wrong though, throw a full conv at it without temporal difference and see what happens! (probably a kinda okayish score)
You could probably throw a small fully connected net at it, and 'train' it via genetic algorithms: ie have a population of nets, let them each run a few games, and pick a few top scorers to seed the next generation with some mutation and cross-over.
This random exploration is less efficient than back propagation, but at least you don't have to muck around with temporal differences. (And it would be hard to figure out how to tweak the weights for backpropagation in this multi-step game; without falling back to reinforcement learning.)
The problem seems simple enough that evolution via mutation and artificial selection has a decent chance to get good in a reasonable amount of computing time.
NEAT? An added bonus of the evolutionary neural search is that you can still explore the solution space with small networks, which isnt really true of backpropogation without evolution. You run out of states.
With the right small neural network architecture you could probably just repeat random weight initialization until you find one that works. This is known to work for some of the Atari games. The trick is, of course, to engineer a good input encoding and to find the right model complexity and architecture in the first place.
The script took, at most, 5 hours to write (inclduing the time for this poster to play the game, understand the game, reverse engineer the game, and implement a bot). What type of "ML solution" to the same problem would take less than 5 hours?
For the record, it took me about 30 minutes of playing, reverse-engineering, and writing the bot. So, indeed, no need for an ML solution :)
You are right about the time, but 1D Pac-Man might still be a good fun toy problem to hone your ML skills on.
This sounds like a simple enough problem that one could work out a provably optimal algorithm in less time than it takes to get even a non-optimal ML solution working.
But both approaches would be an interesting exercise in their respective domain!
Come to think of it, it might even be an interesting toy problem for a SAT solver! Overkill in the same way as an ML approach, of course.
something similar
when tired just call end()probably better to manipulate the score directly but its not fun that way :)
Not sure why you obfuscated it but here's the code for anyone curious:
Why would anyone even consider pasting obfuscated JS into their console, on HN of all places, is beyond me.
Theoretically, what's the worst thing that could happen - given it's a pacman game's console and not your online bank's?
I recall manually entered commmands have access to some APIs that are not normally accessible. Maybe timing related? So in the theoretical extreme, a timing attack to access something in your entire system memory and upload it via HTTP... while you watch the game play :)
Do they? It's easy enough to 'give' any API accessible to the console to the page by setting a variable from the page to point to a console-only accessible function.
Given there aren't many sites that say "just open the console and paste this command to win the big prize", I suspect that any console-only API's aren't very powerful if they exist at all.
No different than clicking on the links, right?
(Admittedly I typically browse with JS disabled...)
This seems to make it possible for Pac-man to stick with the ghosts while still consuming pellets, so maybe still evading the rules of the game :)
How did you (or how does one) discover that the game state consists of the variables enemy.{x,vx,eyeVx}, player.{x,vx}, powerTicks? (And are there others?)
I reverse-engineered it. This whole exercise took me about 30 minutes while half coding and half watching a TV show. Since the JS code was neither minified nor obfuscated, it was easy. Here are my exact steps:
In Chrome, right-click on the page -> View Source. Go to the "Sources" tab. When you open the tree on the left you find a file, that was dynamically loaded (so you wouldn't have found it by just looking at the HTML source of the main page), https://abagames.github.io/crisp-game-lib-11-games/pakupaku/... that seems to contain the game's code. And indeed it's less than 200 lines and implements the core game logic.
There is a main function, update(), containing a conditional "if (!ticks)" that seems to do initialization. "ticks" is probably zero at the start of the game. The code initializes multiple variables, including "player" and "enemy":
Then with a bunch of console.log() statements added to the update() function (just edit the code and type Ctrl-S to save it, Chrome allows this sort of in-place code editing), I got a sense that player.x and enemy.x was the x position, with the scale going from 0 (left edge) to 100 (right edge). Then with more console.log() statements I found enemy.eyeVx was normally 0 when the enemy is a ghost or 1 or -1 if the ghost has been eaten and its eyes are flying respectively right or left.Some of the code is less readable, for example:
I figured by literally typing "play(...)" in the dev console that these functions play a sound. Therefore based on the sound names "powerUp" and "explosion" I realized this code block is called when pac-man collides with a ghost. And either it has a power-up ("powerTicks > 0") or not, in which case it dies.At this point I had the meaning of "player", "enemy" and "powerTicks" and that's all I needed to write the bot.
The source code for the page doesn't have any reference to a game-specific source code, so it must have been loaded dynamically and can be found from devtools [1]. You can then read the actual code and find that every state is in the global scope, so they are directly accessible without any trick. It hasn't been minimized so all variable names make sense and you can even guess their purposes without actually checking.
[1] https://abagames.github.io/crisp-game-lib-11-games/pakupaku/...
Putting comments before the conditionals made my brain parser malfunction for a moment.
Same here -- but I kind of like it. I might adopt that.
I'm kind of a 'why not what' kind of guy when it comes to comments. For example "powerTicks < 10" took much less time for my brain to parse (and I get extra information, I see there is a variable called powerTicks then "/* ...has no powerup or powerup expires in less than 10 "ticks" */" did. Then my eyes had to scan for the closing comment to get to the important part.
Something different, but not for me personally.
I simply changed the distance to be dynamic based on closeness to the edge and it's getting into the many 100's of thousands. This simple change also seemed to fix the edge case of the respawn hitting the player is solved since the player never gets into that magic position (the enemy has to be in a very specific position for the eyes to reach the end just as the player does).
Of course ! Nice.
This is really cool. My bot score stayed low because there was a single dot remaining near the middle which Pac-Man couldn't reach. They went back and forth for a long time until the ghost became fast enough to get him. Still fun to watch.
This is great. That's what Alan Kay envisioned when doing Smalltalk 45 years ago: opening up a computer program while it's running and tinkering with its code directly.
And it's quite funny that JS was designed after Self/Smalltalk well before anyone cared about interactive tooling, towards which we evolved independently.
Alan Kay truly is a genius.
This is true AI! Nice work!