FYI - If this is a true reproduction of Shazam, it’s under patent by Apple through at least March 2025[1].
Recently found Shazam is less accurate - somehow soundhound is giving me better results. On Shazam I'm getting a lot of results from Asian musical traditions which is great, if it wasn't the wrong song. Maybe they need to improve the algo if they've increased the range of music they will select from? Seems now there's a lot more hash table collision[1].
[1] https://github.com/cgzirim/not-shazam?tab=readme-ov-file#resources--card_file_box
Soundhound has always been better than Shazam. It can even pick up people singing and extremely quiet songs
In my sample of one song, I have to disagree. I played Watermelon by Mezerg, which is admittedly not very popular, and Soundhound couldn't get it with two tries, but Shazam picked it up in less than two seconds.
I work at SoundHound. If it didn't get it in two tries it's likely we didn't have that song in the database. Both Shazam and SH have knowledge gaps.
I've championed Soundhound but it has literally stopped working (finding any tune) on my iphone. I've reinstalled, still nothing. It does not appear to 'hear' anything.
Is it possible the microphone permission was turned off?
It's a fair question. Out of an abundance of caution I just toggled it. Definitely on. In fact, in doing so I have today noticed some correlation between swapping between Shazam and Soundhound and mic behaviour. That is to say, Shazam appears to hog the mic and not relinquish it to Soundhound upon request. If all apps are quit, then I launch Soundhound, I can get a response. But if I happen to then use Shazam, which always works, Soundhound fails until I do the double quit.
Shazam seems to have a way bigger database than Soundhound.
For a while, it seems like Soundhound was about to shut down. It wouldn't match anything released in the last 12-18 months, but that seems to be fixed.
I compared Shazam's, SoundHound's, and BeatFind's recognition library in August 2021. (And tried MusixMatch but it crashed on startup apparently.) Don't think I published it anywhere, these are my raw notes I found among saved chat messages. The format is starting to make more sense now that I'm putting it into a wider window than a chat screen, so I can recommend using a wide browser (94 characters per line should do it). Eyeing the song choices, it looks like I tried to find different genres and artist types (ccMixer/youtube celebrities, to indie, to established) but a larger sample size would obviously have been even better. Still, I hope it's one step up from adding another random opinion!
The conclusion appears to be that BeatFind and Shazam know the most songs, but are also somewhat complementary and all of the services had at least one song they uniquely recognised.
---
Fun facts:
* Night Driver (W) said "1 Shazams". I think I was the first person to ever Shazam that. Some of the most obscure things had hundreds, often thousands of shazams!
* You know where they are taking the hobbits but none of the services do!
========
- ABC = found the song
- # = number of attempts
- f = exceptionally fast matching (when it did match, might not be first attempt)
- ~ = knew one of the songs
BeatFind: 2B C 1E 2G 2H 1I 2Jf 2Lf 4M 2Nf 2Of 1Pf 1Rf ~S 2T 1Uf 1Vf 1W 1Xf Y Z
SoundHound: A 1B 1E 2F 1G 2H 2If 1L 2O 3P 2R 1Tf 2U
Shazam: 1Bf 1D 1E 1F 1G 1Hf 1I 2J 1L 1M 4N 1O 1Pf 1Rf 1Tf 1Uf 1V 2W 1X Y Z
MusixMatch: crashes on startup, presumably it realizes it won't be able to show me ads
missingno
Shazam: A C K Q S
SoundHound: C D F J K M/N Q S V W X Y Z
BeatFind: A D F K Q
non-universal finds (repeated letter = unique = counts double; slash means same artist so should be counted as one)
Shazam: DD F J M/N V X Y Z
SoundHound: AA F
BeatFind: CC J M/N S V X Y Z
A: Levan Polkka Epic Orchestral Cover version
B: Pokémon red/blue soundtrack
C: Mayhem (various songs, it seems either they have all or they have none)
D: Art Now ft. Snowflake
E: Hero's Choice
F: Three Days Grace - Scared
G: Syrian - Supernova
H: The Explosion - Here I Am
I: The Von Bondies - C'mon C'mon
J: Frank Klepacki - Scouting (C&C TibSun)
K: Conspiracy - Chaos Theory (demoscene)
L: Cheshyre - Madness6 (remix) (Newgrounds ID 77998)
M: Dimrain47 - Twilight Techno
N: Dimrain47 - Cloud Control
O: DragonForce - My Spirit Will Go On
P: Yuki Kajiura - The First Town (SAO)
Q: THEY'RE TAKING THE HOBBITS TO ISENGARD! THE HOBBITS- THE HOBBITS- TO ISENGARD! TO ISENGARD!
R: Faithless - Insomnia
S: Age of Empires 1 soundtrack
T: Moulin Rouge - El Tango De Roxanne
U: Van Canto - Master of Puppets
V: Slack Bird - Jouni
W: Floppytronic - Night Driver
X: EgoSalad / Kitboga - Breathe in
Y: Floppy Drive music: top 4 hits on yt: sweet dreams, imperial march, ghostbusters, beat it. Only ghostbusters was known to any
Z: Obsidian Shell - Orphanage
---Note that what I did not test/review introducing noise (like people talking through it) or filtering (like when you hear the music through a wall)
This is useless unless you have all the songs on earth
Algorithm don't matter, only data matters
That's like saying the Hutter prize is useless for anyone who doesn't want highly compressed versions of Wikipedia. The underlying code or algorithm is still interesting to study, use, and remix.
AFAIK the underlying algorithm had been implemented multiple times with accompanying explaining blog posts and articles. So this is yet another iteration, done for fun. Nothing wrong with it, just nothing really useful/new I believe.
Although, would be curious how good you could get to isolating to a single artist. If you had say one exemplar fingerprint per artist, could an out of dataset fingerprint from their discography cluster to that artist? Obviously not for artists who transitioned musical styles.
Or is the algorithm more feature hash than a clusterable feature vector?
Isolating a single artist based on a fingerprint sounds challenging but interesting.
Using exemplar fingerprints, a representative sample of an artist's music, is a good approach, but success would require detailed fingerprints, a varied dataset, and a well-chosen algorithm.
For artists who change styles, time-series analysis can capture their evolving sound.
The solution will likely need machine learning.
The current solution doesn't use feature hashing or clusterable feature vectors. Instead, it relies on audio fingerprinting, which breaks down short audio samples into unique patterns or "fingerprints" for quick comparison with a large database of known songs.
Here we have an open-source algorithm that is useful to anyone with data. It doesn't have to be music
If you insert Spotify songs, wouldn't it make more sense to output Spotify songs too?
It would actually. But Spotify doesn't allow direct downloads so I had to find the songs on YouTube and download them from there.
Note that YouTube's ToS doesn't allow this either, be aware what you're potentially getting into by releasing a tool that rips music from there.
Uploaders can choose a video license. Since there is no download button for those videos, I've always wondered if that is a creative commons' violation and therefore Youtube should run into the same legal issue that StackOverflow currently is (ctrl+f "violat" https://meta.stackexchange.com/questions/401324/announcing-a...), namely that this terminated google's (but nobody else's) license to use the video under these creative commons terms
You're a G
All Spotify sees is bytes going to your computer. What happens to those bytes afterward is your own business.
Run it as a daemon that displays every song in a UI notification ?
You deserve reddit gold for this idea
I love spyware!
B-b-but... source code included !
This is cool, but you urgently need to change the name.
Why does OP urgently need to change the name?
Notch of Sam?
Not jus' AM?
A. SeekSound vs B. SoundSeek vs C. SoundScout
Which would be a better alternative?
Isn’t the whole point of Shazam that you don’t know the song and want to find it? If you don’t know the song, hoeven you provide a Spotify link?
this is a demo of the algorithm, not a full app / hosted service using it with a pre-populated database. The spotify link would be to fingerprint the song and add it to the database
You're right. The Spotify link is used solely to get details about the song. These details are then used to search for and download the song from YouTube. Afterward, the fingerprint for the song is created and added to the database.
The idea is that you add every Spotify song in the database, and then run your match against them.
While the project does look nice to use and modify. I'm not sure I personally would have posted it yet.
- The instructions seem not to be the best to get it up and running (e.g. "cd not-shazam" and just a few lines later "cd not-shazam/client")
- MongoDB is needed but information on how to hook it up/use it are absent (I would make the DB swapable and provide something less intrusive like sqlite)
- If replacing MongoDB is not possible, I would provide a dockerfile and a docker compose to allow easy startup and testing.
- The client npm install has 8 critical vulnerabilities, these might not actually matter but it makes me hesitant to continue testing
- You might not care about the patent or the copyright, but I would still change the name at the very least. Github itself is located in the US and will remove the project if they receives a DMCA.
- Last, this might not be as important, I would add a way to add songs from wav files. Not everything I'd want to test this with is on spotify or youtube.
I'm not saying this to discourage you or anything, I just think the project needs that little extra bit of polish. Minor things will cause people to discredit or ignore a project. If I get around to it I might make a PR for the project. I want to experiment with audio matching outside of the music space, and your project seems like it'll be the easiest to modify.
Edit: Formatting
Thank you for the time you took to provide such detailed feedback. I really appreciate your honest input. You've raised some valid points that I hadn't really considered.
I agree that the project could definitely use some polishing. I'll prioritize improving the setup instructions and look into adding a file-based DB for flexibility, as well as resolving the npm vulnerabilities. Adding support for directly fingerprinting wav files is a great idea and something I'll prioritize, too.
Regarding the project name, I understand the potential legal implications and will definitely change it. I'd appreciate any suggestions you might have.
I'm excited about the possibility of your contributions. Please, feel free to open a PR whenever you're ready.
Thanks again for your feedback!
Why not use sqlite instead of mongodb?
You post it on HN to get invaluable comments like yours. Good writeup!
Hardcoded sleeps for some reason, nice /s
https://github.com/cgzirim/not-shazam/blob/888070f3434acbc0a...
It's to go around the ban of the IP / account by Spotify and to be softer with them, you have to wait between two requests to download songs.
I also use sleep a lot in my code when interfacing with third-party services (multiprocess usually so it's not blocking things, though I'd also totally see myself using a callback pattern or so if the caller can handle those). When it's more than an ad-hoc piece of code, it generally measures how long ago the previous request finished to determine how long to sleep if the next call is made within the cool-off period. If you're not doing that... please don't interface with my server
I think you've leaked your developer key here... https://github.com/cgzirim/not-shazam/blob/main/spotify/yout...
Does this mean he could accidentally get a $1 million credit card bill from google from someone using his key without his permission? (I don’t know how it works with google.)
Oops... Thank you. I've disabled it.
I enjoyed this talk at the DAFx17 conference by Avery Wang, co-founder of Shazam. It goes a little into the theory behind the algorithm, and looks at some of the more practical issues (background noise, etc.): https://www.youtube.com/watch?v=YVTnj3OIhwI
Adding this to the watch list. Reading this paper was one of the first times I got a 'wow' moment around computing algorthms.
This is really cool. I’ve been itching to try building this exact kind of thing as part of my bucket list.
Thanks. I'm glad it inspires you! It'd be awesome to see you take it on. You can clone it and develop it further.
Haven't crawled through the repo (yet) but quick question - where does the data that is being searched over come from? Are you loading a library or searching some large library acquired from somewhere else?
The data comes from a database of fingerprints connected to the server. These fingerprints are created whenever songs are added.
Shazam's technology came in part out of CCRMA, which is a very cool and special place on Stanford Campus, with deep connections to early computer history.
I think it is very interesting that so many of the early applications of computer technology have to do with audio. John Bardeen's music box, the first commercial application of the transistor in hearing aids, the HP garage in Palo Alto was originally building audio oscillators, the iPhone evolved from the iPod, the internet was built on copper made to carry analog telephone calls, Bell Labs (ping!), the list goes on.
A friend of mine has the hypothesis that maybe human beings end up figuring out how to do kHz stuff before they go on to do MHz/GHz stuff. Not a perfect explanation but kind of attractive...
IMHO it’s because audio is „easy“ to manipulate electronically.
You can transform every audio signal into an electronic signal relatively easy - for graphics there is so much more complexity involved just in making them visible.
A speaker that translates electronic signal into sound waves is a super simple contraption at its core.
Edit:/ and audio is striking - it has a profound effect on every human (except deaf of course). If I wanted to demonstrate the power of electronics/computers I would choose audio as well.
I'd love a way to use local files instead of spotify/youtube to create the set of fingerprints that is searched.
I've added that to my to-do list and plan to implement it this weekend.
really decent and nicely done Golang! I'll pull and play with it tomorrow!
Thanks! I appreciate the compliment on my Golang; This is actually my first full-fledged project with the language, haha. Feel free to reach out if you have any issues running it.
It would be quite nice if there were a community-based way of sharing fingerprints.
I think musicbrainz supports this https://musicbrainz.org/doc/AcoustID
I’ve heard that the Google phones have a built in music recognition feature that is the best implementation of this stuff. Anyone know what their approach was? Apart from that I always have felt Soundhound was better than Shazam
Iirc there was some small algorithm, or perhaps even a piece of hardware, that should trigger when music is playing so that the phone isn't active all the time. From there, I guess they could use any old detection algorithm; for me, the magic was in this super-energy-efficient bit of the chain, though I never read up on the details (if they ever provided any)
I wonder how long until someone will simply smoosh a billion songs into a "large song model" and make all signal processing knowledge irrelevant.
You mean Suno?
Shazam is historically interesting, but Google's "hum to search" algorithm is far superior, and even that is nearly four years old (since production).
I enjoyed reading the Go source. As opposed to the time I had to read some Ruby code.
“An Industrial-Strength Audio Search Algorithm”, the paper where Shazam describes their algorithm (https://www.ee.columbia.edu/~dpwe/papers/Wang03-shazam.pdf) doesn’t have a clear publication date, but https://www.researchgate.net/publication/220723446_An_Indust... indicates it is from 2003.
That patent was filed in the US on 2004-10-21.
IANAL, but to me, that’s a point against that patent in the USA.
How is it a point against the patent if the paper was written by someone under the employment of Shazam? Isn't the point of the patent to award the innovator with the right to profit from the innovation?
You only get a one year grace period after first public discloser to file for a patent in the US. So if the dates in this scenario are: - paper in 2003 - patent 2004-10-21
Only if the paper was released between 2003-10-22 and 2003-12-31 would it meet the one year grace period requirement.
Looks like it’s not relevant in this case since they got a provisional patent in 2002, but that’s likely what the above was referring to as “a point against that patent”.
Thank you! That's elucidating
... in return for disclosing said innovation for use by others (in the patent) instead of keeping it secret. If the disclosure has already happened without the prospect of royalties (such as in a journal article), then the deal is off. But indeed various places have grace periods for that.
That it was written by someone under the employment of Shazam makes it likely that it describes their algorithm, but for patent protection, what matters is that you can’t apply for a patent for an invention that has been published.
https://www.science.org/content/article/patent-first-publish...: “According to U.S. law, a patent cannot be obtained if an invention was previously known or used by other people in the U.S., or was already patented or published anywhere in the world. Furthermore, publicly using or selling an invention more than 1 year prior to filing a patent application completely bars you from ever winning a patent on that invention.
[…]
In Europe, for instance, there is no 1-year grace period--the chances of winning patent protection is lost the instant an invention becomes public”
You should note that the USA even today still has a one year grace period between when you first published an idea and when you can valodly file for a patent for that idea. So if they published it in June 2003 but applied for the patent in April 2004, then the patent is well within the grace period and the paper doesn't constitute prior art.
As long as you have submitted the patent before publication you are good to go.
Actually the provisional patent # 60/376,055 was filed on April 25 2002.
So if the paper was indeed published in Oct 2003 then all is well.
I was under the impression that patents aren’t enforceable on open source projects. I assume I am wrong?
Yes. Just because something is open source doesn't mean it's unbound by IP laws.
It's not whether "IP" laws apply; it's whether source code itself is in scope for patents. Source code is a description of an algorithm, which in principle is what the software patent is supposed to be providing anyway. Patents shouldn't be relevant here for the same reason they aren't relevant to what you write in e.g. a textbook on signal processing where you might find an exact description of how Shazam works.
Compiling and running that source code on your computer/as part of a wider system may violate a patent, but my impression was that patents are not relevant to the actual code. Are there test cases in the US around pure source distribution of a patented algorithm? Particularly post-Alice?
I wonder if the massive amount of open source software can now be used as elaborate 'prior art' in a way that basically invalidates any software patent that is awarded after the source had been made available?
I.e. if any algorithm was already implemented, in some variation, then the patent is not valid?
For example, for the infamous Amazon 'one click purchase', if a similar pattern was used, maybe a 'one click start vacuum cleaner robot', would it that patent then be invalid?
Depends how much money you have.
For example, Tesla just patented the Robotaxi one year ago, despite having open-source solutions like Apollo Self-Driving platform.
Patents are really an obsolete system that favors the super-rich and the lawyers.
I'd be surprised if they patented the general concept of a robotaxi. Almost certainly, it'd be one specific type they they could say they invented.
Patents are enforceable in the US for even personal use without distribution.
I dunno how software patents work but I was under the impression that unless you basically copy paste their code, the courts wouldn't consider it patent infringement as you can't patent the function, but rather the specific thing itself which for software is the exact code itself. But if I'm not understanding something please correct me.
Software patents are magic, you just start your process with 'on a computer do X' and because computers are a piece of hardware you can patent anything you like
I'm still upset that my "you can't patent software because of the Curry-Howard isomorphism" legal argument never took off. (Basically, software is equivalent to math, and you can't patent math. Therefore, you can't patent software. QED.)
You’re thinking of copyright, which covers a specific creative expression. Patents are more general on how something is done and would cover different code that works the same way.
Overly simplistic: patents is the process not strictly design.
Edit: just to be clear there are design patents too, but I don’t think they’d be granted for software.
This cannot be true because if you copied a physical design and made a few irrelevant modifications then it's still infringement.
Figure 1 looks interesting since it has both a time and frequency axis, when usually signals have either a time __or__ frequency axis.
Now I'm curious how the Fourier (?) transform of a signal at a __single__ given timepoint is even defined ...
Jamessb already linked to the right terms. One thing to add is that there is always a tradeoff between time and frequency resolution on short time Fourier transforms. You just can’t have both. It’s always a somewhat unsatisfying tradeoff that still works well in practice.
I found this out when I was trying to turn an ordinary 5 dollar thrifted musical keyboard into a midi controller by plugging it into my PC, putting it on "sine wave" and using a Goertzel detector
The latency for detecting audio-frequency waves is quite bad
This also stymied my desire to put digital audio onto a vinyl record :( literally not enough bandwidth
By definition there is vinyl quality worth of bandwidth on the record.
The concepts you are looking for are the short-time Fourier transform and spectrogram:
https://en.wikipedia.org/wiki/Short-time_Fourier_transform
https://en.wikipedia.org/wiki/Spectrogram
I remember a popular HN post from 10 years ago, that was pulled or the source was pulled because Shazam legally threatened the disclosure of the algorithm. I think it's actually the Google drive file pdf capture from OP's article.
Here is a link to that HN thread from 2013:
https://news.ycombinator.com/item?id=5723863
It was a Dutch guy. I have seen his talk on this something with fast Fourier transform and getting sued. Was fun
Thanks! Macroexpanded:
Creating Shazam in Java (2010) - https://news.ycombinator.com/item?id=32530056 - Aug 2022 (36 comments)
Patent infringement claim re: “Creating Shazam in Java” blogpost (2010) - https://news.ycombinator.com/item?id=9594480 - May 2015 (18 comments)
Creating Shazam in Java - https://news.ycombinator.com/item?id=5723863 - May 2013 (43 comments)
Implementing Shazam with Java in a weekend - https://news.ycombinator.com/item?id=1702975 - Sept 2010 (23 comments)
Longer list of relateds here: https://news.ycombinator.com/item?id=41132462.
Related. Others?
How Shazam Works (2003) [pdf] - https://news.ycombinator.com/item?id=40029036 - April 2024 (29 comments)
How does Shazam work? (2022) - https://news.ycombinator.com/item?id=38531428 - Dec 2023 (154 comments)
An Industrial-Strength Audio Search Algorithm (2003) [pdf] - https://news.ycombinator.com/item?id=33299853 - Oct 2022 (1 comment)
Creating Shazam in Java (2010) - https://news.ycombinator.com/item?id=32530056 - Aug 2022 (36 comments)
Shazam turns 20 - https://news.ycombinator.com/item?id=32520593 - Aug 2022 (227 comments)
How Shazam Works (2015) - https://news.ycombinator.com/item?id=23806142 - July 2020 (7 comments)
Designing an audio adblocker - https://news.ycombinator.com/item?id=18855029 - Jan 2019 (186 comments)
Show HN: A radio/podcast adblocker featuring ML and Shazam-like fingerprinting - https://news.ycombinator.com/item?id=18459058 - Nov 2018 (2 comments)
Apple has completed its acquisition of Shazam - https://news.ycombinator.com/item?id=18066724 - Sept 2018 (316 comments)
Apple Buys Shazam to Boost Apple Music - https://news.ycombinator.com/item?id=15899065 - Dec 2017 (156 comments)
Apple is close to acquiring Shazam, sources say - https://news.ycombinator.com/item?id=15881896 - Dec 2017 (292 comments)
Show HN: Shazam-like acoustic fingerprinting of continuous audio streams - https://news.ycombinator.com/item?id=15809291 - Nov 2017 (76 comments)
How Shazam Works (2015) - https://news.ycombinator.com/item?id=15350729 - Sept 2017 (13 comments)
Tell HN: Shazam picks up song from my kitchen light - https://news.ycombinator.com/item?id=11593305 - April 2016 (2 comments)
How Shazam works - https://news.ycombinator.com/item?id=9870408 - July 2015 (48 comments)
Patent infringement claim re: “Creating Shazam in Java” blogpost (2010) - https://news.ycombinator.com/item?id=9594480 - May 2015 (18 comments)
The Shazam Effect (2014) - https://news.ycombinator.com/item?id=9593429 - May 2015 (37 comments)
The Shazam Effect - https://news.ycombinator.com/item?id=8634357 - Nov 2014 (34 comments)
Ask HN: Is there an audio search technology that finds exact and similar audio? - https://news.ycombinator.com/item?id=8420141 - Oct 2014 (3 comments)
Source code example of the Shazam algorithm - https://news.ycombinator.com/item?id=5724442 - May 2013 (16 comments)
Creating Shazam in Java - https://news.ycombinator.com/item?id=5723863 - May 2013 (43 comments)
An Industrial-Strength Audio Search Algorithm (Shazam) - https://news.ycombinator.com/item?id=2621103 - June 2011 (4 comments)
Shazam's Search for Songs Creates New Music Jobs - https://news.ycombinator.com/item?id=2215295 - Feb 2011 (1 comment)
How does the music-identifying app Shazam work its magic? - https://news.ycombinator.com/item?id=2214992 - Feb 2011 (2 comments)
How Shazam Works To Identify (Nearly) Every Song You Throw At It - https://news.ycombinator.com/item?id=1727891 - Sept 2010 (1 comment)
Implementing Shazam with Java in a weekend - https://news.ycombinator.com/item?id=1702975 - Sept 2010 (23 comments)
Shazam: not magic after all - https://news.ycombinator.com/item?id=909263 - Oct 2009 (28 comments)
How does the music-identifying app Shazam work its magic? - https://news.ycombinator.com/item?id=893353 - Oct 2009 (16 comments)
Shazam Has 50 Million Users and Secures Investment From KPCB - https://news.ycombinator.com/item?id=882537 - Oct 2009 (13 comments)
So not enforceable anywhere else than in the U.S.
Any seller of software would be liable for selling software to US customers without a patent license.
I'm curious about the legal consequences of freely distributed software (e.g. open source). I wonder if the author/provider could be held liable if they: - knowingly (passively) or actively market to US customers (e.g. provide support) - are aware that US users are using it, and take no actions to prevent its distribution etc.
Can someone share their knowledge on this?
If European software incidentally infringes a US patent, and it is distributed freely, is the provider then liable? E.g. is Github basically liable for restricting US users from access to (distributing) patent infringing software?
As I understand, you can violate a patent just by importing something infringing. Selling it isn't necessarily a requirement for infringement.
https://blog.shp.law/index.php/2021/03/28/open-source-softwa...
I'm sure there's tons of patent violations in FOSS, but patents are usually just used to go after companies with big pockets.
There are other patents for their implementation, and all are filed in multiple countries. Look at the blue sidebar under "Worldwide applications".
The linked one is just for the US version of a singular patent. It had applications in 14 other countries and WIPO, 6 of which are still active (plus US).
It's:
(1) deriving a simple fingerprint from the FFT of the audio signal
(2) simple indexing
(3) simple similarity search
You need the signatures of all music on earth for this to work though ;)
Not that difficult with a Spotify subscription. It will take some time, though.
Would-be Neil Young fans will be left wanting if Spotify is the only reference.
Soon!
So you are saying to clone the repo now