I self-host a (non-critical) mail server and a few other things and occasionally look at live firewall logs, seeing the constant flow of illegitimate traffic hitting random ports all over the place, some hitting legitimate service ports but others just probing basically anything and everything. I decided to setup a series of scripts that detect activity on ports that aren't open (and therefore there's no legitimate reason for the traffic to exist) and block those IP addresses from the service ports since the traffic source isn't to be trusted.
Something that came out of analysis of the blocked IP addresses was that I discovered a few untrustworthy /24 networks belonging to a bunch of "internet security companies" whose core business seems to depend on flooding the entire IPv4 space with daily scans. Blocking these Internet scanner networks significantly reduced the uninvited activity on my open service ports. And by significantly I mean easily over 50% of unwanted traffic is blocked.
Network lists and various scripts to achieve my setup can be found here: https://github.com/UninvitedActivity/UninvitedActivity
Internet Scanner lists are here: https://github.com/UninvitedActivity/UninvitedActivity/tree/...
Large networks that seem responsible for more than their fair share of uninvited activity are listed here: https://github.com/UninvitedActivity/UninvitedActivity/tree/...
I'm semi-aware of the futility of blocking IP addresses and networks. I do believe, however, that it can significantly reduce the load on the next layers of security that require computation for pattern matching etc.
Be aware: there are footguns to be found here.
One thing I do is I blocklist entire countries' and regional ISP' CIDR blocks. Believe it or not: straight to firewall DROP.
China, North Korea, so many african countries who's only traffic is from scammers, tiny islands in the pacific that are used for nothing but scamming...
Straight to DROP.
And I do not care about the whining.
I assume you don’t host anything that could be useful to the 1.5 to 2 billion people that you’re blocking.
Or they host a business site that doesn't do business in those countries and so nothing of value is lost to them. For example, it's literally illegal for me to accept payments from .ru, so why bother wasting their time and my bandwidth?
I live in EU,and a bunch of american sites just block the whole EU due to GDPR laws.
Then someone in US uses my email by accident to subscribe to some newsletter (not the first time, I also get personal emails for that person, since it's just one letter difference, and i'm guessing it's someone old, considering the emails I get), i try to click "unsubscribe", and it just redirects me to "<site> is unavailable in EU, blah blah" page, without unsubscribing.
I make sure to report that site to every goddamn spam list possible.
Which is incredibly reasonable. If the EU didn't try to claim EU law applies globally, those sites might still be up.
The US is just as bad at extraterritorial law, see FATCA for just one example.
https://en.wikipedia.org/wiki/Foreign_Account_Tax_Compliance...
That situation is quite different. The US is using its significant power and weight to coerce those non-US banks into compliance with FACTA. Those banks don't have to comply, but they want to do business with the US and US companies, then they don't have much of a choice.
It's not like they just made a law and now insisted it applies globally, which is what the EU did.
it's effectively the same, small banks just shove you out of the building and refuse to open a bank account for you if FATCA applies to you, their compliance is through just not accepting US tax payers.
This is a real issue that leaves US citizens only able to open accounts at bigger banks (with shittier services but enough budget to hire a FATCA compliance department)
Nope. Not even close.
Practically the GDPR law has no teeth at all because its claim of extraterritorial jurisdiction is nothing but nonsense.
FATCA applies because the US has a carrot or stick to enforce it.
Also, the US law as written is entirely reasonable and doesn't try to claim the law applies to US citizens anywhere in the world.
It absolutely does.
The USA has laws that govern what it's own citizens do abroad like. You aren't allowed to have sex with minors or pay bribes when abroad.
The USA also recently passed a law that allows it to prosecute foreign officials who solicit bribes from USA entities. https://www.ropesgray.com/en/insights/alerts/2023/12/us-cong...
Why is it different?
People don't have to comply to GDPR but if they want to serve EU folks then they don't have a choice.
The EU claims their law applies globally regardless of if people set foot in or do business in the EU. According to the EU, an EU citizen just needs to visit a site and the law applies, regardless of where the site is hosted.
According to the EU, the GDPR applies to some small shop owner in China with a website that harvests all data it can that isn't advertising in the EU, courting EU citizens in any way, has no business with the EU, etc.
Isn’t it actually exactly the same? The website doesn’t have to comply (and many don’t), but if they want to do business in the EU, they have to. How is that different?
No, it's not remotely the same.
The US is using the fact that people want to do business with them to coerce compliance, and as written the law only applies to US persons.
The EU claims the GDPR applies globally, regardless of if people want to do business with the EU, or even if people ever set foot in the EU. It's amusing nonsense.
https://en.wikipedia.org/wiki/CLOUD_Act strikes me an example
What? No
Claiming jurisdiction by server location is the stupidest thing ever if you trying to have any kind of customer protection laws. You have to go by customer location.
However, the claim that they have jurisdiction over EU citizens abroad is very questionable.
I disagree, because that's impossible. That's why the EU's attempt is largely a joke. Literally - it seems to get mocked a lot when I tried reading up on the credibility and practicality of what they claim.
It's the claim that they have jurisdiction over non-EU citizens and businesses in their own countries which is so laughable.
IMO replying unsubscribe should always work for marketing emails and if it doesn’t then I flag the email as spam. Nope, I’m not going to visit that tracked / info gathering unsubscribe link.
I only use unsubscribe links from things I voluntarily and willingly subscribed to.
If I was involuntarily subscribed to something, or subscribed because of an inconspicuous "subscribe me" checkbox that I probably didn't notice, including from a legit business that I purchased an item, it's getting reported as spam in Gmail.
Had a travel insurance do this and when I was in hospital in Asia I couldn't start a claim and the hospital nearly kicked me out. I'm sure the sysadmins thought it was a great way to reduce hacking attempts by blocking Asia.
That’s awful but why is the onus on random sys admins around the world to deal with this correctly and not the government hosting the problem entities?
if the government in question is supportive of said problem entities, they won't "deal" with it
If the government in question has free reign on regulating said traffic, it's an avenue for repressions and censorship
Otherwise it's a legal matter to seek action against such entities, which is already how it works
(... but I'm afraid we're actually mostly talking about "scenario 1 entities" here, which makes it futile to seek action from the very offices that already play a role in making it harder to use existing legal means)
And it’s not like we will invade countries to stop spam calls, although China is probably the closest to getting to that stage given that the scam centers in Myanmar seem to be a deciding factor in who they throw their support behind: https://www.theguardian.com/world/2024/jan/31/myanmar-hands-...
Government needs lobbying to act
That's like asking why don't we expect burglars to not burgle, they won't, but that doesn't mean walling off a whole neighborhood is the solution either.
I would say because it’s their job to serve their customers, even if they’re abroad? Especially for a travel insurance company.
Ironic that GP commenter said "I do not care about the whining" about regional IP blocks and the first reply is just someone whining about it.
If there’s one single business that I might expect to honor traffic from foreign countries, it would be the travel industry. I can suddenly envision using a VPN to route through Asia and check a travel agent’s site access before purchasing.
That's so remarkably stupid for travel insurance, it's unbelievable.
As a Russian, I hate it when people do this. It's extremely annoying when you just click some random interesting-looking link from HN or Reddit or Twitter only to be greeted by a 403 or a connection timeout. Then you turn your VPN on, and magically, it loads just fine.
people here are not thinking in whole systems-- roads have dual purpose.. there is security AND there is trade .. a world without trade is a poor world.. that includes the intellectual arts, civilian institutions cooperating, common issues like Climate.
The voices here that say "I block everyone, don't bother me with your whining" .. it is a security practice.. OK. security is not the whole story of civilizations; obstinate thinking leads to ignorance, not evolution.
The topic is SSH, an administrative and secured access. Yes security applies. to be on-topic
Of course one can obfuscate and secure their own SSH access as much or as little as they want. Run sshd on a different port, require port knocking, ban IPs after failed login attempts, all that kind of stuff.
I'm, however, specifically talking about public-facing services like HTTP(S), which also get blocked with this "I'll just indiscriminately blacklist IPs belonging to countries I don't like" approach.
Malicious traffic is not limited to ssh and comes from the same usual suspects. Automated attacks against web applications is constant. I wouldn't say it's indiscriminate, it's practical.
For many services, the expected value of letting people from Russia access their service is negative. The reality is that Russia contributes a large portion of hacking attempts while providing very little to no revenue for the service. At the end of the day it is just business, and sometimes letting countries access your service is bad for the bottom line.
Had a reddit clone. The amount of Russian spam coming in was nuts.
Blocking the ru language blocked all spam. And since it didn't have Russian users, it was an easy choice to make.
Personal page.. sure.
Business? You're a pain to many people and don't care.
I live in EU and many US pages just block the whole EU due to GDPR laws... then someone (by mistake) subscribes me to their newsletter, and the "unsubscribe" links leads to "this page is unavalable in EU"? I'll goddamn make sure your domain ends up on every goddamn possible antispam filter I can find.
Why? Are they spam pages?
Honestly, individuals can't really do much to change the reputation of a domain.
Maybe petition your representative to adjust the GDPR so they don't claim it applies globally?
The Biden administration needs to explain why they allow ISPs to import data from these countries.
I'm not sure I understand what you're suggesting. Are you saying that the US govt should make it illegal for people in its borders to communicate with people in those countries?
That's very computationally inefficient.
You can trivially maintain a list of the size of the whole ipv4 space by using bitmaps
Just be aware that with your strategy “blocking 50% of unwanted traffic” means blocking non-attack traffic, as these Internet security companies are mostly legitimate. The automated attack traffic that you actually want to block is in the other half and will frequently change IPs.
This is both subjective and highly dependent upon the scope of services being run. My setup would probably progressively create more hassle than it saves as on a scale from small business to large business. For the setup I have, I quite specifically want to block their traffic.
I'm possibly overly militant about this, but they keep databases of the results of their scans, and their business is selling this information to ... whoever's buying. I don't want my IP addresses, open ports, services or any other details they're able to gather to be in these databases over which I have no control and didn't authorise.
To steal an oft-used analogy, they're taking snapshots of all the houses on all the streets and identifying the doors, windows, gates, and having a peek inside, and recording all the results in a database.
I believe all of them are illegitimate. They 'do' because they can, and it's profitable. "Making the internet safer" is not their raison d'être.
Happy for any else to form their own opinion, but this is my current stance.
Would be cool to have a "don't scan me bro" list of IP's that engage in this that we could share - is there such a thing?
The problem is that becomes a concentrator of IPs behind which privacy conscious individuals exist, which probably has higher value to "whoever's buying". It's a conundrum.
It sounds like what GP is suggesting is to collect ips of all the scanners, and share the list of ips among ourselves, so we can collectively route their traffic to /dev/null.
Why not also sell the scans of scanners to the scanners customers and make a little pocket change?
aaaaah, that makes sense. See the links in my original post.
There's a comment downthread discussing something similar; I haven't tried it though: https://news.ycombinator.com/item?id=40695179
You're being sarcastic, right? We did this for telephone numbers and saw how it turned out...
My experience is that after blocking Censys, unwanted traffic on non-standard ports from other IP blocks has basically gone to zero. It appears to me that some bad actors are using Censys scans for targeting.
i get similar results
Act like a bot, get treated like a bot.
You don't block them forever, just enough for them to move on to someone else.
they dont move on to someone else, they scan entire internet on a regular basis, just like gogle crawls web pages
Lol legitimate. As legitimate as door to door salesmen. OP just put up a proverbial "no soliciting" sign.
Note that you're basing your assertion on the motivation of random third parties exclusively on the fact that they exist and they are behind active searches for vulnerabilities.
Just install fail2ban.
For SSH, changing to a random port number resulted in zero connection attempts from bots for months on end. It seems bots just never bother scanning the full 65535 port range.
For most of my VMs there's no ssh running. I use wireguard to connect to a private IP. I haven't done this on the bare metal yet but I might. Though barring exploits like we had recently nobody is getting into a server with either strong passwords or certificates. Fail2ban in my eyes is a log cleaner. It's not useful for much else.
it bans the bad ips, isn't that worth running?
But what does that actually accomplish?
A server with fail2ban can be DOSed by sending traffic with spoofed IP addresses, making it unavailable to the spoofed IP addresses (which could be your IP, or the IP of legitimate users).
That is typically a bigger problem than polluting your logs with failed login attempts.
What would spoofing the IP of a packet when the underlying protocol requires a two-way handshake accomplish?
With CGNAT, a prepaid sim card and some effort, you can make them block a whole legit ISP in a few days without spoofing anything.
fail2ban is another layer which is susceptible to abuse and vulnerabilities. It might keep noise out of your logs but at a huge cost. I'd rather just change the SSH port to something non-standard and write it down.
Add it port knocking and this is how I do it. nftables ftw
Have you considered using crowdsec?
Are there any downsides to crowdsec?
You end up sharing signals (IPs) to their crowd-sourced bad IP databases, but only get 3 free IP lists on the free plan. To get some of the bigger IP lists you need an enterprise plan at $2500 a month.
Essentially they use the free customers to build the lists that drive their enterprise sales, which is fair enough as you get to use their free dashboard and open source software. But to me it seems they're really only targeting enterprise customers as a business.
I set it up in a fairly superficial way, and there are only a handful (two or three) rules that can be applied on the free tier, and I'm a tight-ass.
It's still running, but it doesn't seem to block much - but that might be because I didn't put enough time into "doing it properly".
Good idea. What I do is, I disallowed password login in my ssh server, and I permanently ban whichever address that tries to log in using a password.
I use a bastion host on a VPS as the only source IP address allowed to ssh into my systems, so any attempts to connect to ssh (from any IP address other than the bastion) are both blocked and logged into "the list" to be blocked from connecting to any other service ports.
Try running some of your blocked ips through greynoise, they usually have some interesting information about them
Thanks for the tip. Looks like greynoise use ipinfo.io for IP metadata.
I use https://www.abuseipdb.com/ for any manual IP address checks, and https://hackertarget.com/as-ip-lookup/ for finding what ASN an IP address (range) is a member of. I'll check out greynoise and see what extra info may be provided.
Don't get me wrong, I want to do the same, I run a lot of servers and see all the automated nonsense aimed at public servers. However, you should consider the fact that today blocking an IP is akin to blocking a street, a village or sometimes even a town. For ~better or~ worse we now live in the age of CGNAT.
If your threat model and use case means you only care about a known subset of users with static IPs who are lucky enough to not share IPs then fair enough; but if you are running services intended for wide spread consumption you are likely blocking legitimate users without even knowing it.