I use Firefox whenever I can.
On first install of the browser I usually end up following a hardening guide which includes stuff like blocking cross site cookies, setting a few things in about:config to disable Pocket/etc, and installing uBlock Origin. I’ve taken what I consider a relatively balanced approach, I don’t use anything like noScript, uMatrix, etc that ultimately just cost a lot of time fiddling to get the 10th website of the week working.
I’ve been more or less fine browsing the web this way for years, but around the start of 2024 I’ve started seeing way more “Access Denied” pages than I used to. I think part of it is Cloudflare or similar, but I don’t know exactly what’s changed or what’s triggering it to occur.
It usually goes away and I can re access the site in 10-30 minutes as usual, but I’ve had it occur in really weird instances, such as trying to change my Minecraft skin and getting blocked by the website. The server block often goes away immediately if I switch my user agent, so I know that it has something to do with how I’ve got everything set up.
Not sure what anyone else’s experience with this has been. I’d like to hear some of your thoughts and tips
I for one want to offer a heartfelt apology. As someone that works in this space, bots are becoming more and more sophisticated. I can’t speak for Cloudflare, but we’re definitely not interested in your personal information. As someone who also prefers their privacy on the web, the fact that bot signatures overlap with privacy-centric signatures sucks. I myself have experienced it on my mobile device with Ghostery. It’s frustrating, I know.
Would you mind sharing the guide you used for hardening your Firefox? I’m curious to see what could potentially be triggering the issue.
Also, I just want to say, I think it’s hilarious that a site blocked you but then allows you to continue browsing after changing your user agent. That right there is bot behavior.
To circle back around to the actual block, I bet changing your skin executes JavaScript which flags something from the anti-bot software.
I had my suspicions that the issues I’ve been running into are mostly because of the worsening botting/scraping situation, and in part due to the general very slight preferential treatment Chromium browsers get on the wider Internet, where anything weird coming from Firefox automatically looks more suspicious because it’s an underrepresented browser already.
I typically just look up “Firefox Hardening Guide” and follow what looks like the best of the first few results every time I do a fresh install. Because of that, I don’t know exactly which guide I followed last, but this one echoes a lot of the steps I remember taking. I’ve since turned webRTC back on because it kind of broke discord(… I know, I know, discord is terrible for privacy but it’s where all my peeps are at!) Didn’t tweak everything outlined in guides such as the one linked, but pretty much whenever there was privacy to be gained seemingly without significant website breakage, I’d toggle it.
The user agent thing was bizarre, especially since it was also on Minecraft.net! I swapped to a generic Chrome on Windows agent and it instantly started working again and let me use the site as normal again. That said the user agent thing doesn’t always work… But the fact that it does sometimes may be a clue to why websites seem to hate my configuration.
Thanks for sharing that!
Truthfully, Firefox is fairly easy to detect. Several facets of the API it uses makes for quick identification. For example, Firefox should be able to report its build ID. Also, it won’t report specifics about the WebGL renderer you’re using like the vendor and architecture.
The link you shared is great and really highlights something I was thinking about today regarding this subject. The more you harden and change things the more you stand out. You’re also more likely to trigger bot detection when you alter specifics about your browser like the major version you’re on. I’ve seen some extensions change the user agent to much older major versions like Firefox 60. That’s a big red flag.
The user agent thing was bizarre, especially since it was also on Minecraft.net! I swapped to a generic Chrome on Windows agent and it instantly started working again and let me use the site as normal again.
Yes that is bizarre 😂 It’s not clear to me if Microsoft is using their own anti-bot solution or a third party one, but it doesn’t sound really successful with the way it’s reacting.
Overall, I can’t help but thinking the best route is to use the same thing as everyone else but roll your own VPN and change MAC addresses. Ideally, we would have some laws against all of this but I don’t foresee that anytime soon.
I wish I could do more to help. I’m happy to answer questions you might have, though.
Mull, Librewolf, Mullvad Browser, Arkenfox user.js
Its basically privacy.resistfingerprinting, a generalized useragent, maybe blocked javascript or ads.
Late reply here too. I’m sorry about that.
You can read my comment that I made here in regards to what I think could be causing you problems.
I do take this seriously and will try to find some time to put together a configuration like this for testing, so thanks for sharing.
Librewolf + AdNauseam on linux
Apologies for the late response!
I’ll echo similar thoughts to what I said in another comment. Librewolf, Mullvad, and other privacy based browsers are going to be a double-edged sword. You can take me with a grain of salt but these types of browsers actually do make you stand out in terms of fingerprinting. They have their own unique signatures, and the more you tweak the more you stand out too. Does it protect your privacy? It’s really hard to tell, there’s no data to suggest one way or another that I’m aware of. But, these changes are going to make you more likely to be challenged by captcha and blocked by sites in general.
I wish we didn’t have to try and solve this type of problem. Privacy should be a right.
For work I’ve found that Microsoft Teams no longer works on Debian + Firefox. My workaround is a dedicated VM running debian with Chrome installed (and nothing else). We’ll see how long that works.
Recently Teams is blocking Firefox even on Windows, but changing user agent was enough in my case…
There’s a Linux install of Teams. I’ve never tried it on Debian but it works fine in Ubuntu. Use it for work every day.
The official Linux client has been discontinued. Microsoft’s official solution is to use a browser – they explicitly mention Firefox.
There also seem to be unofficial clients. No idea if those are any good.
Huh, TIL. It’s still working just fine, so I’ll keep using it. Guess there won’t be any updates though.
You can use ungoogled Chromium within firejail to have better performance than a VM.
Yep, I started seeing the new silly puzzle based capcha and even after solving it the page often says ‘you’ve been blocked, fuck off’.
“Select the tools that could be used to install carpet”
Ah, thx. At the beginning of this week I got the first “Access denied”. It was the menu of a restaurant. My friends could open the PDF and my phone got denied (Mull browser).
For me it’s seemed more gradual over the past few years. I keep around a lightly sandboxed firefox install with a clean profile for the occasions where it’s worth going to that much trouble to see whatever cloudflare is blocking.
It also serves to remind me every now and then how much worse the default browser UI is compared to the one I’ve adjusted to my liking.
Yea, I fucking hate Cloudflare
Yes plenty from cloudflar and amazon servers but I am daily using mullvad browser which is hardened firefox
I get it constantly. When I run into it, I just nope out. Isn’t worth my time to fight it
Not with just firefox. Are you also using a VPN?
No, I haven’t had/used a VPN while this has been occurring. Main reason I made the post was that it has started affecting my work computer as well trying to access files from various websites for pdf specs
Most of my issues are because of vpn so I have to connect to fastest or random server again
It’s almost always cloudflare. Fucking cunts won’t ever allow my browser to any site using their “services”. Then there is Hcaptcha, “Solve my puzzles till the end of time in a fucking loop, and no, you’re never getting into the site”. I hate them
I’m using LibreWolf, Firefox and Tor browser since years (I love LW and TB, sometimes FF is needed for less restrictions for some sites). With LibreWolf I don’t see any more access denied messages lately. On Tor browser this happens from time to time, but I don’t see an increase. When using VPN I see these sometimes in LW and FF but didn’t notice an increase. I guess it depends a lot which web sites one visits. I usually stick to mainstream news, tech sites (like Stackexchange) and not much more. With LW and FF I often use the cool LibRedirect Firefox add-on to avoid some tracking and ads, maybe that helps. https://addons.mozilla.org/en-US/firefox/addon/libredirect/
Regarding Cloudflare : I think this really also depends on how the website user has configured their Cloudflare settings. For example with Anna’s Archives where I once in a while check for an e-book Cloudflare has not been offensive at all for the download links. Cloudflare is there checking, but no nagging (captcha), no blocking even with Tor browser. It is even possible that the default Cloudflare settings are pretty hostile towards Tor and VPN users, but that the website admins have no idea that it is so, and just followed some popular howtos how to configure Cloudflare for their site.
I have found that it happens more frequently with sites I’ve either not been to before, or not visited for a long time… Again it does seem to go away after 20 minutes or so for any given website, I just find it weird that it seems to be happening more.
I might have been exaggerating the degree to which this happens… It’s been only around 5-10 occurrences since the start of the year, but it happened so rarely before that point in time I barely noticed. Could also be a coincidence, it’s just barely enough though that I’ve been starting to get suspicious and wonder if anyone else was having issues
But yeah no VPN or anything and it’s occurred across 3 of my devices, only thing in common was Firefox and that I’ve taken steps to harden it on all of them
yeah, lately whenever I upload an image to lemmy I get a 403 error, I’ve had to use imgur in the meantime
Ya same here with librewolf, ublock origin, and a VPN. Decent number of sites block and want me to turn off ad block and/or VPN.
I have a simple policy now: they stay on.
I’ll just go somewhere else that doesn’t want to track people that badly
Bot blocking. Captcha bots are becoming smarter. Even Yandex is at last starting to become stricter with their indigenous captcha, but they do not harass you like Google/Cloudflare captchas (also HCaptcha is CF), rather seem to just want to filter out users that abuse search API in short periods.
Most of the ones I get seem related to my VPN. If I switch servers they usually go away.
I’m using mull with ublock origin on Android. I have to do a recaptcha for basically every cloudflare site which is annoying.
