It is becoming near impossible to find relevant information from search engines. Duckduckgo, SearXNG, Bing, Google, and so many more mainstream engines have a significantly high noise to signal ratio, and it is getting worse.
Here are a collection of the best search engines I know, please add more to the list.
- Forum Search Engine: https://crowdview.ai/
- Non-commercial Search: https://search.marginalia.nu/
- Libre Meta Search Engine: https://librey.devol.it/
- Golden Age Search Engine: https://www.wiby.org/
- Yandex: https://yandex.com/
If no more high quality search engines exist, would it be possible to host your own?
EDIT: Some new discoveries. The addon uBlacklist and filters can block super SEO sites from appearing in search.
Technically the best one was Altavista.
But they are long gone because they came from the old academic & idealistic internet and they never learned to survive in that internet where money rules.
From a very old memory, Google blew AltaVista out of the water no?
I mean we all switched for a reason and it wasn’t the cute logo.
I believe yahoo killed altvista before google came and finished it off
Altavista was ahead of their time. The modern internet desperately needs a technical search engine.
deleted by creator
DDG is ok for most searches, but they have definitely hit a plateau. Programming search results are quite poor, for instance.
I’ve started paying for kagi. Their results are just way better at this point.
It is as useless. After all, it’s just Bing. But if the results are good enough for you, then why bother finding something else.
Some results come from Bing.
most are. just do a side by side comparison. for most queries it’s literally the same results.
Hmm, I did notice a sudden severe drop in quality recently. Perhaps they are A/B testing something.
ddg has been going downhill for awhile now. they changed something significant a couple years back that just made results, especially after the first half-page, absolute shit.
I think it was because they dropped Yandex results.
It’s not as bad as Google yet, but I find myself getting terrible or no results quite a few times.
Ex: if I’m looking for a niche blog post from example.com, just entering the keywords doesn’t return the right result, if anything at all. I have to add “site:example.com” and the right link shows up on top.
It’s kinda amusing when this happens, but I keep using ddg anyway because bing and Google had the same issue for the same keywords when I ran into the issue.
Ddg cannot filter out results. If you don’t want pages containing the term, term you add -term to your search and those results should not be included.
Ddg doesn’t do this. I did a brief test of many search engines, and only google and mojeek filtered results correctly.
Edit: yadex seems to have working filtering.
Google doesn’t do that properly anymore, either, much like the “literal search” (quotes) - used to work, now it’s a crap shoot.
I self-host a SearXNG instance and it works really well.
Correct answer
Why pay for a search engine? I’m getting tired of Kagi astroturfing
Thank you!
Yandex was way better for searches in russian sources, but it came to shit just like google and also excludes whatever russian government don’t like at that point. I searched for some software in it multiple timea and the first link was some noname, probably malware site. It also promotes it’s own malware like browser with questionable russian security sertificates and their own Alexa. I’d honestly not include it in any list.
I like DDG and don’t switch from it that much. I’ve also heard Kagi as paid search engine is good, but I’ve never tried it.
Kagi
I self host a searxng instance and I find the combination of bing, duckduckgo and qwant as the source engines to return decent results. You can use a public instance and choose those engines in settings.
I’m specifically looking to replace DDG, they have really dropped in quality lately and are very clearly going back on their word for not having you in a bubble with specific to you results.
I found public instances often have issues connecting to many search engines at once, will look into self hosting it. Thanks!
Even though there’s a small monthly cost, the results have been consistent for Kagi. But consistency meets only half of my needs for search: I also want to make decisions quickly from what I find within the contents. If I were to to go to a link, wait for it to load, scroll the content, etc. – does that listed forum post have the answer I am looking for? Does this news article cover the nuances I have been tracking and would like to read more of? Kagi offers an AI-based summarize feature that helps. And that’s been meeting the other half of my needs, as well.
EDIT, an opinion: Search services may well be eventually replaced by small, niche LLMs trained to perform summerization tasks, such as Consensus, which I have used for work research, and Perplexity.ai. The AI summarize feature of Kagi is why I see the service as more useful than straight indexes, even when self-hosted. Kagi is a stepping stone toward this for me, and why I recommend it.
Lol kagi bots spotted!
Beep boop, my dude.
Please solve the partial differential equation to continue watching the video:
“I am not a robot” captcha is getting too hard…": https://www.youtube.com/watch?v=ru6fi4O4lp4
How WOULD you differentiate bots from happy users, anyway? They certainly look like organic accounts to me
For everyone who uses searxng, is it great for day to day browsing? Do I require to host my own instance or the setup is as easy as requiring to add “searxng” option on my browser app?
I’m interested to move away from google as it becomes shitty everyday and loses its effectiveness for advanced query (based on my own result compared during 2013 up to pre covid). Bing have weird result on my region so cannot use it, ddg only for occasional use.
Thanks!
pick a host from searx.space
Might be a good idea to bookmark this too because sometimes some hosts go down.
Appreciate it, will read more in documentations 👍🏽
Is “super SEO sites” a catch all term for those 99% filler websites that have a tomato soup recipe (in theory) but actually start out with, “Historical evidence seems to suggest that the tomato was first cultivated in the territory that would eventually become Guam back in 1464…”
I’ve wondered if we had a common reference term for those? I wish it didn’t have a positive connotation though…
Those plus sites that are lists of buzz words that have nothing to do with the actual site.
This site collects quite a few web search engines: https://www.searchenginemap.com/
Come on over to Kagi! You do have to pay but I use a search engine dozens of times per day so I’m not too bothered by it.
I’ve heard a lot of great things about Kagi, though the search limit and subscription is a little off-putting. A self-hosted Kagi would be amazing though!
Kagi has a free trial, 100 searches for free, so you can try it out and see if you like it or not.
They now have unlimited searches for $10/mo. That’s what got me to try it out.
You are correct though. I really do not like having all of my search history tied to my credit card (and then me). What helps me justify that is that instead of me being the product like google, by paying I’ve become the customer. Hopefully that incentivizes keeping them on the up and up.
I did come across searnxg in this thread. It looks like that can be self hosted so I’m gonna give that a try as well.
That’s expensive as shit, what the hell.
Yeah, you may be fine with the $5 plan, but that’s the lowest tier available.
Afaik they are not really running a profit yet, just expanding, so that’s an eye opener to how expensive it is to run a search business and how much value Google and others estimate they get from your personal information.
For now though their user base seems fairly much leaning towards business users that can defend this expense as part of becoming more effective professionally. Hopefully over time they’ll grow large enough to provide cheaper plans for regular persons while staying privacy focused and ad free.
I did 1300 queries which I think would put it at .007 cents per query. Sounds reasonable to me.
Not when they collaborate with brave. Fuck that!
They don’t “collaborate” with Brave, just use its search API.
The owner of Kagi is a homophobic piece of shit.
Just use searxng.
Where did you get that from?
It’s all over the Mastodon side of the Fediverse, shouldn’t be too hard to find with a quick search. I’m not at my pc atm so cbf using a phone to find some links sorry.
The controversy was about Kagi using search API from Brave, whose CEO is supposedly homophobic. Not Kagi’s CEO himself being homophobic.
No, it is also the Kagi founder as per the Mastodon topics.
Can you show some evidence of that?
The sources I find are solely from mastodon threads, and date from this month and the Brave thing. This is the best thing I’ve found
Those AI ones like Perplexity.ai, Kagi FastGPT and even Copilot (Bing Chat) give good results not just in the responses, but also in the links they return.
I’ve been slowly pivoting toward Perplexity.AI as the search engine. It basically does what I do, search + find the resources and summarize it but it is automatic with Perplexity.AI.
I rather pay them 20$ because that is saving me time a lot (and time is money in my case). Kagi’s search is okay but I can get nearly the same by using ublocklist on Bing or DDG for my use case.
I mostly use bing now. I like that it can answer complex queries and provides sources.
Don’t have much trouble not finding things.
Its nice when you are deep in Microsoft already from your company and get BingChat Enterprise included anyways.
Its slower than OpenAI GPT4 at times and its alot more restricted, but it gets the job done mostly.
You need to hack the UI to make it nice, unlock longer inputs, disable search tool at-will, disable synthetic streaming responses (consistent token speed, but takes longer overall)Simple query via DDG, complex stuff and ChatBot stuff via BingChat Enterprise.
I thought the AI assistant was free? I just downloaded the bang app and I just use the AI assistant whenever I search for anything now.
I don’t use any sort of enterprise, or full disclosure know what any of the other things you said are, any of those options or parameters.
But bing’s really impressed me lately, haha, so I still use it.
Actually on my desktop, I just use the regular Bing search, which I do now find superior to Google for just finding quick accurate answers for basic stuff.
It’s still crazy to me a year later after switching all my default search engines to Bing after using Google for so long, but Google search is just such trash now, it’s insane.
It is free, but the Enterprise version doesnt store anything (not even usage statitistics) and runs on separate systems (allegedly) aswell as having no limits.
Oh, I see, yeah that does sound way better
deleted by creator
Why pay? I think the answer is pretty easy. If one doesn’t want to self host. Running any kind of web based service costs the person running it money. Google obviously makes money off of a user doing searches via adds and data collection. I would actually have no issue paying for certain Google services if it meant that as a paying customer they would not double dip and try and profit off the data they are collecting on me.
This is all coming from a person who has a server rack in my basement and multiple PCs scattered throughout my home, so I am no stranger to self hosting.
I would actually have no issue paying for certain Google services
Excuse me sir, do you have a moment to talk about our Lord and Savior, Richard Stallman?
People have spare pc’s or laptops they can sacrifice as a SearxNG instance. I too have a homelab & i have not paid a single penny to have a hobby SearxNG instance on my old hackbook pro that I share with friends.
I feel like, the internet is more than just paying your way for services. It’s about creating and sharing services and decentralization. If Kagi takes off then we have another evil Google that wants to profit at the cost of users, but its name is Kagi. It might be an impractical answer, and it does take some work to create a better internet. End of philosophical rant about the true spirit of the internet
deleted by creator
Removed by mod