Home | Blog | Software | Reviews and Features | Forum | Help | Donate | About us
topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • September 26, 2016, 05:30:28 PM
  • Proudly celebrating 10 years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: YaCy - Decentralised Search  (Read 2512 times)

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,214
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
YaCy - Decentralised Search
« on: July 08, 2014, 07:08:54 AM »
I just found out about this:

http://yacy.net/en/

Quote
Web Search by the people, for the people

YaCy is a free search engine that anyone can use to build a search portal for their intranet or to help search the public internet. When contributing to the world-wide peer network, the scale of YaCy is limited only by the number of users in the world and can index billions of web pages. It is fully decentralized, all users of the search engine network are equal, the network does not store user search requests and it is not possible for anyone to censor the content of the shared index. We want to achieve freedom of information through a free, distributed web search which is powered by the world's users.


comparison_why_is_YaCy_better_en.png

Via: https://www.techdirt...-snowden-world.shtml

Rather timely as I've been finding StartPage doesn't index a lot of things (because it's basically just Google results), so I often end up using DuckDuckGo, which works beautifully. Still, another option besides DDG is welcome.

But, this could be really, really big. If this (or something similar) can effectively compete with Google, we well may see a world where Google actually needs to seriously fight off the censors instead of capitulating so much as they do now. (I've noticed quite a bit of censorship in Google over the recent past - but I am lacking the smoking gun proof...)
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

IainB

  • Supporting Member
  • Joined in 2008
  • **
  • Posts: 6,048
  • Slartibartfarst
    • View Profile
    • Donate to Member
Re: YaCy - Decentralised Search
« Reply #1 on: July 08, 2014, 07:40:53 AM »
Thanks for posting about this. I just read that TechDirt post in my Bazqux feedreader and searched the DCF for references and found that you had just started this discussion thread.
Very thought-provoking.

wraith808

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 8,267
  • "In my dreams, I always do it right."
    • View Profile
    • Donate to Member
Re: YaCy - Decentralised Search
« Reply #2 on: July 08, 2014, 07:52:24 AM »
I just found out about this:

http://yacy.net/en/

Interesting- I'll have to keep an eye on this.  Have you installed it yet?

(re Smoking Gun Proof, check google vs bing vs DDG.  That was enough of a smoking gun for me)

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,214
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: YaCy - Decentralised Search
« Reply #3 on: July 08, 2014, 08:27:23 AM »
Have you installed it yet?

In a word...

AAARRRGGGHHH!!!

No.

My SSD is near full and I don't dare install anything more. I think I'm going to have to get a much larger SSD and clone the drive. I've been cleaning it out forever now, and just can't do much more.

However, this is a very high priority once I have enough space.

(re Smoking Gun Proof, check google vs bing vs DDG.  That was enough of a smoking gun for me)

If I may be so bold, I believe that Google is actively "shaping" search results for ideological purposes. I've done some searches that turn up zero in Google (and it's sidekick, StartPage), but are right at the top of DuckDuckGo. I can't possibly imagine Google being that incompetent that they "missed" some of the things I've tried to look into. I noticed this quite a while back, but set it aside as a possible "quirk". However, having done enough searches on a variety of topics, I can't push that suspicion aside any longer. Google is not a neutral search engine. And neutrality is the most important thing for a search.

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,214
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: YaCy - Decentralised Search
« Reply #4 on: July 08, 2014, 08:43:27 AM »
Here's one (bad) example... "michael meekin alien hat" (I had originally misspelled the name). Put that in StartPage or Google, then check DuckDuckGo. The site you are looking for is this: http://www.stopabductions.com/

(I like UFO videos/documentaries as they're often better than feature length films. I searched for that on a lark.)

BTW - The site there is just WILD! ;D It's quite literally wilder than most good movie plots!  :Thmbsup:

DDG was better & found what I was looking for despite the spelling error. Google is pretty sophisticated, so I'm not quite sure what to make of it. Adding in "alien hat" should get to the proper resource.

But, that's the only example that I can remember at the moment. There have been others where DDG proved better. I should start taking notes...
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,214
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: YaCy - Decentralised Search
« Reply #5 on: July 12, 2014, 07:19:41 PM »
Interesting- I'll have to keep an eye on this.  Have you installed it yet?

I upgraded my drive to a 1 TB SSD and just installed YaCy - seems ok so far. It's not going to replace your regular search engine right now, but it is very promising.

You can browse it at http://localhost:8090/

I find I'm getting a lot of German results, but, it's still small, so I guess I'll run a node for now & hope for more people to jump on board. It will improve 1 person/installation at a time.

In a quick test, I started crawling my blog site, then searched for the title of an essay that I mirrored on in after giving the crawler some time. My blog post with the mirrored essay came up, but the original essay didn't. No real surprise there. But it shows that crawled searches from your local crawl come up darn fast.

The administration interface is very good as well. Lots in there. I've not explored it all yet, but it seems quite robust from a quick browse.

The quality of results is not at the DuckDuckGo or Google level yet, so that's a problem. I suppose that what YaCy needs is philosophically aligned people to start running nodes.

There are some interesting things that you could do with it though. For example, you could run through TOR, crawl sites that you are interested in, e.g. https://publicintelligence.net/ (a subversive site - they even host "Inspire" magazine!), then later search your own local cache in stealth mode while not connected through TOR. Since you'd have that locally stored, you'd never access the Internet (stealth mode) and could get search results without anyone knowing -- it's all done on your local machine.

tl;dr - Great software. Network isn't prime time ready yet. Needs more privacy advocates to add nodes for network effect to start to take hold.
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,214
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: YaCy - Decentralised Search
« Reply #6 on: July 12, 2014, 10:55:23 PM »
Seems ComputerWorld sees a need for something else...

http://www.computerw...n_underground_Google

Quote
Why we need an underground Google
Governments are forcing search engines to show wrong results. It's time for search engines to go rogue so they can be right.

More at the link.
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Posts: 7,642
    • View Profile
    • The Blog of Deozaan
    • Read more about this member.
    • Donate to Member
Re: YaCy - Decentralised Search
« Reply #7 on: July 18, 2014, 02:24:56 PM »
It sounds interesting, but I guess I don't really get how it works. And can't you get in trouble for crawling sites? I recall reading about a guy who crawled the (entire?) internet and later got in trouble for it. I can't remember the details though. . .


intlmedics

  • Participant
  • Joined in 2015
  • *
  • Posts: 1
    • View Profile
    • Donate to Member
Re: YaCy - Decentralised Search
« Reply #8 on: January 14, 2015, 04:27:55 AM »
Hi Guys,

We are currently conducting our own desktop serverless programs, for email, chat, messenger, www search, etc etc etc, and I read about Yacy and others whom just basically index the search with already made SE's out their, what we are conducting is part serverless part node, part server with built in 256 encryption for the uers search engine built in.

Nothing is stored within user Desktop GUI when they perform the www searches, and seeing we own the actual search node, we can then start marketing and add in all users and company www references/indexing, same as they all do.

Our phase 1 launch date is about 2 months from now.

Yes everything is FREE for the complete programs for life incl all updates and other apps. As we will then be building the serverless nodes for any on-line business that requires privacy for them and their users/members.