Welcome Guest.   Make a donation to an author on the site September 30, 2014, 07:01:48 PM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
Read the Practical Guide to DonationCoder.com Forum Search Features
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: [1] 2 Next   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: Do it yourself dropbox  (Read 8955 times)
kartal
Supporting Member
**
Posts: 1,529


View Profile Give some DonationCredits to this forum member
« on: March 08, 2011, 11:19:09 PM »

Hey people

I have managed to create a rig that lets me share my files(via local or distant connections). It is not a simple setup but I think it works well. When I have time I will write a a short tutorial about it. Basically I am using SSH(on Cygwin) and HFS(http://www.rejetto.com/hfs/). I send myfiles to Ssh server and share them with Hfs while I am creating the url for the file on the fly with simple bash console scripts(generated when the file is sent). I endup giving away a url that is similar to any web based sharing tools. I can send files within my network or from outside world.


Does anyone else created a do it yourself dropbox?

I personally like to make life harder for myself so that I can enjoy all the freedoms that comes with custom workflows and methods and I am always interested in what others have to offer as far as custom stuff goes. I do not like to put my data on someone else`s server so I rigged up my own Wink


What is cool is that I can send folders, zip them before sending, pipe other stuff, and even resume files (will use rsync) etc. And I have no MB limit Wink

« Last Edit: March 08, 2011, 11:23:08 PM by kartal » Logged
housetier
Charter Honorary Member
***
Posts: 1,321


see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #1 on: March 09, 2011, 02:06:29 AM »

I have several times thought about creating my own such synchronization service. But I have never actually started on it, that is, I never got past sshfs.

I want to have my personal service because I don't trust 'them'. Sometimes there terms & conditions are not really in my favor (all your stuff belong to us, but you are still liable). Having a system that works for clients on different platforms -- linux, windows, mac, mobile os -- would be awesome.
Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #2 on: March 09, 2011, 06:22:39 AM »

housetier: check out SpiderOak - while even the best providers might disappear without notice, at least the SpiderOak guys have no way to access your files. Client works on multiple OSes.
Logged

- carpe noctem
nudone
Cody's Creator
Columnist
***
Posts: 4,116



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #3 on: March 09, 2011, 08:21:46 AM »

Are we saying that DropBox isn't to be trusted. Not wanting to start an argument, just wondering if I should stop using DropBox and move to SpiderOak.
Logged
Eóin
Charter Member
***
Posts: 1,400


O'Callaghan

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #4 on: March 09, 2011, 08:36:25 AM »

I know I personally never put anything private into my Dropbox folder.
Logged

Interviewer: Is there anything you don't like?
Bjarne Stroustrup: Marketing hype as a substitute for technical argument. Thoughtless adherence to dogma. Pride in ignorance.
argv
Supporting Member
**
Posts: 45


see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #5 on: March 09, 2011, 08:55:19 AM »

And I have no MB limit
That's hard to believe

I know I personally never put anything private into my Dropbox folder.
Personally I'm using a TrueCrypt disk in the dropbox folder for really private data
Logged
kartal
Supporting Member
**
Posts: 1,529


View Profile Give some DonationCredits to this forum member
« Reply #6 on: March 09, 2011, 09:11:00 AM »

I have several times thought about creating my own such synchronization service. But I have never actually started on it, that is, I never got past sshfs.

I want to have my personal service because I don't trust 'them'. Sometimes there terms & conditions are not really in my favor (all your stuff belong to us, but you are still liable). Having a system that works for clients on different platforms -- linux, windows, mac, mobile os -- would be awesome.

My first post was about making data available to outside world on demand(securely and freely). Since you are also wondering about syncing ,I use Unison+SSH for unidirectional syncing(multiple platforms) and works great. I sync GBs of data a day sometimes among 3-4 computers for personal and professional use. I sync big files, text files, video files etc.



@argv
Not sure if you are being sarcastic about the data limit, but really I have alot of empty space, and I can buy new hd as I needed.

Also syncing Truerypt container might be an ok idea for small data, but really  you need to sync the container file every time you change something in the container disc(save a new text file ), for anything bigger than couple hundred MB wont be feasible for my use.
« Last Edit: March 09, 2011, 09:12:52 AM by kartal » Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #7 on: March 09, 2011, 09:20:30 AM »

kartal: perhaps he's using block-based rather than file-based syncing?
Logged

- carpe noctem
mwb1100
Supporting Member
**
Posts: 1,315


View Profile Give some DonationCredits to this forum member
« Reply #8 on: March 09, 2011, 09:25:42 AM »

I think that Unison uses rsync to perform the transfer, so entire files aren't transferred, only the different bits (as detected by rsync).
Logged
kartal
Supporting Member
**
Posts: 1,529


View Profile Give some DonationCredits to this forum member
« Reply #9 on: March 09, 2011, 09:29:18 AM »

I think that Unison uses rsync to perform the transfer, so entire files aren't transferred, only the different bits (as detected by rsync).

Yes Unison is using rsync by default, and I found it to be quite fast since all the clients also acts as servers(Unison always has first hand access to local files).

This is not like syncing files over FTP, Unison knows which files has changed and it can detect moved files.
Logged
timns
Supporting Member
**
Posts: 1,209



Veni, vidi, debuggi

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #10 on: March 09, 2011, 09:41:56 AM »

I achieved something similar on our own Windows servers using VPN, auto-login scripts and a shared drive.
Logged

kartal
Supporting Member
**
Posts: 1,529


View Profile Give some DonationCredits to this forum member
« Reply #11 on: March 09, 2011, 01:05:47 PM »

kartal[/b]: perhaps he's using block-based rather than file-based syncing?

With dropbox?
Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #12 on: March 09, 2011, 02:51:49 PM »

kartal[/b]: perhaps he's using block-based rather than file-based syncing?
With dropbox?
I would have thought any sane service to only transfer modified blocks - SpiderOak certainly does.
Logged

- carpe noctem
jgpaiva
Global Moderator
*****
Posts: 4,710



Artificial Idiocy

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #13 on: March 09, 2011, 06:30:09 PM »

kartal[/b]: perhaps he's using block-based rather than file-based syncing?
With dropbox?
I would have thought any sane service to only transfer modified blocks - SpiderOak certainly does.
Actually, a friend of mine found out that DropBox does even more than that. It only transfers new blocks among all of its members' blocks. That is, if you try to upload a known large file (say a Ubuntu DVD or some known media), it'll upload instantly (regardless of the speed of your connection).
Logged

f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #14 on: March 10, 2011, 07:07:12 AM »

Quote from: f0dder
I would have thought any sane service to only transfer modified blocks - SpiderOak certainly does.
Actually, a friend of mine found out that DropBox does even more than that. It only transfers new blocks among all of its members' blocks. That is, if you try to upload a known large file (say a Ubuntu DVD or some known media), it'll upload instantly (regardless of the speed of your connection).
Good for speed, and storage space reduction at DropBox's servers, bad for privacy.
Logged

- carpe noctem
worstje
Honorary Member
**
Posts: 555



The Gent with the White Hat

View Profile Give some DonationCredits to this forum member
« Reply #15 on: March 10, 2011, 07:22:34 AM »

I guess there might be a privacy element involved with such caching if the extra information involved remains even after the last instance of the data was removed from Dropbox servers. But seriously, practically, what is the privacy leak in this case? The most devious usecase I can think of is some corporation querying dropbox for all possible files they own to see if it was ever uploaded... and even then it does not reveal who has the file. Hell, technically said corporation would probably be liable for stuff like reverse engineering the Dropbox protocols and whatnot, so it would legally be unusable information.

I use the almost exact same technique for JottiQ. Jotti's malware scan has an internal database of files it has already scanned, which allows me to avoid uploading a file a second time.

So, in a nutshell: I am willing to share some file with the service (Jotti / Dropbox), so I would be sharing the private information I worry about with the service to begin with. Thus the service obtains no private information it wouldn't have without such an optimization.

That make sense?
Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #16 on: March 10, 2011, 07:58:43 AM »

SpiderOak has a blog entry somewhere that lines out some security flaws (some which aren't all tinfoil hat) with a scheme that shares blocks between users - too lazy to search for it now, sorry, but it shouldn't be too hard to track down if you're interested. No reverse-engineering is needed, if global hashes are stored all the Evil CorpTM needs is a subpoena.

From the SO blog posts, it sounds like their method and reasoning for it is pretty well considered. Zero-Knowledge ftw.
Logged

- carpe noctem
jgpaiva
Global Moderator
*****
Posts: 4,710



Artificial Idiocy

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #17 on: March 10, 2011, 08:02:53 AM »

Actually, a friend of mine found out that DropBox does even more than that. It only transfers new blocks among all of its members' blocks. That is, if you try to upload a known large file (say a Ubuntu DVD or some known media), it'll upload instantly (regardless of the speed of your connection).
Good for speed, and storage space reduction at DropBox's servers, bad for privacy.
Yes, no question there. It also means that they must store all data unencrypted on their side, or they wouldn't be able to serve you a file someone else uploaded.
Logged

kartal
Supporting Member
**
Posts: 1,529


View Profile Give some DonationCredits to this forum member
« Reply #18 on: March 10, 2011, 09:07:05 AM »

I have nothing to hide! But I prefer SSH and Unison to sync and store my files, because those are my files(at least I own some of them) that sit on my paid hardware and I like to have control over my own system. I know people are going nuts over cloud stuff, I think it is pretty neat and wonderful as well. There can be amazing uses for the future of man kind,  but I prefer to rely on my own ideas and workflows. This way of working also makes me learn and implement new stuff. So I do not become a one way techno consuming machine as well.



You know the cloud can get cloudy, rainy and stormy, then what do you do?
Logged
JavaJones
Review 2.0 Designer
Charter Member
***
Posts: 2,537



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #19 on: June 16, 2011, 10:59:53 PM »

Just ran across this today: http://owncloud.org/index.php/Main_Page
ownCloud is still in development but it looks promising. You need to run your own LAMP stack to host the sync server which then of course has a web UI and there are desktop clients in the works for Win/Mac/*nix. Hopefully mobile apps eventually too. It's a bit too early in dev for me to want to play with it but I'd be interested to hear anyone else's experience.

- Oshyan
Logged

The New Adventures of Oshyan Greene - A life in pictures...
40hz
Supporting Member
**
Posts: 10,675



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #20 on: June 17, 2011, 09:18:06 AM »

Maybe I'm missing something, but wouldn't it be a lot easier to just run a copy of the Amahi server and be done with ?  smiley

On a related note, the Linux Action Show podcast had and episode where they discussed how to go about setting up your own personal cloud. Some very good ideas and solid recommendations in there if you can put up with the silly banter and general loose format of the show. I do put up with it because it's a good show even though Chris Fisher can really get on my nerves from time to time. (Hey Chris? Comedy is a serious business... best leave it to the pros unless you really are funny.)

The episode can be downloaded/watched/listened to here. Thmbsup

Note: apparently a bunch of us blipped Bryan about Amahi since we felt he might be reinventing a wheel. He's agreed there's a good chance he might and plans on doing a show on Amahi in the near future. He's put his project on semi-hold until he checks it out.
 Cool



« Last Edit: June 17, 2011, 09:21:52 AM by 40hz » Logged

Don't you see? It's turtles all the way down!
JavaJones
Review 2.0 Designer
Charter Member
***
Posts: 2,537



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #21 on: June 18, 2011, 05:32:40 PM »

I don't know enough about ownCloud or Amahi to say with any confidence, but Amahi strikes me as a significantly "heavier" system and, while more flexible, perhaps overkill for someone looking for a simple Dropbox-like system. That being said it's definitely a cool option in general.

- Oshyan
Logged

The New Adventures of Oshyan Greene - A life in pictures...
40hz
Supporting Member
**
Posts: 10,675



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #22 on: June 18, 2011, 07:58:41 PM »

I kinda see something like Amahi as less overkill in that all the heavy lifting and configuration has been done for you. So from an enduser perspective it's easier to install and administer. The beauty of it is you only need to use the parts you want. And it's GUI based so you don't need to do all the 'science faire' stuff you'd need to do if you rolled your own.

Sometimes it's easier to go up a level in complexity in order to gain a quantum leap in day-to-day simplicity of operation. This is something I go through with my clients from time to time. Sometimes the simpler something is, the more complex it is to actually use. And vice versa.

Just my 2¢ anyway.  smiley


Logged

Don't you see? It's turtles all the way down!
JavaJones
Review 2.0 Designer
Charter Member
***
Posts: 2,537



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #23 on: June 19, 2011, 02:01:53 AM »

Fair points. I haven't tried either so it's all hypothetical to me. But Amahi does look interesting...

- Oshyan
Logged

The New Adventures of Oshyan Greene - A life in pictures...
40hz
Supporting Member
**
Posts: 10,675



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #24 on: June 19, 2011, 06:04:18 AM »

I tried Amahi back a few years ago and I was very impressed by it. And its developers haven't been sitting on their hands since then.

Nice thing about it is it only requires Fedora, so if you have a spare 32-bit box floating around, you're good to go. Which is convenient since all the new Windows servers have gone over to 64-bit architecture.  
« Last Edit: June 19, 2011, 06:07:36 AM by 40hz » Logged

Don't you see? It's turtles all the way down!
Pages: [1] 2 Next   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.05s | Server load: 0 ]