topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Friday March 29, 2024, 8:41 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: MOANTS Database Design  (Read 8161 times)

Rover

  • Master of Smilies
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 632
    • View Profile
    • Donate to Member
MOANTS Database Design
« on: May 13, 2006, 09:44 PM »
For those not familiar, Superboyac started a big "to-do" with his discussion on Note-Taking software. See the link: https://www.donationcoder.com/forum/index.php?topic=2362.0

So I'm ready to start discussion database engines and database designs and this is where we'll do it.  Fire up your brains... it's time to work.   :Thmbsup:
Insert Brilliant Sig line here

Rover

  • Master of Smilies
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 632
    • View Profile
    • Donate to Member
Re: MOANTS Database Design
« Reply #1 on: May 13, 2006, 09:51 PM »
RE: Firebird
Firebird is a relational database offering many ANSI SQL standard features that runs on Linux, Windows, and a variety of Unix platforms. Firebird offers excellent concurrency, high performance, and powerful language support for stored procedures and triggers. It has been used in production systems, under a variety of names since 1981.

Firebird is a commercially independent project of C and C++ programmers, technical advisors and supporters developing and enhancing a multi-platform relational database management system based on the source code released by Inprise Corp (now known as Borland Software Corp) on 25 July, 2000 under the InterBase Public License v.1.0.

Join the Firebird Foundation    Firebird is completely free of any registration, licensing or deployment fees. It may be deployed freely for use with any third-party software, whether commercial or not.
-From their site
http://firebird.sourceforge.net/

RE: Asksam
askSam is the ideal application organize your information. askSam is a different kind of database - a free-form database designed for users rather than programmers. askSam makes it easy to turn anything into a searchable database: email messages, word processing documents, text files, spreadsheets, addresses, Web pages, and more.

askSam gives you the power of a database without the complexity. No need to program or learn a complicated query language. With askSam, you simply import or enter information, and you're ready to search. askSam users range from individuals organizing email, addresses, and research notes to corporations and government organizations managing meeting minutes, regulations, policy manuals, and corporate databases.
-From their site Product Info
http://www.asksam.com/brochure.asp

If anyone else has other suggestions, now would be the time.  I'm going to email askSam and see if they'd like to donate a copy for us to use, should we choose it.  They at least have a trial version.
Insert Brilliant Sig line here

Rover

  • Master of Smilies
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 632
    • View Profile
    • Donate to Member
Re: MOANTS Database Design
« Reply #2 on: May 13, 2006, 10:10 PM »
Here's another one:  Cache (formlerly MUMPS ??)
  Caché is a post-relational database that uniquely offers three integrated data access options which can be used simultaneously on the same data: a robust object database, high performance SQL, and rich multidimensional access. No mapping is required between object, relational, and multidimensional views of data, resulting in huge savings in both development and processing time. Caché enables rapid Web application development, extraordinary transaction processing speed, massive scalability, and real-time queries against transactional data.

The Caché Executive Overview provides a high-level description of Caché’s architecture and capabilities.

Caché’s technical benefits are briefly laid out in the document Why Developers Choose Caché.
-From their site
http://www.intersystems.com/cache/technology/what-is-cache.html
Insert Brilliant Sig line here

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: MOANTS Database Design
« Reply #3 on: May 14, 2006, 12:57 AM »
im a novice in database design - im looking forward to reading this discussion.  :up:

Rover

  • Master of Smilies
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 632
    • View Profile
    • Donate to Member
Re: MOANTS Database Design
« Reply #4 on: May 15, 2006, 09:44 AM »
askSam update:  Unless someone has serious objections, I'm going to rule out askSam.

1) It's a windows only platform.
2) It's very expensive to distribute with an applications (~$5,000 for unlimited users)
3) It has it's own user interface and targets end users.
4) Using it as a database engine seems to be a 2nd or 3rd priority. (similar to #3)

Insert Brilliant Sig line here

Rover

  • Master of Smilies
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 632
    • View Profile
    • Donate to Member
Re: MOANTS Database Design
« Reply #5 on: May 15, 2006, 09:57 AM »
Looking at Sleepycat Software Berkley DB XML ...

http://www.sleepycat.com/products/bdbxml.html
Berkeley DB XML is an embedded XML database with XQuery-based access to documents stored in containers and indexed based on their content. Berkeley DB XML is built on top of Berkeley DB and inherits its rich features and attributes. Like Berkeley DB, Berkeley DB XML is a library, not a server, exposes a programmatic API for developers, and runs in process with the application with no need for human administration.

Thanks to Berkeley DB as the underlying storage engine, Berkeley DB XML inherits full ACID transactions, automatic recovery, hot standby, XA for distributed transactions, on-disk data encryption with AES, and replication for high availability. In addition, both XML and non-XML data can be stored in Berkeley DB XML, which may be an advantage for some applications. No other XML database in the market is based on such proven, field-tested technology.
-From their site

EDIT:  Added details from the site
Berkeley DB XML is very flexible, easy to deploy and easy to integrate. As a set of C and C++ libraries, it can be installed and configured along with your application. It integrates with Apache and can be accessed directly using C++, Java, or from web pages using PHP, making it ideal for dynamic web systems (much like the popular LAMP architecture). Berkeley DB XML was designed to operate a completely unattended fashion, so all administrative functions are controlled programmatically. It supports a wide variety of programming languages and operating system platforms.

    * Programmatic administration and management - zero human administration
    * Command line tools to load, backup, dump and interact with the XML databases
    * Language support (C++, Java, Perl, Python, PHP, Tcl, Ruby, etc.)
    * Operating system support (Windows, Linux, BSD UNIX, Mac OS/X and any POSIX-compliant operating system)
    * Installer for Microsoft Windows
    * Apache integration
    * Documents up to 256TB
    * Source code, test suite included
Insert Brilliant Sig line here
« Last Edit: May 15, 2006, 10:41 AM by Rover »

Rover

  • Master of Smilies
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 632
    • View Profile
    • Donate to Member
Re: MOANTS Database Design
« Reply #6 on: May 15, 2006, 10:54 AM »
I'm starting to lean pretty heavily toward Sleepycat.

I think an XML Database is an absolute must.  I don't expect much argument so I'll leave off the reasons.  If anyone wants to have that discussion we can.

Sleepycat has a couple of very nice benefits:
1) Cat is in the name
2) It is open source OR closed source depending on how you want to license it.
3) It is based on the rock-solid berkeley DB stuff that's been around longer that you have.
4) It supports document indexing in the database for rapid text search.
5) No user database maint. is required. 
6) It supports almost every language known to man. (Programmatic and lingual)

That was more than a couple.

Any comments, concerns, etc?
Insert Brilliant Sig line here

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 9,153
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Read more about this member.
    • Donate to Member
Re: MOANTS Database Design
« Reply #7 on: May 15, 2006, 12:08 PM »
May I toss in www.sqlite.org ? Free+OpenSource, fast, ACID, used by the military, client-only (no need to install servers), relatively small, et cetera. Doesn't support all of the SQL spec, but usually enough. Binary (fast) databases. Portable across a wide range of OSes and CPUs (and iirc big/little-endian databases are interchangable between big/little-endian systems).

Oh yes, and it has a no-nonsence license - can be used with open- as well as closed-source programs.
- carpe noctem

nevf

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 115
    • View Profile
    • Clibu, accessible knowledge
    • Donate to Member
Re: MOANTS Database Design
« Reply #8 on: May 15, 2006, 03:34 PM »
Seems to me like the cart is being put before the horse. Shouldn't the requirements be clearly defined and documented before any discussion of what database may or may not be appropriate.
Neville Franks, Clibu a better way to collect, use, manage and share information across all devices.

Rover

  • Master of Smilies
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 632
    • View Profile
    • Donate to Member
Re: MOANTS Database Design
« Reply #9 on: May 15, 2006, 05:07 PM »
nevf - What?  You don't read minds?   :P

I have the requirements clearly in my head.  Well most of them....

Database engine requirements:

1) Multi-platform - Minimum Win + Linux, should include Mac and POSIX systems.
2) Light - middle weight.  We're not going to install SQL Server or PostgreSQL just to track notes.
3) Language friendly.  The goal is to be able to create multiple interfaces to the data and people have language preferences.  The database shouldn't require that I program in C.
4) tought/resiliant -  If I'm going to save my life's work in a database, I don't want to lose it becuase the system crashed during a write.  ACID would probably be good.
5) agile - I don't expect to drop and reformat all of my records to add a new interface that uses new fields.  We should be able to extend the schema with legacy data.
6) Exceptional Support for Text searching/query.
7) Blob support
?

As I was looking around for db engines, it occured to me that it should probably support XML natively.  IBM is heading in this direction with DB2.  XML is a great OPEN standard for data exchange.  Open Office and MS Office (next version, 12?) use XML for document formatting.  In short, XML seems like a no-brainner for this type of application.

While it really wasn't a note-taking app requirement (end users shouldn't need to care about the database), it (XML Database) seems to make sense from a development point of view.

Comments?
Insert Brilliant Sig line here
« Last Edit: May 15, 2006, 06:20 PM by Rover »

Jimdoria

  • Charter Member
  • Joined in 2006
  • ***
  • Posts: 257
    • View Profile
    • Donate to Member
Re: MOANTS Database Design
« Reply #10 on: May 15, 2006, 05:22 PM »
Hi Rover -

I think I'm with Mr. Franks on this one. You've defined database requirements - but we're not talking about building a database engine. We're talking about building note taking software. Program functionality comes first, starting with interaction design. Technical stuff follows on as appropriate. Why would you start working on plumbing before the blueprints are even drawn? (Note: "Because I'm a plumber!" is NOT really a good answer!  ;) )

- Jimdoria ~@>@

There are two kinds of people in the world: Those who divide everybody into two kinds of people, and those who don't.

Rover

  • Master of Smilies
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 632
    • View Profile
    • Donate to Member
Re: MOANTS Database Design
« Reply #11 on: May 15, 2006, 05:26 PM »
Database design.  I was going to put off the formal design until after we selected an engine, but it makes sense to have a high level design to help define our engine requirements.  Again this was in my head I don't know why you didn't see it there  ;D

For discussion purposes, a note is the main item in the database.  A user creates a note to record information.  The note could be a web page, image, text, or a combination of these things.  With that in mind the data we need to manage should include:

date/time stamp for each note
note identifier (internal unique)
o note title/name
o category relationship (M-M)
o text
o blob
o template
o user **

o = optional

** opens a sticky issue.  Assuming the database is avilable for anyone who can write an application to talk to it, there is no security.  Should we care that User X can in theory read data from User Y database?  This assumes that the files are on a shared PC and the database files are visible.  Need some discussion.


Normalization: How far do we want to normalize?  Should text be part of the record definition for a note or can it be in a serate linked table?  Same question for any optional field except Category, which should be in a table due to the Many to Many relationship.

Some discussion is probably needed regarding how to maintain formating linkages between images and text.  Also how do you want to store a web page?   We should probably define those standards instead of letting the application interface determine them so you don't have data that is not transportable between two interfaces.

In the end, this becomes more of a standard than an actual database definition.

Insert Brilliant Sig line here
« Last Edit: May 15, 2006, 06:21 PM by Rover »

Rover

  • Master of Smilies
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 632
    • View Profile
    • Donate to Member
Re: MOANTS Database Design
« Reply #12 on: May 15, 2006, 06:18 PM »
You've defined database requirements - but we're not talking about building a database engine. We're talking about building note taking software. Program functionality comes first, starting with interaction design.
Normally I'd say your 100% right.  however in this case, we are designing an application that does not have a single user interface.  We're designing a database where you can roll your own interface to suite your need.  Now, we probably have 1 or 2 in mind to use this, but the application in this case is really data storeage.  How we input the data and how we report it is slightly irrelivant because the interface will vary according to user's individual tastes.

That being said, there are a few non-negotiables for Note-Taking software. That is what we're trying to capture here.  If you want to extend your UI to include a color preference field, then you'll need to do that in your own table.  The basics, however, should be available to any UI.
Insert Brilliant Sig line here

Jimdoria

  • Charter Member
  • Joined in 2006
  • ***
  • Posts: 257
    • View Profile
    • Donate to Member
Re: MOANTS Database Design
« Reply #13 on: May 17, 2006, 12:18 PM »
Sorry, I'd say I'm 100% right in this case, too. :D (Just kidding. Maybe only 93% right.)

You don't pour the foundation (or even set the concrete molds) before you've designed the structure. You're assuming a lot of things as given that I think are far from given.

A post in the other thread suggested using the file system as a data repository. The database needed in that scenario is worlds away from the one needed in the "store all notes in the database" scenario. I don't know that this could work, but if nothing else it shows that some of your "non-negotiables" may actually be "negotiables" at this early stage.

I still think DB req's are premature at this point, and specifying them too early runs the risk of imposing needless limits on the structure of the program. Or at least of spending time doing work that runs a high risk of being completely discarded. But it's your time, I guess. Personally, I don't think I'll be posting to this thread again - at least for a while.
- Jimdoria ~@>@

There are two kinds of people in the world: Those who divide everybody into two kinds of people, and those who don't.