Author Topic: How to maintain a reusable updatable code (Read 23,027 times)

justice · « **on:** July 16, 2008, 08:14 AM »

This is a problem i've been struggling with for years now, and I wonder if any of you found a solution.

I'm trying to maintain a reusable and updatable function library of code. By reusable I mean that I want to use the same code in multiple projects in multiple locations (as any snippets library / function list / custom class?). This part is quite easy. At the moment we share a common onenote workbook that contains a page for each snippet / function / sub (we're talking web development). I'm considering setting up a seperate subversion repository for this so so that you can have it filebased, versionable and local at the same time.

However the way it's implemented is that code is copied into (usually the functions.asp include part of) the project. Then after one of us discovers an error or extends the function I have to update all the other projects with this code by a search and replace and I have no way to ensure that all instances have been updated. Someone might have made a small change in one of the functions for this specific project.

Hardly an ideal solution and I guess we're looking at some kind of extendable classes system but your thoughts would be appreciated.

mediaguycouk · « **Reply #1 on:** July 16, 2008, 08:51 AM »

Wouldn't the answer be to have two seperate include files? Something like

index.asp
companyfunctions.asp
manualfunctions.asp

The company functions file(s) would contain all the code contained in your repository and you would be very very strict in ensuring that the code is read only

If you wanted to edit the code (even by one character) you would write the code into the manual function file for index.asp (in this example).

Once you edit code in the repository you would not need to do a find and replace, you could just replace the companyfunctions file as you can ensure that it will not delete any work (as it is 'read only' and you have only made the code better).

But obviously this is just an overview without really knowing much about your situation.

f0dder · « **Reply #2 on:** July 16, 2008, 10:03 AM »

There are iirc snippet managers that try to maintain a database of "where is this snippet used?", which will then update all the use-sites when you update a snippet. But imho that's still as much a recipe for disaster, as general "snippe use" is by itself.

The only real acceptable thing to do is keeping a properly maintained function library, and keep it under source control. Period. Snippet usage way too easily leads to ill-designed functions, spaghetti code, and redundancy bloat. And, as you've discovered, is a maintenance nightmare.

Sure, it's so deceptively easy to just paste something from a snippet manager, instead of all the annoyance and bother of designing a proper function library, and the boooother of checking out from your source control system, etc. But it's the only sane thing to do.

justice · « **Reply #3 on:** July 16, 2008, 10:39 AM »

Thanks both, I looked a bit more into the suggestion of a seperate
companyfunctions.asp
manualfunctions.asp

It look like this will work well, fortunately you can redefine functions in asp/vbscript quite happily and only the last function will be used, so that it will be compatible with functions inappropriately adapted for a specific project.

Now when I commit a new version of companyfunctions.asp then I want automatically that the other projects receive this update.Am currently thinking about setting up a client side post-commit script using tortoise svn to robocopy over the file into certain checkout folders.

mouser · « **Reply #4 on:** July 16, 2008, 11:01 AM »

As has been suggested elsewhere, duplicating code snippets in different files should trigger a warning siren going off in your head. Whenever that happens it means you have a function that belongs in a separate file that can be shared between projects in some form.

justice · « **Reply #5 on:** July 17, 2008, 10:17 AM »

Maybe others will find it useful. I've set up a separate repository for the shared functions. I set up a post-commit hook batch file that does the following:

* update all projects on the development server to latest version
* then replace all instances of the shared function file with the one just commited
* commit all projects on the development server again with the message that includes the revision number of the shared functions repository.

all this along with the result of all operations, diff of what's changed, log and other basic info that i already generate gets emailed to me. If any projects are committed as a result of the shared functions commit, then they'll sent an email as well with the changes.

That is all setup, except for the post-commit bit haha as it seems any environment variables are not available in the batch files which prove an annoyance, but am sure this will be possible. Good debugging tips

f0dder · « **Reply #6 on:** July 17, 2008, 12:49 PM »

Hm, "development server"? I hope this is only a thing used to test the .asp pages, not a central place where multiple developers edit the code files... and I hope the files there are only updated automatically? Otherwise you risk a lot of interesting problems by doing automated updates that way...

Sounds a bit quirky overwriting files and re-committing, but I dunno if there's smarter solutions when you keep the shared functions in a separate repository and want automated updates of it (is that really necessary, though?)

justice · « **Reply #7 on:** July 21, 2008, 08:15 AM »

Well the development server (when we used to work without version control)'s role is only testing, so we should really call it testing-server but the name stuck.
All two developers (

) check out locally and modify files on their local machines, then commit it, it gets automatically checked out on the testing server in the public root for that site (for new sites) or in a temporary checkout folder and robocopied to the public root. We only ever edit code locally.

The robocopy method is unfortunately necessary because subversion only allows checkouts to empty folders. When we moved to svn, we already had some existing sites with file system permissions set in several places. We're moving webservers soon and until then don't want to mess up any filesystem permissions so are forced to copy changes over for some sites. Soon it will just be a clean checkout from the repository.

We're using beyond compare to move from test to live for an additional safety check (making sure every action is intended)

f0dder · « **Reply #8 on:** July 21, 2008, 09:59 PM »

Quote from: justice on July 21, 2008, 08:15 AM

The robocopy method is unfortunately necessary because subversion only allows checkouts to empty folders. When we moved to svn, we already had some existing sites with file system permissions set in several places. We're moving webservers soon and until then don't want to mess up any filesystem permissions so are forced to copy changes over for some sites. Soon it will just be a clean checkout from the repository.

Once you've done an initial "svn checkout", can't you just use "svn update" - or am I missing something?

justice · « **Reply #9 on:** July 23, 2008, 08:17 AM »

yes thats what's happening sorry if it's not clear., but if you have a folder with files on it and have the same files in a repository you still can't convert the regular folder into a checkout folder, you'll need a empty folder to checkout into which means having to set up all the filesystem permissions. That's my understanding.

f0dder · « **Reply #10 on:** July 23, 2008, 10:57 PM »

Ah, so the problem is you don't want to do the filesystem permission stuff and do an initial checkout in the final location, because the permission stuff is bothersome?

Somebody should write a little tool/script that traverses two filesystem trees, and clones permissions (but nothing else) to the destination tree. Perhaps that's already doable with "XCOPY /T /E /K /O /X", though.

justice · « **Reply #11 on:** September 17, 2008, 09:04 AM »

Just an update, after about 80 tries and two days I had created a post-commit hook for a shared repository that did an update to a server working copy and then committed it for every project that included our shared code. A lot of hassle and now committing shared code was took 4 minutes to come through.

then i learned about svn externals yesterday and in 5 minutes set the same thing up a lot better.

f0dder · « **Reply #12 on:** September 17, 2008, 10:50 PM »

svn externals?

Pray do tell!

justice · « **Reply #13 on:** September 17, 2008, 11:07 PM »

You can use the svn:externals property to specify that a folder in your respository should come from an external repository. so say you have a /trunk/includes/shared structure in repository "test" with another repository "sharedcode" that has shared code, then something like this should do it:

(navigate to /trunk/includes)
svn propedit svn:externals .
(a text editor window opens for you to specify the property)
shared svn://server/sharedcode/trunk
(save and close)
svn update (updates working copy including latest revision from the sharedcode/trunk repository in the /trunk/includes/shared working copy folder)
svn commit -m "added svn:external from sharedcode"

hope i explained that ok.

and from now on all developers have this in their working copy. svn update, checkout, export etc all work with the externals and any change to the /trunk/includes/shared will commit to the sharedcode repository

http://blogs.gnome.org/johannes/2008/02/20/svnexternals-for-noobs/ might explain it better?

justice · « **Reply #14 on:** September 17, 2008, 11:16 PM »

Quote from: justice on July 21, 2008, 08:15 AM

The robocopy method is unfortunately necessary because subversion only allows checkouts to empty folders.

BTW, in the latest subversion they added the --force switch to checkout (and export but maybe that one already existed) that allows you to checkout in populated folders which existing files when already existed will be treated as locally modified files. very nice. this means its easy to create a testing 'build' by using svn update on a working copy on a testing server, which was created from svn co --force.

f0dder · « **Reply #15 on:** September 18, 2008, 12:45 AM »

That sounds like a pretty darned useful feature for larger projects - thanks for bringing my attention to it!

f0dder · « **Reply #16 on:** February 09, 2009, 11:00 PM »

Finally got around to using svn:externals for a project, to include my VS project for unrar - thanks again for tipping me about it, Justice, it's a darn useful feature

justice · « **Reply #17 on:** February 10, 2009, 09:28 AM »

Yeah it's definately the way to go with regards to shared code.

Only cavat I found is that if you got the setup that we have: project 'A' that includes externals 'SHARED' and a testing server + live server and you update SHARED, obviously project A isn't updated instantly. Our 'testing build' is a checkout that gets updated every time a developer commits to it. Changes in SHARED don't update 'testing' until one of us made another commit locally.

change is SHARED -> local working copy update for project A -> make change project A -> commit -> SHARED code commited.
instead of
change in SHARED -> update all testing builds that use SHARED.

OTOH that helps with testing things better but we have no formal testers.

justice · « **Reply #18 on:** February 10, 2009, 09:29 AM »

Quote from: f0dder on February 09, 2009, 11:00 PM

Finally got around to using svn:externals for a project, to include my VS project for unrar - thanks again for tipping me about it, Justice, it's a darn useful feature

Could you detail how you use it? 'for unrar'?

f0dder · « **Reply #19 on:** February 10, 2009, 01:31 PM »

I use unrar.dll in one of my projects, and instead of stuffing unrar.lib, unrar.h, unrar.dll and just using that, I've set up my own VS2008 project for unrar (instead of using their makefile), and I pull svn://my.server/unrar/tags/3.5.8 into project/lib/unrar via svn:externals. The .vcproj for unrardll is then included in my project's .sln file, and everything is hunky-dory.

Well, except that while I can build unrar.exe just fine, something messes up when building the DLL, causing extraction to fail. Need to dig into this, but chrissake the unrar code is messy, poorly structured, and poorly written. It's almost at the level of spaghetti of 7-zip

justice · « **Reply #20 on:** February 10, 2009, 01:33 PM »

Yeah that's similar to how I use it with autohotkey I include the single copy of dcuhelper.exe (to provide dcupdate support) to all my scripts using svn:externals so I can update dcuhelper.exe once and have it current in all projects.