Re: Homegrown synchronization

In case there are others following this thread, I'm going to take this
opportunity to recap what has been implemented so far. I have not yet
implemented the "intermediary app" that David has recommended, because
I don't fully understand what he is saying. Instead I decided to
finish everything else first and then concentrate on David's
recommendation for the intermediary app and in understanding how the
..ldb file really works. Previously there were too many competing
things to do.

So here's where I am so far:

CODE ADDITIONS TO MYAPP
1) Code to create an export update file from the application used on a
remote machine (linked to local PC backend files) has been implemented.
The code works against a snapshot of the local PC backend file that is
created when the app is opened for the session. This snapshot is only
required for determining deleted records. Additions/Updates are
determined by looking for values in the TimeStamp field, which is
present in each table, that are greater than the session start time.
None of this code executes if a user happens to be connected to the
production backend file on the server. I have not added code to
prevent WAN users from trying to link to the production backend file on
the server. In general, this is such a slow process that no one would
do it, but it could happen. So, I'm not sure if I'll try to prevent
WAN users from connecting directly to the production backend file on
the server.

Records (updates/additions/deletions) are written to an update DB that
is created from a model when my app starts up. The model contains
exact duplicates of each table which are empty of data. However there
is one additional field in each table (UpdateorDeleteFlag). If the
record written is to be added or edited within the production backend
on the server, then UpdateorDeleteFlag = 0 (False). If the record is
to be deleted from the production back end, then UpdateorDeleteFlag =
-1 (True).

The way deletions have been implemented in myapp necessitated the use
of a snapshot backend file. Had the DeleteFlag idea been implemented
at the very beginning, this would not have been required. But this
homegrown synchronization scheme is coming well after creation of my
application. The amount of effort required to revamp my delete scheme
was simply too large. It would have required not only massive recoding
but massive retesting of everything.

Creating the snapshot file (for confirming deletions) has significantly
increased the amount of time my app takes to load. But with some help
from others in another forum, I was able to implement a batch file to
actually create the snapshot backend. This removes my app from having
to create it and now the app load time is back to what it previously
was.

I also want to note that when the export update file is created and the
deletions have to be determined by comparing the current backend file
with the snapshot backend file (all files on the remote local PC), this
is a slow process. Each table has to be examined. On my development
machine, right now, this takes 30 seconds for 110 tables. So,
hopefully I can think of a way to speed this up.

Finally, when the export update .mdb file is created it is zipped up
and copied over to the server where the production backend is.

2) Updates to the remote local PC are now done by simply replacing the
entire backend file with an update from the server. In a folder on the
server, myapp code will look for a .zip file and if that file has not
already been downloaded (previous downloaded files are stored in a
table), myapp downloads the zip file and then replaces the existing
backend on the local PC with the unzipped file. I felt it was
important, however, to force my app to wait for this asynchronous
process to be completed, because subsequently then, the app will link
to the tables. Here is a link for a way to do this:

http://vbnet.mvps.org/

The method for creating the zip file, which is on the server, is
explained below under the 'Synchronizer' discussion.


CREATION OF SYNCHRONIZER

The synchronizer app (sync app) handles only the production backend
file on the server. Currently, it can only be launched from myapp by
users who have been given permission to do this (this feature was added
to the custom user setup form). The idea is that only users who are
connected on the same LAN with the server containing the production
backend will be allowed to launch the synchronizer. I'm not sure yet
where I would place the sync app (i.e. either on each LAN user's local
PC or out on the server where the production backend is; this may all
change if and when I implement David's intermediary app idea).

1) The sync app links to the production backend. Using a form timer,
it checks for updates in a folder on the server. These updates were
sent by remote users from my app to an Update Inbox folder on the
server.

2) When an update is found, code in the sync app will go through each
table and look for additions/edits and deletions that need to be
applied to the production back-end file. Whether the record is an
add/edit or deletion depends on the value of the UpdateorDeleteFlag
mentioned earlier.

3) Currently when the sync app applies an update it will subsequently
create zip files of the production backend and put them in an
UpdateOutBox folder. This folder is the one that myapp for remote users
will search for updates to download to the remote user.

A couple of points. Currently, I have not yet implemented creation of
zip files of the production backend on a schedule. Secondly, I need to
test what will happen if someone is actively using data from the
production backend while the synchronizer tries to create a zip file of
the production backend.


Everything seems to be working pretty well. I wanted to document where
I am right now as a "baseline" from here forward and if there are
comments on the method.

Now I want to concentrate on David's concern about what happens if more
than one user tries to synchronize the production backend (which he is
recommending that I avoid by using an intermediary app). The way
things are now, this can happen. It's important to note that I can't
have a software program (like Jet synchronizer) running on the server
to take care of all the production backend synchronization (that's one
of the main points of creating my own synchronization scheme).

So I'll be studying his suggestions on an intermediary app and the .ldb
file.

.

Relevant Pages

  • Re: Homegrown synchronization
    ... more than one synchronizer has a connection to the production backend ... sync app is the currently 'active' one for the purposes of 'managing' ... You need to know who has the synch app opened. ...
    (microsoft.public.access.replication)
  • Re: Homegrown synchronization
    ... to check for update files in the Import DropBox for the server. ... similar to the import code used to update a remote backend). ... code to close the "sync" app. ... synch app, but only one at a time would be able to do synchs. ...
    (microsoft.public.access.replication)
  • RE: Beginners Questions
    ... We do use Windows form on the presentation layer which is on ... terminal server and call web services on the business logic side. ... of using "proxy" authentication on SQL Server. ... > I have written an app with a Windows Forms UI that is deployed to clients ...
    (microsoft.public.dotnet.distributed_apps)
  • Re: Ruby Enterprise App Design Advice
    ... Load balanced to send user to best server. ... FastCGI or SCGI - We would like to replace FastCGI with something ... certain instance and want to have a single session server (if I understand ... deals with login and logout and assigns an app server. ...
    (comp.lang.ruby)
  • RE: C++ Garbage Collector on VMS?
    ... And HW is usually not a problem in th app tier. ... One for App server, one for DB. ... Tiers were a great concept when HW could not keep up with the processing required by each tier and network speeds were slow and unreliable. ... In small to medium app environments, ...
    (comp.os.vms)
Quantcast