Re: Homegrown synchronization


David W. Fenton wrote:
"rdemyan" <rdemyan@xxxxxxxxxxx> wrote in
news:1167694158.878560.213810@xxxxxxxxxxxxxxxxxxxxxxxxxxxx:

David W. Fenton wrote:
"rdemyan" <rdemyan@xxxxxxxxxxx> wrote in
news:1167608261.069673.213770@xxxxxxxxxxxxxxxxxxxxxxxxxxxx:

[]

2) Updates to the remote local PC are now done by simply
replacing the entire backend file with an update from the
server.

I don't understand this. How do you not lose updates this way?
How do you trigger the application of the updates to the server
back-end you've just pushed up to the server? I would do it this
way:

If the updating of the production backend is working correctly, then I
don't think it is so much a question of losing data. Example: WAN User
A changes something in a table. When they shutdown MyApp, the
CreateExportUpdate code is invoked. It creates an UpdateDB and copies
it to the server Inbox and then moves the local UpdateDB (just copied
to the server) to the Archive Outbox on WAN User A's harddrive. Now
User A has shutdown.

SyncApp for a LAN User F finds the update and applies it to the
production backend. It then creates a zip file of the production
backend (okay, I may change the timing on this method, but that's how
it works now). That new zip file is put in the server OUTBOX. When
WAN User A logs in again, MyApp finds the new zip file in the server
OUTBOX, downloads it, kills the local backend and unzips the new
backend.

Now if WAN User B has also sent an update that got applied to
production backend prior to WAN User A logging back in again, then a
new zip file was created of the production backend that contains both
WAN User A and WAN User B's updates. When A logs in again, he gets a
fresh backend (via zipped file) containing both updates.

I think you can argue that this can be a timing issue. But I would
counter that as long as WAN User A's update got applied (gets applied)
to the production backend, he will eventually have it sent back to him
in the zip file. I would also bet that this would work the next time a
user logs on in 99+% of the cases, if I get the timing issues correct
for synchronizing the server and assuming a user doesn't shutdown and
then immediately get back in.

Now, if WAN User A's update doesn't get applied, then no one gets it
including User A. In your scenario, A would still have it. Not sure
if this is better. If the update fails and no one has it, except A,
then it could be a very long time before anyone notices it. Why,
because my WAN users kind of operate as islands. A knows A's data, and
C (over at a different WAN site) knows C's data but doesn't know A's
too well (and vice-versa) but still wants a look at A's data for
comparison purposes. But usually only A will notice if there is a
problem (at least in a reasonable amount of time). With the
incremental update method, A goes merrily along thinking everything is
fine until SOMEONE else points out the problem. But it is A who is in
the best position to notice it.

The alternative, which is to apply incremental updates, could mean that
updates from other user's (like WAN User B) are not applied to WAN User
A's local backend in a timely fashion (which is I think what you are
getting at). Why? Because right now MyApp only checks for updates
(regardless of whether they are incremental updates or zipped copies of
the production backend) at startup. There's no checking going on
during a user's session. If necessary, I can maybe add that later.
Right now I want to get this working smoothly.

My comments above are tentative. I don't disagree with you; I'm just
relaying my thought process. I may have missed something or be looking
at it wrongly. This has been a long and arduous process and I'm
starting to go bonkers :)





1. populate your update database to be zipped and uploaded to the
server and upload it.

2. download a copy of the server data file.

3. compare its data to the local data file and apply updates from
the server to the local database.

The only problem I can see is that you might download the next
server file before the server has applied your updates.

So, I don't understand what you mean when you say you replace the
entire back-end file -- surely you would lose the data you sent
to the server in step 1 above if you did that.

No. The synch app only creates the zipped file updates from the
production backend after an update is applied (and I'll probably
add capability to do it on a schedule as well). It then places
the zipped backend file in the Outbox folder on the server. Any
previous zip files which are in the OutBox on the server will be
replaced by the newly created zip files. These zip files are of
the production backend not any updates files sent by WAN users to
the server. Hence any updates that were previously applied to the
production backend must also be in the zipped file of that
production backend. When a WAN user starts up MyApp it looks in
this OutBox on the server for zipped files.
If it finds any, it downloads them and unzips them. These
unzipped
versions, then replace the local backend file for the WAN user.
Actually, this is what I understood you to recommend previously.
This works very similarly to my Launching program which currently
allows users to download the .mdb production backend and "replace"
the existing local PC backend (i.e. Kill and FileCopy). The zip
file download would be automated, but I'll still keep the option
to download (Kill and FileCopy) the production server backend MDB
to the local PC of the WAN user. That way if the zip code fails
for some reason (or IT somehow disables the zipper), there is an
alternative. This is what users are doing now and as far as I can
tell it has been working very well (latency issues aside).

I still don't understand how you can guarantee that you're not
losing data. The local back end would have the user's updates in it,
and when you synch, that pushes the updates into the MDB that will
zipped and uploaded to the server. All well and good. But if you
then download the server back end and copy it over top of the local
back end, you lose those edits before the new updates have been
applied to the server back end that you've just uploaded.

So, I'm having a very hard time understanding what you're doing.

In a folder on the
server, myapp code will look for a .zip file and if that file
has not already been downloaded (previous downloaded files are
stored in a table), myapp downloads the zip file and then
replaces the existing backend on the local PC with the unzipped
file. I felt it was important, however, to force my app to
wait for this asynchronous process to be completed, because
subsequently then, the app will link to the tables.

I understand that you are downloading the data and why, but I
can't figure out what you're doing with it. I'd think you'd be
keeping your local data file and also a local copy of the server
data, so that you could use the server data as comparison point
to get updates into your local data file.

Why not just replace the local data file with a fresh copy of the
production backend (via the zipping method described above). This
eliminates the need for the sync app to do anything other than
create zipped copies of the production backend. Seems simple and
pretty much guarantees that users get the current state of the
production backend at the time the zipped copy is made.

How do you not lose data? How can you guarantee that the file you
download has the updates the user last uploaded to the server?

2) When an update is found, code in the sync app will go
through each table and look for additions/edits and deletions
that need to be applied to the production back-end file.
Whether the record is an add/edit or deletion depends on the
value of the UpdateorDeleteFlag mentioned earlier.

How do you make certain they get applied in order? That is what
if you are updating from a remote user who deleted a record on
1/1 and then you get an update from another user who updated that
same record on 12/31? And what if that were an update, instead?
Do you just ignore the 12/31 changes? What if they should be
merged, instead?

Well, that's a good question. I actually posted in another forum
if there was some way to get Dir to get the files in folder in a
certain order.

There isn't. Dir() will return files in the directory order, which
is not necessarily in the order of creation. You'll need to sort the
list. Maybe the file system object has better tools for returning
sorted file listings.

The sync app loops through files in the INBOX folder on the
server to find all of the update files sent by WAN users. Each
file does include a date and time, so maybe I could somehow use
that.

But that's only the date and time of the *file*, not the data and
time of the updates included within it. If somebody doesn't synch
until a couple of days after they do updates, the file could be
dated several days after the actual data, and that means you can't
process them in order of file date -- you'd want to process them in
the order that the changes were made.

No, actually each update file has the user's logon ID and the date
which the update file was created in the file name. So I do know when
the update file was created and by whom.


After
an update file is applied to the production backend, a record is
written to a table so that that update is not applied again by the
sync app (as part of the Form Timer code). I may want to allow
admin people, however, to specify that the sync app should try
again on a file that the administrator would select. This
functionality has not been added.

Why not just move the file to an archive once the update has been
applied to the server?

Yeah, that's a possiblility.


--
David W. Fenton http://www.dfenton.com/
usenet at dfenton dot com http://www.dfenton.com/DFA/

.

Relevant Pages

  • Re: Homegrown synchronization
    ... How do you trigger the application of the updates to ... the server back-end you've just pushed up to the server? ... If the updating of the production backend is working correctly, ... WAN User A changes something in a table. ...
    (microsoft.public.access.replication)
  • Re: Homegrown synchronization
    ... server back-end you've just pushed up to the server? ... If the updating of the production backend is working correctly, ... WAN User A changes something in a table. ...
    (microsoft.public.access.replication)
  • Re: Homegrown synchronization
    ... How do you trigger the application of the updates to ... the server back-end you've just pushed up to the server? ... If the updating of the production backend is working correctly, ...
    (microsoft.public.access.replication)
  • Re: Homegrown synchronization
    ... How do you trigger the application of the updates to the server ... The only problem I can see is that you might download the next ... production backend after an update is applied (and I'll probably ...
    (microsoft.public.access.replication)
  • Re: Homegrown synchronization
    ... server back-end you've just pushed up to the server? ... If the updating of the production backend is working correctly, ... WAN User A changes something in a table. ...
    (microsoft.public.access.replication)
Quantcast