Re: Difficulties in deriving a truly universal GUI Scripting Language

Tech-Archive recommends: Speed Up your PC by fixing your registry

It depends on what the purpose of the scripting language is. For example, VBA is designed
to control an application through well-specified interfaces that return error codes if
things don't work (rather than popping up MessageBoxes, for example). You're addressing a
more complicated domain, controlling a program *as if* a human were working at it. In
this case, the interfaces are not cleanly specified, are very ad-hoc, and involve
interactions that depend on the presence of a real live human being to analyze the results
of taking an action, because the problems are presented in human-readable form rather than
in API-interface form. You are going beyond the normal scripting limits and entering new
and challenging territory here. Therefore, you are going to get new kinds of problems
that the ordinary scripting languages have never had to handle, or already handle poorly.

I tried using Microsoft Test many years ago. It was fairly difficult to get a decent test
script out of it.
joe

On Mon, 21 Aug 2006 11:28:09 -0500, "Peter Olcott" <olcott@xxxxxxx> wrote:


"Joseph M. Newcomer" <newcomer@xxxxxxxxxxxx> wrote in message
news:gueje293944memeaq5dunhlib7mimvesha@xxxxxxxxxx
I agree that you are looking at a seriously interesting problem. But having
solved it,
the solution has to be made viable by dealing with the second and third order
effects. As
I tell my students in my System Programming course "third order effects will
kill you".
First order effects are what we focus on. Second order effects are obvious.
It's those
third order effects that screw you over. So if you solve the
screen-inspection problem
but nobody can figure out how to effectively use it, you have a wonder
solution but the
solution is unusable, which is why I think you have to think real hard about
what you're
going to wrap this solution in to make it revenue-producing.

Trust me. I once worked for a company that produced a wonderful solution that
couldn't be
sold because all we had was a solution, not a vehicle that could make the
solution viable.
We lost a lot of money in that company, not because our solution was bad, but
because
nobody wanted a solution that was too hard to use. At least most of it was
not my
personal money, but there are some investors who are going to look real hard
at good
solutions without good delivery mechanisms. You're staking a lot on this, and
I don't
want to see you solve a problem that, while important and critical to the next
stage, is
not itself a usable solution because it relies too much on incredibly clever
programmers
to use it.
joe

It seems to me that all of the problems that you brought up in thses posts are
problems that any scripting language would also face. Since there are scripting
languages that are making money, these problems must not block profitability.
None of these other systems provide a solution that can be applied to every
program that runs under modern operating systems. In other words their solution
is far less universal than mine. Because of this I have estimated the my
technology could eventually dominate the product category. Do you see any reason
why my technology could not eventually dominate the product category, assuming
that I get a business partner with sufficient funding?


On Mon, 21 Aug 2006 06:31:43 -0500, "Peter Olcott" <olcott@xxxxxxx> wrote:


"Joseph M. Newcomer" <newcomer@xxxxxxxxxxxx> wrote in message
news:g247e255d7s7caa5n5s3agblhf4i6dao2u@xxxxxxxxxx
See below...
On Wed, 16 Aug 2006 16:27:42 -0500, "Peter Olcott" <olcott@xxxxxxx> wrote:


"Joseph M. Newcomer" <newcomer@xxxxxxxxxxxx> wrote in message
news:eqs6e25rktibgbk9b3cjofm7sk3vst30av@xxxxxxxxxx
Not all apps display "OVR". Only some do. I'm using one right now to
post
this that does
not report whether it is or is not in overwrite mode. Now, note that
doing
the experiment
destroys the state; for example, doing the change and even if you use
'undo'
as opposed to
putting back what was there, the app may now record the state as "file
changed" even
though it wasn't.

The key here is that for any ONE situation there might be a POSSIBLE set
of
answers, but
"Universal GUI Scripting Language" suggests that there will be ways to
handle
ALL
situations during the scripting.

What I mean by universal scripting language is that the language could
possibly
control any application that executes on modern operating system versions
regardless of the underlying technology of the application. For example,
scripts
could be written to control old MSDOS applications, Applications from MS
Windows
from 3.1 to the current version, applications using Java Swing technology,
fast
paced video games, HTML webpage based applications, all with the same single
basic technology.


What I'm trying to point out here are potential holes in your solution. A
script may have
to deal with asynchronous events (such as unexpected dialog boxes, or even
expected dialog
boxes that may come up at unexpected or unpredictable times), so
algorithmic
scripting
would have to deal with complex interactions such as catch-clauses (e.g.,
catch a
messagebox) that handle unpredictable throws (the act of popping up the
message box).
Then, you have to deal with whether or not such asynchronous events
support
resume
semantics, and if not, the scripting language itself might require
catch/throw
types of
operations to abort the scripting. None of this is trivial. Rule-based
AI
systems
probably come closest to the models that are interesting for this
joe

This would be left as a problem for the script programmer. The basic
technology
(scripting language) could easily handle it. The script programmer would
have
to
decide what to do when anything unexpected happens. One thing that the
script
programmer could do would be if anything unexpected happens, then send the
programmer a screen shot in an email. The programmer could then logon
remotely
and handle the problem, deciding whether or not to let the scripting
platform
simply remember this sequence by incorporating it directly into the script
automatically. The scenario will work many times, and would allow the expert
system rule-base to grow organically.
****
But if it is unrealistically difficult to write the scripts, the acceptance
of
such a
product is problematic. The key here is the phrase "could easily handle
it".
That's the
part I don't quite believe. If the script language is a classical language
in
the

I am only looking at one aspect of the problem, and you are addressing a
different aspect than the one that I need to look at. You are looking at how
difficult it is to write scripts in some cases. I am looking at how difficult
it
is to write scripts that are universal, meaning that they are capable of
directly interfacing with any program the will execute under modern operating
system versions.

C/C++/VBA model, writing an effective script is going to become an exercise
in
pain
management. This is why I suggest you do some experiments in writing actual
test scripts
for humans, and watch what goes wrong. And how they recover. Then figure
out
what their
mental processes were (parameterized rule-based systems is at least one
answer). Test
scripts designed to test "under stress" have to have complex handlers for
unexpected
events, especially if the purpose is to see how well the systems recover
from
errors.
Losing a whole night's test run due to some early failure in the testing
script is not
something most organizations can tolerate, because losing a day of testing
means losing a
day of schedule. Don't be led into the Turing Tarpit, where all things are
possible and
nothing is easy [Alan J. Perlis's characterization of algorithmic languages]

Oh, yes, and what about audio feedback the user may need to respond to?
joe
****


On Wed, 16 Aug 2006 14:23:12 -0500, "Peter Olcott" <olcott@xxxxxxx> wrote:


"Joseph M. Newcomer" <newcomer@xxxxxxxxxxxx> wrote in message
news:u6n6e25dli6fibcoheivk151fkjicoaeao@xxxxxxxxxx
See below...
On Wed, 16 Aug 2006 12:34:09 -0500, "Peter Olcott" <olcott@xxxxxxx>
wrote:


"Joseph M. Newcomer" <newcomer@xxxxxxxxxxxx> wrote in message
news:a2i6e2lulnaf3qvevprd9gvd4hj8f6etnk@xxxxxxxxxx
See below...
On Tue, 15 Aug 2006 20:13:35 -0500, "Peter Olcott" <olcott@xxxxxxx>
wrote:


"Joseph M. Newcomer" <newcomer@xxxxxxxxxxxx> wrote in message
news:9hp4e2lufbla2bpg0fkc0evn4c8r4jj32n@xxxxxxxxxx
A caret means whatever the app *wants* the caret to mean. The APIs
that
create
the caret

From your discussion below it looks more like this statement means
that
most
every aspect of the caret's meaning must be implemented from scratch.
****
True
****
It could
possibly be used in entirely unconventional ways.
****
I thought of another problem dealing with insert/overwrite mode, which
does
confuse the
interpretation, but yes, it *is* possible, although unlikely, the
caret
could
have other
meanings.
****
I would also further estimate
that almost no one ever violates the conventional meaning.
****
True, but as I pointed out concerning horizontal scrolling there can
be
serious and
unexpected behavior of the caret even within those guidelines.
****
I would also estimate
the operating system helps to enforce this conventional meaning to
some
extent,
****
Not really
****
not permitting oddly shaped or sized carets,
****
The caret can be a bitmap of arbitrary shape and size. Consider that
there
are often
special carets to indicate overwrite mode, and in the case of Word,
italic
mode. These
are entirely at the discretion of the application
****
and not allowing more than one
caret at a time.
****
This is the only limitation enforced by the operating system.
****
My estimate is that the caret is simply the MS Windows
equivalent of the MSDOS text cursor.
****
In the abstract, yes. At the detail level, it can vary. For example,
in
our
"stupid
glass tty" emulator, we used a caret that was a character block size.
****
Are these estimates correct?
****
As indicated above.

Now the overwrite problem is a potentially serious one. There is no
way
to
determine if
the window containing the caret is going to accept characters
(readonly),
insert them
("normal" mode) or replace the character to the right (or under) the
caret
(overwrite).
Even the rich edit control, in one of the most glaring API oversights
I've
seen in a long
time, has no way to either report its overwrite state or allow it to
be
explicitly set.
This can lead to interesting problems in scripting, particularly if
the
state
is set by
some mechanism other than coming up by default in insert state (for
example,
the
application comes up in the mode it was last in)..
****

In that case the script author would have to have the script do the same
thing
that people would do, look for an onscreen indication of the current
mode,
or
simply try it an see what happens.
****
Note that this adds whole new dimensions to the complexity of scripting.
Not
that it
can't be done, but emulating human behavior--perception and
feedback--has
been
notoriously
difficult for the field of Artificial Intelligence.
****

I am not talking about AI. All that the script programmer would have to do
if
they are automating for example MS Windows Word 97, is to look for "OVR"
in
the
statusbar. If they are working with an app that does not provide this sort
of
indication, then merely place the cursor in the textbox and type one
character
and see what happens, are the characters slid over, or did you just erase
a
character?

Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm

Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm

Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm

Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
.

Relevant Pages

  • Re: Difficulties in deriving a truly universal GUI Scripting Language
    ... "Universal GUI Scripting Language" suggests that there will be ways to handle ... What I mean by universal scripting language is that the language could possibly ... interpretation, but yes, it *is* possible, although unlikely, the caret ... we used a caret that was a character block size. ...
    (microsoft.public.vc.mfc)
  • Re: Difficulties in deriving a truly universal GUI Scripting Language
    ... "Universal GUI Scripting Language" suggests that there will be ways to ... interpretation, but yes, it *is* possible, although unlikely, the caret ... we used a caret that was a character block size. ...
    (microsoft.public.vc.mfc)
  • Re: IsValid(Object)
    ... It's a scripting language that I'm building, ... database might have been wiped. ... work in a strict order because it's a series of scripting commands. ...
    (borland.public.delphi.language.objectpascal)
  • Re: Difficulties in deriving a truly universal GUI Scripting Language
    ... that the ordinary scripting languages have never had to handle, ... "Universal GUI Scripting Language" suggests that there will be ways to ... This would be left as a problem for the script programmer. ... the caret ...
    (microsoft.public.vc.mfc)
  • Re: Difficulties in deriving a truly universal GUI Scripting Language
    ... problems that any scripting language would also face. ... "Universal GUI Scripting Language" suggests that there will be ways to ... This would be left as a problem for the script programmer. ... the caret ...
    (microsoft.public.vc.mfc)