Re: Multi Threading Options

An idea I tried was to build in an object-based file system back around 1977. The cord
idea was that there was no file system at all involved in a program development
environment, except as a way to store information. There was, to the user, only a
collection of functions (we weren't using object-based languages). You could have multiple
views on the code; for example, a logical view would group the functions in the way we
currently think of as "modules", but without the limitations of files. Another view would
be a "disk representation" view, which determined what underlying objects (call them
"files") saw in terms of grouping. Thus, you, the programmer, worked in terms of "views",
and had no interest at all in the underlying byte stream sent to a compiler. In this
sense, it was going after your idea of computing an "optimal build" strategy. That would
just be another view. The construction of that optimal view should not be of any concern
to the application developer. The project died because what I was attempting would have
taken more time than we could budget to the project. Utlimately, the only budget we had
was about 10% of my time.

I had built a system like this that managed my LISP environment;, in the 1973-1975
timeframe. I could have multiple views of my LISP functions, such as the modular grouping,
the slice-across-call-graph view, and the file view. There was an
alphabetical-by-function-name view useful for printed listings. It made the whole LISP
environment, where I had thousands of functions, manageable. It was the first serious
programming environment I built. The point was that it decoupled the notion of file and
view, something seriously missing in modern programming environments. Even C++ never got
it right, having only two views: "public" and "protected", and the slightly strange
"private" could be construed to be a third view. Key concepts like precompiled interfaces
are missing from the language. This makes it impossible to do true exported interfaces,
because methods shared across internal components must be declared as "public".

We have not graduated very far from punched cards in the ways we think of organizing our
programs. One reason I am a splitter is that I choose views that make sense from the
programming viewpoint, and I never worry about the build time because the projects I work
on, typically 100K-150K source lines, the compilation time is irrelevant for a full build.
It only matters for an incremental build. I want incremental builds to be as small as
possible, so the cost of splitting, using as few .h files as possible in a build (the
smallest possible number), thus minimizing the number of files to be rebuilt on an
incremental build, is what optimizes my time as a developer. I haven't had a full build
take longer than five minutes in the last 15 years. and the latter was a project with
three people with over 250K source lines of C++. I did a full rebuild once a week or so.
The rest of the time incremental builds were very tiny. So I can't really get excited
about repartitioning code to optimize compilation time of full builds. Combining files is
a good way to optimize full builds and pessimize incremental building.

We haven't even achieved the simplest concepts in our program construction, let alone the
interesting organizations that matter. We can't even (a) embed PowerPoint, Visio, and any
other document type we want to as comments in our code and (b) compile a program written
in PowerPoint and produce executable code (I know at least three research projects working
on this right now, but they are far from being more than academic toys). We have weird
ways of trying to annotate code so documentation can be extracted, without considering the
fact that as long as we are limited to thinking of code as sequences of textual bytes we
have missed the point (for example, why can't I write my source files in Word? Yes, I know
the current detailed reasons, but in a truly modern programming environment these would
not have any relevance. Oh yes, the Xerox Mesa programming environment got all of this
right in the late 1970s, because they stopped thinking of input to the compiler as being
card images).
joe

On Thu, 2 Jun 2005 23:53:06 -0500, "Doug Harrison [MVP]" <dsh@xxxxxxxx> wrote:

>On Thu, 2 Jun 2005 18:05:49 +0100, Mark Randall wrote:
>
>> In Update:
>>
>> I have now merged about 15 of the files, unfortunately merging the rest
>> would create a managability nightmare beyond all proportions. I have also
>> turned off both antivirus - Build time has dropped to approx 1 minute 20.
>
>Just out of curiosity, which made the difference? If I ask you to vary one
>thing at a time, it's with hopes the advice will rub off on me. :)
>
>I would bet someone out there has created or at least researched a program
>that combines dependency and context analysis with profiling and uses this
>information to formulate a "build plan". The idea would be to feed the
>compiler the largest chunks of source possible, combining pieces from
>different source files, in order to minimize per-file overhead for multiple
>file builds while not killing true incremental builds, and while still
>preserving language semantics. Without such a build system, you're reduced
>to devising something like this by hand:
>
>// bundle1.cpp - this is what goes into your project
>#include "file1.cpp"
>#include "file2.cpp"
>#include "file3.cpp"
>
>This is a pain in the neck and rarely worth doing, but when it works, it
>can help a lot.
>
>> I have also checked that PCH is included, and it is being used...
>>
>> All headers are #pragma once'd
>>
>> Futher changes:
>>
>> Removed afxwin.h from all classes - No change.
>>
>> It could just be my huge use of afxtempl.h that is causing it? Although this
>> should be compiled just 1 time and placed in the stdafx.
>
>Templates will be instantiated as needed wherever they're used, which
>definitely contributes to compile times. You can try identifying the
>specializations you use and explicitly instantiate them, which in theory
>should reduce compile times. Whether it will be worthwhile for your usage
>pattern I couldn't say.

Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
.

Relevant Pages

  • Re: e to the i pi
    ... a computer without a programming environment - it's just a glorified ... much support it has for Windows-specific programming, ... I think actually the open source restriction applies to anything based on ... I installed the free C compiler. ...
    (comp.lang.c)
  • Re: [OT] e to the i pi
    ... a computer without a programming environment - it's just a glorified ... much support it has for Windows-specific programming, ... I think actually the open source restriction applies to anything based on ... I installed the free C compiler. ...
    (comp.lang.c)
  • Re: [EGN] Numerical Accuracy
    ... The advice worked because in fact the Microsoft compiler had a bug. ... The capabilities of the optimizer are not so ... The language standard itself says ... of "programming" as a nonprofession. ...
    (comp.programming)
  • Re: Brian Kernighan, maybe Im not worthy, maybe Im scum
    ... and just hone in on the stuff related to programming and this newsroup] ... moron that was taken from optimization which does hoist when to do so ... compiler design and optimization, including my 1976 text in graduate ... loop in a language in which the designers messed up, ...
    (comp.programming)
  • Re: Javas performance far better that optimized C++
    ... The compiler is extremely stupid. ... no memory leaks are guaranteed. ... However I have GC in my .NET programming. ... "C.9.1 Automatic Garbage Collection ...
    (comp.lang.cpp)