Re: Data transfer for real time analysis

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

From: Paul Carpenter (paul$_at_pcserv.demon.co.uk)
Date: 04/28/04 

Date: Wed, 28 Apr 2004 22:57:01 +0000 (UTC)

On Wednesday, in article <c6mjmn$r5j$1@news.iucc.ac.il>
     pinit@ee.bgu.ac.il "Pini" wrote:

>The system is developed for academic research of biological signals.
>
>Each channel sampling rate is 25KHz. Each data frame holds 25 samples. The
>data will be processed in frames.

Considering like others you mentioned 25MHz before, I don't really know
what you are really trying to achieve.

How many bits in a sample?
How many channels?
How long do you really need to record all this data for analysis?
Is the analysis the control loop, or post analysis to confirm the
  control loop actions?
What is the analysis output - a decision, vector, matrix, extra
  data stream to store?

>I' planning to use a PC with 4 CPU's. One fore handling the data
>communication and memory allocation, one for handling the HD storage and the
>2 last CPU will be for number crunching. A second alternative is to use a
>PCI board with multiple DSP for data analysis and the PC will do the data
>fusion from all the channels for closing the loop (and data storage).

Why put all the processing and storage in ONE BOX?
As others have said they all share the same datapaths,s o that becomes the
bottleneck.

>We can use FPGA at the sub-units if required.
>We can not afford any data package loss.

Why not store the data there for download after what is likely to be a burst
of data.

>Regarding the selection of the communication protocol:

As I doubt that you really want an hours recording of for example

        4 channels of 8 bits = 32bits = 4 bytes
        sampled at 25KHz = 100,000 Bytes/second

        For an hour = 3600 x 100,000 = 360,000,000 Bytes/hour
                                = just over 343MB!

This does not include saving of data analysis.

Double it for 12/16 bit sample widths, add suitable multiplying factors
for increase in channels or longer data storage times.

>TCP/IP:
>What will be worst the latency of each channel if I use TCP/IP Hub
>(100MBit/sec
>or 1000MBit/sec)?
>Will the use of windows CE guaranty the Ethernet frame overall latency?
>(Did someone tried the real-time TCP/IP extension from:
>http://www.vci.com/products/windows_embedded/rttcpip.asp )

I would only consider this on data sets of this size when using:-

        a) burst capture, then store later
        b) multiple connections on separated network segments
           to achieve the packet rebuilding

>I2C:
>What HW should I use on the PC side?

Practically none as STANDARD controllers for PC use the slow bus speed of
100KHz, whereas most 400KHz and 2MHz bus designs are usually from FPGA
or dedicated controller designs with short signal length within a subsystem.

>What about SMBus? What should I use on the PC side?

>USB:
>I can use the USB protocol but I have a latency of 1msec (can I reduce this
>latency)?
>What is the USB Hub latency?

Depends on the USB configuration and how many layers you have and the
software controlling and number of devices on the bus.

Personally I cannot see that the data analysis of this data set size
realistically being used in a ONE BOX situation.

If the analysis is post experiment run, I would consider each unit having
its own large memory, using some form of seperate communications for the
control linking, and after experiment run download the data sets to the
storage and analysis unit. This method is the basic method of high speed
video capture to ensure no missing picture frames in a sequence.

If the analysis is part of the control loop put DSPs in each sub unit
to process the data, then store original data and results in large memory
for later download. Still communicate results from frames/analysis to either
a central unit, with a reduced data set or analysis decision results.

I think you need to revisit actually what you are trying to achieve,
what you actually need and when and where it is needed.

The current outline looks like a wish list for an ideal world, onto
a single PC as that is available easily, but may not be the ideal
solution.

>??"Pini" <pinit@ee.bgu.ac.il> ??? ??????
>news:c6h98p$3jm$1@news.iucc.ac.il...
>> hi,
>>
>> We are planning a data acquisition system with 4-12 sub units. Each sub
>unit
>> produce data of 25MBit/sec. The distance between the boxes and computer is
>> about 2 meters.
>> I'm willing to transfer and analyze the data in real-time. I'm willing to
>> close a control loop in 1-5msec (I. wish 100usec ......).
>> What is the recommended serial protocol?
>> The sub units will be in-house made the host computer for data analysis
>and
>> storage will be a PC (probably with raid HD).
>> What communication HW should I use in my sub units?
>> What communication HW should I use in the PC?
>>
>> Regards
>> Pini
>> 

-- 
Paul Carpenter		| paul@pcserv.demon.co.uk
<http://www.pcserv.demon.co.uk/>        Main Site
<http://www.gnuh8.org.uk/>              GNU H8 & mailing list info.
<http://www.badweb.org.uk/>             For those web sites you hate.

Relevant Pages

  • Re: Data transfer for real time analysis
    ... Is the analysis the control loop, or post analysis to confirm the ... >fusion from all the channels for closing the loop (and data storage). ... video capture to ensure no missing picture frames in a sequence. ... >> What communication HW should I use in my sub units? ...
    (comp.os.linux.embedded)
  • Re: Data transfer for real time analysis
    ... Is the analysis the control loop, or post analysis to confirm the ... >fusion from all the channels for closing the loop (and data storage). ... video capture to ensure no missing picture frames in a sequence. ... >> What communication HW should I use in my sub units? ...
    (comp.arch.embedded)
  • The Most Powerful Weapon In The World
    ... Strategic Communication Laboratories and the war for your mind ... In a world where the perception is the reality, ... total control over their citizens. ...
    (alt.gathering.rainbow)
  • Re: DNA carries information
    ... signals that require some 'interpretation', ... we could say that the species is ... communication and control theory. ...
    (talk.origins)
  • =?windows-1252?Q?Re=3A_Rupert_Murdoch=3A_=93Internet_Will_Soon_Be_Over=94?=
    ... The government is mainly a tool of the elite and corporate interests. ... I do half of my business communication online, emails, contracts, ... People are easier to control when they have no ties. ... don't like corporations, adjust your spending behavior. ...
    (rec.music.classical.guitar)