Homemade data logging with Arduino ?

[woohoo]

Sorry, I ment Due: https://www.arduino.cc/en/Main/ArduinoBoardDue

There are so many boards these days that choosing one becomes an art in itself.

Thanks Jeff! I think this is the board that is replaced by the Mega 2560 ? We have the 2560 for use
(and a UNO for backup / comparison...)

[johnny99]

We are using the Mega 2560, http://www.arduino.cc/en/Main/ArduinoBoardMega2560

John

[nzjrs]

Arduino are not serious microcontrollers, nor a serious organisation. Their stadard library is poor quality. Arduino are playdough for engineers.

Take a ARM cortex, stm32, mBed or something.

[graham.reeds]

Arduino are not serious microcontrollers, nor a serious organisation. Their stadard library is poor quality. Arduino are playdough for engineers.

Take a ARM cortex, stm32, mBed or something.

Horses for courses.

If they were thinking of making this a professional logger with sales and support, etc. then you might have a point.

As a learning exercise then it is perfectly suited.

I once looked at making a data logger for my kart. The estimated cost was about £400 but would have the capabilities far beyond what commercial ones could do for twice the price. However I had to make it and I would have had to implement the peripherals also. In the end it was easier to buy one rather than make my own so I will be watching with interest.

[johnny99]

Thanks graham.reed. It all started with a chat and we ran with it. Why not, even if it fails, we had fun and learned a huge amount. total budget so far is under 100 Euro and many interesting Saturday afternoon's. If we get something working and it leads woohoo improving his lap times, bonus. If the basics work we can move to better equipment

John

[Brian Coat]

I know the Arduino has built in ADC but maybe the Raspberry Pi could do this well if mated to a good ADC chip.

People on here said the RPi is not up to the job but I've seen it used for very resource-intensive DSP work - e.g. active noise cancellation. That is some serious real-time stuff.

It is a serious device and there is a massive expert community.

You can get all kinds off add-ons for Pi like ADC and i/o CPLD to help with data logging at very high speeds.

Not saying Arduino is not up to it, though.

[johnny99]

Thanks Brian, from checking out what had the best ability and easier to use, we choose the Arduino, we may be wrong, who knows yet. Massive support for the Arduino too, let's see what happens.

John

[VL400]

I have used two arduino based boards in vehicle data loggers, first one used a teensy 2++ and the current one a teensy 3.2. Both have a whole bunch of analog and digital inputs, and an interface to the cars data bus. The teensy 3.2 also has a serial data interface for connection to other modules not on the bus. All analog and digital samples are then added to the vehicle data to form a larger data frame for playback on the PC software TunerPro RT. I also use this for calibration adjustments on various modules in the car, so the arduino data logger records to an SD card in the file format for that software.

I am not recording at the frame rates you are proposing due to the vehicle data being comparatively slow, but am sampling in the KHz range and then using adjustable filter coeffs for fast and slow moving sensors.

One thing to note is the SD library for the arduino is fairly slow to write, you may find gaps in your sampling during a write.

I can provide more info, but you are on the right track

[nzjrs]

I'll also second the teensy recommendation.

[Jef Patat]

I know the Arduino has built in ADC but maybe the Raspberry Pi could do this well if mated to a good ADC chip.

People on here said the RPi is not up to the job but I've seen it used for very resource-intensive DSP work - e.g. active noise cancellation. That is some serious real-time stuff.

It is a serious device and there is a massive expert community.

You can get all kinds off add-ons for Pi like ADC and i/o CPLD to help with data logging at very high speeds.

Not saying Arduino is not up to it, though.

There's a basic difference between arduino (and the likes) and the Pi (and the likes). The Pi runs an OS and the micros do not. For serious datalogging you need as little jitter as possible, of course that is dependent on the task at work, but still, Linux is not meant to be realtime. Certain amounts of jitter makes some postprocessing / correct interpretation impossible.

I work for a company that makes test systems for the off road vehicle industry, hard realtime is everything to us. We use EtherCAT with distributed clocks which allows us microsecond jitter. This makes Fourrier analysis possible.

I didn't mean to say one is better than the other, it depends on so many factors. I just wanted to say this might be an argument in the decision process.

[Brian Coat]

Thanks, Jef.

I think these are very good points.

The Pi runs whatever you load onto it and you are so right about the standard OS being slow.

I forgot to mention that the active noise cancellation application using RPi does not use Linux OS. It just uses compiled C code, including the standard libraries which are available for the hardware interfaces etc.

And it is done this was because the OS is too slow for this very high speed application.

Sorry, an important omission, that.

[Jef Patat]

Are you talking bare metal programming now? The GPU boot loader takes care of starting the system to end with loading from the SD card, so with the correct knowledge and experience you can run 'whatever'. But that's not the intent of the platform, as such you will have minimal community support. I don't think the TS has this kind of competences.

[Brian Coat]

Yes I am. But there are examples of pretty fast data logging using Raspi+Linux, albeit with other hardware like CPLD to unburden the PI+OS a bit.

I think that is a long winded way of me saying "you are right" (crawls back into box)!

[woohoo]

HI,
So, to give an update I have been playing with a Arduino Uno, and have gotten it to do the basics, read analog inputs and spit them into a CSV file. So far so good.
Using very simplistic C++from IDE (the native Arduino language) I was able to get it to extract info at an average of 1.5 milliseconds (ca. 500Hertz)
So far from the desired 2K but not.. hopefully.. not a million miles away. I have not yet done any speed optimizations, so there is, again, hopefully some space there to speed up the data collection.

Now then, in order to get the date to work with anything, I have the following question:

Does anyone know the CSV order for dlog99 ?
Our ideal software, GEMS opens dlog99 files, which in turn opens CSV files. These however have to be formatted a specific way.
Not having a car with a logging system using dlog99 / GEMS means it is difficult to second guess the formatting.

But, any other solution is also welcome

[Tim.Wright]

Why do you need a sampling rate of 2khz? Do you even have a sensor that is responsive at such a high frequency?