Homemade data logging with Arduino ?

[woohoo]

HI all,
We have boldly gone whereto plenty of others have no doubt gone before...
My race engineer and I want to build our own data capture device and system using Arduino as the collection point.

Has anyone here tried using either an Arduino or Raspberry Pi for this and has any experiences to share ?

Considering three areas:

The physical sensors / wiring:
This is to be done my my engineer, as I have very little knowledge in this area.
So I don't even know what to ask
Is there anything we should be aware of when we look into this ?

Programming the device.
The Arduino device itself is programmed in its own language (which is based on C++ , argh!! ) and is not able to do multitasking and it is my task to write the code for it all to work.
We need to be able to sample data at 1KHz (1000 times a second) which should be within the theoretical limits of the hardware, but is it practical to use ?
Obviously the device is to sample all data from all channels, write to a file, then repeat, 1000 times a second.
So far I am counting 19 analog channels, 12 ODB channels, plus two internal inputs on the wishlist.

Any opinions on using OBD to sample some of the data ? (target car still has OBD bus for engine and gearbox)
For the Arduino to take in OBD, most of what I have see use the USB ELM327 protocol / device. Is it a good idea for high sampling rates with no data loss ?

Are there any projects out there that are interesting to watch / participate in ?
I have seen the "loguino" project, but the author has not updated his posts since 2014, so it is safe to assume it is abandoned (for reference https://github.com/irvined1982/loguino )

Computer software
This is, I think, the biggest hurdle. The companies involved with data logging / telemetry on a professional level that have created software for reviewing data keep their software as a exclusive to their line. You can either not legally buy the software without also buying their hardware, or if you can buy the software alone, it is not setup to accept foreign data, with no specs published for the file format / type.
Has anyone found a solution for this ? Either by making their data fit something like GEMS software (which is kinda cool, and we'd love that) or a software solution that is open ?
Creating your own is of course not viable option, even creating it as a rudimentary online app with PHP/MySQL would be a tall order. (Making a graph in PHP from MySQL is simple, but you have about 30 of them, and you have to have the option to zoom in / out, make math channels, have a circuit map overlay, and maybe a camera viewer...)

(personal rant: proprietary data formats should be outlawed. This is the 21st century FFS)

[andylaurence]

Watching with interest. There are other solutions to an online graphing app. Bastardise Grafana, perhaps? There are good graphing libraries that do most of the work client side. I like the idea of a unit with WiFi built in that connects to an access point in the paddock and uploads data directly. With a Pi, you could do a configurable dash too, as it has HDMI output. An app on your phone to do analysis or just the web interface. Why do I need a laptop to do data analysis when a touch interface would give a much better interface?

Why do you need 1000Hz sampling, BTW?

[SameSame]

MATLAB has a nice Arduino package that makes thing very easy. You can start trialling code by programming the Arduino through MATLAB. You can also programme an Arduino in plain C once you know the code works and you want to condense it. MATLAB does have a function to convert code to C but it is absolutely horrific at doing it, it creates such bloated C code. You can also use Simulink and run with hardware in the loop which is also a nifty feature.

Next off, do not ever try and use a Raspberry Pi for real time signal processing. It is just plain rubbish at anything of the sort.

1kHz sampling rate is extremely high. What are you trying to monitor?

If you want to log from the ECU check this out: http://forum.arduino.cc/index.php?topic=171058.0

[woohoo]

The online graphing apps are worth a look, certainly. I discounted them (maybe too quick) as I could not see how you could synchronise data, nor are they able to say, have a track overlay, or understand laps, corners etc. But it is a case of having something is better than having nothing.

The WiFi idea is good to remember. I have considered it, but:
Assuming we capture 32 channels at 10bit (which is four digits, or ca.1.2 bytes) 1000 times a second we get:
(32*10*1000)/8 = 40000 (bytes a second)
over a 2 minute lap that is:
(120 * 40000)/1024/ = 4.6 megabytes.
Though the above excludes headers etc. it should still be theoretically transmittable in the few seconds that a car is going through the pitstraight, even if the data has to be sent two-three times to cover off data loss etc. You could send it continuously throughout the lap, but then you would have to get something stronger than commercial WiFi to transmit signal 2km away...
(I think my math is ok, but if not let me know, its been a while since I did computer science )
It would also be difficult to program the Arduino to transmit at certain times, as it does not do multitasking. But even with a Raspberry Pi the problem is, what do you do with the data when it leaves the unit ? Again, you need a piece of software on the receiving end that will make heads and tails of the data that is being transmitted...

You can also connect displays to the Arduino, and I am hoping that if this project works half as well as intended, the next level will be to add a display for the driver to get crucial warning messages (temperatures, pressures etc. ) shift lights, and also the anticipated laps remaining with the fuel onboard (for endurance) and sector / laptimes. But that is secondary, for the actual detailed data analysis, you need to have it on a big screen and dwell into the details.

[woohoo]

MATLAB has a nice Arduino package that makes thing very easy. You can start trialling code by programming the Arduino through MATLAB. You can also programme an Arduino in plain C once you know the code works and you want to condense it. MATLAB does have a function to convert code to C but it is absolutely horrific at doing it, it creates such bloated C code. You can also use Simulink and run with hardware in the loop which is also a nifty feature.

Next off, do not ever try and use a Raspberry Pi for real time signal processing. It is just plain rubbish at anything of the sort.

1kHz sampling rate is extremely high. What are you trying to monitor?

If you want to log from the ECU check this out: http://forum.arduino.cc/index.php?topic=171058.0

Well the 1KHz questions are surprising me
MATLAB! I haven't looked at that in detail, thanks for that

[Mattdk]

I'd be careful using Matlab. I had a collage project that we had to program an Arduino using Matlab and another group tried to use an accelerometer and Matlab would get behind and couldn't process the data fast enough to give real time feedback. This might not matter for what you're trying to do, but thought I would give you a heads up about it.

[ChrisDanger]

What are typical logging frequencies of commercial systems? I did a quick search and the highest I saw was 50 kHz.

[andylaurence]

That's peak sampling rate. Why would you sample coolant temp at more than 1Hz? On the flip side, why would you sample suspension travel at only 20Hz?

[livinglikethathuh]

You'd only want 1 kHz sampling rate if you are measuring changes at 500 Hz frequency. I can't think of any parameter at 500 Hz, only the rotating parts (wheels, crank, intermediate gears etc.) will have a freq of 500 Hz.

Work out the highest frequency your suspension will be working at (kerbs at Vmax?) and multiply that by 3 to get your sampling rate.

In theory, having a sampling rate twice that of the thing you are measuring would be enough to capture the changes in that thing, but in practice, you need a bit more of that. Google Nyquist frequency.


Sent from my iPhone using Tapatalk

[Jef Patat]

The simple 8bitter is not up to that task. Check carefully if the hardware is capable of doing what you expect. Is the ADC capable of 1kHz for all channels? One thing I almost certain of is that the libraries are not capable of that. I think you'll have to look at the more capable variants (dure, teensy,...)

Sidenote: the Pi is not using a real time OS. For measurements you want realtime behavior.

[Tim.Wright]

I would add the requirement of a few digital/counter inputs for the wheel speed and RPM channel if possible.

I've found that 100Hz works well on most handling channels. Dampers might need to be higher depending on what analysis you do.

The problem of the post processing software is a perpetual one. I have searched, but have not found a decent free/open software that takes a text file as input without too much fuss. WinDarab (of Bosch) takes ASCII inputs but the channel names are limited to 8 characters so you need something to translate and shorten your channel names. For now I use Matlab at work and its a pain in the ass if you just want to quickly view, chop, and zoom through a few channels.

What was the reason for making instead of buying the acquisition system? These days you can get going with hardware + software quite cheaply. Self made electronics are often touted as being cheaper but in my experience, if you really add up the costs off all the little things you need to buy to arrive at a fully operating system (box, connectors, switches, memory, analogue filters, miscellaneous components) it always ends up being a lot more expensive than expected.

If you want to learn how to put together an acquisition system then it's a reasonable route. If your prime aim is to develop the car then in my opinion it's a waste of time & money in my opinion.

[LVDH]

I have once worked on a larger project using a Paparazzi UAV autopilot computer and I have done some small private things with an Arduino. In my opinion you should have a look at the Paparazzi autopilot system. It is much more expensive but will do much more of what you need, is faster and I think it has more analog and probably more accurate inputs.

[johnny99]

I'm the other guy involved in the project. I see what Tom is saying, and I had similar thoughts too. Chatting to woohoo and Frank (the 3rd in the group) put the worry to rest. Sampling rates will need to be 500 Hz, maybe 1 KHz for suspension but other channels will need sampling a lot less, 100 hz for wheel speed, 10 for brake pressure, steering input etc. I have wired in both front wheel speed sensors which are magnetoresistive. They switch from 0.8 to 1.2V, which is a digital signal. I have got a brake pressure sensor from a BMW, 0-5V, steering input is 0-5V linear pot, throttle position is coming from the throttle pedal, and engine speed also. Add a 2 axis acclerometer and we have g measurement. I have spent less than 100 euro so far. If this works we can then add other channels and some math channels too.

John

[woohoo]

Hi Guys,

thanks for the suggestions, and keep them coming!

The teensy board is interesting (could not find the dure module) and it has Arduino libraries available, which is good.

WinDarab is also something to look into, its available for download from Bosch motorsport website, so that is a good thing.

In regards to the sampling rate, I have found this thread at the arduino forums stating that sampling rates of 125KHz (!) are theoretically possible. of course that is the THEORETICAL rate, not including things like multiple channels (reads) writing data to file, etc. etc. I have not tested it yet, so that will be interesting...
http://forum.arduino.cc/index.php?topic=6549.0

I understand the concerns about affordability, but with a homemade system you can scale it as you see fit, and add features as you go along to create your bespoke solution that you cant anywhere else, and as johnny99 said, so far the costs are low. And its a lot cooler to make it yourself

[Jef Patat]

Sorry, I ment Due: https://www.arduino.cc/en/Main/ArduinoBoardDue

There are so many boards these days that choosing one becomes an art in itself.