Any embedded engineers around?

[alexx_88]

Given that this is one of the better engineering forums around, I was wondering if there are any members working in embedded development, automotive or not. It would be interesting to share experiences about the tools that we use, projects that we work on or even anecdotes from the time in the field.

I'll begin. Started working on firmware about 4 years ago, currently using ARM Cortex-M4 chips and the MQX operating system to develop a new aftermarket ECU for 4-cylinder engines. As a funny story, in one of my first projects, we've chased for about 3 days a bug in the firmware update routine in which a couple of bytes (<5) in 30 KBytes written, were corrupt. After banging our heads against the walls, we've discovered that the problem was that the flash controller's clock was bigger than the maximum permitted by about 10 KHz.

[langwadt]

Given that this is one of the better engineering forums around, I was wondering if there are any members working in embedded development, automotive or not. It would be interesting to share experiences about the tools that we use, projects that we work on or even anecdotes from the time in the field.

I'll begin. Started working on firmware about 4 years ago, currently using ARM Cortex-M4 chips and the MQX operating system to develop a new aftermarket ECU for 4-cylinder engines. As a funny story, in one of my first projects, we've chased for about 3 days a bug in the firmware update routine in which a couple of bytes (<5) in 30 KBytes written, were corrupt. After banging our heads against the walls, we've discovered that the problem was that the flash controller's clock was bigger than the maximum permitted by about 10 KHz.

I find it hard to believe that exceeding the max clock a bit is the root cause of your problem, remember that maximum is set to guarantee that it work work over the the full voltage/temperature range, IC process variations and that it will last for ~10 years after being programmed

[jbolenbaugh]

I work at a large American manufacturer. I worked in hybrid controls for three years prior to my current assignment. As many in the industry know there is a large switch from hand coding software to code from models. Having been able to be a part of from scratch Simulink developed advanced controls was an amazing learning opportunity and changed my problem solving approach at a fundamental level. I have since moved into a different group working on powertrain cooling and thermal for hybrid and electric vehicles. Additionally I work on the side for a sports car team in the TUDOR United Sports Car Championship / Continental Sports Car Challenge. All of this to say that I hope to be able to work in vehicle development for a race team as the main job at some point . Its also reassuring that dedicating my work to hybrid vehicle controls and thermal for hybrid vehicle is looking like a good decision. I've always hoped to be able to work in high performance hybrids and these are the two main challenges as we have seen form this years F1 powertrain development.

EDIT: I just realized that after lurking for so long this is my first post! So Hello everyone. I have to be careful about posting on technical forums do to my employment agreement, but I figured this is an ok exception.

[alexx_88]

I work at a large American manufacturer. I worked in hybrid controls for three years prior to my current assignment. As many in the industry know there is a large switch from hand coding software to code from models. Having been able to be a part of from scratch Simulink developed advanced controls was an amazing learning opportunity and changed my problem solving approach at a fundamental level. I have since moved into a different group working on powertrain cooling and thermal for hybrid and electric vehicles. Additionally I work on the side for a sports car team in the TUDOR United Sports Car Championship / Continental Sports Car Challenge. All of this to say that I hope to be able to work in vehicle development for a race team as the main job at some point . Its also reassuring that dedicating my work to hybrid vehicle controls and thermal for hybrid vehicle is looking like a good decision. I've always hoped to be able to work in high performance hybrids and these are the two main challenges as we have seen form this years F1 powertrain development.

EDIT: I just realized that after lurking for so long this is my first post! So Hello everyone. I have to be careful about posting on technical forums do to my employment agreement, but I figured this is an ok exception.

Welcome, first of all!

What you're saying is very interesting, as it seems to be a general trend right now. Without going into too much details, how much of your work right now is still traditional embedded development and how much is just creating the right models to solve a problem and then letting Simulink create the code for it? I must admit that I don't know very much about this approach to controllers, so I'll try to simplify my view. If I am to create a traditional engine controller, I normally allow the user to customize its behavior using different maps that are editable through a PC application. If I understand correctly, the Simulink model actually replaces those maps, so tuning such a controller actually means changing parameters inside the models?

You said that moving from the traditional approach to Simulink fundamentally changed your approach. Could you elaborate please on the advantages/disadvantages of this and how you see the whole industry going forward.

In the end, sorry for picking your brain so aggressively, but I haven't found a lot of relevant literature on what goes on in this field (I'd appreciate some suggestions), only the same old articles, with the same old "don't do too much in ISRs".

Given that this is one of the better engineering forums around, I was wondering if there are any members working in embedded development, automotive or not. It would be interesting to share experiences about the tools that we use, projects that we work on or even anecdotes from the time in the field.

I'll begin. Started working on firmware about 4 years ago, currently using ARM Cortex-M4 chips and the MQX operating system to develop a new aftermarket ECU for 4-cylinder engines. As a funny story, in one of my first projects, we've chased for about 3 days a bug in the firmware update routine in which a couple of bytes (<5) in 30 KBytes written, were corrupt. After banging our heads against the walls, we've discovered that the problem was that the flash controller's clock was bigger than the maximum permitted by about 10 KHz.

I find it hard to believe that exceeding the max clock a bit is the root cause of your problem, remember that maximum is set to guarantee that it work work over the the full voltage/temperature range, IC process variations and that it will last for ~10 years after being programmed

You were right. I've looked in the git repository and the value was exceeded by much more, the wrong version had the flash clock at 8 MHz, exceeding the recommended maximum value by almost 20 times. I don't know why I've remembered just 10Khz.

[jbolenbaugh]

Welcome, first of all!

What you're saying is very interesting, as it seems to be a general trend right now. Without going into too much details, how much of your work right now is still traditional embedded development and how much is just creating the right models to solve a problem and then letting Simulink create the code for it? I must admit that I don't know very much about this approach to controllers, so I'll try to simplify my view. If I am to create a traditional engine controller, I normally allow the user to customize its behavior using different maps that are editable through a PC application. If I understand correctly, the Simulink model actually replaces those maps, so tuning such a controller actually means changing parameters inside the models?

You said that moving from the traditional approach to Simulink fundamentally changed your approach. Could you elaborate please on the advantages/disadvantages of this and how you see the whole industry going forward.

In the end, sorry for picking your brain so aggressively, but I haven't found a lot of relevant literature on what goes on in this field (I'd appreciate some suggestions), only the same old articles, with the same old "don't do too much in ISRs".

When I was working in hybrid controls I would say about 75% of the work being done was code generated from models, there are trade offs resulting from this such as code that isn't as efficient as well executed hand coding. This results in more computational power and memory. The portions that were hand coded were usually items that ran at high loop rates where efficiency was key, or the boot code portions. The process was effectively modeling the system you wanted to control, and creating your control logic visually in a model format. From there Simulink with some help can generate C-code that can be flashed into the controller. The software retains conventional calibration access that can be used in vehicle development and tuning, that you spoke of. So to the controller there is no functional difference in code generated from models and code, aside from it having to work a bit harder due to the slightly inefficient code.

The reason why the code from model method of controls is growing is because it allows you to run simulations with the EXACT content that will be in the controller in a much more simulation friendly environment. This allowed most of the effort to be put into problem solving rather than fighting to get handcode to execute exactly in the way your simulation desired. This is allowed me to change my way of problem solving dramatically. What I could think of and model in simulink I could have the controller execute exactly, rather than having to have someone translate my work into c-code and try to find what the small differences and nuances were between the two.

[Jef Patat]

there are trade offs resulting from this such as code that isn't as efficient as well executed hand coding. This results in more computational power and memory.

This is very doubtfull at least. We have done experiments on that and both were equally efficient. Testing was done 3 years ago. What is true is that the complexity of the whole increases. It's easier to make better models in model based design than it is in code. The better models are more complex and that results in higher demands on HW.

The reason why the code from model method of controls is growing is because it allows you to run simulations with the EXACT content that will be in the controller in a much more simulation friendly environment. This allowed most of the effort to be put into problem solving rather than fighting to get handcode to execute exactly in the way your simulation desired.

That is a very important aspect. Hardware in the loop and software in the loop.

[Jef Patat]

As a funny story, in one of my first projects, we've chased for about 3 days a bug in the firmware update routine in which a couple of bytes (<5) in 30 KBytes written, were corrupt.

Didn't you use some kind of CRC?
Side note: I think my record was 5 weeks to track an error in a WinCE driver IST. Not proud of it, I made the error. But still, we were looking into it with 3 people for 4 weeks and eventually I found my own error.

[jbolenbaugh]

This is very doubtfull at least. We have done experiments on that and both were equally efficient. Testing was done 3 years ago. What is true is that the complexity of the whole increases. It's easier to make better models in model based design than it is in code. The better models are more complex and that results in higher demands on HW.

Hmm, you must not have had very good hand coders . Just kidding of course. When we converted what used to be hand coded to "functionally identical" models, the generated code was then flashed back into the controller and throughput was measured and it appeared from our measurements that the efficiency had decreased, and code size had increased. When we are talking about an entire controllers worth of software being converted the changes were larger than anticipated.

Interesting to see the differences from another perspective though.

[Jef Patat]

I can understand what you are saying, it doesn't surprise me. I must say, that my previous statement was made on the base of a quite basic test (well ... define basic), not the complete SW into model and then compare. The fact is that the model is made by humans, so it is prone to errors, inefficiencies, … Plus the fact that your typical modeler is not SW engineer. And one needs to know the details of how the simulink blocks are mapped to code patterns, which is probably the biggest contributor to inefficiencies. In fact you are probably right, an experienced SW engineer will be unbeatable. But, to quote my colleague: looks like we have the same situation as when people were moving from assembler to C: you compromise speed and (especially) efficiency for ease… And how many of us still master assembler?