|
I'm running into a wall getting my template based library covered through testing. It's not so much runtime issues I'm concerned about, but compile warnings and even errors that may only appear when certain components of the code are used a certain way - or on a certain platform.
I ran into a platform specific compile warning in a release I had *just made* because I don't have unit testing in place for each device.
I *can* put unit testing in place for each device, but it would require a PC with enough USB ports free, running PlatformIO with every single device hooked up (about a dozen right now)
I figure it would cost me around $400-$500 to get that all set up. For a hobby project that's considerable, but worse is the space it would take up. I don't have room for another PC.
I'm not sure what to do about this: 
To err is human. Fortune favors the monsters.
|
|
|
|
|
One does not test one's own code. 
|
|
|
|
|
One does when one doesn't have contributors.
One makes do with what one has.
To err is human. Fortune favors the monsters.
|
|
|
|
|
Not sure who sees the replies but here goes.
How about this plan?
1. Determine what all the units have in common.
2. Determine which units have the least in common.
3. Pick one or two of those units to develop, test, etc.
4. Develop a pass/fail test(s).
(not easy, but you should know success and failure)
5. Continue unit testing and code development until the code meets
the pass/fail criteria for all the chosen unit(s).
6. Select another unit and begin the process again
while assuring the tested units still handle pass/fail criteria.
not everything needs to be done in parallel, just make the
unit change-out be as smooth and consistent as possible.
This is sort of high level and not original, but got me thinking.
|
|
|
|
|
I can't cross section the similarities and differences between the units because it amounts to differences in the standard header files and Arduino hal headers.
For example, I caught a warning because size_t is unsigned on certain platforms. I didn't catch it because only one platform - one I recently added support for, has size_t as unsigned.
Another more major issue, was in some build environments, some jokers defined "putc" as a macro, making my virtual function of the same name break the build.
Then there are differences in the actual SPI bus characteristics which are part of the Arduino HAL. That's largely opaque to me.
Now first two things can be caught purely through compiling. I don't need the devices. But if I want to test the drivers on the devices and catch that last thing i mentioned, I do need those devices hooked up.
And really there's about a $50 cost difference between hooking them up in parallel or doing one at a time manually. I'll take the former, but issue again is space, unless i start wall mounting everything. still need room for a PC. Or at least a second USB hub. I already have like 30 com ports registered on this machine though. Meh.
To err is human. Fortune favors the monsters.
|
|
|
|
|
Many unit tests run against “mocks”, so if you could mock your hardware somehow…
Do you need the hardware? Or just need to know that the driver would emit the same sequence onto the bus?
I guess the problem is with cross compiling and the like.
One of the last projects I did, I had 100% coverage on all of the utility functions that built the data structures for the execution, but I did not bother unit testing the main flow.
Different note on the unsigned problem. If that does cause issues, then it seems like there should be some execution time checks that would alert you to these things.
|
|
|
|
|
englebart wrote: If that does cause issues, then it seems like there should be some execution time checks that would alert you to these things.
I can't afford them, and there's no such thing as a debug build on this gear, for a number of reasons.
It was a stupid mistake on my part:
size_t whoops = 10;
for(int i = 0;i<whoops;++i) {
}
To err is human. Fortune favors the monsters.
|
|
|
|
|
I am thinking of a validate() function.
You never even have to call it. It just needs to be compiled for the case you documented or other edge cases.
You could omit validate() as you get closer to final product.
Once you discover an issue, you need to fix it and you need to prevent it from happening again. ( team motto)
|
|
|
|
|
Interesting, although I'm not sure if I never call it if it will compile what it uses. The g++ compiler is awfully good at avoid work it doesn't have to do.
To err is human. Fortune favors the monsters.
|
|
|
|
|
BTW: when I do macros in C, I preface their names like my initials JM_putc. Avoids nasty name clashes especially with other people's code.
Wow, you do have complex situation. Your software and hardware is so close together, I see why it's hard to compare and contrast. I was speaking mostly of hardware comparisons, but that line with software in your case is pretty thin. If you can do two hardware units at a time, then yes that is preferable. Still need good pass/fail criteria to keep testing consistent.
Hope you have good AC. It gets mighty hot here in summer, so 2 heavy-duty number crunching desktops and 2 laptops running at once can be a challenge. I also have very beefy UPS units for each desktop. A must. Space is an issue as well so I completely understand. You definitely need more with that many units involved.
One giant workbench with power everywhere. My dream.
|
|
|
|
|
I think whoever put that macro in intended it to stand in for the C function putc, but why it's a macro and not a function is beyond me. If they wanted to inline it they should have used a GCC attribute, after all that macro came from a compiler specific header. 
I really want to like, make whoever did that watch it break my build on travis-CI and then show them why, and then ask them to explain to me why they used a macro instead of a function as God/FSM/Kevin intended.
To err is human. Fortune favors the monsters.
|
|
|
|
|
|
was being inclusive
To err is human. Fortune favors the monsters.
|
|
|
|
|
|
FSM? Flying Spaghetti Monster.
To err is human. Fortune favors the monsters.
|
|
|
|
|
LOL
FORTRAN 66 was my first language. Spaghetti code was our professor's favorite term used earlier versions of FORTRAN.
|
|
|
|
|
Redefining C functions with macros is permitted, but must be done properly and well documented as such.
Grrrr. Been burned as well.
|
|
|
|
|
Quote: ... and even errors that may only appear when certain components of the code are used a certain way - or on a certain platform
Is that not the reason why one does tests? 
|
|
|
|
|
Yes, but normally most of your code will compile. There are significant swaths of GFX that don't compiled, only parsed.
It means, compile errors can hide in my code.
To err is human. Fortune favors the monsters.
|
|
|
|
|
I use one of these. It's inexpensive, gives you 7 USB 3.0 ports, and will work with or without being powered by the wall wart. That should solve your USB port issue.
Amazon.com: USB 3.0 Hub, RSHTECH 7 Port Powered USB Hub Expander Aluminum USB 3.0 Data Port hub with Universal 5V AC Adapter and Individual On/Off Switches USB Splitter for Laptop and PC(Black) : Electronics[^]
Are you intending to run all of the devices at once, or just a few at a time? If it's a few at a time, you could set up multiple VMs running minimal linux distros and install VSCode/PlatformIO on those. You'd probably need to bump up the memory on the host machine to allow the memory for that.
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013
|
|
|
|
|
These need to be connected to a powered USB hub, particularly when all of them are turned on. I have one I swear by. I can run them all, testing each in probably in serial with each other, though parallel might be possible. The biggest issue I have is actual physical space. This project really needs its own room.
To err is human. Fortune favors the monsters.
|
|
|
|
|
|
I can't wait! 
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
Hmm, you sound like a procrastinator 
|
|
|
|
|
I'm so lazy that I even put off procrastination!
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
