Category: Chibi-mikuvan

Chibi-Mikuvan: Experimenting with the Trackstar 200A and the Hysterical Current Limiter

chorl

As usual with something I’m working on, let me open this update by yelling at Sparkfun Electronics. Remember when I said:

I couldn’t get it to change baud rates, however, despite trying that command a few times…

This was regarding the serial LCD kit I got for the Chib-Mikuvan digital dashboard. I meant that in the sense of no matter what I tried, I couldn’t get the baud rate change command (0x81) to take effect. So what the hell? It seems like it acknowledged the command, but it wouldn’t carry through a power cycle no matter what. There was nothing on the tutorial page about why this might be the case, but there were definitely a bunch of confused commentors, a few of which were having the same issue (but no answers) The firmware link didn’t go to the firmware, but to the command page. I found, along the way, a few custom user-designed firmwares for the LCD that claimed to fix all of the ills, but I was in no mood to deal with those.

As usual with the “open source” product world, it seems like forum comments and Git repositories are an acceptable form of documentation.

Finally, after some fumbling, in a product page comment post that was three years old, I found:

Tip: ensure that you are not sending any data to the backpack for the first second after powerup. If this happens, the backpack will revert to 9600 baud as a safety measure in case you don’t know what baud rate you set it to. Note that connecting this to an Arduino’s serial port WILL send data to it when the system is powered on, as the bootloader will try to connect to a host computer if present. If you connect the backpack to another set of pins using NewSoftSerial, this won’t be an issue.

Wait, could that have been the issue? Is that why everyone, no matter what, tells me to use a software serial port, without explanation as to why? Or if there is one, it’s “Well because it might interfere with the USB communication when you’re programming the Arduino”. Why the hell is this information not front and center on the front page of the product, or at least mentioned in the “datasheet” page? How is this a remotely acceptable form of product support, making your customers rile through forum posts and internet comment threads to find relevant thread that a comprehensive datasheet or product page should have in plain view? This isn’t the first time I’ve had to deal with this kind of “let the community be your tech support” from open source hardware producers either. Anyways, stay tuned for more details as to why I had to change the baud rate on this thing. It’s been working fine after I wasted about an hour trying to make up for undocumented behaviors.

In the last post, I had mentioned that the digital dash was a critical ingredient in seeing how much current this thing actually pulled during acceleration. I wanted to understand if I could run a race entirely using “tuned finger control” or not: the vast majority of R/C type motor controllers, like the Trackstar 200A unit I decided to use, don’t have current control or even current limiting, so a little twitch of the throttle at higher throttle levels can result in huge battery current spikes, which would cause trouble with the PPPRS regulation fuse. I’m glad to say that it served its purpose well:

This video featuring test pilots Dane and Mike. Unfortunately, it showed that my predictions were true: That the motor could pull spikes of 150-200 battery amps at most speeds just by blipping the throttle. This would cause some pretty instantaneous embarrassment, even though the MIDI type fuses specified are slow-blow types.

Now, the system also happily drew 200+ (pegging my current meter) amps at low speeds too, so the motor-side current must have been incredible. I tried, near the end of the video, to “current limit” by carefully ramping in and out of throttle. This only worked to a degree – any mess-ups resulted in spikes of high current. That’s what you get when your motor is more or less a dead short.

It was clear from that night of testing that, if Chibi-Mikuvan wasn’t going to consume 50A fuses every lap, I had to do something in software or hardware to limit the battery current draw. But what?

My ideal system would let the motor current be whatever it felt like – to exact maximum acceleration at low speeds – so long as the battery side current didn’t exceed 50 amps for more than a brief period. The MIDI type fuses are actually very tolerant of overloads, as their datasheet shows – the minimum opening time for a 200% overload is 3 seconds. That, for me, means hopefully enough time to push several hundred amps out to the motor as it spooled up, before falling back to a more reasonable current limit (which it seems like is 60A or so).

Since I don’t have direct PWM percentage access, and the DHAB-S/34 current sensor pulled out of the Fusion battery only has a 250Hz (???!!!) bandwidth, and I can only update an R/C controller so fast, I can only do “average” current control – there will always be spikes and transients. This wasn’t going to be some kind of kilohertz-bandwidth torque controller, but that isn’t the idea anyway.

I spent a few on and off moments thinking about how this might be feasible, including producing quite a few whiteboard sketches that ostensibly seem to be full of science, but which I assure you are mostly made of gibberish (Engineering folks, back me up here…)

What that diagram-like abhorrence says is basically the following: Don’t limit the battery current unless it exceeds a certain minimum threshold; then, manipulate the input signal such that the current draw stays under a time-dependent limit that starts high (such as 100 or 150 amps) but decays quickly to a sustainable current like 50 or 60A.

Sounds simple enough? I was mostly in battles over how to accomplish the “manipulate the input signal” part. How should it be manipulated? The downside of not having access to the PWM % of the output directly is you don’t really ever “know” how fast the motor is spinning OR how much average voltage is being put out by the controller at any point in time. Now, I did consider rigging a sensing line to the output of the motor to capture the PWM % information, but I wanted first to see if it’s possible to do with ONLY the (non contact, not soldered or hard-connected) current sensor. Sensing the PWM % would be a direction to take if I truly wanted to shave the yak and build an intricate torque control loop around an R/C controller, which seems… excessive. Fuses, being thermal systems, don’t care that you’re carefully controlling your current at thousands of Hz.

After a while, I decided to be sensible and work on this in stages. First, I can’t make the current draw stay under anything without an algorithm that first can manage it. For this, I turned back to Ragebridge and its “Hysterical current limiter“.

I designed RageBridge on the same principles – the motor current can be whatever it wants, up a point, then that is all it’s ever getting. It was done this way as a rebellion against the recent trend in “self protecting” robot controllers which shut down if you go over their current limit. As someone who drives robots like a maniac, I couldn’t stand this at all. To me, “current limiting” meant behavior like a bench supply: Once you go over a current threshold, the voltage floats to whatever is needed to maintain that current, including shorting through your stalled motor. That’s basically what Ragebridge does, usually with a +/- 5 or so ampere accuracy. Good ’nuff for the application.

The gist of how it works is: The output command is only allowed to “ramp” up or down at a certain rate, related to the controller’s loop time (delta-t) and the amount of output units it’s allowed to deviate by each time the loop runs. At each timestep, current is sampled; if the current is over the limit, then the controller will forcibly reduce (or increase) the output command at a rate much steeper than the user-controlled ramp rate. Aside from some second-order effects such as propagation delay/leg present in the output, motor inductance, etc., the current will change on the next cycle. If it wasn’t enough (still over current), the controller will modify the output command again. And so on. The result is a current waveform that looks a bit like a sawtooth.

In that screenshot from RageBridge testing, the current sensor is reading negatively i.e. the downward slopes indicate more current being drawn. As you can see, the current increases to a certain level, then is knocked back down again, then takes several cycles to build back up to that level.

RageBridge handles this bidirectionally, and it handles both regeneration and driving current, so there was a whole ton of state machine code in that thing that I don’t hope to repeat again. In Chibi-Mikuvan, I really only care about driving current, since the way the controller performs braking doesn’t seem to “regenerate” much based on what we saw in testing. That made the code easier.

However, it was still more math that had to be done in the loop. Up until this point, I had no clue 1. how long my code took to run, or 2. if the Trackstar 200A controller could even handle high-frequency inputs. If it was limited to 50 or 100hz R/C command inputs, there was no point in trying this at all – by the time the controller got around to processing your new command, your motor is already halfway back to China. From keeping tabs on the low cost multirotor & drone world, I know that the majority of R/C controllers these days handle servo pulses via interrupt and so can run right up to 499Hz (500Hz being where a 2ms-long servo pulse signal goes continuous).

So step 1 was finding that information out.

I changed the REFRESH_INTERVAL in the Servo library for Arduino to run at 490Hz. Didn’t want to get too crazy right away, in case the Trackstar needed time to think between inputs.

Notice that the drive chain has been removed – if it somehow interpreted this as “LOL OK”, I didn’t want it taking off on me.

Success! The Trackstar took the 490Hz input without issue. I decided this was close enough to start developing the rest of the code.

The next issue was seeing how much time the code, as it stood, took to run. I wanted to increase the loop frequency to 500Hz to take advantage of the fast inputs.

Well isn’t that lovely! Something like 5% processor usage at the 100Hz it was running at. I diagnosed this by inserting a fast pin turnon/turnoff at the beginning and end of the loop. I could conceivably increase the loop to 500Hz with no problems, but I did want to see if I could optimize the code a bit to lessen the run time anyway.

The extra wide band seen above is the LCD writing code. Remember my beef with Sparkfun? That whole hour was spent trying to see if I could shrink this time by increasing the baud rate. Here was the random-dude’s-firmware-free hack I had to make to get it to, you know, work as advertised.

This is a hardware “enable” line I made with a dinky little 2n3904 transistor that prevents the LCD from getting power until I explicitly activate it in the code. That way, the Arduino can shit packets everywhere when it starts, and it doesn’t turn on the LCD until after setup() is done. Oh, and then it will artificially wait a second to make sure before issuing commands!

With this, I was finally able to change the baud rate.

…but I found out it was basically incapable of talking at 57.6K baud. So not only does it have undocumented behavior, but it doesn’t even meet its advertised speeds. Well then. I settled on 38400 baud, which was good enough.

Another speed-up-the-code thing I did was to knock off a significant figure from each of the readings. The minimum resolution of the current sensor with a 10 bit ADC is something like 0.05 amps anyway, so it wasn’t worth 2 digits of decimals to tell me. Likewise, with such imprecision in the V and A readings, there was no use having 2 digits on the watts reading either.

In the end, all this effort only took the write time down 500us. It’s certainly not nearly ratiometric between 9600 and 38400 baud, so I suspect the overhead of Serial.print and Serial.write might be more at play here. Nevertheless, there’s still 1000us left to do other stuff, including a few lines of subtraction for the current control code, so I left it.

After inserting Ragebridge genes into Chibi-mikuvan’s code and seeing it seemingly, allegedly working without the chain drive hooked up, it was time for SCIENCE

This. This is actually the most terrifying thing I have ever built. Worse than LOLrioKart, Segfault, or Deathcopter. This is Sciencekart.

Well, okay, I didn’t have a DSO Nano or one of those fancy cordless oscilloscopes, but I did have a 12v lithium module, an inverter, and lots of gorilla tape.

I scoped a bunch of things while driving around the empty hallways staring intently at the oscilloscope screen, with one hand on the throttle and the other perched over the RUN/STOP button. If they say texting while driving is bad, they clearly have never tried oscilloscoping while driving.

The signals I looked at were the R/C pulsewidth output in microseconds and the raw reading from the current sensor. I basically just gunned it from a stop for a bit, let go, then gunned it again, and so on. Square waving the throttle is the biggest rookie mistake for people not used to driving R/C throttles, and this basically eliminates that problem. It doesn’t account for “startup jump” since the controller’s starting routines are often dead-reckoned, but that’s where I come in.

I’m glad to say that it works! Clearly, not with decimal amps accuracy… perhaps now I don’t even need the time decay curve – the error of the controller alone is enough -_-

This is the sawtooth waveform the Hysterical Current Limiter generates. This was captured at around 25% top speed after I punched the throttle to 100%. The current sensor’s zero point is at +5 divisons (0.5V) on the vertical scale; I’ve moved the zero voltage line waaaaaay down to capture the peaks of the waveform more effectively. At 10mv/A sensitivity, the max current, as can be seen, is actually peaking at 100a. It’s interesting to see that the minimum current in this case was more or less 50A, the programmed limit!

My guess as to why? I think this is in part because the motor is so low inductance that the current changes much faster than the controller can keep up, and there is little back-EMF from the motor to help at low speeds. The controller only gets a little time below the current limit as a result: any increase in the command results in a ton more current flowing again. I will need to find a somewhat emptier hallway to capture higher speed waveforms, but based on eyeballing, the spikes decreased as speed increased.

I’ll continue experimenting to see how to reduce the error, but right now, it works great as-is (if not a little anemic now because it can no longer pound 150 amps). If the error is large, then I won’t really be able to utilize the ‘time decay’ method as well, but it could be fine just setting the current limit to “60A, pretty-pleeeease” and keeping it there.

Here’s a test of me driving straight towards a wall:

This was one of the early tests in which I set the current to only 40A to make sure the concept even had a chance of working. From still frames, I could see numbers like 40.6, 45.0, 32.5, and the random spike to 79.6 at the very beginning when the startup loop exits. During the cruise portion, the amps ranged from 29 to 48 or so. Promising start for now!  Tuning can occur from here as I drive around more with sciencekart!

Expect some more testing video this weekend possibly with the oscilloscope still attached.

Chibi-Mikuvan The Extended Universe Edition: Water Cooling Loop and Digital Dash

chorl

It must be awkward observing your own viscera from the perspective of your own husk.

Now that the summer season here in the shop has settled down and people are well on their way to wrecking my laser cutter, I’m advancing on the checkoff list before the Detroit Maker Faire and its associated PPPRS race. I’m like, signed up noncommittally and everything. Through a web form. That’s some serious confidence right there.

water cooling

Last time, I mentioned the water cooling circuit and electronic dashboard. Recall that the motor I ended up investigating and deciding on is a water-cooled boat motor, and I’ve been running it without any water cooling. This has been somewhat tolerable – it is actually quite efficient, and even during the hard garage runs, I was able to do 2 or 3 back to back and not feel that the motor was nearing burnout temperature. However, all of this is was done in mild weather, at night. I’m sure that if the daytime temperatures were much higher, it would be a greater risk to run the motor that hard. Bottom line is, why buy a water cooled motor and not water-cool it!? I could have sprung for its longer air-cooled cousin if I didn’t want to mess with that.

The water cooling circuit I had imagined was going to be quite simple. This wasn’t going to have to dispose of a kilowatt or more. I ran some rough numbers and expected that at 100 amps continuous (all day, which will never happen), the motor is going to only dissipate about 150 watts. A modern workstation PC’s graphics card along can shit more than that at any point in time. The plan was to base this system off a computer liquid cooling rig. I scouted eBay for used water cooling kits for a little while before Mike handed me a Powermac G5 pump and radiator, first revealed a few posts back.

So that was taken care of! The only missing from this kit was a reservoir. Now, I could have been crafty and just made a whole closed loop using the pump and radiator alone, but a reservoir would be helpful for topping off the system and allowing expansion space (I have zero cred here – real-Mikuvan’s coolant expansion tank is the hose looped back onto itself and pointed away from the battery so any spurting pressurized coolant goes straight down…)

So that’s how I went to the grocery store, bought a jar of pickled olives, and ate the whole thing. That was a very uneasy evening.

I made some “panel mount” barb fittings for 6mm ID Tygon tubing and attached them to the can lid using my favorite adhesive/sealant, Goop. It’s rubberized, so it’ll be both watertight and somewhat flexible. The other end of these fittings end in a non-barbed tube that I extend down into the jar as a pickup and……dropoff? tube.

It took a while to come up with a component placement, since the only left over space was immediately behind the seat and in front of the electronics case, which is not a very good airflow spot. So I did what I could for the location. The big board it all sits on is 6mm hardboard. I purchased several (dozen) sheets from a local wood products distributor as prototyping material for 2.00gokart and the shop in general, so plenty was left over. I’m just going to use this hardboard as-is. It’s not very afraid of water (unlike MDF), and is quite rigid.

The jar mounting bracket was produced on an Up Mini, one of the handful of hobby-class 3d printers accessible by students in my shop. I designed it while shuffling component placement in my head – I knew this bracket would be needed, just didn’t know exactly where yet.

Installing the tubing was quite straightfoward. Near the motor, the 6mm tubing steps down via a custom 3d printed adapter to 4mm tubing to attach to the motor. Hello, flow restriction. Designing these adapters was a fun excursion into the world of fluid systems design. They were made using the lab Objet (high resolution resin printing) machine.

McMaster wanted to charge me $7 each for these damn things, so I figured having a $33000 state of the art rapid prototyping machine knock one out …uhh, I guess I really have no case here. I think these fittings will still work if made with a hobby FDM printer, but there would likely be sealant involved to fill the filament low spots.

Testing on a power supply, I found out that the pump wanted 0.4 amps at 12 volts by itself. Yikes – the little hacked 5V BEC units I was using to produce 12V (I have hogsheads of these) can realistically only sustain about 1 amp when jacked to their new output voltage. I was already needing 1 amp for the contactor.

Solution: Just whip up another one in like 5 minutes. AUX runs the pump, LOGIC is dedicated to the Arduino, contactor, and sensors. I have 0.5 amps left to add gaudy lighting.

Here’s the completed water cooling loop! After a little leaking, all the O-rings seem to have found their seats and I haven’t noticed any more drops on the ground.

I ran this thing around the shop “track” (meaning the conveniently rectangular closed loop building at night when everyone’s gone) for half an hour, doing 55 lap or so (I may have lost count). I ran the battery straight down, and the motor could best be described as “lukewarm”. Maybe warm water for hand and face washing temperature. This was with no additional airflow to the radiator besides what leaked behind the seat through that cutout in the mounting plate. Without power data to back it up, I can’t really draw firm scientific conclusions, but yes.

As the weather gets hotter here, I’ll most likely set up the 2.00gokart test track one weekend and just try and keep running as long as possible. I think the addition of this system will help the reliability greatly.

the digital dash of science

One thing that has been bugging me about Chibi-Mikuvan is indeed the utter lack of science on it. Because of my use of the 150A Anderson connectors, I don’t have any instrumentation that can jack in the middle between the battery and motor controller. I mean, besides making yet another why-did-you-even-bother-adapter.

I conceived the idea of rolling my own wattmeter, or perhaps Anal Cyclist Cycle Analyst, after finding the datasheet for the current sensors that are integrated inside the Ford Fusion hybrid battery, the LEM DHAB S/34 series. It was easy enough to read, as an isolated Hall Effect loop sensor, and I could put it any where. It’s also easy to rig a voltage divider to sense the battery voltage. With those two pieces of information, I was basically satisfied (I could go totally insane and add way too many features and datalogging to boot, but…)

First off, removing the ridiculous connector shell on the current sensor and attaching a pigtail of wire to it. Protip: The cheapest source of 4 conductor shielded high quality cable is old USB cables. Almost all of the signal cabling on this thing is made of hacked up computer cables!  8 conductors? Ethernet. 9 conductors? DB-9… nobody uses those any more. USB has 4 conductors and an outer shield that could be another.

Next, I swung by Microcenter and got a Sparkfun serial LCD kit. Imagine my surprise when I found out it was not one of their integrated one-board sLCDs, but a cleverly packaged Arduino. Well hell, I could have made that with the onboard Arduino. Oh well.  I was looking for a sLCD in particular because I wanted the simpler interface, and not actually having to resort to using all 8 pins of an Ethernet cable to run the otherwise parallel signals.

I designed this mounting bracket in about half an hour, to be slowly worked on by the Up Mini, while I wrote LCD code. For some reason, I’ve been all about those flexural snap fits lately.

Here it is being produced. I’m basically an Up salesman, so I’ll take this opportunity again to plug these machines. They just work. Extremely well – they’re still the only machines I’ve dealt with which produce truly breakaway support lattices. Even an old Dimension BST machine.

I have a personal Up Plus, and the shop has an Up Mini.

Getting this LCD up and running was quite easy. I basically followed the Sparkfun datasheet and instructions. I couldn’t get it to change baud rates, however, despite trying that command a few times. So for now, it’ll just talk slowly. (I typically run my Arduino-related serial things at 57.6Kbaud).

I used the remainder of the USB cord to run the LCD from the handlebars to the electronics box. This wiring job is almost approaching unserviceable – I’m strongly considering a full refactor into something more accessible and serviceable prior to the Detroit Maker Faire, now that I know what connections need to be made and what could be done better. For instance, I have a few 12V, 10A DC/DC converter modules that could replace the two smaller hacked ones and run all the gaudy lighting I want. 

On the breadboard are some of the signal interface circuitry for the current sensors and voltage sense. Just simple R’s and C’s, they form low pass filter networks to help smooth the sensor readings. I’m not sampling thousands of times a second here…

With two cables now snaking up to the handlebars, I used some left over wire-twizzlers (proper name: spiral wire or cable loom) to constrain them. It looks almost professional!

Here’s the end result!

I spent way too much time in the code measuring the length of the digits to keep them at the correct offsets such that the units’ letters (V, A, W) didn’t move. The digits have enough space in front to add a negative sign for regenerative braking. Volts have 2 digits, Amps has 3, and Watts has 4. The LCD refreshes 5 times a second, which makes it a bit flickery, but I still can catch glimpses of the levels that the measured physical variables hit.

I’d say this setup is definitely more dashboard than instrumentation. Instrumentation would imply I investigated and know the level of precision attainable with the parts and the potential error and uncertainty in each one of those indicated measurements. Which, no. This was calibrated with a Harbor Freight voltmeter. There is literally nothing between it and just flat out guessing.

Here is the current version of the Chibi-Mikuvan running code. I’m posting it primarily because of the LCD interface code – the rest of it is a big fancy servo tester knob (I have fathoms of those).

I’ve been thinking of a potential next step with this code, now that I have the ability to read the battery current directly. The idea is to use something similar to RageBridge’s current limiting scheme in order to follow the time-current curve of the PRS series fuse. The fuses specified, MIDI style fuses from Littelfuse, are slow-blow ones which give you a moment of surge current capability, so I’m thinking about a way to try and push that envelope with current limiting – no matter how hard you punch the throttle, it will stay (ideally, just barely) within the transient limit of the fuse.

Something I have to think about now, though, is how to make a “Chibi-Mikuvan adapter” to hitch it to real-Mikuvan, though if we have enough go-karts coming from here, it’s easier to just borrow or rent a trailer.