Category: Motor Controllers

Null Hypothesis: Finishing Up and Testing the Ragebridge v1

chorl3 Comments

This it!

The post where my project gets stuck in motor controller hell. It happens to everything I try to build a motor controller for, so be prepared to read about exploding FETs and endless nondeterministic troubleshooting!

Well, the story is actually slightly happier than that. In the past few days,

  • The frame has been put together, all the wheels mounted, and battery pack made
  • The Ragebridge has been installed and the robot successfully tested and driven using it, seemingly exhibiting no adverse noise issues
  • The Ragebridge has been successfully detonated when a motor suddenly locked up.

Let’s start from the beginning.

With the wheels bored out, making hubs was the next step. I decided to make these more consistently on a Nice Thing with digital readouts, courtesy of the Edgerton Shop (They were also the nearest straight knurling tool, incidentally). The knurls will help the hubs not slip and strip out against the soft plastic bore of the McMasterbots wheels.

The hubs are installed with an arbor press.

Two wheels in. The hubs are very conventional “drill hubs” with a 3/8″-24 internal thread and internal shoulder that lets the reverse-threaded 5mm locking screw hold the hub in place against both threads. I’ve always found it funny how the ‘cheap drill’ converged solution ended up using an English and a Metric thread on the same part. The outer thread is not M10 or a metric thread, and the internal thread is definitely 5mm x .8 and not #10-32, which only fit down to a few threads. I’m mildly interested in how shady Chinese factories converge towards which design to copy and paste.

It’s starting to look like a robot now.

I temporarily secured the front wedge on with a few screws in preparation for a test drive.

Okay, robot’s done. Nothing more to see.

…

Using a spare 7S battery pack (from the long-retired Segfault) and the Botbuttz controllers I bought a while back, I threw together this test rig just to try out how the robot handles. Conclusion: It handles like a pushybot with a wide wheelbase and near-midpoint center of gravity! Unlike Clocker, which is back-heavy out of necessity and so oversteers very easily, this thing is extremely stable in turns. The braking effect of the BB controllers also contributes to snap turning response and smooth starting and stopping. These are all features I want to make sure Ragebridge can duplicate.

I spent a while scooting this around before moving onto the real challenge: the aforementioned Ragebridge.

I soldered up another board (the first one having been put into the Silly Media Lab Vehicle). There were no changes made to the board layout, only the modifications addressed in its first post which included moving the low-side gate drive bypass capacitors into the right place.

After verification of the hardware functionality, I spent the majority of 2 days hammering out an R/C input firmware for the board. I added features in the following order:

  • Radio signal-loss failsafes! I figured if I just wrote a dumb Servo-to-PWM program, it would stay that way forever. ESCs for robots need to have a way to shut off the motor outputs on recognition of signal loss, so the robot doesn’t just keep rolling. This part wasn’t difficult: every time a valid servo pulse is received in the pulse-reading interrupt, a flag is set to indicate the reading was good. A checking routine runs once per half-second to see if there were any channels which did not set their validity flags – if there are, then the gate drivers are shut off until all signals are good again.
  • Command ramping, meaning the ESC takes a small amount of time to slew from one direction to another. This is to prevent instantaneous “step reversing” which can cause huge current draws, up to twice the DC bus voltage to appear at your ESC’s power inputs, exploding motors, and other terrible things. I created quite an epicly named function, mapWithRampingAndDeadBand(); to perform this task.
  • Linear stick response. Directly mapping of the pulse widths to PWM output %, through the ramping limiter. This was just to get the robot driveable so I could add…
  • Invert mode, a feature which allows you to redesignate which end of the robot is forward. Useful for invertible bots like NH because otherwise your robot would drive backwards when upside down, as well as have swapped turning behavior. This is activated by the channel 5 toggle witch on my radio, and just swaps the left and right commands while adding a negative sign.
  • Exponential stick response. This makes the initial response ‘softer’ so the robot can be driven more precisely when moving slowly. It tends to reduce spin-out and oversteer tendencies. It is implemented as a simple lookup table – raw command goes in, expo command comes back.

I began to add a ‘stick calibration’ mode, but decided against it for now. Stick calibration is a luxury, because radios can generally be trimmed and have their endpoints modified to suit just about any desired pulsewidth range.

The current version of the RB firmware is here. It’s written in Arduino 1.0.1 (which finally got rid of the CTRL+SHIFT+M bug!). I generated two exponential response tables, one a little more expo than the other. I found the “more expo” one to be a little sluggish in responding without moving the sticks really far, but your experiences may vary.

Ragebridge v1 installed in the robot on its little standoffs…

After testing the controller out on a power supply first (which has nice features such as non-infinite current output in case of a fault) I put the test battery right back in because I hadn’t gotten to making the actual robot pack yet, then made some test driving videos!

The “basic test” was done right after I finished the linear stick mapping, and the more advanced testing is with exponential.

I like Ragebridge’s functionality in those tests. While open-floor driving is not as strenuous as pushing another robot (trying to push you back), high-current transients like from reversal and stopping will be what causes noise-induced failures and latchups. I didn’t observe any of those issues during that test driving, which bodes well for more sustained heavier loads. Unfortunately I neglected to put a Wattmeter on the thing, so I don’t actually have power throughput figures. Typical drill motor drivetrains in this size robot will draw about 30-40A per motor accelerating or changing directions and about 15 amps spinning out the wheels.

With at least a slim prospect of the robot working properly, I started putting together the battery pack:

The voltage of this pack will be 8S2P in lithium iron phosphate, so 25.6V nominal and 28v peak. This high voltage was just to get the speed I wanted out of the motors, but it could prove to be the proverbial Achille’s Heel… or perhaps Achille’s A123 Cell for the robot.

Those yellow things are PTC thermal fuses. I usually just lay more busbraid across the cells to put them in parallel, but the PTCs add another layer of failsafing by limiting the rate at which the cells can differentially discharge. If one cell suddenly fails (or gets punctured) and becomes a short, then the very low resistance of the bus braid will let the other healthy cell short into it, potentially causing even more damage. The PTCs will at least limit this to a reasonable current.

I chopped up some spare 4S JST balance extension cables for the cell taps. Generally I make my own using JST-XH connector shells and pins (e.g. Digikey’s 455-2217-ND), but it takes longer (and most chargers don’t have slots for 8S pack taps anyway). One JST lead is the lower half of the pack and the other is the upper half.

The balance leads are insulated by some Kapton tape layers as I soldered them in. I didn’t want them laying directly on the cell tops any more, after this happened, but on this pack there’s no space to route the wires only down the sides of the cells (it will be laying flat in the robot).

Since this pack will be more enthusiastically jostled inside the robot, I’m giving it a layer of foamy armor.

And after Giant Heatshrinking! I gave the pack its initial balance-and-charge wakeup routine with my 1010B+ charger doohickey.

While the pack was slow-roasting to perfection, I took care of the last bits of electrical work on the robot itself.

This adorable ABS-printed connector holder houses a Deans connector and an XT-60 connector. The Deans is going to be the master switch via the “Georgia Tech Key” method. In order to prevent myself from being confused at which terminal to charge the battery from versus to bridge together (hint: bridging the battery is a bad thing), I just used a totally different connector for the charge plug. The connector housing is mounted to the bottom plate.

This is the “Georgia Tech Key”, so named because I’ve only seen it on random GT creations before. Technically it’s called a removable link.

I made this loop a little too long and it sticks out the top of the robot by a little, but some tape should keep it down. In the worst case I’ll just replace it with a directly shorted Deans connector.

Alright, here it is. That’s it. The robot’s done.

Now to hand it off to other people and drive it around until it blows up! The best way for me to find the weaknesses of any of my builds is to give it to other people. I subconsciously go easy on things I know may be unreliable (the so called Creator’s Mercy), but other people have no concept of, say, the drills being severely overvolted, or for instance in the case of the Chibikarts, that flooring the throttle from a standstill may not result in satisfactory behavior.

I wasn’t able to whip out the video camera in time to capture the magic moment, but what happened was this:

And this:

What triggered the failure was a MITERS member driving (very happily and enthusiastically) NH into a solid machine frame at pretty close to top speed. The robot stopped moving, and I instructed him to give a little throttle to see if the controller was still working. It’s actually very difficult to judge what people consider ‘a little throttle’ based on experience, so I can only assume that it was near full stick or something. A small explosion was seen inside the case.

The superficial damage was the blown ground trace and one very, very unhappy MOSFET. The actual damage, though, as I found out through subsequent hours of checking and debugging, was pretty much the whole thing. The Arduino Nano was fried (noted when I powered it back on and the ATMega started drawing half an amp…), as were all 4 FETs on the exploded side (gate-to-source shorted through), both of the IRS21844 gate drivers on that side, the gate-to-source protection zeners (which I’m sure did all the protection they could before exploding), as well as half of the 74LS08 quad AND gate that is involved in piping the PWM signal to the gate drivers.

Damn.

After replacing most of the semiconductors (and the Nano) failed to make the controller work consistently again, I decided to just scrap this board. Some times, enormous transients damage components in ways that are unexpected

And that’s the motor after I took it apart.

I think a combination of being oversped (18v overvolted to 28v) and sudden impact (running into the solid object) must have blown off the commutator segment. These motors are not really what I would call “well made” since the whole drill they come in costs $20. Running 28 volts is certainly up for compromise. But, I enjoy the speed of the bot on such a voltage.

Good thing I have a small forest of these drills!

So the final version of the story is, NH was finished and blown up, and is now unfinished. Putting together a new controller (with some mods I will discuss below) and pitching the bot  back together should not take long.

ragebridge mods

Now that I’ve gotten to know Ragebridge a little better, I have in mind a list of issues and changes I’d like to make when it’s time for version 2. Since V1 actually seems to work fairly well, this is not an immediate task.

First issue to take note of is of course the misplacement of the gate drive bypass caps. This is what led to the very first board not working at all until I changed it. Clearly this is a routing matter and the second version of the board has to address it, but for now it is easy to hack by hanging the little capacitor off the pins of the 21844 chip.

Second, the row of input headers at the top of the board are not in a very useful order. Two of them were designed for “servo” style connectors which are terminated signal-power-ground, and two for ‘potentiometer’ style inputs, terminated power-signal-ground. Well, I got the servo input order wrong anyway – those two headers actually are connected signal-ground-power. So I had to make two different styles of deceptively incompatible servo cables to suit the board because I needed 3 channels of input. I’d just line them all up signal-power-ground in the future, even the “analog” inputs, for better consistency.

Next, and an issue I can hack around for now, is the way that the power traces narrow awkwardly and haphazardly:

Those yellow highlights is where the trace width drops significantly from the average. The ones at the left and right bug me in particular – I was definitely trying to keep the controller within a reasonable width, but the heavy power traces around them are easily double the width and very short too. I was intending to reinforce those with braid or wire, I think, but didn’t actually do it for this board.

The circle near left center is a really obvious design derp. Most of the power stage of the board is basically rotated 180 degrees from eachother, except that current sensor placement. The right side of the board has no corresponding narrowing – current flows down through the vias on the right center two FETs for MA2, and the right side’s current sense path is a straight shot.

Given that these traces are all made in 1 oz copper, they’re exceptionally prone to sudden current overloads, such as a near doubly-overvolted drill motor suddenly slamming to a halt. The current sense path on the left is a little more difficult to reinforce since it’s under that FET, but as long as I’m not using the current sensor on these boards yet, I could hard-jump it with a piece of wire.

Overall, I’m satisfied with how Ragebridge has been behaving (short of being inductively crowbarred by an unhappy drill motor), and I’m going to be building more of them to test on other bots soon…and of course fixing this one.

 

Ragebridge: It Needs More RAGE (and Testing)

chorl3 Comments

After a brief detour to Otakon 2012, I’m back and ready to push forth on the everything ever that needs to happen for Dragon*Con 2012 and its associated Robot Battles. There’s actually alot that has been going on lately, but I haven’t been keeping up with the site updates. I’m going to fix that now… but notice how there’s no update on Null Hypothesis yet? That’s because I haven’t started on it. Hmm…

First up is a new chapter in my neverending fairy tale of chasing after the mythical working motor controller, Ragebridge. First addressed in the NH post, but actually designed several weeks ago, Ragebridge is my newest attempt at finally making a reasonably reliable small H-bridge controller for robots or certain DC powered vehicles. My first (and only) successful H-bridge was one hell of a start – it was a 60V, 100-200A (the amperage never having been truly pushed) unit for LOLrioKart. I have not produced a single working H-bridge since, and I really can’t say I understand why even after this board.

The last update concluded with myroPCB taking eons to process my order. I’m glad to report that since then, I’ve gotten the boards!

…all 4 panels of them. Wait, I only orderd one panel of 4 boards, and now I have 16 boards. Is Myro’s excuse for fucking up your order in-production (their excuse when I called them to inquire, 2 weeks past the 1 week leadtime guarantee) to just send you an asston of boards so you’re happy?

Well, I’m sure as well happy. Who can stay mad when there’s more boards?

Now, these things had better work… If they do, I really might consider releasing some of these to other robot builders to test, because 16 boards is alot of door decoration.

I begin the board population by placing all of the ICs and flat stuff.  The FETs and capacitors, since they stick up pretty far, will go on last. On this board, I placed almost all of the passive components (the Rs and Cs, and some other things) on the bottom side. Diodes, such as the zeners for the FET gates, are all on the top side, since they’re semiconductors.

If I were actually to try and get this board made by an assembly house (or maybe I’ll just sell it to Hobbyking), I’d try my darndest to put all of the components on one side. Double sided boards are more challenging and expensive to populate because they involve flipping the board. So if the design proves reliable, I might make some more layout permutations and possibly even form factors/sizes.

I tend to test my boards in stages – getting the logic power working is one such stage. Here’s some verification that my 15V gate drive and pre-logic rail is indeed working – it’s lighting a 12V LED incandenscent-replacement.

Incidentally, this step took a little while to finish. I accidentally replaced a logic bypass capacitor with a 0 ohm jumper resistor. That explained why the regulator was only outputting 50 millivolts and becoming very unhappy. It took me a while of searching for accidental net overlaps in Eagle and solder bridges before I thought that the number of resistors did not quite match up with the board design.

After that minor issue was fixed, the logic power began working fine.

Now with more Arduino. I first wrote a “dumb” PWM testing program to make sure the outputs were all working. This just entailed putting a 50/50 PWM on each side, alternating between forward and backward every 5 seconds to test if the gate drive and PWM AND’ing circuit worked. During this process, I discovered that one FET was dead – which was unsurprising, since it was salvaged from another controller.

The FETs I used on this thing are some 3.4 mohm units from Infineon. I’m generally a die-hard IRF fan, but their shit is getting too expensive for me to keep blowing up ($6-7 each, and I went through a whole rail of 25 for the Tinytrollers and LBS boards…) and these D2PAK-7 FETs were only $2.18 each in reasonable quantity… And it’s not like I’ve had a controller fail from overheating yet.

Since my outputs were all valid, I decided to throw a small test motor on it. This small 24v random motor worked fine, even on a bench power supply with hard stepped reversing.

Mostly because it stores too little energy to actually do damage. I next tried this exercise with a classic robot motor, the EV Warrior. Hey, I figured if it can hard-punt an EV Warrior on 24 volts (by this time, I was running 75% duty cycle on a 30 volt power supply), it could survive anything. Except it was still on a power supply. Power supplies, generally, do not “sink” current very well – if there is an attempt to reverse current flow, they either tend to crowbar (protection circuitry) or the voltage on the output will surge uncontrollably as the reverse current charges the output capacitors. In my case, I briefly saw the supply flash to over 60 volts before it shut off its own outputs.

I was left with an entirely dead side  – the spike itself was likely much higher than 60 volts. After appraising how much of a dipshit I was for trying that stunt, I replaced the dead IRS21844 chips (and 1 blown FET) and all was good again.

The original application for Ragebridge is to drive the Silly Media Lab Vehicle project I’m working on this summer (like… just about every summer, now that I think about it). Now, while I like documenting and publicizing things I do, this isn’t really my project to do so with. So here’s a picture from a little later when the test board has been installed into the vehicle. The drive motors are some 500W wheelchair motors which are rated to 100A stall, so they should be a pretty healthy load for this board.

Ragebridge’s first test was a little inconclusive. I began running into the same regulator latchup problems as Tinytroller – the LM2594 circuit would, seemingly randomly (but vaguely correlated with surge loads) enter a state where it outputs approximately 6-7 volts and makes an audible buzz. The situation is only resolved with a power cycle. These things are supposed to switch at 150kHz, so if I hear it, something’s gone terrbly wrong.

I spent a while trying to figure out what might be causing it, going through my usual motor controller differential diagnosis processs like…

  • checking that I didn’t swap my Zener diodes with 100V bootstrapping Schottky diodes, or vice versa (which would actually be bad, as I was running about 30 volts and my Zener stock is all 16-17 volts)
  • adding more capacitance to the regulator circuitry, since the amount I had on was under the recommended bus capacitance in the datasheet
  • adding zener diodes to the 15V rail in case for some reason power was backflowing into it,
  • and adding a “precharge resistor” to the regulator’s input in case it was a dV/dt problem during direction changes – the resistor and input bus capacitors form a RC circuit which should absorb any sudden input spikes.

I never got to that last one. Since Tinytroller exhibited the same kind of regulator latchup, and seemingly nobody else in the world has ever seen this problem happen, I decided it must be some kind of systemic terrible layout fuckup that must be obvious. I submitted the design files for Ragebridge to He Who Has Actually Reliable Motor Controllers and Stuff, and the verdict we agreed upon was that nothing substantial was wrong with the design, except this weird arrangement of capacitors:

This is where I need to explain how the IRS21844 gate drive chip works. It has two different grounds, one for logic and one for power. Hypothetically, they are allowed to float 5 volts apart, providing some semblance of isolation between logic and power. It’s not as pure as fully optocoupled logic in terms of noise robustness, but they sell these things to people so they must work, right?

Anyways, the pin on the chip that is supposed to be logic ground is called VSS. I generally call my boards’ power-side return (/battery negative) VSS also. The actual pin on the 21844 which is supposed to be on the power ground side is called COM, and the datasheet recommends a ceramic bypass capacitor between VCC and COM to accommodate the pulsed gate drive current, which can reach about 1.5A for a few dozen nanoseconds (read: super-fast dI/dt). If you’re lost, don’t worry – clearly so was I.

I’ve generally copied and pasted my gate drive & power schematics from one controller to another. Looking back, this particular nugget of the schematic was taken from Tinytroller. After Melontroller 2’s success, I decided to start totally over with the layout. This is where the mistake occurred: Instead of routing the “low side” bypass cap from VCC to COM, I bridged it between VCC and logic ground, the chip’s “VSS” or my “AGND”.

Hmm.

So basically what this means is, first, the low side had no bypass capacitance at all. Looking at the Ragebridge original design picture, gate drive return current has a hell of a long way to go before it can reach the regulator’s return, all through narrow traces with lots of corners and vias.With pulses of 1.5A needing to evacuate in matters of nanoseconds, I’m not exactly surprised that substantial voltage transients could have been induced in the ground traces. The job of bypass capacitance is to absorb these rapid voltage transients very close to their source (the drive chip) so they do not propagate to the rest of the board.

Second, should the local 15V gate drive supply become unstable because of the pulsed current draw, its effects are capacitatively coupled into my logic ground. Again, while possibly small, it can’t be that good.

 

I had to execute a little bit of ninja soldering using the finest SMT tip I could locate in order to fix the (not even certain) problem. The caps were removed from their pads on the underside and then carefully jumped across the requisite 21844 pins (VCC and COM) on the top side.

And it worked. I’m guessing moving the errant cap to the correct side of the divide made the VSS-COM distinction actually useful, because this seems to have cured all of the ills that was plaguing the regulator.

While I can’t really post video of the SMLV working, I have yet to get Ragebridge to reset a single time after this change was made, even under rapid forward-backward direction switching. The maximum batttery-side current (measured with a Wattmeter) was 76A, though this was surely very brief. But it proves that Ragebridge can at least handle those kinds of surges. The 3.4mohm FETs used on this board should be good for at least 40A continuous duty. The components on the board are selectable for a wide range of voltages – the board I put together above can probably run 36V robot systems without problems, with 60+ volt rated major power devices.

I’m confident enough now in this design that I want to pitch it on a robot. Even as powerful as the SMLV motors were, they were fighting against alot of inertia (literally, my morbidly obese ass) and generally moving very slowly. I can’t just whip the whole thing back and forth on a timescale of milliseconds. A robot motor, such as the venerable 18V DeWalt motors, have much less inertia and can change directions (and currents) very quickly. This test would be to see how well the Ragebridge design stands up to these higher, faster transients. If they prove to be no problem after the changes were made, then I will rage-quit everything because god that was so obvious of a fuckup.

Too bad I took apart my only generic 30lb drive base last month. Fortunately, I’m building another one soon. Overall, I’m glad that to fix at least 90% of the problem, no traces and wire jumps were needed because that would have seriously made these boards unattractive for use.