The Dramatic Story of How I Saved the Ragebridges and Suck at Reading Datasheets, and an Überclocker Update

Alright, here we go. The post where I have to fix everything, because I have to leave for Atlanta and Dragon*Con 2012 (and its associated Robot Battles) in 8 hours. After assembling 2 more Ragebridge boards, finding that all of them had Resetting Regulator Disorder, I tried several increasingly desperate hacks to get the boards reliable before discovering a small nuance in the datasheet. In all, I managed to blow up another board (due to a slight… assembly problem), but I believe I have finally found a stable operating plateau for the Ragebridge boards.

Oh, and Clocker works. Here’s a recap:

One of the first problems I had to solve before actually getting to work on the Ragebridges was how to mount them. Formerly, Clocker’s “eBays” held 2 Victor 883s each. I wanted to group the Ragebridge boards on one side of the bot to avoid having to have long runs of distribution wiring, and because mounting two mostly bare PCBs horizontally greatly increases the risk of robot grunge getting places they shouldn’t.

I couldn’t really stack two of the Ragebridge boards horizontally, though, because they were slightly too tall to accommodate the thickness overhead of mounting standoffs. If I had shorter capacitors, this could have worked fine, and I was planning on building a small enclosure around the boards with fan cooling.

Then I realized that the boards are just short enough to mount vertically. I still needed a way to retain them, though, so cooked up very quickly this “rack” that has slots which the boards fit in.  This was whipped out on the Lab Replicator™ and the standoffs made with a lathe.

First tries always reveal where parts run into eachother. On the upper board, the capacitors on the underside touch the standoff above them. And on the lower board, the Arduino Nano’s ISP headers touch the standoff below them.

Hmm.

The second revision moved the boards a little closer together and away from the standoffs. With the mounting solution taken care of, it was time to attack the boards themselves. I first had to attach wires to them:

Instead of soldring wires to both battery inputs and then joining them in parallel, I used a single 12 gauge input wire and then a 16 gauge ‘jumper’ which brought power to the other half of the board. The constraints of rack-mounting my controllers this time meant I couldn’t make big Y-cables.

After populating the boards, I began to run into regulator drama yet again. With my Spektrum receiver (Clocker’s receiver for 3 years now) attached, the LM2594-based switching regulator wouldn’t. This was worse than it was on Null Hypothesis (details in the middle) where it seemingly had a habit of not wanting to power cycle. The 15v rail was totally unstable, and the 5v power LED on the Arduino nano would flicker as the regulator kept latching up and resetting.

Even with the Arduino Nano but no receiver, it was still erratic, taking several seconds to stop resetting the microcontroller. The whole time, the regulator was also getting hot. Extremely hot.

Both boards displayed the same symptoms. When the regulator was in latchup mode, the switching frequency was audible and also very unstable. Basically, it seemed that any load was causing the whole thing to go crazy. I began attributing it to being a design error on the board or in the layout – maybe I had crossed my grounds somewhere else?

I began pulling desperate hacks suck as capacitance in places where it really should not have needed capacitance (i.e. where a slightly out of spec component value should not have a first-order effect on functionality), and even hacking a 7815 linear regulator onto the board to test if it worked at all (Result: Yes, the board itself is fine, and the linear regulator would be also fine if I wasn’t wanting to run 30+ volts).

Defeated for the night and about to switch Clocker back to Victor 883s, I read once more through the datasheet hunting for possible thermal limitations or some indication that I had totally fucked up the component layout and selection. The datasheet had a section on inductor selection, which I’ve handily ignored in the past. Within it was this equation:

Okay, sure, let’s try it… For my setup, with Vin = around 25 volts and Vout = 15 volts, I got a V-us value of 38. Now let’s see the inductor selection table:

Oh dear.

For as long as I’ve used the LM2594, I’ve just tossed a 100uH inductor onto it because it was cheap on Digikey for the package size I designed with initially. And for as long as I’ve used the LM2594, it’s always been a nervous wreck.

My voltages and load clearly put me in 330uH territory. The microcontroller and receiver combined generally draw 50-70mA combined, with the beefy Spektrum receiver demanding 100mA when it is searching for a transmitter. So for the purposes of modelling, 0.1A is a good number to use for load current.

So what does too-small an inductor do to the system? Several possible things. First, current can change (ramp up or down) faster – there’s less “mass’ to it, so to speak. Second, as a result, the regulator can enter “discontinuous” mode, where the energy needed by the load per switching cycle results in a duty cycle so small that the current (which can now change faster) falls to zero instead of continuing to flow (“continuous” mode). The LM2594 should be able to handle discontinuous; but next, because the current can change faster due to the inductance being too low, it could very quickly reach a level which throws an overcurrent fault in the LM2594 regulator. In other words, it wakes up, turns the inductor on, goes OH FUCK, enters shutdown, wakes up, turns the inductor on, goes OH FUCK….

This really all speculation, but the observation that most supports the inductor-too-small theory is the fact that the LM2594 only reduces its switching frequency if it is in the overcurrent condition. I shouldn’t be able to hear it otherwise. I tested this by finding a voltage input which would run comfortably at 15v out and 100uH – about 17 to 18 volts seemed to do it. And there were no problems whatsoever. As soon as I crested about 20-22 volts, though, all hell would break loose.

Well then.

Could I have been defeated by just not reading the datasheet? Let’s find out. Luckily, I had some 330uH through-hole inductors left over from the construction of Segfault’s controller long ago:

Through hole, surface mount, same thing… just some bent legs.

This was the only change I made to the whole regulator circuit – all of my other hacks had been undone by this point, because I wanted to make sure there were no other layers of mods that needed to be made. Surely just an out-of-tolerance component value cannot be the root cause of my controllers’ undoing!

Except it was.

Whatever, I’m just gonna stuff it in the robot.

I’ve yet to experience another hiccup or reset since replacing the inductor. At all – the only thing which causes a reset is a sudden applied load on the logic side – plugging in the Spektrum receiver when the board was not doing anything previously, for instance. In *that* case, certainly more buffering capacitance could help, but if the receiver is suddenly plugged in during battle, something has gone very, very wrong.

I test drove Clocker for a few minutes in this configuration (only the drive – the other controller isn’t installed) just to test the robustness of the fix. This test included several power cycles both “hot” and “cold”, to see if the thermal issue could have been part of the problem. The regulator does still get hot, but it seems to be a normal hot.

It’s time to install the next board! By this picture, I had already pulled the 100uH inductor off every single Ragebridge board current in existence and replaced them with the 330uH thru-hole ones. I was now test lifting various chairs and handtrucks with Clocker.

Playing with the current limit setting, I got the main output gear clutch tuned to the point where the bot could quickly lift 30 pounds on top of the fork, but if I stalled the fork against the frame, there wouldn’t be enough current to do any damage. The way it was accidentally tested was by wiring up one lift motor backwards so the two motors were fighting eachother the whole time, while I stared dumbly on and went “Hmmm, I wonder if something’s jammed”. I am vaguely glad for the existence of (my and only my) current-limited controllers.

The prescribed current limit for the fork is 30A, at which point the two motors will produce 50lb of upward force at the tip of the fork.

Alright! I’ve fixed the electrical problem, so let’s bounce the problem back to mechanics. That is my right side output gear, not being where an output gear is normally found.

How did this robot *ever* work?

Remember at Dragon*Con 2010, the left-side output shaft of my custom DeWalt frakenboxes lost its shoddy press fit, leaving the bot mostly immobile. That side was repaired by knurling the output shaft where it meets the planetary gear carrier, and it hasn’t been a problem.

I’ll admit that the discovery of this problem was just a little unconventional:

Hey, if it has a current-mode, I’m riding it, okay?

Definitely a useful way to stall-test your drivetrain. Each drive side’s current limit is set to 55A, which actually seems kind of low for the DeWalts. They definitely want more when spinning up. Maybe I should have double-bypassed the current sensors to get peak 90A limits, but then I’m pushing the board’s physical layout in terms of current density in the copper layers.

During some enthusiastic turning, the right side suddenly stopped producing torque, and I heard the dreadful sound of “motor spinning up unloaded”.

How did this robot EVER work?

Clearly the answer is it never really worked, if I’m hammering out one problem after another. I repeated the knurling and re-fitting operation on the right side, and all was good.

Now let’s return to the electronics.

During another driving and lifting stress test, something exploded in the bot.

Crap. It’s already like 3 in the morning by this point, and I’m wondering if now that the fundamental disability of the controller has been solved, the problems have moved down the line to the next weak link. Now, the controller that bit it was the drive controller, a.k.a the one I did all the nasty experimentation on before finding the inductor issue, so maybe it’s just weakened from being an experimental subject.

The failure was just indicative of massive shoot-through. Aren’t my gate drivers supposed to prevent that? What could possibly cause the FETs to suddenly overlap with disasterous consequences, besides like, leaving half of the gate drive chip unsoldered because I forgot to go back and solder all the pads after anchoring it?

Oh, it was that? Okay, carry on.

Here’s the final wiring arrangement for Clocker. It looks kind of like the same jumbled mess as it has been, but the wiring runs are alot better packed.

I made another spare battery afterwards, because this pack is getting high in years (and in number of dents in the cells – a little disturbing). Not only that, but Clocker has no less than 3 550 class motors and two DeWalt motors now, so it’s more stress on the battery than ever. It’s not going to run back-to-back matches for sure, so I need to have a swappable battery.

Here’s the bot all closed up for this year!

And now, a bit of test video, showing some more handtruck yoga.

So that’s where all those dents in the cells are coming from.

And here’s a shot of the 2012 fleet: Null Hypothesis and Überclocker Unicorn. Sorry, the nickname just kind of stuck – and it was also partially inspired by the fact that Gundam Unicorn is a real thing.

So that’s it. 30lbers only this year for me, with none of the ants and beetles returning. Instead, I have become the arena; more on that later!

Dragon*Con 2012 Schedule

Observant attendees will notice that I am a panelist in the Robotics & Making Things & Engineering & We’ll Find a Snappier Name For This Next Year track. It’s been a long peeve of mine to go to conventions only to discover that all the panels suck, so I’m out to fix it. In the past 2 years, I’ve been complaining that nobody really had a panel focusing on the where to get and how to use (not to mention the even go want to do look more like) – the resources of engineering projects, which for me at least was perhaps the number 1 contributor to actually being able to get nice stuff done. What to buy, where to buy it, where to scrounge and salvage it if you can’t buy it, and why it doesn’t work the way you think it does.

So I’m proud to announce the MAKER RESOURCES panel, which will probably be something like all of my Instructables slammed into an hour. Focusing primarily on parts procurement and fabrication resources for the non-shop-endowed maker, I’ll also touch on design software and design methodologies.

Next is the ELECTRIC VEHICLE TECH panel, co-hosted by real EV guy Adam Bercu. My role on this panel will probably be dealing with the smaller end of rideable objects, discussing the nuances of using hobby R/C parts and shitty e-bike controllers. We’ll likely cover basic EV drivetrains and power system choices as well as math for estimation and calculation of drivetrain properties. Basically, 2.00scooter in an hour except probably more 2.00motorcycle.

I’ll also be on the DIGITAL FABRICATION AND WATERJETTING panel along side long-time robot buddy Simon Arthur (some of you may know him as Big Blue Saw), in which we’ll touch on ways to abuse the waterjet and laser cutter, and the coolness of digital fab in general.

The 1100-mile haul begins soon.

Überclocker’s Fork is Still Dysfunctional Along With Every Other Part of the Bot

Hmm, well that ain’t good.

T-minus right about 48 hours before I be trippin’, and Clocker has just begun its journey back into one piece. I suppose this is really nothing new.

Here’s what the bot looked like at the beginning of this week. It’s not exactly complete… In order to finish up, I needed to remake several parts and wait on McMaster. Luckily, neither of these processes takes very long. The parts to be remade included the drive wheel assemblies from 0.25″ steel and aluminum, the top fork parts and new lower fork ‘tines’ from 1/4″ aluminum, and new sprockets from 1/8″ steel…the same piece of 1/8″ steel I’ve made Clocker’s previous sprockets and Cold Arbor‘s drive sprockets from, incidentally. It finally reached the end of its waterjettability with this round.

The first part up for rework is the top fork:

I played with the geometry a little more after the CAD images last time. The fork is now 2.5″ taller in the “gullet’ section, allowing it to park a bot up to 7.5″ tall (if the robot’s small enough to fit totally in past the grabby bit, that is). To reduce weight of the forward part of the bot, I’ve went back to standoffs for this part, as the .25” side plates provide most of the stiffness.

Here are the new front and rear wheel parts, in super-convenient pile-o-plates form. I machined the Delrin bushing from round stock on the venerable tinylathe. The wheel holes were drilled in-situ after a layer of sprockets has been piled onto the hub, since the hub plastic is dense air polypropylene and exceptionally soft. The weird flowery cutout in the steel is for weight reduction only.

One issue that came up is that these wheels are manufactured….not very consistently. The hub on the one I used as a model seemed to be a few hundredths thinner than this set that I’ve gotten, with the result that my cap screws protrude further than the plastic hub. This meant they would have ground against the frame instead of letting the bushing take up the thrust load.

Maybe it’s my bad for designing something so close in a combat robot, but whatever. A quick hit with the appropriate diameter drill let me “counterbore” the holes enough to sink the bolt heads down below the bushing.

Yep, it fits.

Once I made a full wheel, then I repeated the process production-line style for the rest.

Again, what the hell are with these wheels? They have excellent traction and predictability on smooth ground, but they are so inconsistent. There are also massive voids in the molding near the center. Someone either just does not care at all, or they know their industry well enough to know what costs to cut…

The sprockets were “rotationally filed” to achieve the tooth chamfer that is critical to making your chain not explode off.

Here’s the chassis on all 4 wheels as a fit test.

SO MUCH GROUND CLEARANCE. A whole 3/4″ of it. That’s alot for one my bots – Null Hypothesis itself was my first design to venture above 0.5″ on purpose. I’m a fan of low and wide bots because of my arena combat history.

Onto the fork actuator. Again, this thing is Cold Arbor’s old saw actuator, which I saved for this exact purpose.

There was one problem with the actuator, though. The spacing of the sprockets did not result in an integer number of chain links in the chain pitch circle – this was one of those design oversights I’m unfortunately too prone to. The end result is that the chain has always been extra floppy. Cold Arbor used to regulator ditch these chains because they would bunch up at high speeds and then get ripped apart by the drive motor when they got jammed against the side of the actuator frame.

I decided that there was no real value in trying to move the output center up a little, which would have resulted in me needing to machine some kind of weird adjustable linear bushing. Instead, I elected to add these ninja standoffs to act as crude chain tensioners. They’re made of some random steel round and screw onto the protruding motor mounting bolts. They keep the chain squeezed around as much of the sprocket as possible.

It’s worked beautifully so far.

Now with the leadscrew cut to the correct length.

And installed in place – note the former plate spanning the two inner tines now replaced by the motor mount.

Say hello to Clocker Unicorn (or perhaps Clocker Cyclops). The big rubber bumper doesn’t look as out of place as I had anticipated, and it’s very squishy – good for hanging onto people.

One problem I discovered was that the higher ground clearance meant the fork no longer got anywhere near the ground. The front structure of the lifter gearbox mount was just getting in the way. I counter-milled (milled with a countersink…) a chamfer onto the front and also took the height down a little bit. The amount is set such that the front fork will just barely not scrape the ground – I don’t want to deal with it potentially impeding the robot’s movement.

Starting to put everything back together now…

To reassemble just this part of the robot requires at least 2 sizes of ball-ended hex wrenches. Design win.

SO

MUCH

GROUND CLEARANCE.

And holy shit those chains. No wonder I lost both of them alternately!

The solution is a set of miniature eggy-cam chain tensioners. Unfortunately not roller, but Delrin’s slippery tendencies should make up for it. These can be adjusted on their mounting bolts to press down on the chain in varying amounts. Way better than my very non-adjustable failed tensioner attempt of yestercon.

A full set of installed eggy cam things (more formally just called cam tensioners) totally removes the slack from the chain, with plenty of takeup capacity left. If it stretches more, I might as well throw out the chain.

Alright, it’s time to test if my totally refreshed lifter parts can do anything! I clamped the robot to the table and hooked up the still assembled electronics deck to the fork. The bot’s 7S battery was reconnected, and…

… well then.

Okay, so I did managed one successful lift. Then this happened on the way back down when the fork hit the frame and the gearboxes kept torquing, even though the clutch slipped when that happened. The failure of the pins was entirely brittle – the break is clean across the diameter and there was no bending seen in the pins at all. The sound I heard from the outside was quite an impressive series of cracks.

Alright, alright. I get it. I suck at the mechical engeerning. A little more thought and analysis (and reading of the Chief Delphi forums of archived Banebots 56mm carrier plate threads) led me to decide that whatever the pin is made of and whatever the shaft is made of is simply too disparate in carbon content and possibly other alloying components to consider as one unit. One thing I could have done was to heat treat the shaft separately – pop the pins out, carburized/case harden the low carbon steel shaft to gain some strength, and shove the pins back in.

For now, however, I’m going to restore the system back to the way it was before I tried to fix it. When the gears are pushed tightly together as they are inside the gearbox, loose pins aren’t as much of a problem as them just shearing the fuck off.

I took apart two of my older spare HF motors for these replacement shafts. They have a carrier plate that is a full millimeter thicker than the ones I used, and which are more commonly found in newer cheap drills. Damn cost cutting measures – really, it’s an extra millimeter of length of a shaft that is machined from a single piece of steel and is over 50mm long so it’s not like that much material is actually being saved compared to what is being removed to get the shaft diameter itself.

And damn these pins if they come loose inside.

I put the gearbox back together with the output stages submerged in grease so at the very least there’s hydrostatic pressure keeping the gears in line (Not really, but if the pins do loosen, the gear face sliding against the previous carrier stage provides alot of support to the whole assembly).

One more mod I made to Clocker today as I rage-reassembled it was some stiffer springs on the ‘front shocks’. The old ones were actually way too soft and didn’t provide much support for the bot as it lifted an opponent, so Clocker faceplants were a more common occurrence than I would have liked. These are about twice the spring rate of the old shock springs.

Clocker is now mechanically back together and alot less jiggly than it was before. What is it time for now?

ragebridges.