Saving Private Überclocker: Adventures in TIG Welding

With Null Hypothesis pretty much squared away except for the occasional motor detonation, and no real leads on that regulator problem besides “wait more than 1-2 minutes before turning it back on”, I’m going to start fixing up my current ‘flagbot’ (like flagship, I suppose) Überclocker Remix.

And I’ll start off like I always do:

poor Überclocker.

Having been mostly neglected during my focus on building more silly vehicles than silly robots, Clocker has always been on the receiving end of last-week hacks and fixes to address only the issue that came up in the competition previously. In 2010, to fix the problems from 2009, I replaced the shitty 775-type motors with DeWalt drill motors. In 2011, to fix the issue of the DeWalt motors being used with improper torque transmission methods, I just press fitted harder. And in 2012, to fix the problem with the chain falling off now that the motors are relatively reliable, I’m gonna…

Fuck, I need to make actual upgrades to this bot. Changes that should have occured in 2010.

The actual story, though, started a few days ago when I wanted to spar Clocker against Null Hypothesis just to shake down the bot a little more to see what else might have been creeping up on it. Long story short, I couldn’t get the lifting fork to actually lift anything – something, it seems, was slipping before the big gear clutch. Having seen this problem before, I quickly decided it was the tapered gear shaft on the Integrated Dual Frankenb0x – the retaining screw probably loosened or stretched or something. But it was an excuse to open the robot up and examine it completely.

Often times, what you don’t know and can’t see is better off being undiscovered.

Alright, so here’s the bot. It’s just looking worse and worse too – dirt and grime has settled into all the little metal dings and gashes. Shiny new cuts in aluminum look ballsy and cool, but dirty oxidation does not.

Clocker is a great example of one of my bad habits of designing things very sequentially. In order to take out the lifter gearbox, I have to execute the following sequence of events:

  1. Remove the top lid of the bot and take out the batteries to access 2 of the 4 mounting screws
  2. Unscrew the other 2 mounting screw from the front
  3. Take off the front suspension legs so I can
  4. Take off the left and right side
  5. Remove the wheel and drive gear to
  6. Loosen the drive motors just a little
  7. Remove the top plates of the electronics deck to clear the wires and give space to the taper-retaining screws on the lifter gears
  8. Push the whole lifter assembly, fork and all, towards the back in order to clear the slot and tab mate
  9. Finally, wiggle it out of the bot while making sure none of the wires that are threaded through holes get cut or twanged.
  10. Un-press the bearing fit on the gearbox itself and push it out of its mounting cage.

Basically I unbuild the robot to replace a critical component. If I just dive in and go, it takes about 10 minutes to go one way.

And after all that is said and done, it pops out!

…with one of the bearings permanently fused to it. Hurray?

Ah, the IDF, one of my finest machining examples. Combining the parts from what must be like 6 cordless drills, it’s a twin 3-stage 216:1 reduction all made of shitty sintered iron and maybe steel.

When I test ran the motors still attached to the gearbox, something sounded bad. Really, really bad. Time to take those gears out…

Uh oh.

Gearbox 1 is missing all of the pins ever on its output stage. This is a known failure mode of the cheap drills when overtorqued, because the carriers are seemingly made of mild steel and the press fits just explode instantly, causing the pins to fall out. Some times, if the loads are light, wiggly pins are enough when backed by the intermediate stage to not cause much noticeable trouble.

Not good at all.

Gearbox 2, unfortunately, displays the same failures.

The funny thing is, I bet these gearboxes have been running like this since at least Motorama 2010 when Clocker might have experienced the highest shock loading forces on the fork. I’ve literally never touched this gearbox since when I closed it up in 2009 and now.

Well, tits.

I went from possibly having something which was still working now to either a few hours of machining to replace just those shafts (which might just fail once again), or cooking up a new and potentially way more expensive solution.

Time to take apart the gearbox some more.

I really liked the tapered shafts for these gears. They take some amount of screw pressure to transmit a ton of torque using the full width of the gear box (unlike, say, a pin or partial keyway), and in the worst case they just let loose and slip. They took a lot of effort to get back off – I had to carefully secure the gears by their teeth in a vise and tap on the shaft with a hard punch and hammer.

After getting the gears off and appraising the condition of the carriers, I spent a little while brainstorming about what to do from here.

Solution 0: Just remachine the damn thing from one of n drill motor shafts I have sitting around. Prone to failing in the same fashion? Yeah, definitely, because cheap drills are cheap drills. It would take less than 30 minutes because taper-cutter.

Solution 1: Machine a tiny Wubba-wubba drive! Prone to failing: Probably not. Probability of me actually getting it right: Ummm….

Solution 2: Overnight myself some Banebots P60s and emergency-redesign the whole system to accommodate them.

Solution 3: Attempt to weld the pins to the carrier plate, just the way they are.

I decided to try solution 3 first, since it involved 1. trying to use our TIG welder because this is a very precise and delicate piece, and 2. The shafts were already broken and could only get broken even more in the name of learning and practicing TIG welding, or fixed.

 

Say hello to the blob.

I’ve only used our cheap Harbor Freight TIG exactly once, and it was to weld the tungsten to the random plate of scrap practice steel 15 minutes after we set it up for the very first time. This was going to be fun!

Since MITERS is used to MIG and very few people (save for Amy) had aptitude with the TIG machine, it was a little neglected. I resorted to using a chunk of mild steel MIG wire as the filler rod, and it definitely took a few tries to not weld the tungsten. But after practicing on some more angle scrap, I began to like it better than MIGing. It’s quiet. Deceivingly quiet – just an eerie blue glow. There’s no disgusting splatter and crackling.

Next, I practiced on a scrapped drill shaft from some project I can’t recall, but it was in my little baggie of drill parts.

Taking Solution 3 can actually have very indeterminate results depending on what kind of steel the Shady Chinese Power Tool Co. Ltd. had lying on its shop floor when the shafts were made. Welding different alloys, especially a very hard high-carbon alloy like the pin (….presumably) to a low carbon soft alloy like the squishy shaft steel will most likely cause cracking in the heat-affected zone. Welding high carbon steel is just terrible in general.

But the pin steel, as tested with some pliers anyway, didn’t seem to be that hard. I’m not exactly surprised – given how shitty the shaft steel is, I wouldn’t expect nice hardened pins. I decided to preheat the whole mysterious assembly to a nice golden brown color (approx. 250 celsius)  based on this site’s recommendation for a medium carbon steel.

The results of my first practice run is shown above. It’s definitely blobby – I didn’t let the puddle form for nearly long enough. I started with a fairly low current of 40 amps, too, so it took a little while to reach even that stage. I might as well have MIG’d it.

Attempt number 2. I think I’m getting a little better at this. I upped the current to 60A, and it just looked better. The liquifaction starts at the top of the pin (where I aimed the electrode) and propagates to the body. Once the puddle became a few millimeters in diameter, I dipped in a little MIG wire.

Since the IDF runs with just 1 0.5mm shim of clearance between the carrier and bearing, I decided to machine down this weld to see if I made it reasonably far into the steel. Interesting effect – the pin’s weld puddle is much harder than the surrounding steel.

Here’s 3 attempts on 3 scrappy drill shafts, all machined down. The first one on the left clearly shows some porosity and lack of filler, and I’m really starting to like the 3rd on the far right. Overall, I’m liking Option #3 as a potential saviour technique for my drill shafts (as well as preserving them from being damaged in the first place). While it may be metallurgically unsound and may earn me cold stares from certified welders, I’m going to guess that any metallic inteference is better than a slouchy press fit.In the best case, the metals join reasonably strong and I up the failure ceiling a few more ft-lbs and I win. In the worst case, I’ve made a mushroomed metal head on the pin so at least it can’t just slip out backwards, and I…. win?

Enough practice – hows about some production parts? These are the two drill shafts after welding and machining. The one on the right I kind of skimped on surfacing a little.

Unfortunately, my joy was short lived. I’m now going to admit a slightly embarassing mistake I made with all of these practice pieces: they were water quenched. Like straight off being red hot and into the sink.

Anything I’ve welded in the past has been gigantic, ugly gorilla welds in mild steel which can’t possibly heat treat in any way, and dunking the parts in water just became second nature to cool them down quickly. Well, with alloy steels and medium to high carbon steels, I think that was a dumb.

The pin there broke off after I put two stages of planetary gearing back into the gearbox and turned it by hand. Sadness.

Alright, it looks like I’m going to have to remachine these shafts anyway. However, before I do so, I will weld the pins in place and then more gently cool them down, possibly even applying a proper tempering process. MITERS does have a tiny oven which has been used by some members to cast aluminum.

So, how’s about them TIG welders?

Null Hypothesis: Finishing Up and Testing the Ragebridge v1

This it!

The post where my project gets stuck in motor controller hell. It happens to everything I try to build a motor controller for, so be prepared to read about exploding FETs and endless nondeterministic troubleshooting!

Well, the story is actually slightly happier than that. In the past few days,

  • The frame has been put together, all the wheels mounted, and battery pack made
  • The Ragebridge has been installed and the robot successfully tested and driven using it, seemingly exhibiting no adverse noise issues
  • The Ragebridge has been successfully detonated when a motor suddenly locked up.

Let’s start from the beginning.

With the wheels bored out, making hubs was the next step. I decided to make these more consistently on a Nice Thing with digital readouts, courtesy of the Edgerton Shop (They were also the nearest straight knurling tool, incidentally). The knurls will help the hubs not slip and strip out against the soft plastic bore of the McMasterbots wheels.

The hubs are installed with an arbor press.

Two wheels in. The hubs are very conventional “drill hubs” with a 3/8″-24 internal thread and internal shoulder that lets the reverse-threaded 5mm locking screw hold the hub in place against both threads. I’ve always found it funny how the ‘cheap drill’ converged solution ended up using an English and a Metric thread on the same part. The outer thread is not M10 or a metric thread, and the internal thread is definitely 5mm x .8 and not #10-32, which only fit down to a few threads. I’m mildly interested in how shady Chinese factories converge towards which design to copy and paste.

It’s starting to look like a robot now.

I temporarily secured the front wedge on with a few screws in preparation for a test drive.

Okay, robot’s done. Nothing more to see.

Using a spare 7S battery pack (from the long-retired Segfault) and the Botbuttz controllers I bought a while back, I threw together this test rig just to try out how the robot handles. Conclusion: It handles like a pushybot with a wide wheelbase and near-midpoint center of gravity! Unlike Clocker, which is back-heavy out of necessity and so oversteers very easily, this thing is extremely stable in turns. The braking effect of the BB controllers also contributes to snap turning response and smooth starting and stopping. These are all features I want to make sure Ragebridge can duplicate.

I spent a while scooting this around before moving onto the real challenge: the aforementioned Ragebridge.

I soldered up another board (the first one having been put into the Silly Media Lab Vehicle). There were no changes made to the board layout, only the modifications addressed in its first post which included moving the low-side gate drive bypass capacitors into the right place.

After verification of the hardware functionality, I spent the majority of 2 days hammering out an R/C input firmware for the board. I added features in the following order:

  • Radio signal-loss failsafes! I figured if I just wrote a dumb Servo-to-PWM program, it would stay that way forever. ESCs for robots need to have a way to shut off the motor outputs on recognition of signal loss, so the robot doesn’t just keep rolling. This part wasn’t difficult: every time a valid servo pulse is received in the pulse-reading interrupt, a flag is set to indicate the reading was good. A checking routine runs once per half-second to see if there were any channels which did not set their validity flags – if there are, then the gate drivers are shut off until all signals are good again.
  • Command ramping, meaning the ESC takes a small amount of time to slew from one direction to another. This is to prevent instantaneous “step reversing” which can cause huge current draws, up to twice the DC bus voltage to appear at your ESC’s power inputs, exploding motors, and other terrible things. I created quite an epicly named function, mapWithRampingAndDeadBand(); to perform this task.
  • Linear stick response. Directly mapping of the pulse widths to PWM output %, through the ramping limiter. This was just to get the robot driveable so I could add…
  • Invert mode, a feature which allows you to redesignate which end of the robot is forward. Useful for invertible bots like NH because otherwise your robot would drive backwards when upside down, as well as have swapped turning behavior. This is activated by the channel 5 toggle witch on my radio, and just swaps the left and right commands while adding a negative sign.
  • Exponential stick response. This makes the initial response ‘softer’ so the robot can be driven more precisely when moving slowly. It tends to reduce spin-out and oversteer tendencies. It is implemented as a simple lookup table – raw command goes in, expo command comes back.

I began to add a ‘stick calibration’ mode, but decided against it for now. Stick calibration is a luxury, because radios can generally be trimmed and have their endpoints modified to suit just about any desired pulsewidth range.

The current version of the RB firmware is here. It’s written in Arduino 1.0.1 (which finally got rid of the CTRL+SHIFT+M bug!). I generated two exponential response tables, one a little more expo than the other. I found the “more expo” one to be a little sluggish in responding without moving the sticks really far, but your experiences may vary.

Ragebridge v1 installed in the robot on its little standoffs…

After testing the controller out on a power supply first (which has nice features such as non-infinite current output in case of a fault) I put the test battery right back in because I hadn’t gotten to making the actual robot pack yet, then made some test driving videos!

The “basic test” was done right after I finished the linear stick mapping, and the more advanced testing is with exponential.

I like Ragebridge’s functionality in those tests. While open-floor driving is not as strenuous as pushing another robot (trying to push you back), high-current transients like from reversal and stopping will be what causes noise-induced failures and latchups. I didn’t observe any of those issues during that test driving, which bodes well for more sustained heavier loads. Unfortunately I neglected to put a Wattmeter on the thing, so I don’t actually have power throughput figures. Typical drill motor drivetrains in this size robot will draw about 30-40A per motor accelerating or changing directions and about 15 amps spinning out the wheels.

With at least a slim prospect of the robot working properly, I started putting together the battery pack:

The voltage of this pack will be 8S2P in lithium iron phosphate, so 25.6V nominal and 28v peak. This high voltage was just to get the speed I wanted out of the motors, but it could prove to be the proverbial Achille’s Heel… or perhaps Achille’s A123 Cell for the robot.

Those yellow things are PTC thermal fuses. I usually just lay more busbraid across the cells to put them in parallel, but the PTCs add another layer of failsafing by limiting the rate at which the cells can differentially discharge. If one cell suddenly fails (or gets punctured) and becomes a short, then the very low resistance of the bus braid will let the other healthy cell short into it, potentially causing even more damage. The PTCs will at least limit this to a reasonable current.

I chopped up some spare 4S JST balance extension cables for the cell taps. Generally I make my own using JST-XH connector shells and pins (e.g. Digikey’s 455-2217-ND), but it takes longer (and most chargers don’t have slots for 8S pack taps anyway). One JST lead is the lower half of the pack and the other is the upper half.

The balance leads are insulated by some Kapton tape layers as I soldered them in. I didn’t want them laying directly on the cell tops any more, after this happened, but on this pack there’s no space to route the wires only down the sides of the cells (it will be laying flat in the robot).

Since this pack will be more enthusiastically jostled inside the robot, I’m giving it a layer of foamy armor.

And after Giant Heatshrinking! I gave the pack its initial balance-and-charge wakeup routine with my 1010B+ charger doohickey.

While the pack was slow-roasting to perfection, I took care of the last bits of electrical work on the robot itself.

This adorable ABS-printed connector holder houses a Deans connector and an XT-60 connector. The Deans is going to be the master switch via the “Georgia Tech Key” method. In order to prevent myself from being confused at which terminal to charge the battery from versus to bridge together (hint: bridging the battery is a bad thing), I just used a totally different connector for the charge plug. The connector housing is mounted to the bottom plate.

This is the “Georgia Tech Key”, so named because I’ve only seen it on random GT creations before. Technically it’s called a removable link.

I made this loop a little too long and it sticks out the top of the robot by a little, but some tape should keep it down. In the worst case I’ll just replace it with a directly shorted Deans connector.

Alright, here it is. That’s it. The robot’s done.

Now to hand it off to other people and drive it around until it blows up! The best way for me to find the weaknesses of any of my builds is to give it to other people. I subconsciously go easy on things I know may be unreliable (the so called Creator’s Mercy), but other people have no concept of, say, the drills being severely overvolted, or for instance in the case of the Chibikarts, that flooring the throttle from a standstill may not result in satisfactory behavior.

I wasn’t able to whip out the video camera in time to capture the magic moment, but what happened was this:

And this:

What triggered the failure was a MITERS member driving (very happily and enthusiastically) NH into a solid machine frame at pretty close to top speed. The robot stopped moving, and I instructed him to give a little throttle to see if the controller was still working. It’s actually very difficult to judge what people consider ‘a little throttle’ based on experience, so I can only assume that it was near full stick or something. A small explosion was seen inside the case.

The superficial damage was the blown ground trace and one very, very unhappy MOSFET. The actual damage, though, as I found out through subsequent hours of checking and debugging, was pretty much the whole thing. The Arduino Nano was fried (noted when I powered it back on and the ATMega started drawing half an amp…), as were all 4 FETs on the exploded side (gate-to-source shorted through), both of the IRS21844 gate drivers on that side, the gate-to-source protection zeners (which I’m sure did all the protection they could before exploding), as well as half of the 74LS08 quad AND gate that is involved in piping the PWM signal to the gate drivers.

Damn.

After replacing most of the semiconductors (and the Nano) failed to make the controller work consistently again, I decided to just scrap this board. Some times, enormous transients damage components in ways that are unexpected

And that’s the motor after I took it apart.

I think a combination of being oversped (18v overvolted to 28v) and sudden impact (running into the solid object) must have blown off the commutator segment. These motors are not really what I would call “well made” since the whole drill they come in costs $20. Running 28 volts is certainly up for compromise. But, I enjoy the speed of the bot on such a voltage.

Good thing I have a small forest of these drills!

So the final version of the story is, NH was finished and blown up, and is now unfinished. Putting together a new controller (with some mods I will discuss below) and pitching the bot  back together should not take long.

ragebridge mods

Now that I’ve gotten to know Ragebridge a little better, I have in mind a list of issues and changes I’d like to make when it’s time for version 2. Since V1 actually seems to work fairly well, this is not an immediate task.

First issue to take note of is of course the misplacement of the gate drive bypass caps. This is what led to the very first board not working at all until I changed it. Clearly this is a routing matter and the second version of the board has to address it, but for now it is easy to hack by hanging the little capacitor off the pins of the 21844 chip.

Second, the row of input headers at the top of the board are not in a very useful order. Two of them were designed for “servo” style connectors which are terminated signal-power-ground, and two for ‘potentiometer’ style inputs, terminated power-signal-ground. Well, I got the servo input order wrong anyway – those two headers actually are connected signal-ground-power. So I had to make two different styles of deceptively incompatible servo cables to suit the board because I needed 3 channels of input. I’d just line them all up signal-power-ground in the future, even the “analog” inputs, for better consistency.

Next, and an issue I can hack around for now, is the way that the power traces narrow awkwardly and haphazardly:

Those yellow highlights is where the trace width drops significantly from the average. The ones at the left and right bug me in particular – I was definitely trying to keep the controller within a reasonable width, but the heavy power traces around them are easily double the width and very short too. I was intending to reinforce those with braid or wire, I think, but didn’t actually do it for this board.

The circle near left center is a really obvious design derp. Most of the power stage of the board is basically rotated 180 degrees from eachother, except that current sensor placement. The right side of the board has no corresponding narrowing – current flows down through the vias on the right center two FETs for MA2, and the right side’s current sense path is a straight shot.

Given that these traces are all made in 1 oz copper, they’re exceptionally prone to sudden current overloads, such as a near doubly-overvolted drill motor suddenly slamming to a halt. The current sense path on the left is a little more difficult to reinforce since it’s under that FET, but as long as I’m not using the current sensor on these boards yet, I could hard-jump it with a piece of wire.

Overall, I’m satisfied with how Ragebridge has been behaving (short of being inductively crowbarred by an unhappy drill motor), and I’m going to be building more of them to test on other bots soon…and of course fixing this one.