X

Nov 13th – Torque and Diameter Sensor Bugs

Hi everyone,

We’re taking a break from production this week since there’s not much to report there – we’re moving along slowly but surely, shipping units consistently but still slower than we’d hoped. See the previous update if you missed it for any news here.

Instead, we’d like to address another issue that we’ve noticed – a huge thank you to our existing customers for their help in tracking things down, and in their patience and understanding given they never signed up to be debuggers! For those that just want to know what’s wrong, and what we’re going to do about it, read everything until the line break – after that, we’re going to get technical.

The first issue has to do with minor variations in our torque sensor, which can affect user experience and may also have a minor performance affect in terms of throughput. We’re still looking into exactly what it affects, as it is admittedly pretty minor. So far what we know is that some units have been skipping auger steps based on improper torque readings – which should never happen, of course! – and others have slightly less throughput than we’d like to see. If you notice your unit skipping (a loud, violent “click” from the auger during extrusion) please let us know immediately and we’ll work with you to tweak your settings so that your unit performs as it should. Long term, we will likely address this issue by calibrating the torque arms more rigorously before they leave – so we don’t expect any lasting issues here.

The second issue, slightly more upsetting, is clear filament. Despite our best efforts, some pretty thorough investigation has revealed that our diameter sensors aren’t tracking clear filament as accurately as they could be, resulting in fake noise that can confuse the control system and decrease performance, at times considerably. This is of course not acceptable at all, so we’ll be throwing some extra weight behind it until we’ve figured out how to best manage this (almost for sure, it will be a software upgrade – we will keep anyone posted if we have to change any physical parts to fix things, and will send replacement parts to any units affected if it comes to that, free of charge). In the meantime, we think it’s fair to just give everyone that already has a unit a packet of Masterbatch and some more pellets so that they can extrude anything other than clear filament (white is fine), so that they can use ProtoCycler until we’ve gotten this one bug sorted out without issue. If you really, really demand clear filament, see the technical section below…it’s not totally useless, just not the quality we’ve promised you. We’ll be e-mailing everyone who’s already got a unit or will be getting one shortly on their color preference, and should have gotten stock and supply sorted out within a week or so. For everyone who’s placed an order but has yet to get a unit, your pellets will come pre-coloured – we’ll reach out to ask preference, but will not hold your order until we hear like we do with address confirmation (you’ll just get blue…ReDeTec blue, of course!)

We understand this can be a bit upsetting to hear after such an incredibly long wait – shouldn’t they work flawlessly by now?! – but unfortunately this is one of the realities of bringing a brand new product to market. We’re doing everything we can to keep you all honestly informed, to remedy the situation in the short term with a bit of a band-aid in terms of free pellets / masterbatch, and to get it sorted out as quickly as possible to minimize any disruptions. While it would be much easier to either call it “insignificant” or just add a disclaimer that performance is only confirmed with opaque filament…that just wouldn’t be fair to all of you, especially after waiting so long for the product you were promised.

That wraps up the high level update, but if you want to really dive down into the specifics, see below – and apologies again for this less than stellar news.

-The ReDeTec Team

___________________________________________________________________________

Tech overview – Torque Sensor

As you may already know from our previous updates, calibrating the torque sensor is important – the orientation of the lever arm that rides along the chain at “0 torque” can vary enough to change the scale of readings. The obvious solution and one we’ve implemented on all units to date is to allow some adjust-ability in the chain tension, by moving the stepper motor in it’s sliding mount, to ensure that the torque arm is always at the same angle at rest. Here’s four extruders, showing exactly what we mean…and the new issue, if you can spot it!

DSC01630

Note the angle of the torque arms (bottom left) – the “long” leg should be ~5 degrees off of vertical, like the right three extruders – the left most is exactly vertical, which would affect the torque readings, and is usually fixed by moving the stepper ever so slightly to adjust the chain tension until the lever just – and only just – rests right on the main spur at zero torque (the left most extruder here will never reach that point, meaning the torque sensor is always “pre loaded” – totally messing up our measurements).

Did you spot the new issue? If not, don’t worry – we didn’t either until we started digging pretty deeply. Look closer at the screw head that the right side of the chain passes over. The left most is covered – because the chain tension is a bit too tight. The second from the left is just uncovered where the hex head would go – a tiny bit too loose, as we could take up some slack and still have the torque idler rest against the spur. The 3rd one over is perfect – the torque arm is at the right angle, but there’s no excess chain tension. The 4th one (right most) is the kicker! While it’s very hard to see, the angle of the right most torque idler is actually a few degrees closer to vertical than the middle two torque arms. This would suggest the chain is a little too tight…but there’s already too much slack in it, as you can see from the un-covered hex head of the screw. This means our torque sensor has a constant ~2-3 degree offset.

Does this seem incredibly minor? It is. Are we really splitting hairs over 2-3 degrees? We are. While it might not seem like much, across a max range of ~40 degrees, that’s 5-7 of our value%!All it does is change the 0 offset of the torque function in code – so your machine might always run a little higher pressure than normal, or a little lower. It still responds quickly and actively to any changes in torque, the readings are equally accurate and the function is still relatively linear. The only change is that, instead of reading “0-50″ for torques of 0-50, now it might read “2-52″, or “-2 – 48″. In the first case, the reading would always be artificially high – so your unit would slow down slightly, and end up being a bit slower than normal. In the second case, the readings are always a bit low, and it might try and take a step with the auger when it actually shouldn’t, and skip.

Of course, the solution here is to just calibrate the torque sensor for each one operationally, instead of adjusting chain slack when they’re put together. The cause, most likely, are slight variations in the sprockets we’ve sourced – their spec isn’t overly tight as it generally doesn’t matter within ANSI chain standards, but for us it turned out to be a minor inconvenience. We will of course continue to monitor and test this issue to ensure there aren’t any further negative effects we don’t yet know, but as of right now it’s thankfully extremely minor, and should be “already solved” going forward.

Tech overview – Clear Filament

This one involved quite a bit of hair pulling. It’s no secret that we spent a very long time on our sensor algorithms because, well, detecting filament diameter is hard! Unless you have a perfectly collimated light source or a laser micrometer, shadows, excess background light, and lensing effects with clear filament can all make things *very* difficult to get an accurate estimation.

Sensor responses

In the ideal response, the edges are perfectly flat, and exactly at the edges of the filament. In real life though, you get a much “softer” response – which makes things much more difficult! To add to the problem, clear filament can have a “lensing” effect – where it acts like a lense and creates multiple other “high points” of light in the pattern, throwing off everything. Never mind dead pixels, poorly aligned lighting, etc…

 

Using some tricks from AI, our current algorithm effectively looks for “edge features” and weights them in terms of their sharpness, magnitude, and relative location. This allows the algorithm to be pretty robust against dead pixels, lensing effects, varrying amounts of background light, etc – and in normal situations the algorithm always achieves sub pixel accuracy, resolving to about 0.01mm across a range of sample rod gauges – from 1.5 to 3.0mm, with the lid open or closed, and even with lensing filament, we’re almost always within 0.01mm of the calipers…which is all good news!

The bad news is that, relatively recently, we’ve discovered that the algorithm isn’t actually as robust against lensing in practice, as it is in controlled tests. What we mean by this is that if you hold a piece of clear filament stationary, the sensor will resolve it perfectly. But if the lensing pattern continues to shift and change, it can confuse the algorithm (likely based on the weighting of relative positions, which change too quickly, but we’re not 100% sure yet), leading to excessive noise. This noise confuses the feedback algorithm, and it ends up trying to compensate for problems that aren’t there – sometimes with bad results.

Of course, discovering this issue took quite a bit of time – whenever we’d check the sensors or do a study on the algorithm, it would perform near flawlessly…but our clear filament kept having issues. We initially thought it was a calibration error – maybe the colourant somehow made it easier to extrude consistent filament? – but after really diving into the issue we realized that the filament being created was equally consistent whether it was clear or opaque – ti was just the sensor readings that could vary. We ended up logging 30 second samples across many different units, filaments, etc, and then comparing the 30 second samples with the actual filament extruded. We did this with the control systems off – to ensure that it would not in any way alter the filament from what our sensors had read – and then again with the systems on, to see how the noise affected the control system and therefore the output.  Here’s what we found:

Black filament, no control system:

Black control

Here’s 30 seconds of black filament if you extruded it from a system without feedback…not to mention any competitors! We happened to catch this one on a “downswing” that would normally be compensated for in feedback control, so try to ignore the general downward trend…it will climb back up the same amount over the next 30 seconds, we promise! The filament generally wavers within a 0.1mm range, though across the entire snippet our range is 1.74mm – 1.91mm – a 0.17mm spread and hitting diameters that are a little too large for some printers! The main point, however, is this – there aren’t any sudden jumps in filament size. Even completely “unintelligent” extrusion isn’t awful – it’s just not very good. Compare this to the clear filament response below…

Clear filament, no control system:

Clear no feedback

First and foremost, please note the scale of the Y axis! It is not the same as above, and we do not want to mislead people by changing our axes…that’s just how excel spit them out and given our perpetual shortage of time, we just left it with this disclaimer. That being said, the error is pretty clear – at around 2027, the filament drops .15mm in the span of 3/10ths of a second, and then jumps back up just as much about 5 seconds later. This is an exemplary failure as it’s much larger and longer than most – the brief blip around 2037 is more normal. But it’s very easy to see that this response pattern is nothing at all like what we got for the black filament…even though the physical measurements for both are nearly identical. That huge drop at 2027-2032? It doesn’t actually exist – this is what the filament actually looks like when measured with calipers (keep in mind this is a bit of an estimate based on our measurement ability, but you get the idea):

Clear no feedback actual

Now our range is 1.62mm – 1.82mm, a 0.2mm spread and pretty much in line with what the black filament does.

What about the control system?

This is where things get interesting. The control system is actually designed to ignore massive jumps in filament diameter, because it presumes they are impossible, and are an error, and so it reacts to them a little slower than it otherwise would (a long, long, long time ago our sensors used to crap out and throw garbage values every once in a while, and at the time this was the quick fix…it’s stayed ever since). So while it gets confused for a second, it doesn’t *completely* screw things up. We’ve yet to find it so out of spec the control system fails outright. That being said, the quality of our clear filament is pretty much bottom of the barrel right now – +/- 0.1mm to +/-0.15mm based on extraneous factors – and that’s just not good enough. So, as we mentioned, we’ll be going back to the drawing board somewhat on our algorithm to give it a fighting chance with clear filament as well, and we’ll keep everyone up to date on our progress in the meantime. As mentioned earlier, we’ll also send some master-batch of your choice so you can extrude high quality filament. If you really need clear, please e-mail us at support@redetec.com and we’ll see what we can do behind the scenes for you. Ultimately, we’ll get our clear filament to the same quality all of our coloured stuff is: +/- 0.05mm throughout any perturbations.

Black feedback

Of course, we’d like to get things even better, and just before we started shipping we’d done some very interesting work on smith modeling and predictive feedback, to try and get rid of those short term 0.05mm oscillations you can still see in the above response. We’ve had to put that off while we get through production, as we already meet our performance targets but are way behind schedule in shipping! But as we switch some more effort back to our sensor algorithm to solve the clear filament issue, we’ll also be getting back into our work with Smith Predictions to try and improve everything even further – and will of course release all of that as a free firmware update once it’s ready!

If you’ve made it this far, give yourself a pat on the back – and thank you for being as interested in high performance filament extrusion as we are. That’s it for this week, but we will continue to update everyone on both these two bugs and our shipment progress as we make headway on both. Thanks everyone!

Oct 31st – Quick Update on Shipping Progress, etc

Hey everyone!

What a crazy fall we’re having –  We’re actually somewhat on time with this update, and we’re actually still on time with shipments! Heck, if the Toronto Maple Leafs weren’t starting to choke already, I’d start looking for flying pigs…

DSC01405
Another skid of ProtoCyclers, ready to head out the door!

We’ve currently shipped out 25 units and should have another 30 shipped out by the end of this week (we might only make 10, with the other 20 leaving start of next week – but we’re trying to clear 50 unit by weeks end). FedEx has been quite sneaky about picking up the units without us noticing (unless there’s a loud knock on the door, it can be hard to hear visitors over the machines!), but we did manage to finally catch them in the act and get a few shots of some units heading out the door:

DSC01407
David managed to catch FedEx trying to grab our boxes – they’ve been sneaky about it in the past!

DSC01408
Another cartload going onto the truck! 

DSC01409
Loaded up and on their way :)

 

The rest of production is continuing to move along at a relatively steady pace and we aren’t hitting any more delays, though we aren’t quite at our goals for throughput yet. We are still looking for one more staff member to help us with production and do have a few more tools / machine hacks left to keep speeding things up but at this stage we’ve shifted towards making this run go as smoothly and quickly as possible…then we can re group, double down, and get the next one out even faster. This is a bit of a safer, more reliable approach than trying to improve our processes in “real time”, as it ensure we’ll get through the whole run before we start trying to fix things that aren’t broken!

DSC01406
Machining and assembly happen in parallel now – so while units for this run get assembled, tested, and shipped…hot sections for the next run get bored, drilled, and tapped!

 

Last but not least, some housekeeping on our website, customer service, progress tracking, and updates. Up until this point, as you all know, we’ve been fighting to get production up and running steadily pretty much non stop…so things like forums, “normal” social media posts (“Look at this cool thing we printed!” , “Watch as we try and recycle water bottles!”, “Check out what highschool X did with their ProtoCycler”, etc), and of course customer communication and relations have all fallen by the wayside. With some extra staff helping us out, fully trained, and production now ticking away trouble free…it’s time to start focusing more on these. The first step is our production tracker, which is nearly done. It’s taken a lot longer than we thought because the logic behind it is actually quite complicated – with over 200 parts, all of different batch sizes and production schedules, lining everything up so that you can find out when your unit is ready is quite a challenge! That being said, our database is now complete, and we’ve only got one little logical snag left keeping us from rolling it out (“Batch size” vs. “Run size” – In other words if we decide to do runs of 200 instead of 100, then the first run will take twice as long to produce, but the last run will be sent out earlier, as we’re eliminating half our setup time for the overall backlog). Long story short, that should go live next week! It will kick off a series of change through November where we’ll switch over most of our social media away from production updates as the progress tracker takes over, and categorize our e-mail a little better so we can divide and conquer there as well (some people know we tried this in Sept/Oct to less than steller results – don’t worry, we’re doing something different this time!).

Progress tracker screenshot
This is a brief look behind the scenes at our database for the progress tracker. The top sheet holds the relationship between all of the different parts, sub assemblies, and assemblies (for instance, roughly halfway down you can see that a UI panel requires a UI Idler, UI Light Guide, UI Puller, etc to complete). The bottom sheet holds information on each part – like how long it takes to make or setup, and the minimum / most you can make at once (to save setup time). Putting it all together will allow us to track upcoming runs based on run sizing, and the “yellow” columns get populated with exactly how much of each part we need, and how long that will take. We can then take that information, compile it together based on what units are in the queue, and then present an overall timeline for each run!

 

All of that means that by the end of November we should – almost exactly 3 years after we launched our campaign – be starting to operate more like a normal business. We’re not there yet, but it’s something we’re incredibly excited for, and have been waiting on for quite a while…as have you. So while we’re still going a little slower than hoped, this massive saga of getting units produced and shipping seems to be finally drawing to a close…and we can’t wait to start the next chapter when things get really exciting!

Thanks,

-The ReDeTec team

 

Oct 19th – 20 units ready to go out!

Hi Everyone,

As promised last week, we’re just about to ship out another 20 units today to start off this next batch of 100! Here they all are assembled and ready for UL testing and final inspection…

DSC01403
Our first rack of 10, ready to go – one unit is out for inspection and is getting its lid, on the right.

Once they get inspected, the lids go on, firmware gets loaded, and they’re run through a quick diagnostic one by one – before getting loaded into the testing chamber for a break in. How many ProtoCycler’s can you spot in this photo?

DSC01404
We’ve got one unit getting firmware loaded (the laptop and white cable on the left), and Alex is actively calibrating a unit on the right…how many others can you find? Note also all of the completed extruders, waiting for the next 20 chassis to be assembled and sent out!

We’re still rolling out our tracking system, finishing up firmware updates, and fleshing out the support section of the website – but those should come pretty soon over the next few weeks as we continue to ship out this run!

Thank you all for your patience and support as always, and stayed tuned as we shift into a more continuous production environment…things are finally starting to get exciting :)

-The ReDeTec Team