Collecting Timelapses - Need your help

[yasas]

Hi all,

I am Yasas, a software engineer working with Duet3D.

We are working on a part quality monitoring project (which is also my Master's research) and we need your help with a few things things.

• A set of calibration images for your printer (step 1 of Instructions.pdf)
• A calibration print (step 2 of Instructions.pdf)
• Timelapses that you've been recording/will record as a collection of images and and the associated gcode that you printed.

Attachment - Instructions.pdf

For sharing, feel free to use Dropbox/WeTransfer or a similar tool.
I can also arrange a Google Drive link upon request.

PS:
- Make sure not to upload any prints with commercial sensitivity.
- Any images/emails/files you provide will be handled/stored/used safely.

Thanks very much!

[stuartofmt]

@yasas

A couple of aspects are unclear:

Lay the page on a flat surface. Take several (4-5 would be ample) photos of the printed pages (from step 1) using the camera used for recording 3D prints.

In many timelapse captures, the camera is fixed to the printer (or some adjacent location). The bed and camera then move relative to each other.
Is the intent of these calibration images to understand changes in perspective as a result of this relative movement ?

If that's the case, would it be better to instruct the tester to perhaps place the calibration pages on the bed (with one edge facing towards the camara) and take images at various positions of the z-axis? Would it help to know what the Z axis offset is for each image?

Note down the XYZ coordinates (of the extruder) position where timelapse pictures are recorded Relative to what? the camera? I'm asking because its not clear because XYZ is an arbirary concept. To some printer setups X = 0 Y = 0 is the center of the bed, for others it may be one of the corners. Similaryly, pretty much all prints start at Z=0.

[yasas]

Hi @stuartofmt

Thanks for the questions/suggestions.

So in Step 1, what we're trying to do is to calculate intrinsic camera parameters so we can get rid of as much distortion as possible. It's in Step 2 that we're trying to determine the perspective.

I believe that in the event the camera is fixed, one workaround would be to print the grids on a smaller paper(or 1/2 of letter/A4) and fixing it on a board(ex: chopping board) using clips, so the paper would be flat, and capture an image from the camera.

About XYZ coordinates: As most of the time timelapses are taken by taking the extruder out of the way and moving the bed towards the camera, it would be helpful to know those coordinates. For example img, in this case roughly X=0, Y=180. Now if we assume X=0, Y=0 is in the center. In that case, for the above image roughly X=-0.5bedsize, Y=-0.5bedsize.
(And yes, Z should be disregarded)

I'll update the document/post. Thanks very much for raising these. Hadn't occurred to me earlier.

Cheers

[stuartofmt]

@yasas said in Collecting Timelapses - Need your help:

Hi @stuartofmt

... As most of the time timelapses are taken by taking the extruder out of the way and moving the bed towards the camera,, it would be helpful to know those coordinates.

SInce I have done research, I appreciate the need for good data and hence clarity and consistency in the capture of that data. My questions are intended to help (and potentially reduce confounding variability in your study) so please bear with me.

There are a couple of potential issues with the quoted statement:

1. Moving the extruder out of the way (before capturing an image) can lead to poor prints. Poor layer adhesion, zits, stringing etc. I'm not sure that "most of the time" is true. If this is a requirement (or preference) of your study, you should include it in the instructions
2. Not sure what moving the bed towards the camera means. In my setup (without moving the extruder out of the way), the bed does not move relative to the camera, other than progressively away, as the layer height increases. If the extruder is moved out of the way (pause in printing) then the bed is moved away from the camera by a fixed amount, the image is taken, then the bed is moved back. In any case, the position of the bed, at the time of image capture is a fixed offset plus the layer height.
3. Are you asking for the coordinates of the extruder nozzle at the time the image is taken ? Would you mind explaining what the purpose for this information would be?. The reason I ask is that, again, I think it depends on the physical setup. In my case (Ender 5 Pro) with the extruder out of the way at the time of image capture, there is zero relative movement extruder to camera (for each and every image). This is because the camera is mounted to the fixed frame of the printer.

[yasas]

Hi @stuartofmt
Definitely. Appreciate the feedback. It allows me to think from many other perspectives which I didn't consider before. So thank you.

To address the points.

1. It's not a requirement. Step 2 in the instructions are so we can perform real world to image relation and find the perspective (elevation, azimuth, distance)
2. Mentioned it as moving the bed towards the camera because usually for bed slinger printers when timelapses are taken the bed is moved a bit forward for a clearer image of the print. Wanted to generalize a bit but seems I have done the exact opposite.
3. Yes, exactly. This information will be helpful in finding an offset per setup as we can make a more accurate connection using perspective data we determine/calculate from point 1 above.

Again, thanks very much for questions. Super helpful.. Will change the document appropriately

[stuartofmt]

@yasas

That helps, although unless the extruder is 'parked" at a specific location when the image is captured, you really have no way of telling where it (the extruder) is. Images can typically be captured on layer change, at specific time intervals or a combination. There is also the question of latency between when the printer event (layer change / time lapse) occurs and when the image is actually captured (could be several seconds). So right there, you don't know where the extruder is, other than "somewhere".

I'm wondering if you are not better off with a calibration mark to make azimuth / elevation / distance determinations ? E.g. some sort of sticker attached to the bed and in view of the camera ? It seems that could work with many types of printers, bed slingers, standard cartesian etc.

Expanding on this last thought. Would the calibration excercise be better served by placing the calibration images ont he printer bed and then capturing images from various X Y Z coordinates. For sure, in the case of my cartesian printer, X and Y would never change but Z would. For othe types of printers, the results would be different.

Provided one of the calibration shapes remains on the bed during an actual print, does this then provide accurate information for what you want to achieve ?

[yasas]

@stuartofmt
Yeah I agree. But having an estimate would be sufficient to support the calibration process.

So the test print we ask the users to print and take a timelapse of does actually contain markers so we can come up with a mapping between real-world and pixel coordinates.

Yeah you're right. It was also our initial though to use the calibration image and take images from different coordinates but then later we realised that it would potentially involve a lot of human errors. Hence, the test print(test_model.stl). But definitely it would be helpful to take different images of it from different coordinates as opposed to only taking from the timelapse coords.

For the time being, yes. The calibration print(test_model.stl) and its timelapse would allow us to determine the camera coordinates/perspective to some extent. It's not perfect but it gives an estimation that's within the tolerance levels we accept.

[T3P3Tony]

@stuartofmt said in Collecting Timelapses - Need your help:

you really have no way of telling where it (the extruder) is

We could extract that from the gcode as long as it's annotated in some way to show the points that the images are captured.

E.g. on layer change

[stuartofmt]

@T3P3Tony said in Collecting Timelapses - Need your help:

@stuartofmt said in Collecting Timelapses - Need your help:

you really have no way of telling where it (the extruder) is

We could extract that from the gcode as long as it's annotated in some way to show the points that the images are captured.

E.g. on layer change

Yes, or pause the print on layer change, capture an image, resume print ( one of the capabilities of DuetLapse3).

I wanted to alert @yasas to the potential problem of latency in image capture and hence variance in the results. I've seen latencies of several seconds especially if the images are being streamed from a web service. Specifically, even if the image capture is requested at the time of layer change and we know XYZ, the actual image may be of some time later and the extruders XYZ could be some distance away. The question is, would that be significant?

This may or may not matter, perhaps close enough is good enough. In any case, its something that likely ought to be mentioned in the final paper.

@yasas One thing that would help is to more fully understand the nature of the study. Its clear that accounting for perspective distortion is part of the puzzle. Just why the position of the extruder matters is a mystery to me, other than perhaps its a marker for relatative position. But relative position of what?
Consider the case of a camera fixed to (say) the printer gantry, and using a cartesian printer with a "drop away" bed (E.g. Ender 5). The printing plane never changes relative to the camera, the extruder moves around in that plane. Some elements being printed will be closer / farther left / right in the camera's field of view (but always in the same plane). At any decent print speed, the extruder has move on from the e.g.layer change position, maybe by quite a lot by the time the image is captured. Does this matter? What perspective information is to be calculated.

On the other hand, as the print progresses, already printed portions are moving away from the camera (in the above printer example) and perspective (of the already printed portion) is dominated by z height and independent of the extruder X,Y position.

Again, I'm trying to tease out the objective. Frankly, the hard part of most studies is ariving at a robust design for data capture. To the extent that @yasas can (or needs to) be prescriptive, the better the results can be. Maybe perspective does not matter that much (relative to the objective for this study) but that's a claim that has more weight if the method of assessing perspective does not have a lot of variation.

Sorry for being so long-winded (and repetative).
I'll have some time next week to gather some data for you.

[T3P3Tony]

@stuartofmt AFAIK the position of the extruder only mattered in the case where it was a bed slinger, because its really the Y position of the bed in that case. other than of course the extruder could obscure the print i the images if it was in the way of the camera when a picture was taken.

[stuartofmt]

@T3P3Tony said in Collecting Timelapses - Need your help:

@stuartofmt AFAIK the position of the extruder only mattered in the case where it was a bed slinger, because its really the Y position of the bed in that case. other than of course the extruder could obscure the print i the images if it was in the way of the camera when a picture was taken.

Agree that is likely the case. Given that this is a test model (sorry, I've only just looked) there is only a small number of layers, so Z height variation looks negligible. Also print quality is not really an issue - so perhaps the best approach is to use pause on layer change (where the time lapse software allows) and get the extruder well out of the way

Given this is a masters paper, I was trying to address the more general situation, of an arbitrary model, so that the study as a whole could be more robust.

[T3P3Tony]

@stuartofmt sure yes I think the point of the calibration print is simply to use it as a start point for your individual system so the position of the camera relative to the bed can be determined as a system calibration step. then as long as the camera position does not change relative to the bed, the actual time lapses can be of all sorts of models

[stuartofmt]

@yasas

I have not forgot about your request. I was in the middle of some changes to my setup, finishing off a plugin (for timelapse) and life ... it all took time.

I've got to revamp my camera mount since it is at way too shallow an angle (relative to the bed) for your purposes (looks straight across the bed at bed level). As soon as that is done, I'll create some output for you.

[yasas]

Hi @stuartofmt

Sorry I missed this.

No no that's totally fine. And I really appreciate it.

Take your time.
(And I know it's kind of a big ask given the calibration prints/printouts and everything. But it will definitely be helpful. Thank you.)